Influential Cases in Multilevel Modeling. A Methodological Comment.

Abstract

A large number of cross-national survey datasets have become available in recent decades. Consequently, scholars frequently apply multilevel models to test hypotheses on both the individual and the country level. However, no currently available cross-national survey project covers more than 54 countries (GESIS 2009). Multilevel modeling therefore runs the risk that higher-level slope estimates (and the substantial conclusions drawn from these estimates) are unreliable due to one or more influential cases (i.e., countries). This comment emphasizes the problem of influential cases and presents ways to detect and deal with them. To detect influential cases, one may use both graphic tools (e.g., scatter plots at the aggregate level) and numeric tools (e.g., diagnostic tests such as Cook’s D and DFBETAS). To illustrate the usefulness and necessity of these tools, we apply them to a study that was recently published in this journal (Ruiter and De Graaf 2006). Finally, we provide recommendations and tools to detect and handle influential cases, specifically in cross-sectional multilevel analyses.