Incorporating Complex Sampling Weights in Multilevel Analyses of Education Data

Abstract

Large-scale assessment survey (LSAS) data are collected via complex sampling designs with special features (e.g., clustering and unequal probability of selection). Multilevel models have been utilized to account for clustering effects whereas the probability weighting approach (PWA) has been used to deal with design informativeness derived from the unequal probability selection. However, the difficulty of applying PWA in multilevel models (MLM) has been generally underestimated and practical guidance is scarce. This study utilizes an empirical as well as a Monte Carlo simulation investigation to examine the performance of the multilevel pseudo maximum likelihood (MPML) estimation based on information derived from the Early Childhood Longitudinal Study Kindergarten cohort of 2010-2011 (ECLS-K:2011). Variance components and fixed effects estimators across four estimation methods including three MPML estimators (i.e., weighted without scaling, weighted size-scaled and weighted effective-scaled) and the unweighted estimator are provided. Practical guidance about the use of sampling weights in MLM analyses of LSAS data is also offered.

Department(s)

Psychological Science

Keywords and Phrases

Large-Scale Assessment Surveys; Monte Carlo Simulation; Multilevel Models; Multilevel Pseudo Maximum Likelihood; Sampling Weights

International Standard Serial Number (ISSN)

1531-7714

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2022, All rights reserved.

Publication Date

01 Jan 2022

Link to Full Text

Share

 
COinS