Complex Sampling Designs in Large-Scale Education Surveys: A Two-Level Sample Distribution Approach
Abstract
Large-scale education data are collected via complex sampling designs that incorporate clustering and unequal probability of selection. Multilevel models are often utilized to account for clustering effects. The probability weighted approach (PWA) has been frequently used to deal with the unequal probability of selection. In this study, we examine the performance of an intuitive, easy to implement approach named the sample distribution approach (SDA) that utilizes Markov Chain Monte Carlo (MCMC) methods and Bayesian inference. Our simulation design focused on clustering effects, represented by the Intraclass Correlation (ICC) and on the sample size of the cluster. We analyzed a large-scale educational assessment dataset (Early Childhood Longitudinal Study - Kindergarten 2011) to compute estimates for the simulation. Findings reveal that the SDA overall generated reliable posterior distributions of parameters and had small error variances. In addition, although design informativeness is important, the ICC and cluster sample size factors had little impact on the performance of this model-based approach.
Recommended Citation
Shen, T., & Konstantopoulos, S. (2021). Complex Sampling Designs in Large-Scale Education Surveys: A Two-Level Sample Distribution Approach. The Journal of Experimental Education, 89(2) Taylor & Francis.
The definitive version is available at https://doi.org/10.1080/00220973.2021.1891007
Department(s)
Psychological Science
Keywords and Phrases
Sampling designs; Multilevel models; SDA; MCMC; Bayesian analysis
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2021 Taylor & Francis, All rights reserved.
Publication Date
22 Mar 2021