Complex Sampling Designs in Large-Scale Education Surveys: A Two-Level Sample Distribution Approach

Abstract

Large-scale education data are collected via complex sampling designs that incorporate clustering and unequal probability of selection. Multilevel models are often utilized to account for clustering effects. The probability weighted approach (PWA) has been frequently used to deal with the unequal probability of selection. In this study, we examine the performance of an intuitive, easy to implement approach named the sample distribution approach (SDA) that utilizes Markov Chain Monte Carlo (MCMC) methods and Bayesian inference. Our simulation design focused on clustering effects, represented by the Intraclass Correlation (ICC) and on the sample size of the cluster. We analyzed a large-scale educational assessment dataset (Early Childhood Longitudinal Study - Kindergarten 2011) to compute estimates for the simulation. Findings reveal that the SDA overall generated reliable posterior distributions of parameters and had small error variances. In addition, although design informativeness is important, the ICC and cluster sample size factors had little impact on the performance of this model-based approach.

Department(s)

Psychological Science

Keywords and Phrases

Sampling designs; Multilevel models; SDA; MCMC; Bayesian analysis

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2021 Taylor & Francis, All rights reserved.

Publication Date

22 Mar 2021

Link to Full Text

Share

 
COinS