Linguistic complexity and differential item functioning (DIF) for English language learners (ELL) in math word problems

This dissertation presents three papers about differential item functioning (DIF) in math word problems for English Language Learners (ELLS). The purpose of this work is to examine non-mathematical linguistic complexity as a source of DIF for ELLS in the Massachusetts Comprehensive Assessment System (MCAS) 4 th -grade math test. Chapter 1 describes the relationship between item measures of linguistic complexity, non-linguistic forms of representation and uniform DIF measures based on Item Response Theory (IRT) difficulty parameters. This study revealed that the greater the item non-mathematical lexical and syntactic complexity, the greater are the differences in difficulty parameter estimates favoring non ELLS over ELLs. However, the impact of linguistic complexity on uniform DIF is attenuated when items provide non-linguistic schematic representations that help ELLs make meaning of the text, suggesting that their inclusion could help mitigate the negative effect of increased linguistic complexity in math word problems. Chapter 2 provides an in-depth analysis of specific items identified as showing DIF for ELLs using two non-parameterc DIF-detection methods, Mantel-Haenszel (Holland & Thayer, 1988) and standardization (Dorans & Kulick, 1986).

Through textual analyses and children's responses to think-aloud protocols, this chapter illustrates some of the linguistic characteristics of math word problems that pose disproportionate difficulty for ELLs. Among them are the use of complex multi-clausal sentences with long noun phrases, unfamiliar vocabulary, polysemous words, and cultural references. Based on findings reported in previous chapters, Chapter 3 proposes the use of a designated anchor, comprised of linguistically simple items, for detecting DIF for ELLs. Since linguistic complexity of math items represents a source of construct-irrelevant difficulty for ELLs, the unbiased conditioning variable for DIF detection should only include linguistically simple items. The chapter describes a replicable process for selecting anchor items based on the analysis of items' linguistic complexity. It also compares IRT uniform DIF indices obtained through the proposed designated anchor with those obtained through the commonly used all-item anchor in the context of IRT likelihood-ratio test DIF estimation. The dissertation findings have important implications for guiding test construction and analysis.