Exploratory Research Findings
Analytic Essay Coding
RQ: (1) What are the dimensions of text-based analytical writing of secondary students? (2) How do the various dimensions of analytical writing predict overall writing quality? (3) How do demographic factors predict performance in different dimensions.
Data: 206 text-based analytical essays.
Analysis: Analytic coding of 20+ items related to writing quality; Confirmatory Factor Analysis and Structural Regression.
Findings: Writing quality is best represented by three related dimensions: Ideas/Structure, Evidence Use, and Language Use (Figure 1). All dimensions predict holistic scoring. Females perform better than males in all dimensions. EL students perform worse in all dimensions (Figure 2).
How these findings inform the Development Phase: The three factors may be considered as the categories represented in evaluative rubrics. All of the skills comprising each factor should be considered for instructional purposes. ELs need writing instruction that focuses on all elements of writing, not just language.
Analytic Essay Coding Part 2
RQ: How did Pathway intervention affect discrete writing skills?
Data: 206 students’ pre and post text-based analytical essays from a RCT
Analysis: Analytic coding of 20+ items related to writing quality. Descriptive statistics. Multiple Regression.
Findings: All discrete items (except “demonstrates control of conventions”) related to writing quality were affected by the Pathway intervention, with the strongest effects on Ideas/Structure and Evidence Use dimensions of writing quality.
How these findings inform the Development Phase: All skills other than Language Use skills (e.g. diction and syntax) improve in a year-long intervention. The stronger relation between Evidence Use and holistic scores suggest skills related to integrating and interpreting textual evidence are especially important. Overall, results suggest a holistic approach to writing development.
RQ: (1) To what extent do 7th grade students incorporate perspective taking in source-based analytical writing? (2) How does the extent of perspective taking portrayed in writing differ by students’ demographic backgrounds, such as sex and English language learner status? (3) Is perspective taking in writing related to overall writing quality, controlling for demographic backgrounds?
Data: 195 7th grade source-based analytical essays.
Analysis: Analytic coding of perspective taking in T-units; multilevel analysis.
Findings: Students wrote more from their own perspective than that of others. Female students exhibited more varied perspectives, but there was no meaningful difference by English learner status. Greater perspective taking, particularly that of higher level of perspectives (i.e., dual perspective), was associated with better writing quality, after accounting for students’ demographic backgrounds and essay length.
How these findings inform the Development Phase: These results underscore the importance of writing from multiple perspectives on writing quality. Explicit instruction on perspective taking should be considered.
For more information: See our blog and the full article:
Cho, M., Kim, Y. S. G., & Olson, C. B. (2021). Does perspective taking matter for writing? Perspective taking in source-based analytical writing of secondary students. Reading and Writing, 1-21.
RQ: (1) Does the MacArthur, Jennings, & Philippakos (2019) model for basic college writers also predict human ratings of quality for secondary school writers in ELA and in history? (2) How are ELA and history papers different? (3) What constructs did the Pathway intervention impact from pre and post essays in the RCT?
Data: Essays from 7-12 grade students in Pathway RCT and history classes piloting WRITE Center prompts.
Analysis: Structural equation modeling of natural language processing information from Coh Metrix and holistic quality ratings.
Findings: MacArthur et al. (2019) model described both ELA and history essays, with somewhat different loadings on the four constructs of connectives, referential cohesion, syntactic complexity, and lexical complexity. Students wrote about 100 more words in the history essays, with higher level of cohesion, more logical connectives, and fewer adversative/ contrastive connectives compared to ELA essays.The Pathway intervention primarily increased students’ writing fluency, improving their word count especially for female students. English learners’ only received about half the treatment effect, however.
How these findings inform the Development Phase: With respect to the differences in history and ELA, there were No clear implications, because of the different populations in each set of papers. However, we discussed with history teachers and disciplinary experts about the possible need to support increased use of either the logical connectives and/or the adversative/ contrastive connectives through the use of additional sentence starters or frames.