![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
Someone brought "They Don’t Read Very Well: A Study of the Reading Comprehension Skills of English Majors at Two Midwestern Universities" by Susan Carlson, Ananda Jayawardhana, Diane Miniel (CEA Critic, Volume 86, Number 1, March 2024) to my attention today. a pdf of the paper is here
This is an appallingly badly designed study and a seriously flawed paper and the authors should be fucking ashamed of themselves. (Carlson is an English professor, Miniel was one when the research was conducted in the teens, and Jayawardhana is a statistician, all at Pittsburgh State U. Why they ventured all the way to Kansas to conduct their research is a mystery).
They interviewed 85 students majoring in English or English education (mostly juniors and seniors with some sophomores and four freshmen) at two Kansas universities. Each participant took a reading test designed to determine 10th grade literacy, filled out a survey, and then were asked to spend 20 minutes reading the opening seven paragraphs of Dickens' Bleak House aloud, pausing to explain the meaning of each sentence as they went along. They were given dictionaries and allowed to look things up in those or on their phones. They were told that it didn't matter if they did not finish going through the passage before time was up. (I'll put the opening paragraphs they used in the first comment below so you don't have to google it on project Gutenberg)
The authors recorded the students reading aloud and explaining the passage to an interviewer, transcribed those recordings, and then tagged and analyzed the transcripts. Their conclusions were that only five percent of the students were able to properly understand the dickens passage ("proficient readers") another 38 percent understood about half the passage ("competent readers"), and 58 percent struggled to understand the passage ("problematic readers"). "Problematic readers often described their reading process as skimming and/or relying on SparkNotes" (page 6).
They conclude that most of the people majoring in English do not have the reading skills necessary for such a major and do not gain those reading skills from their first and second year classes.
This paper has two huge and one large problems.
1. I am probably in or near their top 5% of readers, and I would have struggled and been incredibly frustrated if someone asked me to read the beginning of a Dickens novel one sentence at a time, explaining each sentence as I go along. That's not how novels are intended to work, especially densely written 19th century novels. Reading a whole paragraph, or the whole passage, then going back and working through it bit by bit, sure. But expecting to extract meaning from each sentence in isolation without knowing what's coming in the next sentence -- no fucking way. Fictional prose is intended to *flow*, you can't ask someone to chop it up into bits based on the punctuation and expect those bits to make sense without the context of what comes next.
2. They assigned 21st century American students a passage from a mid 19th century British novel, thereby turning it from a test of whether or not they could read and understand a chunk of complex literary prose, into a test of that *plus* whether or not they had adequate working knowledge of an archaic and foreign prose style, culture, vocabulary, and setting. OF COURSE the students struggled and did poorly in figuring out the meaning of "Michaelmas term," "the Lord Chancellor," and "Lincoln’s Inn Hall" in just the first sentence. Not because they're poor readers, but because they're not 19th century Londoners.
If the researchers had been serious about trying to gauge the students' reading ability without confounding their results with the student's poor familiarity with 170 year old prose style, setting, and culture, they could have assigned them a dense passage from a 20th or 21st century American literary novel. But they didn't. This is the worst kind of bigoted, classist, prior knowledge based intelligence testing. Shame on the authors, and shame on the journal for publishing this crap.
The not so huge but still large problem:
3. Forty-one percent of the study's participants were "English education" majors, not traditional English majors. Even if both majors are taught by the English department, they are very different beasts with very different course requirements that attract very different types of student. One is a major for people who wish to become primary and secondary school teachers of English (aka grammar, literacy, writing, and maybe also age-appropriate novels), the other is for people who enjoy reading and analyzing literature. The authors do not say anything about how the two majors differed in their ability to understand the opening of Bleak House, an omission that makes me raise my eyebrows very high.
In sum: Those who can't, teach. Those who can't do research, write poorly conceived papers on how their students are bad at reading.
***
That said, I am unsurprised that significant numbers of students struggled with the passage. Even aside from the design flaws which artificially lowered the scores of the students in the study, some people who aren't actually interested in reading and thinking about novels get a degree in English as a job credential (like those education majors who were included in the study, who may only be in the English department because it's a path to getting a teaching certificate that doesn't involve math).
And it is a sad fact that secondary schools turn out tons of students each year who have never really learned to read well, some of them with high GPAs. Now that a 4 year degree is required for many jobs that used to be open to high school graduates, they end up taking classes they're not really equipped for.
Which reminds me of something my sister said about her year teaching English at a magnet school in Austin Texas. She was one of the only teachers who stocked her classroom with age appropriate books and encouraged her students to pick out and read ones that appealed to them. The administration and most of the rest of the teachers regarded this sort of thing as a bad idea, because students reading books for fun were not spending time honing the skills needed for doing well on the next standardized test.
To the extent that it's a real problem, and not one manufactured to produce a shocking academic article, it's multi generational at this point. Those who don't read books themselves, have become school teachers and are teaching children to read but not to be able to read very well. Those children grow up, go to college, and frustrate their professors with their lack of reading skills.
This is an appallingly badly designed study and a seriously flawed paper and the authors should be fucking ashamed of themselves. (Carlson is an English professor, Miniel was one when the research was conducted in the teens, and Jayawardhana is a statistician, all at Pittsburgh State U. Why they ventured all the way to Kansas to conduct their research is a mystery).
They interviewed 85 students majoring in English or English education (mostly juniors and seniors with some sophomores and four freshmen) at two Kansas universities. Each participant took a reading test designed to determine 10th grade literacy, filled out a survey, and then were asked to spend 20 minutes reading the opening seven paragraphs of Dickens' Bleak House aloud, pausing to explain the meaning of each sentence as they went along. They were given dictionaries and allowed to look things up in those or on their phones. They were told that it didn't matter if they did not finish going through the passage before time was up. (I'll put the opening paragraphs they used in the first comment below so you don't have to google it on project Gutenberg)
The authors recorded the students reading aloud and explaining the passage to an interviewer, transcribed those recordings, and then tagged and analyzed the transcripts. Their conclusions were that only five percent of the students were able to properly understand the dickens passage ("proficient readers") another 38 percent understood about half the passage ("competent readers"), and 58 percent struggled to understand the passage ("problematic readers"). "Problematic readers often described their reading process as skimming and/or relying on SparkNotes" (page 6).
They conclude that most of the people majoring in English do not have the reading skills necessary for such a major and do not gain those reading skills from their first and second year classes.
This paper has two huge and one large problems.
1. I am probably in or near their top 5% of readers, and I would have struggled and been incredibly frustrated if someone asked me to read the beginning of a Dickens novel one sentence at a time, explaining each sentence as I go along. That's not how novels are intended to work, especially densely written 19th century novels. Reading a whole paragraph, or the whole passage, then going back and working through it bit by bit, sure. But expecting to extract meaning from each sentence in isolation without knowing what's coming in the next sentence -- no fucking way. Fictional prose is intended to *flow*, you can't ask someone to chop it up into bits based on the punctuation and expect those bits to make sense without the context of what comes next.
2. They assigned 21st century American students a passage from a mid 19th century British novel, thereby turning it from a test of whether or not they could read and understand a chunk of complex literary prose, into a test of that *plus* whether or not they had adequate working knowledge of an archaic and foreign prose style, culture, vocabulary, and setting. OF COURSE the students struggled and did poorly in figuring out the meaning of "Michaelmas term," "the Lord Chancellor," and "Lincoln’s Inn Hall" in just the first sentence. Not because they're poor readers, but because they're not 19th century Londoners.
If the researchers had been serious about trying to gauge the students' reading ability without confounding their results with the student's poor familiarity with 170 year old prose style, setting, and culture, they could have assigned them a dense passage from a 20th or 21st century American literary novel. But they didn't. This is the worst kind of bigoted, classist, prior knowledge based intelligence testing. Shame on the authors, and shame on the journal for publishing this crap.
The not so huge but still large problem:
3. Forty-one percent of the study's participants were "English education" majors, not traditional English majors. Even if both majors are taught by the English department, they are very different beasts with very different course requirements that attract very different types of student. One is a major for people who wish to become primary and secondary school teachers of English (aka grammar, literacy, writing, and maybe also age-appropriate novels), the other is for people who enjoy reading and analyzing literature. The authors do not say anything about how the two majors differed in their ability to understand the opening of Bleak House, an omission that makes me raise my eyebrows very high.
In sum: Those who can't, teach. Those who can't do research, write poorly conceived papers on how their students are bad at reading.
***
That said, I am unsurprised that significant numbers of students struggled with the passage. Even aside from the design flaws which artificially lowered the scores of the students in the study, some people who aren't actually interested in reading and thinking about novels get a degree in English as a job credential (like those education majors who were included in the study, who may only be in the English department because it's a path to getting a teaching certificate that doesn't involve math).
And it is a sad fact that secondary schools turn out tons of students each year who have never really learned to read well, some of them with high GPAs. Now that a 4 year degree is required for many jobs that used to be open to high school graduates, they end up taking classes they're not really equipped for.
Which reminds me of something my sister said about her year teaching English at a magnet school in Austin Texas. She was one of the only teachers who stocked her classroom with age appropriate books and encouraged her students to pick out and read ones that appealed to them. The administration and most of the rest of the teachers regarded this sort of thing as a bad idea, because students reading books for fun were not spending time honing the skills needed for doing well on the next standardized test.
To the extent that it's a real problem, and not one manufactured to produce a shocking academic article, it's multi generational at this point. Those who don't read books themselves, have become school teachers and are teaching children to read but not to be able to read very well. Those children grow up, go to college, and frustrate their professors with their lack of reading skills.
