In the previous post I looked at how the difference between a reading comprehension approach and an information-processing approach plays out in an example test question. This difference isn’t simply about test questions and testing approaches. It has a direct impact and a series of consequences on the LBS system as a whole, as the difference also plays out in the curricular and accountability frameworks that shape day-to-day teaching and learning. The unique curricular and accountability approach taken by LBS has effectively disconnected and marginalized the LBS system from the broader education system in Ontario.
The disconnection is most apparent when comparing standardized literacy testing in both systems. (Standardized testing designed for accountability may be different from day-to-day methods used; however standardized testing has tremendous influence.) A comparison between the Ontario Secondary School Literacy Test (OSSLT), a mandated test for Grade 10 students, and the OALCF Milestones and ESEE illustrates the system-wide divergence.
By focusing on the OSSLT, I am not suggesting that the test offers a better approach to literacy assessment; there are numerous issues and concerns with the high-stakes test raised here and here, for example. The comparison is another way to look at the unique reading pedagogy introduced through the OALCF curriculum reform. It is also another way to recognize the perversity of a pedagogical approach that disengages literacy learning in LBS from literacy learning in the broader education system, the very system the majority of students in LBS are preparing to re-enter or enter for the first time. (To learn more about the OSSLT I examined sample tests available on the Education Quality Accountability Office (EQAO) site.)
Texts to be read
Both assessments use various documents and texts that one could encounter in daily life, including newspaper articles, maps and written procedures. But the similarities end there.
The OSSLT includes a wider range of text types and avoids the narrow focus on rules and procedures found in the ESEE and ESDC skills indicator. OSSLT texts include subject-based content and first-person narratives. In general, the texts in the OSSLT are unaltered and intact containing various references to place names, people and organizations. The choice of content connects to the curriculum , general knowledge about living in Ontario and current events, all of which are likely the focus of learning activities and discussion in the school system. The texts maintain some important formatting and navigational cues, including photos, so one can readily recognise a newspaper article or a flyer about recycling or a textbook excerpt.
In comparison, the OALCF Milestones and ESEE contain altered texts focused mostly on work-based procedures, rules and protocols. Both also contain a few texts related to attending an education program, but these are focused on rules and processes rather than content knowledge. When the texts are altered, important cues and supports are removed. In addition, all direct references to people, place names and organisations, which can also help the reader make meaning of the text, are removed, except for a reference to the college’s online high school equivalency program in the ESEE. In the Milestones, test-takers are directed to read altered secondary school course descriptions with made up course codes and titles rather than an actual excerpt from the Ontario curriculum.
Questions to be answered
Similar to the example test question described in the previous post, it is the questions to be answered that demonstrate the more substantial differences between the information-processing approach used in the ESEE and Milestones and the reading comprehension approach used in the OSSLT.
In the OSSLT, test questions require the test-taker to explain, describe, summarize and draw conclusions. To formulate a response a deep reading of the text from beginning to end is required. Then the test-taker may have to re-read sections. Questions are also designed to demonstrate particular linguistic technical skills and knowledge such as word meanings, idioms, metaphorical meanings, synonyms, grammatical conventions, etc. Two types of responses are used: multiple-choice and fully composed written responses up to a paragraph in length. Both reading and writing are assessed in the OSSLT.
In the ESEE and Milestones (and all other international literacy testing spin-offs) questions require the test-taker to superficially scan the text to locate a piece of information that matches the test question. Test-takers may need to scan more than once, but deep reading and understanding is not required. In order to respond to test questions, test-takers don’t have to have knowledge of the topic and content. For example, the ESEE contains questions about polymers and heat recovery from milk. Although the Milestones do attempt to integrate more general content, it may not be familiar to individual students (this can also be a problem with the OSSLT).
The Milestones do include some questions focused on word meanings, which deviates from the international literacy testing methods, but emulates reading comprehension approaches. Milestones also incorporate other types of responses such as written responses requiring one word or phrase, yes/no responses and ordering responses. No responses require fully composed sentences or paragraphs. The adaptations made to the Milestones mean they don’t fully adhere to either the information-processing approaches or a reading comprehension approach. Neither the Milestones nor the ESEE integrate writing; both are only reading tests, and not literacy tests.
What is being learned about reading in both approaches?
To successfully complete the OSSLT the following skills, strategies and understandings are drawn upon:
- Reading intently from beginning to end
- Paying attention to detail
- Developing knowledge of various text structures and how they are organized to navigate the text as a whole
- Drawing on content knowledge and cultural knowledge
- Developing new knowledge from the text in order to respond to questions
- Making personal connections with the content in order to relate experiences and opinions with the author or content
- Learning how to use text to understand other points of view and actions
- Integrating reading with writing to support deeper reading and thinking.
However, when preparing to complete tests like the OALCF Milestones and ESEE, a different collection of skills, strategies and understandings are developed by readers.
- Scanning the text quickly and superficially
- Searching for potential matches and responses without getting side-tracked
- Setting up a mental system to determine an ideal match
- Learning to discard extraneous and interfering information
- Disconnecting personal experiences, thoughts and opinions from the reading process.
Does this sound more like computer processing than reading? It was the intent of test developers to develop a textual processing model based on the idea that our brains function like computers. The error analysis provided the details for the overall model, and an information-processing metaphor provided the general framework.
The decision to align the LBS system with international literacy levels, scores and standards statements (i.e. descriptions of complexity) means adults are learning a strange and perverse pedagogy of reading that completely contradicts literacy learning in Ontario’s K-12 system. It also contradicts predominant approaches to teaching literacy used in all education systems. Did system developers set out to marginalize LBS from the broader education system? Is it their goal to support the development of unthinking, disengaged and superficial readers? Likely not, but the decision to embrace international testing methods without fully understanding the implications has serious consequences for learners, educators and the system as a whole.
(Looks like I will need to add part five to this series in order to address the consequences more thoroughly.)
Fair Assessment Practices 