Gathering meaningful evidence of students’ thinking about reading
By Nicole Marie and Judy Douglas
As teachers, we are familiar with some version of a teaching and learning cycle that includes:
assessing or gathering data about student learning
using that knowledge to target planning to meet students’ learning needs
monitoring or tracking student progress
analysing or evaluating the outcome
The evidence and data collected is usually something that is directly observable and ‘submittable,’ such as a piece of writing for evidence of writing skills, or an observation of a small-group discussion or oral presentation to assess speaking (presentation) skills.
But this is not as easy in practice when we are gathering meaningful evidence of students’ thinking about reading. When we compare it to something that seems more tangible like writing, the thinking processes involved in reading comprehension happen in a student’s mind and are ‘invisible.’ Because of this, it can be challenging to find observable ways for students to show us their reading processes without also having to demonstrate oral or writing skills. What’s more, what happens in a students’ mind when they make sense of what they are reading is dependent on so many variables that are constantly fluctuating, such as the reader’s prior knowledge, their command of the English language, their familiarity with the genre they are reading, and their memory and stamina, to name a few (Serravallo, 2018). As Alex Quigley surmises in his book Closing the Reading Gap:
“…reading comprehension can prove vast, diverse and therefore devilishly hard to pin down and assess. We can only ‘observe indirect symptoms’ of said comprehension” (Quigley, 2020: 67). As a result, we might find ourselves falling back on ‘traditional’ measures such as a text response essay or comprehension questions that seemingly give us a good ‘picture’ of the students reading skills and knowledge. Through these measures, we determine what students know about the content they are learning about and their ability to express this in writing, but obtain a less clear picture about students’ reading processes. For example, when we ask students to summarise a text, we are often just looking at if they have accurately summarised the content of a specific text, rather than finding out about their process in summarising a text. In other words, the focus is on the product (the summary) rather than how they went about it (the process of summarisation). This can make it harder for us to ‘diagnose’ and plan for the improvement of their reading skills, and to give quality feedback that contains a “recipe for future action”, to quote Dylan William and Siobhan Leahy (2015: p. 143). While comprehension questions and text essays can give us a glimpse of student understanding of what they have read, these measures are still asking students to demonstrate that they can ‘answer’ something very specific about a text (or texts) and can limit the scope of student response and interpretation. As Marilyn Pryle suggests (when considering reading comprehension questions), for some students, “once the answer is found, the work is done and no additional thought need be invested” (2018: p.6).
Other measures such as NAPLAN results and benchmarking tests (like PAT-R, On Demand, Essential Assessment, Probe, PM testing and Fountas and Pinnell, to name a few) have also been used to gain a snapshot of students’ reading comprehension at specific points in time. While NAPLAN can be useful for looking at system-wide trends and progress over time and can prompt some reflection on general strengths and weaknesses regarding curriculum and teaching across a whole school, it is limited in identifying each individual students’ learning needs in a frequent enough and timely manner, not to mention in a way that is immediately ‘usable’ for teachers to inform their day-to-day practice so they can target their teaching. While benchmarking tests are taken more frequently than NAPLAN at specific times across a school year and can provide an indication of growth over time, it is not long before they are out of date when it comes to providing a valid snapshot of current student progress. Therefore, their use in targeting immediate student learning needs and their ability to inform teaching in a way that builds on student growth achieved throughout the school year is limited.
While we recognise the limitations of the above measures, at no point are we suggesting that we ‘throw the baby out with the bathwater’ and get rid of them. What we are suggesting though, is to approach the assessment of thinking about reading with these limitations in mind, with the awareness that there is no such thing as the ‘perfect’ assessment tool that is going to assess every skill every time. Instead, we need to seek out multiple sources of evidence, broaden our repertoire of assessment tools, be cognizant about what we are looking for and determine what conditions, and which tools, will give us the most valid evidence and information about what students can do, need support with, and what they are ready to learn next. In this way, we can approach the assessment of students’ thinking about reading with more confidence and precision.
To gain an overview of eight ways in which we can assess and monitor students’ thinking about reading, please see the examples below;
1. Reader’s notebook entries and low-stakes reader responses
Asking students to record their thinking in a low-stakes environment gives us observable evidence of what, and how, a student is thinking about when they are reading. These entries and responses are made as students are reading, or pausing as they read; ensuring they are immediate and ‘unfiltered.’
The focus of notebook entries and reader responses can range from teacher-directed (and modelled) to completely student-led (or somewhere in-between). Assessment does not include writing skills but is on the level of metacognition demonstrated (except for the students’ use of metalanguage to articulate their thinking—which is an option for assessment only when students become more proficient in their recording)
Purpose:
To collect written observable data of student thinking in an organised way
Get an understanding of how a student is applying comprehension strategies; able to see confidence and competence, or misconceptions and hesitancy
Gain a sense of a students’ engagement with texts, and what it is about a book that interests them
What they can tell us:
Understanding of the application of comprehension strategies
Effectiveness of teacher instruction in reading
Possible misconceptions or misunderstandings of a text or a strategy
Ability to convey thinking through writing
Students’ focus of interest in a text (or across texts) - how a text is ‘speaking’ to that student through what they select to write about
What they don’t tell us:
All their thinking - we can only observe what they have written down (and are not able to prompt for further information as we are not observing the data immediately)
Student thinking beyond what they can express in writing
The source/reason of a misconception or hesitancy (or lack of entries/responses)
What to look for and monitor:
Application of reading comprehension strategies
Interest and engagement in reading
Sharing thinking about things beyond what has been taught (or asked/prompted)
Use of metalanguage to convey their thinking in writing
Applying wider/multiple reading strategies to a text
2. QuickWrites
A QuickWrite response to reading provides an opportunity for students to ‘grow their thinking ‘ by processing, reflecting, exploring and expanding on their thinking. It is different from low stakes reader responses and reader’s notebook entries as it is not always directly related to their thinking processes. Completed in less than ten minutes - with little concern about writing conventions - students take a thought and write about it. What they write about can be prompted or guided by the teacher or a supporting resource (such as a matrix of QuickWrite prompts), or determined by the student (selecting an idea or thought they would like to explore through the QuickWrite). A QuickWrite is a great way to activate and consolidate learning from a lesson and to evaluate a students’ depth of understanding.
Purpose:
To collect written observable data of student thinking
Foster exploration and expansion of thinking
Develop analysis and critical thinking skills
Evaluate how engaged or how closely a student is working with a text
Depth of thinking about their reading
Encourage students to develop their own interpretations of texts
What they can tell us:
Student thinking about what they are reading
Depth of thinking
What it is about a text that is ‘sticking’ with a student (and level of engagement)
How a student is seeing themselves as a reader
Level of analysis and critiquing skills
How students are navigating a text
Connections being made with a text (self, text-text, world etc)
What they don’t tell us:
All their thinking - we can only observe what they have written down (and are not able to prompt for further information as we are not observing the data immediately)
Application of individual reading strategies
The source of any misconceptions or misunderstandings
What to look for and monitor:
Students’ views of themselves as a reader
Depth of thinking
Reading engagement
Skills in analysing text
Critical thinking skills
Text connections
Developing interpretations and perspectives
3. Thinking routines that provide a targeted snapshot of reading skills and processes
According to Harvard Graduate School of Education’s Project Zero, a thinking routine is “a set of questions or a brief sequence of steps used to scaffold and support student thinking.” In this specific sense, we are talking about sets of questions, sequences or activities that help guide the development of students’ thinking about reading, usually in relation to a specific skill or combination of skills (eg: like summarising, synthesising or inferring, for example). In terms of the ‘assessment product’, we are talking about quick written or spoken examples of students using the routines that provide evidence of student thinking.
Some examples include:
Six word summary (focus is on summarising and interpreting. See the next image below on the left)
3-2-1 (focus can be on questioning, inferring main ideas, and determining important information/facts/vocabulary/concepts. See the next image below on the right)
What makes you say that? (focus is on justifying inferences and interpretations with evidence)
I used to think/Now I think (focus is on synthesising and interpreting)
See, think, wonder (focus is on inferring, interpreting and questioning)
SQ3R (focus is on setting a reading purpose, questioning, determining important information and reviewing)
It says, I say, and so… (focus is on inferring while using textual evidence)
‘Somebody wanted to... but... so…’ (focus is on summarising plot with a focus on character and narrative conflict)
Purpose:
To collect written observable data of students’ application of a particular reading skill/set of skills
To provide an example of the thinking processes students engage in when demonstrating particular reading skills
To scaffold deeper thinking about reading --can provide students with a series of cognitive and metacognitive prompts to develop their thinking
What they can tell us:
How students apply a particular set of reading strategies or demonstrate a particular set of reading skills
Provide information regarding the depth of student thinking about texts
Gives an indication of further scaffolding that might be needed in relation to specific aspects of a particular reading skill or set of skills (eg: interpreting indirectly stated information to form an inference about a text, how to determine important information and concepts before summarising, comparing earlier thinking to more developed thinking when synthesising understanding, etc.)
What they don’t tell us:
How students synthesise their thinking about reading into more sustained reading responses
About skills and strategies students use outside the scope of the particular thinking routine applied
What to look for and monitor:
How students engage in particular reading processes
Student application of reading skills or strategies relevant to the thinking routine used
Depth of thinking about texts
Opportunities to further develop student thinking
Engagement in a text
See Harvard Project Zero’s Thinking Routine Toolbox for more routines that can be easily adapted for assessment of thinking about reading.
4. Graphic organisers
Graphic organisers provide students with space to record their inner voice (a conversation they are having with a text) and prompt students to read more purposefully and interactively (Tovani, 2004; 2011). They can also encourage students to organise their thinking in a more systematic way and track the development of their ‘inner conversation’ as they read. When recording their thinking using graphic organisers, students might respond to quotations or specific textual elements and:
Record a reaction to something that strikes them
Ask questions about something that has happened in the text
Give an opinion on how they might respond in a similar situation
Make a connection to the information they read and to what they know
Predict what they think might happen next (Tovani, 2011: p. 50)
Connect cause and effect
Record prior knowledge, questions and learning
Synthesise their thinking
The best examples of graphic organisers used to assess comprehension are the ones that move beyond simply recording information from the text, and instead invite students to interact with elements of the text, providing ample opportunities for meaning-making and justification of thinking. Please see a few examples below:
Inner voice sheet (as seen in Cris Tovani’s Do I Really Have to Teach Reading?)
Say/mean/matter (as seen in Kelly Gallagher’s book Deeper Reading)
Important vs Interesting T-charts (as seen in Stephanie Harvey and Anne Goudvis’ book Strategies That Work - see the yellow-shaded headings in the first image on the left for a variation of this)
Purpose:
Show thinking about a text in a specific way or context
See how deeply students are thinking about what they are reading
Encourages students to have a dialogue with the text
Gives students an easy location to go back to and reflect on their thinking
A source for immediate feedback or to be moderated after a lesson/week
Can prompt students to interact with quotations or other specific elements of the text (eg: characters, main ideas, text structure, plot elements, conflict, etc.)
What they can tell us:
Progress of students’ thinking about a text across short or lengthy periods of time
Identify students who may need intervention or extension
Provide information to prepare for a conference conversation
Engagement with a text
Depth of thinking about a text
Which comprehension strategies they are using; which they are not
How students interact with textual evidence/elements of text
What they don’t tell us:
Knowledge of the text (what it is happening, what it is about)
If the student is reading a ‘just right’ text (if they are reading a student-choice book)
What to look for and monitor:
Depth of thinking about texts
Engagement in a text
Variety of reading comprehension strategies being used
Reading pace (how much they are reading)
For more examples of easily-adaptable examples and for tips on how to use graphic organisers effectively, please see Cult of Pedagogy’s post ‘The Great and Powerful Graphic Organizer’ here.
5. Annotations and sticky notes
An annotation is a “written record of how readers think as they read instead of after they read” (Tovani, 2011; p. 77).
When students annotate, they are required to read the text closely and be conscious about how they are making meaning. They also record questions or thoughts as they come to their mind, which they can always come back to later.
Where annotating directly onto a text is not possible, annotating can be done on sticky notes that can be stuck on the text at the point where the thinking occurred. Annotation can also occur electronically, or in a collaborative context (eg: group annotation of a text, with each student using a different coloured marker).
Purpose:
To support close reading of a text (as it is difficult to comprehensively annotate when skimming and scanning a text)
Helps students to remember what they have read
Gives students an opportunity to record thinking they can go back to and reflect/expand upon
Identify misconceptions or questions that can be addressed immediately
For students to determine what is important
What they can tell us:
Engagement with text (‘fake’ annotations are able to be observed)
What reading comprehension strategies students are using
Which students need intervention
Provides information about points of need for a student, or a class
Pace of reading
How students are navigating a text
Comprehension of texts representing various levels of complexity
What they don’t tell us:
‘After reading’ thinking - bringing it all together
About the students’ ability to construct a sustained, comprehensive reader response or interpretation
The source of students’ misconceptions or ‘breakdown’ in meaning
Student understanding beyond what they can express in writing
What to look for and monitor:
Quality of annotations
Engagement with texts
Depth of thinking about a text
Being able to identify main and supporting ideas
Comprehension of different texts with varying levels of complexity
6. Reader reflections
A student’s reflection about how they are reading can provide the most honest formative assessment data, as well as help students monitor their own learning and set their own goals. One way for students to reflect on their learning in a way that also provides valuable information for the teacher, is by asking students to go back over their records of thinking (such as through their reader’s notebook, annotations, graphic organisers, sticky notes etc) at the end of a week and having students select which ones they think are the best, that they want to expand on, and want the teacher to see. Students then have a chance to reflect on their reading in multiple ways, such as:
Why they selected those notes
Expanding on their notes
How they feel their reading went that week
What they noticed about their thinking over the week
What they are feeling more confident in doing
What they are still struggling with or would like some further support
Purpose:
Opportunity for students to take ownership over what the teacher ‘sees’ about their reading
Understand what students are thinking and how they are applying reading comprehensions strategies
Time for students to go back over their notes and evaluate their quality (and how they are making their thinking visible)
Students can set their own goals
What they can tell us:
What students are thinking about as they read
What students aren’t thinking about (and could be an area of need)
Depth of thinking
Metacognitive awareness of thinking in relation to reading
What is engaging or of interesting about a text for the student
Which strategies students are confident in using
Which strategies students are not using
How a student is seeing themselves as a reader
Effectiveness of teacher instruction
What they don’t tell us:
All their thinking (notes or annotations)
Student understanding beyond what they can express in writing
What to look for and monitor:
Students’ views of themselves as a reader
Students’ understanding of themselves as a reader
Variety of reading comprehension strategies being applied
Depth of thinking
Reading engagement
7. Reading conferences
Having a meaningful conversation with a student about their reading provides an opportunity for students to verbalise their thinking. A purposeful conversation with a student is probably the quickest and most efficient way to:
gather formative data (researching the reader) about their comprehension and understanding of reading strategies;
give immediate feedback;
be able to explicitly teach to target a point of need (teaching the reader) and;
evaluate the outcome of a goal (after setting individualised reading goals).
It is also the ultimate differentiation tool as each conversation or conference can be designed based on the individual student, and questioning and instruction can be adjusted throughout the assessment. However, what we love most about a conversation with a student (and not a question-answer model), is that it provides space for valuing talk about texts and stories.
Purpose:
To explicitly teach reading comprehension strategies directly at the point of need for an individual student
Set, monitor and evaluate individual reading goals
Can track the progress of reading skills and strategies quickly
Find out, and address, reasons for misconceptions or uncertainties
Learn about the student as a reader and build a positive relationship with the student about their reading
Foster engagement with texts
Providing immediate formative assessment data and opportunities for feedback. This could be used as a way to track students’ ability to verbalise their reading processes, and determine how they are applying reading strategies (either generally, or more specific ones that have been the focus of classroom instruction).
What they can tell us:
Students’ level of engagement and interest in what they are reading
Reasons for misunderstandings or uncertainties around reading comprehension strategies
Ability to decode and clarify
Reading fluency
Reading processes for different text genres and text types
About students’ ability to read texts of varying complexity (and to help us guide them to suitable texts)
How students are feeling about their reading progress (reflection)
About students’ depth of understanding of a text
A students’ level of metacognition; their awareness of how they are reading (strategies they are using)
Flexibility in using reading strategies (this can be observed over multiple student think alouds with a variety of texts)
Reading processes for different text genres and text types
Effectiveness of reading instruction
What it doesn’t tell us:
Anything the student is thinking but can’t express verbally
Anything beyond what they demonstrate from just that piece of text
Unable to monitor strategies used when reading across a longer text
What to look for and monitor:
Decoding
Fluency
Application of reading comprehension strategies
Text selection and complexity
Engagement
Level of scaffolding/prompting needed by the teacher; moving towards more independence (with the student leading the conversation)
Increase in text complexity
Willingness to share thinking about reading/volume of sharing
Sharing thinking about things beyond what has been taught (or asked/prompted)
Use of metalanguage to verbalise their inner voice
Applying wider/multiple reading strategies to a text
8. Observing collaborative discussions around texts
It’s one thing to collect evidence of student learning from teacher-student conversations; it’s quite another to analyse students’ collaborative discussions with each other about the texts they are reading. When students engage in collaborative discussion, it allows them to put their thoughts about reading into words and make the ‘invisible’ evidence of their reading processes visible to us. Students can also consolidate and further develop their thinking about texts through discussion.
Collaborative discussions around texts can be held in pairs or in larger groups and can be as formal or informal as you like. They tend to work best when:
A safe, open and inclusive environment for talk has been established
Clear talk guidelines or protocols have been provided (eg: respond to a specific prompt, one voice speaking at a time, listen actively and respond to your partner with a probing question, etc.)
Students have been provided with ample thinking and preparation time beforehand so they have something to share (eg: jot down three thoughts before sharing them with a partner, select an annotation to share, rehearse for a larger group discussion with a partner, etc.)
Questions or prompts are clear and specific, but open-ended enough to provide space for students to express thinking that encompasses a range of possibilities
Students are guided to engage in the language and behaviours typical of academic discussion (asking follow-up questions to elicit deeper thinking, for example. For some examples of question starters students can use, please see this page from the Victorian Government’s Literacy Teaching Toolkit)
The teacher actively observes and monitors talk and provides feedback afterwards
Some examples of collaborative discussion formats/strategies for this purpose include:
Students sharing their think alouds
4As: Assumptions, Agree, Argue, Aspire to
For more examples that can be adapted for assessing students’ thinking about reading, check out Cult of Pedagogy’s article ‘The Big List of Class Discussion Strategies’ (available here) and EL Education’s classroom protocols (available here)
Teachers can watch and record their observations of common strengths, misconceptions or areas for improvement. Alternatively, students can also be guided to observe and record their peers’ demonstration of particular reading behaviours or processes (eg: making connections between different parts of a text or to global issues/broader aspects of the human experience; determining the difference between what’s important and what’s interesting; determining main ideas; using textual evidence and prior knowledge to infer meaning; synthesising textual evidence to build an interpretation; posing questions, etc.).
Purpose:
To observe levels of student engagement/students’ reactions to the texts they are reading
To observe how students interpret meaning when they read
To observe how students justify their thinking about reading
To discern patterns within students’ thinking
To find out about any misconceptions students might have
To inform future lessons and feedback
To track students’ use of reading and/or discussion skills
What they can tell us:
How students demonstrate particular reading behaviours or processes
About patterns of student thinking about reading
About how students consolidate their understanding of texts
Can highlight misconceptions to address
Provides insight into students’ engagement regarding what they’re reading
Depth of thinking about a text
What they don’t tell us:
Anything the student is thinking but can’t (or won’t) express verbally
Whether the quality of student responses is primarily a reflection of their reading comprehension, or if it is more reflective of their level of confidence when engaging in discussions
Student understanding beyond the scope of the questions or prompts posed by students and the teacher
What to look for and monitor:
Students’ application of particular reading strategies or thinking processes to make/build meaning
Level of scaffolding/prompting needed for discussion to flow
‘Aha’ moments (moments of sudden insight or understanding)- reflect on what prompted them
Patterns of student reasoning
Misconceptions to address
Connections students make —to other parts of the text, their lives, the world, other texts, other human experiences, others’ interpretations
Evidence of higher-order thinking (eg: synthesising, analysing, evaluating, etc.)
…What formative assessment strategies and tools do you use to assess reading comprehension?
References:
Gallagher, K. (2004). Deeper Reading: Comprehending Challenging Texts 4-12. Portland: Stenhouse Publishers.
Goss, P., Hunter, J., Romanes, D. & Parsonage, H., (2015). Targeted Teaching: How Better Use of Data Can Improve Student Learning. Parkville: Grattan Institute.
Harvey, S. & Goudvis, A. (2017). Strategies That Work: Teaching Comprehension for Engagement, Understanding, and Building Knowledge, Grades K-8. 3rd Ed. Portland: Stenhouse Publishers.
Quigley, A (2020). Closing the Reading Gap. London: Routledge.
Pryle, M. (2018). Reading with Presence: Crafting Meaningful, Evidence-Based Reading Responses. Portsmouth: Heinemann.
Serravallo, J. (2018). Understanding Texts and Readers. Portsmouth: Heinemann.
Tovani, C. (2004). Do I Really Have to Teach Reading? Content Comprehension, Grades 6-12. Portland: Stenhouse Publishers.
Tovani, C. (2011). So what do they really know? Assessment that Informs Teaching and Learning. Portland: Stenhouse Publishers.
Wiliam, D. & Leahy, S. (2015). Embedding Formative Assessment: Practical Techniques for f-12 Classrooms. West Palm Beach: Learning Sciences International.
