Profiling: understanding the nature of the students' learning strengths and needs in detail.
We're lucky in NZ that our PAT maths have been designed with appropriate expertise and "helps teachers determine their students’ levels of achievement in the knowledge, skills and understanding of mathematics in the New Zealand curriculum." (NZCER Rangahau Maautranga o Aotearoa)
Our NZ PATs prevent biases that have been found in poorly designed multiple choice questions, which have been caused by a lack of clarity through unclear wording, using poorly designed options that telegraph the answers to students, testing recall rather than accumulated knowledge, and using questions that inadequately measure skill attainment etc.
Given the expertise, it's clear the PAT structure cannot be blamed for my students' disappointing results. That makes it doubly important to analyse the answers (to come at a later date) to guide my teaching and programme development to strengthen their weaknesses and close the gaps that exist in their learning.
PAT testing is our school-wide assessment and as teachers we rely on multiple assessments other than PATs (observation, bookwork, individual conferencing, group and class discussions) to decide our overall teacher judgement of students progress. Notwithstanding the different forms of assessment, and given students must sit PATs, it's important to teach my students how to navigate the multiple-choice format.
I believe it's also important for teachers to be aware of both the advantages and disadvantages of multiple choice formats to make best use of any analysis work of test results, and to more concisely define their levels of trust in the PAT structure. Simkin & Kuechler, 2005 (quoted in Multiple-choice questions: Tips for optimsing assessment in-seat and online, a research paper by Xiaomeng Xu, Samantha Tuby, Sierra Dawn Kaur) mentions some advantages of multiple choice tests:
- it allows "test-givers to ask a greater number of questions on a broader set" of topics in a shorter amount of space and time,
- it makes the "administering and grading the exam simpler" with students having to choose only one of the prerecorded answers and the computer having to mark only what is correct,
- it reduces the "subjectivity / inconsistency / human errors in scoring" when "machine graded" thus guaranteeing more accurate results,
- the research also quotes Marsh, Roediger, Bjork & Bjork, 2007 who state multiple choice tests have "been shown to positively enhance retention of the material that is tested (a testing effect) and to boost performance on later test."
The research also mentions multiple choice format disadvantages such as:
- students can "choose answers based on the process of elimination" which could point to them not having the knowledge ingrained,
- because each question has a pre-written, pre-determined correct answer, it can make it "difficult to pinpoint a student's true knowledge,"
- It also quotes Ozuru, Briner, Kirbe & MCNamarra, 2013 who point out that students may have the skill to perform an activity, but lack comprehension skills to understand what has been asked,
- One other thought: multiple choice questions gives students the opportunity to guess, and if they guess right, they're given credit for something they don't know.
Taking these advantages and disadvantages together serves to enhance my personal perspective: I find the PATs very useful because they have been proven by the test of time; machine marking means I'm guaranteed accurate results; they guide me on the individual's/group's strengths and weaknesses; I can carry out the next learning steps with more confidence; the PAT Maths cover all the maths strands in one test and gives me an overall view of each students' progress. Finally, as an aside, it helps students develop the discipline to sit and complete formal tests written in a formal way before they get to high school. In short, preparing our students to cope with future exams gives them a valuable skill they can fall back on with every exam they take.
