Access to Information Orders
Decision Information
NATURE OF THE APPEAL: The Education Quality and Accountability Office (the EQAO) received a request under the Freedom of Information and Protection of Privacy Act ( the Act ) for access to a copy of the Ontario Secondary School Literacy Test (the OSSLT) administered in February 2002 to the requester's son. The EQAO located the requested record and granted access to portions of it. Access to the majority of the record was denied pursuant to the discretionary exemption in section 18(1)(h) of the Act (examination questions). The requester, now the appellant, appealed the EQAO's decision and raised the possible application of the "public interest override" provision in section 23 of the Act . The appellant also indicated that he would be prepared to "view my son's original test with the questions blacked out" rather than receive a complete copy of the test with the answers and questions included, in accordance with section 30(2) of the Act . As mediation of the appeal was not successful, the matter was moved into the adjudication stage of the appeal process. I decided to seek the representations of the EQAO, initially, as it bears the onus of demonstrating the application of the exemption claimed. During the inquiry stage of the appeal, the EQAO disclosed to the appellant those portions of the requested records which have been made public and posted on its website. These parts of the records are no longer at issue. The EQAO submitted representations, which were disclosed, in their entirety, to the appellant along with a copy of the Notice of Inquiry. The appellant also made submissions that were shared with the EQAO, which then made further submissions by way of reply. RECORDS: The record at issue consists of the undisclosed portions of the OSSLT comprising four booklets. The undisclosed information consists of 11 reading selections that form most of the reading component of the February 2002 literacy test. DISCUSSION: EXAMINATION QUESTIONS The EQAO submits that the undisclosed portions of the records are exempt from disclosure under the discretionary exemption in section 18(1)(h) of the Act , which reads: A head may refuse to disclose a record that contains, questions that are to be used in an examination or test for an educational purpose; The EQAO's representations I received extremely detailed submissions from the EQAO in response to the questions posed in the Notice of Inquiry. It has described in detail the protocol and formula developed to formalize its policy of re-using OSSLT test questions and the structure of the test forms themselves. To assist in understanding the position taken by the EQAO, I will set out much of these submissions verbatim. The EQAO begins its representations as follows: EQAO has always had a policy of re-using OSSLT test questions after their initial use. The agency has recently developed a protocol and schedules to formalize this policy, to be implemented starting in October 2004, when the first cohort to have taken the test officially (those who entered Grade 9 in September 2000 and took the test in February 2002) shall have graduated. . . . Under the protocol, all test questions, subject to very limited exceptions, are considered secure. Test booklets are never returned to students, and in fact are void of grades or marks (which are entered by hand-held devices), a deliberate policy to minimize bias when marking is required. Students who fail the test are provided with feedback through the Individual Student Report, . . . [which] details the student's performance under the marking rubric but does not incorporate matter from the test itself. The re-use policy allows the EQAO to control long-term development costs and ensures that different tests are comparable over time and that high-stakes testing remains equitable and consistent. The EQAO also provided a lengthy outline of the structure of the OSSLT test forms themselves and the techniques employed by it in ensuring that the tests which are administered are "equitable and consistent over time." The object of this exercise is to ensure that the test forms used are rendered interchangeable. It indicates that Form 3, the test administered in February 2002 which is the subject of this request, includes test questions to be used in Form 4 (October 2002), Form 5 (October 2003), Form 6 (October 2004) and Form 7 (October 2005). It further submits that Forms 9, 10 and 11 will be run in October 2006, 2007 and 2008 and will also include elements taken from Form 3. In describing the protocol which it has developed for the re-use of test materials, the EQAO submits that: Ideally, test items and even whole tests could be re-used from year-to-year, a practice not uncommon in high schools and university examinations. However, since a student who fails has two opportunities to retake the OSSLT before graduating, items cannot be re-used until at least the third year after their most recent use (in practice it will often be four years). New forms will consist of an equal number of newly developed and re-used items. Rarely, an item can be dropped if the subject matter by its nature becomes stale-dated or irrelevant (items of this type are no longer being developed), or if an item for some reason does not 'play well' during the live administration or is found to be defective or flawed. As indicated previously, some items are made public in order to develop field supports and are never re-used. The re-use protocol has been endorsed at several management levels at EQAO, and is pending approval by Senior Management Committee. The principal components of the protocol are as follows: Items and not test forms are re-used, new test forms consisting in equal part of re-used items and newly field-tested items (the 50% re-use rule); The reading component of each new test form shall consist of six re-used reading selections (three selections being re-used from each of two previous forms) and six newly field-tested reading selections; The writing components of each new test form shall consist of two re-used writing prompts and two newly field-tested writing prompts; Items cannot be re-used until at least the third year after their most recent use; and Items may be dropped from the schedule only if they contain material that is no longer current or relevant to students or if a defect or flaw is identified. In support of its position that section 18(1)(h) applies to the records, the EQAO relies on the decisions in Orders P-1284 and M-1116 where the records included a series of questions which were to be used in future examinations. In both cases, it was held that the examinations were for an educational purpose and that the questions were exempt from disclosure under section 18(1)(h) and 11(h) of the Municipal Freedom of Information and Protection of Privacy Act respectively. To summarize, the EQAO submits that it has provided "specific supporting evidence" that the questions included in the records will be re-used in future tests. It suggests that the re-use protocol described in its submissions "is not mere speculation or a series of general sta
Decision Content
NATURE OF THE APPEAL:
The Education Quality and Accountability Office (the EQAO) received a request under the Freedom of Information and Protection of Privacy Act (the Act) for access to a copy of the Ontario Secondary School Literacy Test (the OSSLT) administered in February 2002 to the requester’s son. The EQAO located the requested record and granted access to portions of it. Access to the majority of the record was denied pursuant to the discretionary exemption in section 18(1)(h) of the Act (examination questions).
The requester, now the appellant, appealed the EQAO’s decision and raised the possible application of the “public interest override” provision in section 23 of the Act. The appellant also indicated that he would be prepared to “view my son’s original test with the questions blacked out” rather than receive a complete copy of the test with the answers and questions included, in accordance with section 30(2) of the Act.
As mediation of the appeal was not successful, the matter was moved into the adjudication stage of the appeal process. I decided to seek the representations of the EQAO, initially, as it bears the onus of demonstrating the application of the exemption claimed. During the inquiry stage of the appeal, the EQAO disclosed to the appellant those portions of the requested records which have been made public and posted on its website. These parts of the records are no longer at issue. The EQAO submitted representations, which were disclosed, in their entirety, to the appellant along with a copy of the Notice of Inquiry. The appellant also made submissions that were shared with the EQAO, which then made further submissions by way of reply.
RECORDS:
The record at issue consists of the undisclosed portions of the OSSLT comprising four booklets. The undisclosed information consists of 11 reading selections that form most of the reading component of the February 2002 literacy test.
DISCUSSION:
EXAMINATION QUESTIONS
The EQAO submits that the undisclosed portions of the records are exempt from disclosure under the discretionary exemption in section 18(1)(h) of the Act, which reads:
A head may refuse to disclose a record that contains,
questions that are to be used in an examination or test for an educational purpose;
The EQAO’s representations
I received extremely detailed submissions from the EQAO in response to the questions posed in the Notice of Inquiry. It has described in detail the protocol and formula developed to formalize its policy of re-using OSSLT test questions and the structure of the test forms themselves. To assist in understanding the position taken by the EQAO, I will set out much of these submissions verbatim.
The EQAO begins its representations as follows:
EQAO has always had a policy of re-using OSSLT test questions after their initial use. The agency has recently developed a protocol and schedules to formalize this policy, to be implemented starting in October 2004, when the first cohort to have taken the test officially (those who entered Grade 9 in September 2000 and took the test in February 2002) shall have graduated. . .. Under the protocol, all test questions, subject to very limited exceptions, are considered secure.
Test booklets are never returned to students, and in fact are void of grades or marks (which are entered by hand-held devices), a deliberate policy to minimize bias when marking is required. Students who fail the test are provided with feedback through the Individual Student Report, . . . [which] details the student’s performance under the marking rubric but does not incorporate matter from the test itself. The re-use policy allows the EQAO to control long-term development costs and ensures that different tests are comparable over time and that high-stakes testing remains equitable and consistent.
The EQAO also provided a lengthy outline of the structure of the OSSLT test forms themselves and the techniques employed by it in ensuring that the tests which are administered are “equitable and consistent over time.” The object of this exercise is to ensure that the test forms used are rendered interchangeable. It indicates that Form 3, the test administered in February 2002 which is the subject of this request, includes test questions to be used in Form 4 (October 2002), Form 5 (October 2003), Form 6 (October 2004) and Form 7 (October 2005). It further submits that Forms 9, 10 and 11 will be run in October 2006, 2007 and 2008 and will also include elements taken from Form 3.
In describing the protocol which it has developed for the re-use of test materials, the EQAO submits that:
Ideally, test items and even whole tests could be re-used from year-to-year, a practice not uncommon in high schools and university examinations. However, since a student who fails has two opportunities to retake the OSSLT before graduating, items cannot be re-used until at least the third year after their most recent use (in practice it will often be four years). New forms will consist of an equal number of newly developed and re-used items. Rarely, an item can be dropped if the subject matter by its nature becomes stale-dated or irrelevant (items of this type are no longer being developed), or if an item for some reason does not ‘play well’ during the live administration or is found to be defective or flawed. As indicated previously, some items are made public in order to develop field supports and are never re-used.
The re-use protocol has been endorsed at several management levels at EQAO, and is pending approval by Senior Management Committee. The principal components of the protocol are as follows:
• Items and not test forms are re-used, new test forms consisting in equal part of re-used items and newly field-tested items (the 50% re-use rule);
• The reading component of each new test form shall consist of six re-used reading selections (three selections being re-used from each of two previous forms) and six newly field-tested reading selections;
• The writing components of each new test form shall consist of two re-used writing prompts and two newly field-tested writing prompts;
• Items cannot be re-used until at least the third year after their most recent use; and
• Items may be dropped from the schedule only if they contain material that is no longer current or relevant to students or if a defect or flaw is identified.
In support of its position that section 18(1)(h) applies to the records, the EQAO relies on the decisions in Orders P-1284 and M-1116 where the records included a series of questions which were to be used in future examinations. In both cases, it was held that the examinations were for an educational purpose and that the questions were exempt from disclosure under section 18(1)(h) and 11(h) of the Municipal Freedom of Information and Protection of Privacy Act respectively.
To summarize, the EQAO submits that it has provided “specific supporting evidence” that the questions included in the records will be re-used in future tests. It suggests that the re-use protocol described in its submissions “is not mere speculation or a series of general statements”. The EQAO adds that the questions which comprise the record at issue are not simply “dumped” into a test bank for possible, unscheduled future use. Rather, the re-use protocol sets out in detail exactly when and under what circumstances questions would be re-used. It submits that 50% of the test questions in the record will find their way into future tests according to the re-use scheme set out in the protocol.
The Appellant’s representations
The appellant maintains that he is seeking access to the records in order to determine whether there is some “design flaw in the format” of the test. He states that his son did not complete a portion of the test and that he has been advised that other students failed to do so as well. The appellant states that:
If the above is true I feel it is imperative that the public know and more importantly it is acknowledged and appropriate steps are taken so the same thing does not happen in the future.
Findings under section 18(1)(h)
Based on the extensive information provided by the EQAO in support of its contention that the test questions which appear on the record at issue will, in fact, be re-used on future tests, I find that the questions in the record are to be used in an examination or test for the purpose of section 18(1)(h). I am satisfied that the EQAO has provided me with sufficient evidence to establish that it intends to re-use the questions in future examinations. The description of the re-use protocol and the steps taken by the EQAO to ensure the integrity of the testing system demonstrate that the questions will be re-used until such time as they are removed in accordance with the provisions set forth in the protocol.
Similarly, I have no difficulty in making a finding that the questions which form the record at issue are to be used in an examination or test “for an educational purpose” within the meaning of section 18(1)(h). The mandate of the EQAO described in section 3 of the Education Quality and Accountability Office Act includes an evaluation of the quality and effectiveness of elementary and secondary education. Part of that mandate, which clearly has an educational purpose, includes the development, administration and marking of testing materials completed by elementary and secondary students. The test which forms the subject matter of the request was created as part of that educational mandate.
Accordingly, I find that the EQAO has provided me with sufficient evidence to find that the record in question is exempt from disclosure under section 18(1)(h).
PUBLIC INTEREST IN DISCLOSURE
The appellant takes the position that there exists a public interest within the meaning of section 23 of the Act in the disclosure of the record which outweighs the purpose of the exemption in section 18(1)(h). Section 23 states:
An exemption from disclosure of a record under sections 13, 15, 17, 18, 20, 21 and 21.1 does not apply where a compelling public interest in the disclosure of the record clearly outweighs the purpose of the exemption. [my emphasis]
For section 23 to apply, two requirements must be met. First, there must exist a compelling public interest in the disclosure of the records. Second, this interest must clearly outweigh the purpose of the exemption [Order P-1398, upheld on judicial review in Ontario (Ministry of Finance) v. Ontario (Information and Privacy Commissioner), [1999] O.J. No. 488 (C.A.)].
In Order P-984, former Adjudicator Holly Big Canoe discussed the first requirement referred to above:
‘Compelling’ is defined as ‘rousing strong interest or attention’ (Oxford). In my view, the public interest in disclosure of a record should be measured in terms of the relationship of the record to the Act’s central purpose of shedding light on the operations of government. In order to find that there is a compelling public interest in disclosure, the information contained in a record must serve the purpose of informing the citizenry about the activities of their government, adding in some way to the information the public has to make effective use of the means of expressing public opinion or to make political choices.
If a compelling public interest is established, it must then be balanced against the purpose of any exemptions which have been found to apply. Section 23 recognizes that each of the exemptions listed, while serving to protect valid interests, must yield on occasion to the public interest in access to the information that has been requested. An important consideration in this balancing exercise is the extent to which denying access to the information is consistent with the purpose of the exemption. [Order P-1398]
The appellant argues that, as an alternative to viewing the actual test questions, he seeks access only to the “format” of the test. He indicates that there is a public interest in ensuring that the OSSLT was administered fairly and that students were not misled by problems with the way the test was “formatted”.
The EQAO takes the position that the interest expressed by the appellant is, in fact, a private one and that there is no public interest in the disclosure of the test questions. It argues that the appellant is seeking access to the record in order to evaluate why his son failed the test and that this represents a private, rather than a public, interest in the disclosure of the record. It adds that:
Individual test performance does not touch the public domain and therefore is not within the scope of section 23. Parents in general are not lacking sufficient performance feedback to enable appropriate remedial activity. EQAO provides parents of unsuccessful students with detailed feedback through the Individual Student Report. Furthermore, as indicated previously, all writing prompts and one of the reading selections for the February 2002 administration have been made publicly available through EQAO’s Internet site, and have been provided to the appellant, together with the answers thereto. A significant portion of the test is therefore available to the appellant, together with his son’s responses, and can be used for diagnostic purposes.
The EQAO goes on to address the appellant’s concerns with the “format” of the test as follows:
. . . the views of the appellant on the design validity of the OSSLT do not in our opinion constitute an issue of compelling public interest as defined by the Commission, i.e., “rousing strong interest or attention” in the public domain. We strongly submit that the onus is on the appellant to show that a compelling public purpose would be served by the disclosure of secure test items.
The EQAO also made submissions in favour of what it considers to be “a strong public interest in the non-disclosure of the secure test items.” In Orders PO-2014-I and PO-1871-I, it was found that a public interest weighing against the disclosure of certain types of information might also exist alongside a public interest in disclosure. The EQAO points out that the creation of test questions is a lengthy and expensive process. By re-using test questions, a significant portion of those costs are saved in not having to undergo the process of creating and field-testing questions prior to their incorporation in the materials. The re-use of questions also allows for the direct comparison of test results from year to year. The EQAO submits that even if a public interest in disclosure was established, there is also a significant public interest weighing against disclosure.
The Act is silent as to who bears the burden of proof in respect of section 23. However, former Commissioner Linden has stated in a number of Orders that it is a general principle that a party asserting a right or duty has the onus of proving its case. This onus cannot be absolute in the case of an appellant who has not had the benefit of reviewing the requested records before making submissions in support of his or her contention that section 23 applies. To find otherwise would be to impose an onus which could seldom if ever be met by the appellant. [Order P-241]
In the present case, the appellant has presented his own concerns with the validity of the format of the OSSLT test. He also indicates anecdotally that he has received comments from other individuals which echo these concerns. In my view, this evidence is not sufficient to establish that there exists the requisite “public interest” in the disclosure of the test questions sought in this appeal. I have not been provided with sufficient evidence from the appellant which would substantiate his contention that there was a problem with the format of the test, based on the considerable disclosure of the requested information already released to him. I find that the appellant has failed to demonstrate that a public interest, compelling or otherwise, exists in the disclosure of this information. As a result, I find that section 23 has no application in the appeal.
ORDER:
I uphold the decision of the EQAO to deny access to the records.
Original signed by: September 19, 2003