Behind the Scenes of SEA-PLM in the Philippines
The Southeast Asian Ministers of Education Organisation (SEAMEO) and the United Nations Child Fund (UNICEF), as well as the Australian Council for Educational Research (ACER), the technical consortium of SEA-PLM have spent many years working with participating countries to develop, trial and improve the technical and logistical procedures and roles needed to successfully implement the large-scale regional assessment.
This being said with SEA-PLM being embedded within national education systems the practicalities of preparing for the assessment differs slightly in each country, depending on its individual education structures and mechanisms.
So what does SEA-PLM actually look like at a country level?
To take a
closer look at the in-country roles and functions of SEA-PLM we head to the Philippines who implemented main survey data collection during February. With the involvement of 16 regions, 119 Divisions, and 176 schools the archipelagic nation has had to be meticulous in its approach for preparing all relevant parties for the first round of SEA-PLM.
For each participating country, a Technical Team Manager (TTM) and Technical Team (TT) including positions such as school contact person, logistics coordinator, sampling coordinator, coding coordinator, data manager, and administration staff are appointed.
At the heart of SEA-PLM coordination in the Philippines sits the Bureau of Education Assessment (BEA) of the Department of Education (DepEd), with Chief Education Program Specialist Ms. Gretchen Cordero assigned as Technical Team Manager. Responsible for the coordination of all activities at the national level and the supervision of the Technical Team, Ms Gretchen is coordinating not one but three large-scale learning assessments. A priority of this year is to look into our system of assessment and see how our students are faring compared to other countries in the region.
Core tasks of the Technical Team include the selection and training orientation of School Coordinators and Test Administrators, the translation, review, printing, and distribution of test booklets and questionnaires, student sampling and quality monitoring.
Ms. Gretchen commented that overall the preparation process had run smoothly, a minor challenge however was that half of the test booklets had to be printed in-house due to procurement processes. This meant a lot of late nights for my staff to ensure that all booklets were printed in time!
So what about the school level?
Similar to the Technical Team there are multiple roles required at the front line of school testing. Two essential positions are that of the School Coordinator (SC) and Test Administrator (TA). In the case of the Philippines, the additional position of District Testing Coordinator (DTC) plays a significant role in supporting School Coordinators and preparing schools for testing.
DTC Ms. Remylinda T. Soriano explained to us how she had coordinated the orientation session with teachers, parents, and students ensuring they understood the purpose of SEA-PLM and what would be required from them on the testing day. With the testing day finally upon them, Ms. Remylinda shared what a happy day it was and how proud the school was to have been selected to represent the Philippines.
For security purposes test booklets and questionnaires are not delivered to the school until the testing day meaning that parents, teachers, principals, and students all complete the test and questionnaires on the same day. The task of assessment material delivery and collection is that of a National Quality Control Monitor (NQCM). NQCMs are from the central level and also fulfill the purpose of school observation, filling out standard checklists and reporting back to the Technical Team.Â
When the testing day finally arrives it is the Test Administrators moment to shine. Test Administrators are selected from different schools than those participating in the assessment and all undergo training to orient them towards their role. One Test Administrator Mr Benjamin Cruz Junior described the testing process to be straightforward, with the detailed instructions and script provided it is simply a matter of following it.
It is because of these people and many more that national data collection for SEA-PLM is made possible. A core priority of SEA-PLM is the capacity building of national teams to prepare and implement learning assessments at an internationally recognised standard. In the Philippines, the impact of the preparation, training and support process of these key roles, as well as the utilization of existing DepEd structures and mechanisms have all contributed to the strengthening of local capacity and the successful collection of school data for SEA-PLM.
SEA-PLM 2019 Begins in Myanmar
The first regional assessment of the Southeast Asia Primary Learning Metrics (SEA-PLM) entered its final stretch on the 28th January, with Myanmar being the first country to begin data collection in schools. This large-scale survey will take place in Myanmar, Philippines, Lao PDR, Vietnam, Cambodia and Malaysia during 2019 according to the different school calendars.
Led through the cooperative tradition among Ministries of Education in the region, SEA-PLM is a project jointly managed by the Southeast Asian Ministers for Education Organisation (SEAMEO) and United Nations Children's Fund (UNICEF). With the shared vision of improving and redefining learning outcomes, SEA-PLM provides metrics that are founded within the regional context and inclusive of 21st Century skills.
The implementation of the first assessment is the culmination of 5 years' worth of research, capacity building, and field trials, all to develop a suitable set of assessment and survey instruments specifically designed for the region. For each of the participating countries, approximately 3500 grade 5 students enrolled in an average of 150 schools are randomly selected to represent their country. The Australian Council of Education Research (ACER) an independent technical operator is used to select schools and students, providing technical support to national teams as they prepare and implement the assessment.
Deputy Director of the Department of Myanmar Examinations Mr. Aung Htike has been working as a SEA-PLM Technical Team Manager since June 2015 and is highly experienced in the management of large-scale learning assessments. Leading a Technical Team made up of 35 members from seven different Departments and Universities across Myanmar, Mr. Aung with his team coordinated the first assessment in 202 schools across the country. To successfully achieve this huge undertaking, Myanmar uses the Six C approach; Commitment, Capacity, Communication, Collaboration, Consistency, and Coordination.
Over a three-week period, 5715 students undertook standardized tests in Myanmar language in reading, mathematics, writing, and global citizenship. One Myanmar teacher shared how she could not sleep the night before the test because she felt so nervous and excited to be chosen to participate. To paint a fuller picture of the factors contributing to learning outcomes the principals, teachers, and parents of selected students also complete background surveys. While all tests and questionnaires are anonymous and not designed to assess the individual performance of students or teachers, it is still a nerve-racking experience to be chosen to represent your country.
An important element of SEA-PLM is to ensure comparability and consistency of the meaning of test items between countries. Questions are similar in all participating countries and have been designed in collaboration with national and international experts, reflecting the disciplinary domains and cognitive processes considered important to have been achieved by students by the end of primary school.
The road to developing comparable and suitable test items and assessment tools have been a long and complex one. In order to design suitable metrics, surveys and assessment tools were trialed during 2018 in 7 different countries on reduced school samples. In Myanmar, Mr. Aung and his team participated in this process by joining in multiple capacity building workshops and implementing field trial assessments in 35 schools during November 2016.
Countries have committed considerable human and financial resources to this assessment because they can see that it has something very important to offer them. SEA-PLM will generate key data on each country's context on how students perform in school and what factors are affecting learning outcomes. From these gaps in national education policy and systems can be identified and improved.
What is impressive about SEA-PLM Mr. Aung says is that it both assesses the four different domains and that the items of these four domains are specifically tailored for Southeast Asian students, allowing them to apply their knowledge and learnings in real life situations.
