Delivering high quality and reliable data
SEA-PLM is a common methodological approach to assessing learning outcomes of primary Grade 5 students in Southeast Asia. The assessment framework and tools are designed to meet international best practices and technical standards in large-scale assessment—delivering high quality and reliable data.
Methodology, instruments, procedures and quality control processes are produced collaboratively by countries and external assessment experts. Phases include framework and item development, translation and verification, sampling, data collection operations, data processing, scaling, data analysis and reporting of results.
The first assessment round, SEA-PLM 2019, has been implemented in six participating countries from Southeast Asia. The Australian Council for Educational Research (ACER) managed the scientific expertise, technical support and quality control for the design and implementation of SEA-PLM 2019.
Scientific protocol and survey parameters are outlined and reported for each round of SEA-PLM in the framework documentation, technical report, database, data user manual and released instruments. While the principal assessment methodology remains consistent from one round to the next, the possibilities exist to revise, develop and explore new domain areas and techniques during a new assessment round.
SEA-PLM develops cognitive and contextual questions on the basis of a conceptual framework designed by experts and countries. The framework defines the contents to be measured and is contextually suited to the region, national curricula and current research in education. Framework and instruments are designed to be used by all participating countries and between different assessment rounds allowing for comparison of trends over time.
SEA-PLM 2019 covers four content domains
SEA-PLM Mathematical literacy is a person's capacity to translate a problem (given in a context of interest or importance to them) into a suitable mathematical formulation. The individual will apply mathematical knowledge and skills to find a solution, interpret the mathematical results concerning the context, and review the merits or limitations of those results..
SEA-PLM Reading literacy is a person's capacity to understand, use, and respond to written texts to meet personal, societal, economic, and civic needs.
SEA-PLM Writing literacy is a person's capacity to construct meaning by generating a range of written texts to express oneself and communicate with others to meet personal, societal, economic, and civic needs.
SEA-PLM Global Citizenship literacy is a person's attitude to appreciate and understand the interconnectedness of all life on the planet and act and relate to others—with this understanding— to make the world a more peaceful, just, safe and sustainable place.The instruments also include background questionnaires that gather essential information from students, parents, teachers and school principals.
Countries and experts develop questions and coding materials based on the SEA-PLM assessment framework to measure specific content and cognitive domain. All instruments are first experimented with during field trial operations in schools, ensuring that questions function well across the different participating countries and language of instruction and maintaining consistency and reliability in the methodology.
SA-PLM sampling obtains accurate Grade 5 learning outcomes estimates in the Southeast Asian region. Sampling is consistent across countries and accommodates requested local and national objectives. Assessment data is collected from a representative sample of the national target population. This population is defined by UNESCO (2012) as "all students enrolled in the grade that represents five years of schooling counting from the first year of ISCED Level 1." This is referred to in SEA-PLM as Grade 5.
All participating countries apply the same sampling procedures, which consist of defining the national target population, constructing the sampling framework, definition of stratification variables, sampling schools and classes, and the allocation of booklets between students in a class. SEA-PLM sampling has two stages:
1. Schools are selected following a systematic procedure whereby selection probability is proportional to the number of enrolled Grade 5 students. A minimum of 150 schools are sampled from each participating country.
2. One Grade 5 class is selected at random in each sample school. All students of the chosen class participate in the assessment.
The final SEA-PLM booklets are administrated in the language(s) of instruction of each country to students via paper-pencil testing. The testing period is one hour, with students completing a series of multiple-choice and open-ended questions. The assessment uses a rotating booklet design (a total of 18 cognitive booklets) that includes 30 minutes for each learning area (reading, writing, and numeracy). The exact instructions are provided to all students at the beginning of the test. In addition, students, parents, teachers and principals complete contextual questionnaires to gather background information.
Countries are responsible for training government staff, school coordinators and test administrators, ensuring that the assessment standards are applied consistently. The government and external observers undertake quality control of the assessment. Standardized guidelines and instructions guide countries at each stage of the assessment administration.
Data collected during the field trials and the main survey follows the same entry, cleaning, verification, scaling, and analysis process. This process allows for constructing valid measurements and datasets, including generating variance estimations and computing estimators of descriptive and factorial analysis.
Final reports are prepared through a collaborative process involving countries and experts. Results are reported in regional, national and specific reports (thematic report, policy brief and research paper). Researchers and education stakeholders are consulted to interpret and comment on the results, formulating messages and recommendations based on the data.
