PISA Methodology: Sampling, Scoring & Comparability

PISA does not attempt to test every 15-year-old in a participating education system. It selects a probability sample and uses that sample to estimate characteristics of the larger population, including mean performance, proficiency distributions and differences between student groups.

The target is primarily age-based rather than grade-based. In PISA 2025, eligible students were between 15 years 3 months and 16 years 2 months at the time of assessment and had completed at least six years of formal schooling. Students can attend different grades, academic or vocational programs, public or private schools, and full-time or part-time education.

A representative sample depends on more than sample size. The students omitted from the sampling frame, excluded after sampling or absent from the assessment can matter if they differ systematically from participating students.

PISA 2025 Technical Standards specify that combined school-level and within-school exclusions should normally remain below 5% of the desired target population. Permitted exclusions can include circumstances such as inaccessible schools and students whose disabilities or limited experience with the assessment language prevent meaningful participation under the defined criteria.

Response rates are evaluated separately for schools and students. When standards are missed, weighting adjustments can reduce some observed imbalances, but weighting cannot automatically eliminate every source of non-response bias. PISA therefore uses a technical adjudication process and places warnings, asterisks or reporting restrictions on results when the evidence calls for them.

A PISA score is not a percentage of questions answered correctly. Students receive different combinations of assessment material, and the difficulty of the items they encounter varies. Statistical scaling is therefore required to place student responses and item difficulties on common reporting scales.

Item responses

Depending on the item, a response may receive full credit, partial credit or no credit. Open-response items can require coding according to standardized coding guides, while many other items can be scored automatically.

Item Response Theory

PISA uses Item Response Theory models to relate observed response patterns to unobserved proficiency. The scaling process estimates characteristics of assessment items and places performance on a common latent scale. This allows information from different forms of the test to contribute to estimates for the same population even though individual students have not answered every item.

Adaptive testing in PISA 2025

Most PISA 2025 students completed computer-based testing. Science, reading and mathematics used a multi-stage adaptive approach: initial material was assigned without using prior performance, while subsequent routing could depend on performance in earlier sections. Students performing differently could therefore encounter different later item blocks.

The full PISA 2025 pool contained far more material than any one student could complete. Each student generally spent about two hours on the cognitive assessment, while the combined pool of science, reading, mathematics and Learning in the Digital World material covered more than 11 hours. Matrix sampling and statistical scaling make this broad content coverage possible without requiring every student to complete every task.

PISA performance databases contain plausible values rather than a single definitive proficiency estimate for each student. Plausible values are multiple draws from an estimated distribution of proficiency based on the student’s responses and the statistical model used in scaling.

Since PISA 2015, ten plausible values are provided for a student’s performance in each reported cognitive domain. They are intended for population-level statistical analysis. They should not be interpreted as ten repeated test scores, nor should they normally be averaged within each student to create a single personal score before conducting an analysis.

For a population statistic such as a country mean, the statistic is calculated across the plausible values and the resulting estimates are combined. This preserves uncertainty arising from the fact that each student answers only a limited sample of all possible assessment items.

PISA score points are scale units, not physical units such as years, percentages or numbers of correct answers. The scales were constructed so that their distributions were centered around roughly 500 points with a standard deviation near 100 points in the reference populations used when the scales were established. There is no theoretical minimum or maximum PISA score.

Large score differences can sometimes be interpreted through proficiency-level descriptions. Small score differences require more caution. A difference of a few points between two estimated means can be smaller than the uncertainty surrounding those estimates.

A country mean also summarizes a distribution. Students within the same education system can be spread across many proficiency levels, so a national mean cannot be used to state that every student—or even a particular individual student—can perform the tasks associated with the scale region containing that mean.

Because PISA results are estimated from samples, every reported statistic has uncertainty. OECD tables therefore provide standard errors for many estimates and use confidence intervals and statistical tests when comparing countries, groups or assessment cycles.

For many estimates, a 95% confidence interval can be approximated by the estimate plus or minus 1.96 standard errors. When a reported difference is described as statistically significant, the comparison incorporates the estimated uncertainty rather than relying only on the numerical difference between two rounded scores.

PISA’s complex sampling design also means that ordinary formulas designed for simple random samples are not adequate for many analyses. Replicate weights are used to estimate sampling variance. Analyses of cognitive performance additionally need to incorporate the variation across plausible values.

International comparability depends on several parts of the assessment working together. Using the same nominal score scale is not enough by itself.

Measurement invariance and item differences

An item does not necessarily behave identically in every country and language. Statistical analyses examine whether response patterns are consistent with common item parameters. When an item shows substantial differential functioning for a particular group, PISA can estimate group-specific parameters rather than forcing that item to contribute to the international link in exactly the same way.

This approach is often described as partial measurement invariance. International comparisons are supported by the remaining items whose parameters can serve as common anchors. Comparability is therefore a property of the measurement links across groups, not merely the use of a shared questionnaire or identical numerical scale.

PISA reports results from successive assessments on linked scales so that changes can be studied over time. The reference scale for each core subject was established when that subject first served as a major PISA domain: reading in 2000, mathematics in 2003 and science in 2006.

Trend items

A central part of trend measurement is the reuse of assessment items from earlier cycles. These trend items provide anchors between assessments. If an item’s measurement characteristics remain sufficiently stable, its responses help connect the newer assessment to the established scale.

Items that no longer behave consistently across cycles do not have to be forced to retain identical parameters. Scaling procedures evaluate these constraints and can release them when response patterns indicate that the common-parameter assumption is not appropriate.

Link error

Comparisons across cycles introduce an additional source of uncertainty because the link between two administrations is estimated rather than perfectly known. OECD trend analyses account for this link error when evaluating changes in performance over time.

A numerical difference between two cycles should therefore not be interpreted only from the two rounded mean scores. Sampling error, measurement error and uncertainty in the cross-cycle link can all affect the precision of the estimated change.

Science was the major domain in PISA 2025, and adaptive testing was introduced in science for the first time. New science material was administered alongside retained trend items from earlier assessments. OECD also assigned 25% of students to a non-adaptive version of the science design, providing evidence for examining whether adaptivity itself affected student behavior or measured performance.

Initial test material was assigned randomly, and test-domain positions were rotated across students. These design features allow several checks of whether observed changes persist when comparisons are restricted to parts of the assessment that are less affected by a design change.

For interpreting longer-term movement, OECD distinguishes direct cycle-to-cycle comparisons from trends estimated across three or more assessments. Adjacent-cycle comparisons are strongest when the assessments share substantial content and design features. Over longer periods, multi-cycle trend estimates can reduce reliance on the circumstances of any one administration.

Computer-based administration was the primary delivery mode in PISA 2025. Eighty-five of the 91 participating countries and economies used the computer-based assessment, while six used the paper-based version.

OECD reports performance from the computer- and paper-based assessments on comparable scales. However, comparability of score estimates should be distinguished from comparability of every proficiency-level interpretation. The computer assessment included material that was not available in the paper version, particularly in mathematics and reading and, to a lesser extent, science.

For example, OECD states that mathematics performance from the PISA 2025 paper-based assessment should be described using the PISA 2012 proficiency-level descriptions rather than the newer mathematics descriptions established with the later computer-based assessment. This distinction prevents a common measurement error: assuming that an equal numerical score automatically supports every newer task-level interpretation regardless of assessment mode and item coverage.

Country means are estimates, so their ordering is also uncertain. A table sorted from the highest reported mean to the lowest can describe score-based ordering, but it does not imply that every adjacent country has a measurably different level of performance.

PISA 2025 therefore reports confidence sets for ranks. The procedure reflects uncertainty in estimated country means and the multiple comparisons involved when many education systems are ranked simultaneously. A country’s defensible position can span several ranks even when a single rounded mean would place it at one exact position in a simple sorted table.

Before interpreting two PISA values as directly comparable, the comparison should satisfy the relevant measurement conditions:

Passing these checks does not mean that two estimates are perfectly error-free. It means that the comparison is being made on the measurement basis for which PISA was designed and reported.