Official syllabusPaper II · Unit 1 · Psychological Measurement of Individual Differences
Characteristics and construction of standardized psychological tests.
Definition
“A psychological test is essentially an objective and standardized measure of a sample of behavior.”
Anastasi and Urbina (1997)
Construction turns a defined construct into items that sample it; standardization fixes the procedure and the reference group, so that a score means the same thing wherever it is earned. Each step shuts out one source of error, which is why one sequence serves an arithmetic test for children, an aptitude test for administrators and a personality inventory.

The bottom row is the top row’s sequence applied three times, to three different purposes.
Characteristics of standardized psychological tests
- Standardization — a uniform procedure of administering and scoring, with materials, time limits, instructions and the handling of queries fixed by the manual, plus norms from a large, representative standardization sample (Anastasi and Urbina, 1997).
- Objectivity — the score does not depend on the scorer; fixed-response items, a key and high inter-scorer agreement secure it.
- Reliability — consistency across occasions, forms and items; Nunnally (1978) asked for .80 in basic research and at least .90 where decisions about individuals rest on the score (Reliability of Psychological Tests).
- Validity — evidence that the intended interpretation is justified for a stated purpose, a property of a use rather than of the test (Messick, 1989; Validity of Psychological Tests).
- Norms — a representative, recent and relevant reference group, without which a raw score means nothing (Test Norms and Norm-Referenced Measurement).
- Adequate sampling of behaviour — an arithmetic test of five problems, or of multiplication alone, would poorly measure computational skill (Anastasi and Urbina, 1997).
- Discriminating power — items of moderate difficulty that spread examinees; an item everyone passes is uninformative.
- Fairness — the Standards of the AERA, APA and NCME (2014) make it a foundation beside validity and reliability: no construct-irrelevant barrier for any group.
- Practicability — reasonable time and cost, easy administration and scoring, and a manual documenting the evidence for everything above.
Factors determining the efficacy of psychological tests
Efficacy is what a test achieves in use: psychometric quality sets the ceiling, and four groups of factors decide how much is reached.
Test factors
- Length and item quality — more items raise reliability; flawed items add construct-irrelevant variance. Haladyna, Downing and Rodriguez (2002) distilled 31 multiple-choice item-writing guidelines from 27 textbooks and 27 studies.
- Difficulty and fit — a test too easy or too hard for its group yields ceiling or floor effects; cultural loading and outdated norms damage some groups selectively.
Examiner and administration factors
- Fuchs and Fuchs (1986) meta-analysed 22 controlled studies of 1,489 children tested by familiar or unfamiliar examiners.
- Familiarity raised scores by .28 standard deviations, more for low-SES children, on difficult tests and with longer acquaintance.
- Limitation: small studies of individually tested children, so the size of the effect in group testing is unknown.
- Masling (1959) found examiners scored confederates’ identical ambiguous answers higher when the confederates acted warm; Sattler, Hillix and Neher (1970) found ambiguous Wechsler responses earned more credit inside a “bright” protocol (IQ about 130) than an average one.
- Extra time, paraphrased instructions or hints leave the norms describing conditions the examinee never met.
Testee factors
- Test anxiety — von der Embse and colleagues (2018), pooling 238 studies, found a small-to-moderate negative relation with performance, most pronounced in the middle grades, with anxiety higher on high-stakes tests.
- Practice and coaching — Hausknecht and colleagues (2007), across 107 samples and 134,436 people, found retest gains of .26 standard deviations, larger with coaching and with identical forms.
- Response sets and faking are taken up under the methodological issues; stereotype threat (Steele and Aronson, 1995) is treated in Use, Misuse and Limitations of Psychological Tests.
Situational factors
- Noise, light and time of day; the stakes, since selection invites faking and counselling candour; and the mode of administration, which needs its own equivalence evidence (International Test Commission, 2017). User competence is in Ethical Issues in the Use of Psychological Tests.
Constructed versus standardized tests
Construction ends when the items work; standardization fixes procedure and reference group for everyone who will take the test. A constructed test that gains try-out evidence, representative reliability and validity studies, norms and a manual has become a standardized one.
| Dimension | Constructed (teacher- or researcher-made) | Standardized |
|---|---|---|
| Purpose | One class, course or study; local decisions | Many settings; comparison with a defined population |
| Planning | Informal; content chosen by the constructor | Written specifications and a blueprint, reviewed by experts |
| Items | Written once, rarely pre-tested | Large pool, pre-tested, selected by item analysis |
| Reliability and validity | Assumed, or estimated on a convenience sample | Estimated on fresh samples and reported |
| Administration and scoring | Varies with the user | Fixed instructions, time limit and scoring key |
| Norms | None; read against the class or a pass mark | Percentile or standard-score norms from a stratified sample |
| Documentation | None | Manual, technical report, periodic re-norming |
| Example | A Class V teacher’s fractions test | National Achievement Survey booklets; the Adolescent Adjustment Inventory |
Steps in the construction and standardization of psychological tests

Each step closes one door to error; skip it and the door stays open, whatever follows.
- Downing (2006) set out twelve steps, from an overall plan to a technical report; the classical sequence below has eleven.
- 1. Purpose and construct — what is measured, in whom, for what decision, and whether norm- or criterion-referenced; the construct is defined by the behaviours that will count as evidence of it.
- 2. Blueprint (table of specifications) — content areas against the objective levels of Bloom (1956) or Anderson and Krathwohl (2001), with items allotted by teaching time and importance (Downing, 2006); format, length, time and scoring are fixed here.
- 3. Item writing — a pool well beyond the final length, written to tested guidelines (Haladyna and colleagues, 2002).
- 4. Expert review — relevance, accuracy, clarity and bias. Lawshe (1975) quantified it as the content validity ratio, CVR = (ne − N/2) / (N/2), where ne of N panellists rate an item essential; fifteen panellists require .49.
- 5. Pre-try-out — a small sample checks wording, instructions and time.
- 6. Try-out and item analysis — difficulty, discrimination between the top and bottom 27 per cent (Kelley, 1939), item–total correlation and distractors; Ebel (1965) read D of .40 or more as very good and below .20 as poor (Item Analysis and Item Validation).
- 7. Final form — items assembled to the blueprint, easy to hard for a power test, with parallel forms, instructions, time limit and key.
- 8 to 10. Reliability, validity and norms — each estimated on fresh, representative samples.
- 11. Manual and revision — every claim documented, and the test re-normed as content and populations change.
Worked design: an achievement test in arithmetic for Class V
- Purpose — a group test of arithmetic attainment at the end of Class V, reporting norm-referenced standing and mastery of each learning outcome.
- Domain — the Class V mathematics learning outcomes of the National Council of Educational Research and Training (2017), on which the national survey also builds: numbers beyond 1,000, the four operations, estimation, fractions, decimals, unit conversion, and problems on money, length, mass, capacity and time.
- Blueprint — forty items over six content areas and four levels: seven remember, twelve understand, fourteen apply, seven analyse. Computation and the money-and-measurement problems carry ten items each, as they take most teaching time.

Rows are what is taught, columns the thinking it calls for; every cell is a promise the items must keep.
- Items — about thirty four-option multiple-choice and ten short-answer items in the child’s medium; distractors come from known errors, as when 1/2 + 1/3 becomes 2/5.
- Review and pre-try-out — sixty items for forty places, rated by Class V teachers and a mathematics educator by Lawshe’s (1975) procedure, with urban or English-medium content rewritten; thirty to forty children then fix the wording and an hour’s time limit.
- Try-out — about 400 children across government and private, rural and urban, Hindi- and English-medium schools. Items are kept at p of .30–.80 and D of .30 or more, which Ebel (1965) rated reasonably good; weak distractors rewritten and items behaving differently by medium flagged.
- Reliability — KR-20 plus a two-week parallel-form retest; .80 suffices for reporting groups, .90 for placing individual children (Nunnally, 1978).
- Validity — content evidence from the blueprint; concurrent correlations with school examination results and teacher ratings; construct evidence from scores rising across Classes IV, V and VI.
- Norms — a sample stratified by region, school management, location, medium and gender gives percentile ranks and stanines, while cut-scores on each outcome give performance levels from below basic to advanced.
- Anchor — in the National Achievement Survey 2021, Class 5 pupils averaged 44 per cent in mathematics, against 53 per cent in 2017 (Ministry of Education, 2022); an outcome-by-outcome test shows what that single average conceals.
Worked design: an aptitude test for the Civil Services
- Purpose and criterion — predicting success in training and early service, so the criteria come first: training assessments at the Lal Bahadur Shastri National Academy of Administration, probation records and supervisors’ ratings.
- Job analysis — the critical incident technique of Flanagan (1954) gathers officers’ accounts of decisive conduct in flood relief, land disputes or riots; task inventories and officer panels then rate the abilities each task needs. Mission Karmayogi (2020) is mapping posts to competencies.
- Aptitudes and blueprint — about a hundred items in two hours: twenty each for comprehension, logical reasoning and decision-making, fifteen each for quantitative reasoning and interpersonal judgement, ten for general mental ability, with numeracy at Class X level so that an engineering degree is not measured.
- Item writing — decision and interpersonal items as situational judgement tests built from the incidents and keyed by officer consensus; McDaniel and colleagues (2001) found such tests predict job performance at ρ = .34, overlapping .46 with cognitive ability.
- Two languages from the start — parallel Hindi and English writing, back-translation (Brislin, 1970) and bilingual review under the guidelines of the International Test Commission (2017), never a literal translation at the end.
- Try-out on aspirants — a sample stratified by medium, discipline, region, gender and rural or urban schooling; differential item functioning by medium and discipline is tested by the Mantel–Haenszel procedure (Holland and Thayer, 1988), and offending items are rewritten or dropped.
- Reliability and forms — internal consistency by section, and a new parallel form each year equated to a common scale.
- Validation — predictive correlations with the training and probation criteria, corrected for range restriction; incremental validity over the general-studies paper; and differential prediction, since a fair test predicts equally well for each group (Cleary, 1968).
- Cut-score — set from judges’ item-by-item estimates for a minimally competent candidate (Angoff, 1971), not by fiat.
- Fairness, tested in 2014 — Hindi-medium Main candidates fell from 5,082 in 2008 and 4,156 in 2010 to 1,682 in 2011, the aptitude paper’s debut (UPSC reports, 2008–2011). Disparity alone is not bias; item-functioning and prediction evidence, gathered before use, decides it.
Worked design: a self-report personality inventory
Strategies of construction

The strategies differ in who decides what an item means: the theorist, the criterion group or the correlation matrix.
- Burisch (1984) classified the strategies as deductive (rational or theoretical), external (criterion-group) and inductive (factor-analytic).
- Deductive — items written from a definition or theory. Woodworth (1917) took the Personal Data Sheet’s 116 items from symptom lists to screen First World War recruits; Edwards (1954) wrote items for 15 needs of Murray (1938), paired as forced choices of equal social desirability.
- Strength: short and plain in meaning. Weakness: plain to the respondent too, and it assumes self-insight.
- External (criterion-keyed) — an item is kept if criterion and comparison groups answer it differently, whatever its content.
- Hathaway and McKinley (1940) cut over 1,000 items to 504, contrasting eight diagnostic patient groups (50 hypochondriacs for the first scale) with 724 visitors to the University of Minnesota Hospitals; the MMPI appeared in 1943.
- Meehl (1945) gave the rationale: an answer is verbal behaviour whose meaning is learnt empirically, not a self-description.
- Weakness: heterogeneous scales, a narrow comparison group and items whose predictive power nobody can explain, so cross-validation is essential.
- Inductive — items grouped by factor analysis of their intercorrelations, as in the Sixteen Personality Factor Questionnaire of Cattell (1949); factor naming is subjective (Factor Analysis and Multivariate Techniques, Paper I).
- The comparison — Hase and Goldberg (1967) built scales from one pool by each strategy and found them roughly equal in validity; Burisch (1984), reviewing over a dozen such studies, found no consistent superiority and preferred the deductive route for its economy.
- Combination — Jackson (1970) proposed a sequential system: rational definitions, a large pool, item–total selection, suppressed social desirability, convergent–discriminant checks.
- Goffin and colleagues (2000) found such an inventory predicted 487 managers’ performance better than a factor-analytic one; Costa and McCrae (1992) joined theory to factor analysis in the NEO-PI-R.
Steps, with an Indian example
Venu Gopal, Ashok and Madhu (2013) built the Adolescent Adjustment Inventory in Telugu and English through the general steps.
- Construct — adjustment as harmonious relations with home, academic and social contexts and with health and emotional life, the five areas fixed from expert opinion and earlier inventories.
- Pool and review — 600 items from existing measures plus new ones on domestic violence and tobacco chewing; four senior faculty cut duplicates and irrelevant items to 332.
- Comprehensibility — Class 6 pupils flagged what they could not follow; each item needed 75 per cent understanding, and “siblings” became “brothers or sisters” (Venu Gopal and colleagues, 2013).
- Translation equivalence — 82 bilingual Class 7 pupils took both forms in counterbalanced order; 42 items answered differently by language were revised, and two back-translations confirmed meaning.
- Item selection — on 2,073 pupils of Classes 7 to 10, an item survived only if it correlated at least .30 with a scale, and highest with its own: 186 items, five scales, 24 subscales.
- Validity and norms — convergent correlations with a culturally adapted Bell inventory (Ashok, Madhu and Suneetha, 2004); percentile norms on 9,082 pupils from three regions of Andhra Pradesh, by location, gender and class.
- General additions — balanced keying against acquiescence, and validity scales like the MMPI’s L (favourable self-presentation) and F (infrequent responding).
Major methodological issues in test construction
- Construct definition — construct under-representation (too little of the construct) and construct-irrelevant variance (something else as well, such as reading speed in a mathematics test) are the two ways a test fails (Messick, 1989).
- Level of measurement — measurement assigns numerals by rule (Stevens, 1946, paraphrasing N. R. Campbell); test scores are largely ordinal treated as interval, so means assume equal units that scoring does not guarantee.
- Sampling — of content, which the blueprint controls, and of people: a try-out in one city’s English-medium schools yields difficulty values that mislead everywhere else.
- Response sets — Cronbach (1946) identified acquiescence; Edwards (1953) found endorsement correlated .87 with judged social desirability, which Edwards (1957) made a variable in its own right.
- Remedies: balanced keying, forced choices matched on desirability, the scale of Crowne and Marlowe (1960), and validity scales.
- Faking — Viswesvaran and Ones (1999), across 51 studies, found all Big Five dimensions equally fakable, by roughly half to three-quarters of a standard deviation, with the largest shifts on social-desirability scales.
- Translation and adaptation — the International Test Commission (2017) set 18 guidelines in six groups (pre-condition, test development, confirmation, administration, score interpretation, documentation); equivalence must be shown, not assumed.
- Construct, method and item bias (van de Vijver and Tanzer, 1997) are in Use, Misuse and Limitations of Psychological Tests.
- Speed and the measurement model — a tight time limit changes what is measured and inflates split-half reliability (Types of Psychological Tests); classical item statistics are sample-bound, which item response theory avoids (Item Response Theory, Paper I).
Criticisms and limitations of standardized tests
- Uniform procedure is not uniform meaning — every test tends to favour the culture that produced it (Anastasi and Urbina, 1997), so identical instructions can set a first-generation, regional-medium child a different task.
- Each strategy has a blind spot — deductive scales are transparent, criterion-keyed scales atheoretical, factor scales hostage to factor naming (Burisch, 1984), and all self-report depends on insight and candour.
- Norms age — rising population scores (Flynn, 1987) make old norms flatter today’s examinees.
- High stakes corrupt the indicator — Campbell (1979) argued that an indicator used for social decisions becomes distorted and distorts the process it monitors; teaching to the test raises scores, not attainment.
- Disparity is not yet bias — group differences call for item-functioning and prediction evidence before either the test or the group is blamed (Cleary, 1968).
- Construction is costly — national norms across India’s languages need samples few developers can afford, so many instruments rest on regional samples, as the Telugu inventory does.
Indian context
- National Achievement Survey 2021 (Ministry of Education; NCERT) — 3.4 million pupils of Classes 3, 5, 8 and 10 in 1,18,274 schools and 720 districts, tested on 12 November 2021 with outcome-linked items scaled by item response theory (Ministry of Education, 2021).
- Principle: content validity by blueprint, every item tracing to a stated outcome.
- PARAKH — the national assessment centre that NEP 2020 called for, set up within NCERT on 8 February 2023 to set norms and standards for student assessment (Ministry of Education, 2023).
- Its Rashtriya Sarvekshan (4 December 2024) tested 21.15 lakh pupils of Classes 3, 6 and 9 in 74,229 schools, averaging 60, 46 and 37 per cent in mathematics; Class 5 was dropped (Ministry of Education, 2025). Principle: standard-setting across boards and languages.
- Civil Services (Preliminary) Examination — the Khanna Committee (2010) aptitude paper replaced the optional paper from 2011; English comprehension left the merit count on 4 August 2014, and the paper became qualifying at 33 per cent from 2015 (Department of Personnel and Training, 2015).
- Principle: fairness as part of validity, since a selection test must not reward the medium of schooling.
- Mission Karmayogi (Department of Personnel and Training, 2 September 2020) — shifts capacity building from rule-based to role-based through a Framework of Roles, Activities and Competencies: behavioural, functional and domain competencies graded by proficiency level. Principle: job analysis, the first step of any personnel test.
Previous Year Questions
- 2026What do you understand by Individual Differences? Comment on the major methodological issues in construction of tests for assessment.
- 2025What are the factors that determine the efficacy of psychological tests? Discuss the ethical issues in the use of psychological tests.
- 2024How will you construct an achievement test in Arithmetics for the fifth standard?
- 2023Explain the characteristics of standardized psychological tests. Highlight the limitations of psychological tests.
- 2018Discuss the different steps in the construction and standardization of psychological tests. Illustrate your answer with suitable example.
- 2016Analyze the factors determining the efficacy of psychological tests. Discuss the limitations in the use of psychological tests.
- 2015How will you go about constructing a test for assessing aptitude for Civil Services? Discuss the details.
- 2012Explain the characteristics of standardized psychological tests. Discuss the major steps in developing self-report personality inventory. Illustrate your answer with a suitable example.
- 2009What are the methodological differences between a ‘constructed’ and a ‘standardized’ psychological test?
- 2007Explain the characteristics and types of standardized psychological tests. Critically comment on their utility and limitations.
- 2004List the characteristics of psychological tests. Explain the process of construction of a standardized psychological test with an example of your choice.
- 2003Discuss the salient features of a psychological test. Identify the process involved in construction and standardization of a psychological test. What are the ethical issues involved in the use of a psychological test?
- 1999Describe the steps and procedures involved in construction of psychological tests.
