Computer Application in the Psychological Laboratory and in Testing

Computer application in the psychological laboratory and psychological testing.

Official Syllabus: Paper I · Unit 14 Issues and Perspectives in Modern Contemporary Psychology

“A test is a systematic procedure for comparing the behaviour of two or more persons.”

— Cronbach (1949)

Every clause of that definition names something a machine now does. The procedure is executed by software rather than by an examiner; the comparison is made against a stored norm table; and systematic becomes literal, because a program cannot vary its tone, skip an item or mis-add a column. What follows is an account of what is gained when each step of measurement and experimentation is handed over, and what is lost or newly risked in the handover.

The computer enters psychology at four separate points, and the objections at each point are different in kind: timing in the laboratory, construct validity in testing, model specification in analysis, and consent in data handling.

Computer application in the psychological laboratory

The five jobs the computer took over

• The mechanical laboratory had instruments already. Wundt’s Leipzig laboratory of 1879 timed reaction with a Hipp chronoscope, and Donders (1868) had derived processing durations by his subtraction method, subtracting simple from choice reaction time to isolate the decision stage.
• What those instruments could not do was run the experiment. Five functions passed to the computer, and each removes a specific source of error.
  • Stimulus generation and presentation — exact luminance, duration, position and sequence, replacing the tachistoscope and the memory drum.
  • Timing — presentation and response latency recorded to the millisecond, replacing the experimenter’s stopwatch.
  • Response capture — keypress, voice onset, touch, gaze or physiological channel, logged without transcription.
  • Sequencing — randomisation of order, counterbalancing of conditions, and adaptive staircases that set the next stimulus from the last response.
  • Logging and analysis — a trial-level data file that goes straight into statistical software.
• Reactive sequencing is the genuinely new capability. A paper-and-pencil procedure must fix its order in advance; a program can decide the next trial from the response just given, which is what makes psychophysical threshold tracking and adaptive testing possible at all.

The experimenter’s judgement does not disappear — it moves upstream into the script, where it is written once and applied identically to every participant.

Experiment-generator software and the timing question

• Experiments are now written in dedicated generators rather than in general-purpose code. PsychoPy (Peirce, 2007) is a Python library with a graphical builder; OpenSesame (Mathôt, Schreij and Theeuwes, 2012) is its contemporary; E-Prime, Presentation and Psychtoolbox are the other standard tools.
• Millisecond precision is an empirical claim about hardware and software together, not a specification. Bridges and colleagues (2020) ran a comparison across generators and found Psychtoolbox, PsychoPy, Presentation and E-Prime achieving mean precision under 1 ms for visual, audio and response measures in the laboratory, with browser-delivered studies more variable throughout.
  • The qualification belongs with the finding: the authors include PsychoPy’s own developer, so the comparison is not independent, and their own recommendation is that each laboratory measure the timing of its own machines rather than trust a published figure.
• Computerised versions of classic tasks became the standard instruments: the Stroop colour-word task of Stroop (1935), the Wisconsin Card Sorting Test, the dot-probe and visual search, all depending on latencies too short to time by hand.
• Automated neuropsychological batteries followed. Luciana (2003), reviewing use of the Cambridge Neuropsychological Test Automated Battery in paediatric populations, located its gains in standardisation and precise timing of timed subtests, not in replacing the assessor.

Data collection beyond the laboratory room

• Online samples. Buhrmester, Kwang and Gosling (2011) evaluated crowdsourced participant platforms and reported data at least as reliable as that from traditional recruitment, at lower cost and far greater speed — the finding that moved much of experimental psychology onto the web.
• Experience sampling. The experience sampling method of Csikszentmihalyi and Larson (1987) signalled participants at random moments to record current activity and affect; the smartphone made it cheap and dense, giving ecological momentary assessment.
• Passive sensing. Keystroke dynamics, movement, sleep and call patterns logged from a phone make up what Jain and colleagues (2015) named the digital phenotype — behaviour inferred from device data, not self-report.
• Simulation is a fourth use, in which the computer is not an instrument but the theory itself: a running model whose output is compared with human data. That use is treated in Artificial Intelligence and Psychology.

Computer application in psychological testing

How computer-assisted assessment developed

• The first operational system was built at the Mayo Clinic. Rome and colleagues (1962) had MMPI answer sheets read by an optical scanner, scored by computer and returned as printed descriptive statements keyed to scale elevations — scoring plus a crude interpretation, with no human in the loop.
• Three capabilities developed separately and are still worth separating.
  • Computer-assisted administration — items presented and responses captured on screen.
  • Automated scoring — raw responses converted to scale scores, standard scores and indices.
  • Computer-based test interpretation — narrative statements generated from the score pattern. The first programs producing narrative reports from whole scale configurations, rather than single-scale statements, appeared by 1969.
• Examiner-administered instruments such as the Wechsler scales use the computer only at the second and third stages: the examiner enters scores and the program returns the indices.

Administration, scoring and interpretation are three decisions, not one. The evidence supporting automation is strongest for the middle layer and weakest for the top.

Computerized adaptive testing and item banking

• Computerized adaptive testing selects each item from a calibrated pool according to the ability estimate derived from the responses already given. Its basis is item response theory, in which an item’s difficulty and discrimination are estimated on the same scale as the person’s ability, so items and people can be compared independently of the particular set administered.
• The algorithm is four steps, repeated.
  • Begin with an item of medium difficulty, since nothing is yet known about the examinee.
  • Administer it and re-estimate ability from all responses so far.
  • Select from the pool the item carrying most information at that estimate.
  • Stop when a precision criterion, usually a target standard error, is met.
• The consequence is measurement efficiency: the same standard error is reached with far fewer items, because nobody is asked questions too easy or too hard to be informative. Different examinees therefore sit different tests, which is the point and also the administrative difficulty.
• Item banking is the precondition. A pool must be calibrated on large samples, monitored for item drift as populations change, and governed by exposure controls so that frequently selected items are not compromised.
• The operational record is long. The adaptive Armed Services Vocational Aptitude Battery programme began in 1979, entered operational test and evaluation in 1992, and replaced the printed battery at all Military Entrance Processing Stations from 1996; the adaptive nursing licensure examination of the United States began on 1 April 1994.

Automated scoring of constructed responses

• Scoring free text by machine is older than it looks. Page (1966) argued that essays could be graded by computer and published results from his Project Essay Grade program in 1968; the work then lapsed for two decades on grounds of cost.
• The modern generation began with the e-rater engine, developed under Jill Burstein and launched for operational scoring in 1999.
• The limitation is structural, not technical. These engines score features that correlate with human judgements of quality — length, lexical variety, syntactic complexity, discourse markers — not the argument, so a fluent, well-signposted essay saying nothing can score well.

Delivery, supervision and security

• Bartram (2001) distinguished four levels of control over internet-delivered testing, and the level decides what a score supports: open (no identification, no supervision), controlled (identified, unsupervised), supervised (a proctor confirms identity and conditions) and managed (a dedicated centre controlling the environment).
• The International Test Commission issued International Guidelines on Computer-Based and Internet-Delivered Testing in 2005, organised under four headings — technology, quality, control and security — which remains the reference framework for practice.
• Remote proctoring adds identity verification, a locked-down browser and automated flagging of webcam and audio anomalies. Its problems are documented: false flags from ordinary behaviour, intrusion into the home, and dependence on a connection that is not uniformly available.
• Mode equivalence cannot be assumed. Scores from a screen and from paper are comparable only where comparability has been demonstrated for that instrument and population; where it has not, the computerised form is a new test requiring its own norms.

Statistical software and the analysis of data

A general-purpose package models observed variables one equation at a time; a structural equation program models latent variables and all equations at once. That single difference generates every entry in the comparison.

SPSS in psychological testing

• The Statistical Package for the Social Sciences was written in 1968 by Norman Nie, Dale Bent and Hadlai Hull, incorporated as a company in 1975, and acquired by IBM in 2009; it is now distributed as IBM SPSS Statistics.
• Its architecture is deliberately restrictive, which is why it is easy to learn. Data sit in a two-dimensional table with cases as rows and variables as columns; only numeric and string types exist; and three windows — Data Editor, Output Viewer and Syntax Editor — separate data, results and commands.
• In the construction and evaluation of a test it does the following work, which is the substance of its use in psychometrics.
  • Item analysis — item difficulty as an item mean, item discrimination as the corrected item-total correlation, and the effect on internal consistency of deleting each item.
  • Reliability — Cronbach’s alpha and split-half coefficients for internal consistency; a Pearson correlation between administrations for test-retest reliability.
  • Dimensionality — exploratory factor analysis with rotation, scree plot and eigenvalues, to establish how many constructs the items measure.
  • Norming — cumulative percentages and percentiles for a norm table, and saved standardised values to convert raw scores into z-scores and from there into any derived standard score.
  • Validity — correlation and regression against a criterion for predictive validity; group comparisons by t-test or analysis of variance for construct validity.
• One caution matters because the default hides it: the factor-analysis routine extracts principal components unless told otherwise, and principal components analysis is data reduction, not a common-factor model. The two give different loadings and support different claims about latent structure.
• The syntax file, not the menu, is what makes an analysis reproducible. A saved command file records exactly what was done, can be corrected and rerun, and is the audit trail that menu clicking does not leave.

LISREL and the analysis of multivariate data

• LISREL — LInear Structural RELations — implements Jöreskog’s method for the analysis of covariance structures, written with Dag Sörbom, originally in FORTRAN for mainframe hardware, with the multi-group version following in 1972 and commercial distribution from 1974.
• Its object is different from a general package’s. It estimates a measurement model, relating observed indicators to latent variables, and a structural model, relating the latent variables to each other, simultaneously, by maximum likelihood.
• Three consequences follow, and they are the whole case for structural equation modelling.
  • Measurement error is modelled rather than ignored. A regression on observed scores attenuates every coefficient by the unreliability of its predictors; a latent-variable model estimates the relation between the constructs.
  • Whole systems are estimated at once, including mediated paths, reciprocal effects and correlated errors, instead of one equation at a time.
  • The model as a whole is testable. A chi-square and approximate indices — RMSEA, CFI, TLI, SRMR — assess how well the specified structure reproduces the observed covariance matrix.
• Hu and Bentler (1999) are the usual authority for the thresholds in general use, around CFI ≥ .95, RMSEA ≤ .06 and SRMR ≤ .08. Two honest qualifications travel with them: the authors did not propose them as fixed rules, and their own simulations showed the maximum-likelihood RMSEA and TLI over-rejecting correct models at small sample sizes.

• Neither is a substitute for the other, and practice uses both: the general package for item analysis, reliability and exploratory structure, the structural program for confirmatory factor analysis and construct validation. R, Python, JASP and jamovi now give the same procedures in open-source form, with Mplus, AMOS and lavaan the usual alternatives for covariance-structure work.

The computer-human interface and behaviour

• The interface is not a neutral window. Who or what is asking changes what people disclose. Turner and colleagues (1998) randomly assigned 1,690 adolescent male respondents to an audio computer-assisted self-interview or a written self-administered questionnaire and found reported prevalence of the most stigmatised behaviours three times higher or more under computer administration.
  • The effect is not uniform across ages. Donohue and colleagues (1999) had children interviewed about a staged event by a computer program, by the program with an adult present, or by a standard verbal interview, and found no difference in words recalled, features recalled or disclosure of a planted secret, with the children preferring the computer.
  • Powell, Wilson and Thomson (2002) extended this to a second interview fourteen days later and found children then less willing to disclose the secret to the computer than to a person, concluding that the computer added little over face-to-face questioning of children.
• Transactive memory reorganises around the machine. Sparrow, Liu and Wegner (2011) found that people primed with hard questions thought of search engines, recalled where information had been stored better than the information itself, and remembered less of what they had been told would be saved.
• Claims about screens and wellbeing need the size of the effect, not its direction. Orben and Przybylski (2019) applied specification curve analysis to three large adolescent datasets, roughly 355,000 respondents in all, and found digital technology use explaining at most 0.4 per cent of variance in wellbeing — comparable to regularly eating potatoes, and about a quarter the size of the association with being bullied.
• The reciprocal influence of computing on psychology’s own theory — the mind as information processor — and the questions raised by conversational systems belong with Artificial Intelligence and Psychology.

Indian context

• Norms do not travel. The NIMHANS Neuropsychology Battery (Rao, Subbakrishna and Gopakumar, 2004) was built because performance on neuropsychological tasks in Indian samples varies strongly with education and literacy, so an imported computerised battery scored against Western norms misclassifies. Automation reduces administration error; it does not supply a norm group.
• Large-scale computer-based testing is now the default in Indian recruitment and admission, conducted through the National Testing Agency and comparable bodies. The psychological questions it raises are computer familiarity as a source of construct-irrelevant variance, and connectivity as a determinant of who can sit a remote test at all.
• Digital Personal Data Protection Act, 2023 (Ministry of Electronics and Information Technology), operationalised by the Digital Personal Data Protection Rules, 2025 notified on 13 November 2025 with an eighteen-month compliance window — requires itemised, purpose-specific consent carrying a means of withdrawal, erasure once consent is withdrawn or the purpose served, verifiable parental consent for a child’s data, and a Data Protection Board of India.
  • It gives statutory force to what were professional obligations: informed consent, purpose limitation and the confidentiality of test data.

Criticisms and limitations

• Interpretive reports are nomothetic, not idiographic. A narrative report says what is generally true of people with a similar score pattern. Written in the second or third person about a named individual, it reads as a statement about that person and is not one.
• The validity of interpretation is the weakest link. Lanyon (1984) objected that computer interpretations are clinical rather than actuarial in construction while being presented with actuarial authority. Epstein and Rotunda (2000) gave 155 practising psychologists one of four information packets about a patient and found that those given a clinician-authored report matched hospital staff ratings of symptomatology most closely; where the profile was ambiguous, those given a computer-generated report were less accurate than those given no report at all.
• Consumer satisfaction is not validity. Users rate narrative reports as accurate descriptions of themselves partly because generically true statements feel personally apt — the Barnum effect — so satisfaction surveys cannot substitute for criterion evidence.
• Reliability claims are often misstated. Automation removes scorer variance, and that is the whole of what it removes — it says nothing about the stability of the construct over time, which is what test-retest reliability measures. For an interpretive program, inter-rater reliability is not high but undefined: there is only one rater, so agreement has nothing to be agreement with.
• What the examiner was doing is lost. A human administrator establishes rapport, checks that instructions were understood, and observes the test-taking behaviour — persistence, strategy, distress — that frequently carries more clinical information than the score. Matarazzo (1986) pressed this point as a matter of professional integrity rather than convenience.
• Access outruns competence. Software that administers, scores and interprets makes testing available to people with no training in psychometrics, which is the standing argument for restricting test purchase by qualification.
• Security and equity cut against efficiency. Adaptive testing concentrates exposure on the most informative items; remote proctoring buys supervision at the cost of surveillance; and both assume hardware, bandwidth and computer familiarity that are unevenly distributed.
• The menu makes a procedure easy, not appropriate. Software will compute a multivariate analysis on data that violate its assumptions and report it cleanly. The default extraction method, the untested assumption and the undisclosed sequence of analyses are errors a package cannot catch, and the saved script is the only defence.
• What survives is substantial. Standardised presentation and millisecond timing made whole classes of experiment possible; adaptive administration gives equal precision from fewer items; automated scoring removes clerical error; and latent-variable modelling changed what counts as evidence for a construct.