Objective and Self-Report Assessment of Personality

Measurement of personality (pencil-paper test).

Official Syllabus: Paper I · Unit 11 Personality

A personality inventory is “essentially a standardized self-report… in which the subject describes his own behaviour.”

— Cronbach (1960), on what an inventory actually is

A personality inventory asks people to report on themselves and then treats the answers as data. That is either the most sensible thing to do — nobody else has watched the person for as long — or the least, since the respondent is the one party with a stake in the result. The whole technology of inventory construction is an attempt to keep the first while limiting the second.

The family of assessment methods

• The classification that organises this is Cattell’s: Q-data from questionnaires, L-data from the life record, and T-data from objective tests whose purpose the testee cannot guess. He regarded T-data as the least falsifiable and it is the least used, because such tests are laborious to build and often weakly valid.
• The maximum-typical distinction matters more than it is usually given credit for. Ability tests are maximum-performance measures: there are correct answers and the instruction is to do one’s best. Personality inventories are typical-behaviour measures: there are no correct answers and the instruction is to report honestly. Almost every technical difference between the two literatures follows from this.

How inventories are built

• Four construction strategies exist, and an inventory’s strengths and weaknesses are largely determined by which was used.
  • Rational or content strategy. Items are written because they appear, on their face, to measure the construct. The Woodworth Personal Data Sheet, developed during the First World War to screen recruits for susceptibility to breakdown, is the ancestor of the whole enterprise — a list of yes/no questions drawn from psychiatric interview content, yielding one overall score. It is simple and entirely dependent on the constructor’s judgement being right, and on the respondent being candid.
  • Criterion-keyed or empirical strategy. Items are selected solely because they discriminate a criterion group from a comparison group, irrespective of whether the content looks relevant. The Minnesota Multiphasic Personality Inventory — Hathaway and McKinley, developed around 1940 and distributed from 1943 — is the exemplar: over a thousand candidate items reduced to about five hundred by testing which ones separated diagnosed patients from ordinary visitors to the hospital.
    • The strength is that it makes no assumption about item meaning, so an item that works is retained even if nobody can say why.
    • The weakness is that a scale built this way may have no interpretable content, and that it is bound to the criterion groups used — which is why the inventory’s original diagnostic categories became dated and the revision was necessary.
  • Factor-analytic strategy. Items are retained according to how they load on factors recovered from the data, as in the sixteen-factor inventory, the five-factor inventories and Eysenck’s questionnaires. The structure is empirical and the resulting scales are internally coherent; the number of factors remains a decision of the analyst.
  • Theoretical or deductive strategy. Items are generated from a theory’s constructs — an inventory built on Murray’s list of needs, or one built on Jung’s types. The scales mean what the theory says they mean, and inherit whatever is wrong with it.
• Most modern instruments are hybrids, using a theory to generate items, factor analysis to select them, and criterion groups to validate them.

The validity-scale apparatus

• The problem is that a respondent can simply not tell the truth, and three distinct motives produce three distinct distortions: faking good for selection, faking bad for compensation or exculpation, and random or careless responding from indifference.
• The classic solution is to measure the distortion itself. The Minnesota inventory introduced the apparatus most instruments have copied.
  • L, the Lie scale: items describing minor faults almost everyone has. Denying too many of them indicates an unsophisticated attempt to look good.
  • F, the Infrequency scale: items endorsed by very few people in the normative sample. A high score suggests exaggeration, carelessness, confusion or severe disturbance — and because it bears on the whole protocol, a high F puts every other scale in doubt.
  • K, the Correction scale: a subtler index of defensiveness, developed empirically, whose score is used to adjust several clinical scales.
  • Cannot Say, the count of unanswered items, which at high levels invalidates the profile by itself.
• Forced-choice format is the other main defence: the respondent chooses between statements matched for social desirability, so the desirable answer is not available. It constrains faking and introduces ipsativity — scores become relative within the person rather than comparable between people, which limits what can be done with them.
• The limits of all this are real. Detection is imperfect; sophisticated respondents and coached ones evade it; and statistically correcting scores for faking has repeatedly been found not to improve, and sometimes to reduce, the validity of the prediction. Detecting distortion is more defensible than adjusting for it.

The problems with paper-and-pencil assessment

• Response sets and styles are systematic ways of answering that have nothing to do with content.
  • Social desirability — answering to present well, which may be deliberate or may be an honest but flattering self-view. The two are hard to distinguish and have different implications.
  • Acquiescence — a tendency to agree regardless of content, countered by balancing positively and negatively keyed items.
  • Extreme and moderate responding — habitual use of the ends or the middle of a rating scale, which varies systematically across cultures and so contaminates cross-national comparison.
  • Careless or random responding, which is common in long inventories and low-stakes settings.
• The limits of self-knowledge are the deeper problem, and no validity scale touches it. People have imperfect access to how they typically behave; self-ratings and informant ratings of the same person agree only moderately; and the gap is largest for the traits that are most evaluative, where the person has the strongest reason not to see clearly.
• Items do not mean the same thing to everyone. “I am often tense” requires the respondent to decide what counts as often and what counts as tense, and those judgements vary with the person’s standards and comparison group.
• The reference-group effect is a specific and underrated instance. Asked whether they are conscientious, people compare themselves to those around them — so members of a highly conscientious group may rate themselves lower than members of a less conscientious one. This makes mean comparisons between cultures on self-report scores close to uninterpretable, and is one reason national-character ratings fail to match aggregated self-reports.
• Transparency. Most items are face-valid, so the respondent generally knows what is being measured. In high-stakes selection this is a standing invitation to distortion.
• Language, literacy and translation limit applicability directly. A pencil-and-paper inventory requires reading ability, and translation must preserve not just meaning but difficulty and connotation — a serious constraint in multilingual settings, and the reason Indian use rests so heavily on adapted rather than indigenous instruments.
• Norms are a property of the sample that produced them. An imported norm table misplaces a respondent systematically, and the error is largest at the extremes where decisions are made.
• The situational problem: items ask about typical behaviour and so average over exactly the situational variation that the person-situation debate established to be substantial.

What answers these problems

• Multi-method assessment is the principal remedy, and it has a formal rationale. Campbell and Fiske’s multitrait-multimethod logic requires that measures of the same trait by different methods converge, and that measures of different traits by the same method diverge — which exposes how much of an apparent finding is method variance. Any single-method assessment, however good, cannot make that separation.
• Informant reports add genuine incremental validity, predicting outcomes over and above self-report rather than merely agreeing with it — which is the strongest argument for collecting them routinely.
• Behavioural and life data avoid the self-report problem entirely, at a cost in time and in breadth.
• Objective performance measures — tasks whose purpose is not apparent — are the ideal in principle and scarce in practice.
• Implicit measures, which infer dispositions from reaction times rather than from endorsement, bypass deliberate self-presentation. Their reliability is modest and what exactly they measure is contested, so they supplement rather than replace.
• Structured interviews and standardised observation restore reliability to methods that are unreliable in their informal versions.

Can personality be measured justly by projective and objective tests together?

• The answer has to separate two questions: whether the two methods complement each other, and whether testing of any kind can do justice to a personality.
• On complementarity, the case is good and specific. The two families fail in different directions, which is precisely the condition under which combining them helps.
  • Objective inventories are vulnerable to deliberate distortion and to limited self-knowledge; projective techniques are much less exposed to the first.
  • Projective techniques have contested reliability and depend on examiner judgement; inventories are reliable and objectively scored.
  • Inventories describe a person against norms; projectives generate individualised hypotheses.
  • The supported projective variables assess perceptual-cognitive process, which inventories do not reach at all; inventories assess self-concept and typical behaviour, which the projectives predict poorly.
• So the combination is defensible where each is used for what it is good at — and indefensible where a projective is used to produce a trait score or an inventory to diagnose thought disorder. The operative test is incremental validity: whether adding the second method improves the inference enough to justify its cost.
• On the deeper question, the honest answer is no. Tests support inferences about specified characteristics; they do not capture a personality. A battery can establish that a person is high in neuroticism, shows disordered thinking under ambiguity, and reports a particular pattern of relationships. It cannot supply what the person’s life means to them, what they are trying to do, or how they came to be as they are — which is what the idiographic and humanistic traditions insist is the subject matter. Doing justice requires the test to be one input into a formulation, with interview, history and observation — and requires the assessor to treat every score as an estimate with an error attached.

Merits and demerits of psychometric measures

• The merits are shared by ability and personality measures alike. Standardised administration and scoring remove the assessor’s idiosyncrasy; norms make a score interpretable; reliability and validity are quantified rather than asserted; the standard error is stated; the procedure is economical, reproducible and auditable. Against unaided clinical judgement — which is demonstrably prone to bias, overconfidence and illusory correlation — a test is usually the more defensible instrument.
• The demerits are also shared: norms mislead when transplanted; scores invite over-interpretation as fixed properties of persons; cultural loading is unavoidable; and the apparent precision of a number conceals the error around it.
• Where the two diverge is instructive, and the divergence follows entirely from maximum against typical performance.

• Personality measurement is intrinsically the harder problem, and the lower reliabilities and more contested validities are not a failure of effort but a consequence of measuring typical behaviour in a domain with no correct answers. The fuller treatment of the intelligence side belongs with the measurement of intelligence.

Criticisms and limitations

• Self-report remains the foundation of the field despite every known objection to it, because the alternatives are costly and narrower.
• Validity scales detect rather than correct, and the practice of statistically adjusting for faking is not supported by the evidence.
• Criterion-keyed scales inherit their criterion groups, so an instrument built on one era’s diagnostic categories becomes dated in a way a factor-analytic one does not.
• Forced-choice formats trade one problem for another, buying resistance to faking at the cost of ipsative scores that cannot be compared between people.
• The reference-group effect undermines cross-cultural comparison of mean scores, which is routinely done anyway.
• Indian practice depends on adapted instruments whose norms, and sometimes whose translations, have not been re-established on representative samples — a measurement problem with direct consequences in selection and clinical use.
• Tests are used beyond their validated purpose more often than any other failure in this field, and the remedy is procedural rather than technical: state the inference the score supports, and stop there.