“How can one resolve the issue of reliability and validity in the context of sociological research on inequality?” (2017)
- Pierre Bourdieu’s call to turn sociology’s own analytic instruments back on the researcher herself anchors this answer: studying inequality is never a neutral technical exercise, because the researcher’s own location within a hierarchy of class, caste, and educational capital shapes what she notices, measures, and is even willing to call “inequality” in the first place.
- Reliability is the consistency and repeatability of a measure — would a second investigator, or the same investigator on a second occasion, get the same result; validity is the correspondence between an indicator and the concept it claims to measure.
- Research on inequality is a genuinely hard case for both, because inequality is not one settled variable but a contested, multidimensional construct, and because the population under study spans dominant groups whose position it flatters to overstate and subordinate groups whose position is often risky to disclose accurately.
- This answer argues that resolution is not a single technical fix but a two-part discipline: methodological triangulation at the point of measurement, and sustained reflexivity about the researcher’s own position relative to the hierarchy she studies.
The Contested Operationalisation of “Inequality” Itself
- A single-indicator study can be perfectly reliable relative to itself while still lacking validity, because inequality is inherently multidimensional — income alone, a multidimensional deprivation index, and subjective status or prestige are three different constructs, not three measures of the same thing.
- Amartya Sen’s capability approach is the classic argument for why income-only measures under-capture real deprivation: two households with identical income can have starkly different capabilities to convert that income into functioning lives, depending on health, education, and social discrimination.
- Peter Townsend’s concept of relative deprivation pushes further — poverty and inequality must be assessed against the standard of living customary in a given society, not an absolute income line alone.
- A concrete illustration of how the choice of indicator changes the picture: India’s official multidimensional poverty index, built on health, education, and living-standard indicators rather than income alone, has recorded a substantially larger and differently-distributed decline in deprivation over the past decade than income-poverty estimates alone would suggest — precisely because it is measuring a different, broader construct.
Three ways of operationalising “inequality”
| Indicator | What it captures | Reliability | Validity risk |
|---|---|---|---|
| Income / consumption expenditure | Economic position; comparable across surveys | High — standardised, repeatable | May miss non-economic deprivation (health, education, dignity, discrimination) |
| Multidimensional deprivation index | Overlapping disadvantages across sectors | Moderate — composite of several indicators | Choice of weights and cut-offs is itself a value judgement |
| Subjective status / prestige | Lived experience and self-perceived standing | Lower — varies across respondents and interviewers | Captures meaning but is hard to standardise across groups |
Social-Desirability Bias and the Reliability of Self-Reported Data
- Respondents systematically under-report or over-report depending on what the answer costs or gains them — an upper-caste respondent may under-report the extent of discriminatory practice, a household may over-report deprivation where survey responses are linked to welfare eligibility.
- Kirk and Miller’s distinction between types of reliability sharpens this problem: a single method that keeps producing suspiciously uniform, “rehearsed” answers about caste discrimination exhibits what they term quixotic reliability — consistency that signals a validity failure rather than genuine measurement.
- Interviewer effects compound this: the caste, class, and gender of the person asking the question shape what a respondent is willing to say, so the same instrument administered by different interviewers may not actually be measuring the same thing.
Access Barriers and the Representativeness of the Sample
- Marginalised groups — Dalit and Adivasi households, migrant labour, women in seclusion — are structurally harder to access, so a sample that skews toward the more visible or vocal risks both unreliability (different investigators reach different, non-representative sets of respondents) and invalidity (findings that do not represent the population they claim to describe).
- Access to a locally dominant caste’s own gatekeeper is often the price of entry to a village or community study; the gatekeeper who opens one door quietly closes others, and the researcher inherits his blind spots and his position within local inequality along with the access itself.
Measurement Invariance: Borrowed Instruments and Local Meaning
- Standardised stratification instruments developed on a different population can lack validity when applied to India’s specific hierarchy — an occupational-prestige scale built for a class society organised around income and skill does not automatically capture a hierarchy organised significantly around ritual purity and pollution.
- Max Weber’s three-dimensional model of class, status, and party is a useful corrective here: a Western import that measures only economic class risks entirely missing the status dimension through which much of Indian inequality is actually lived and enforced.
- Aaron Cicourel’s charge of “measurement by fiat” names the danger precisely — imposing a numerical category on a phenomenon whose local meaning has not first been established, buying apparent precision at the cost of validity.
Resolution: Triangulation Across Data, Method, and Investigator
- Norman Denzin’s typology of triangulation — across data sources, investigators, theories, and methods — supplies the working resolution: because standardisation and immersion trade off against each other (standardised instruments buy reliability at the cost of validity; immersive fieldwork buys validity at the cost of reliability), combining them lets each correct the other’s characteristic blind spot.
- Concretely: measure the distribution of inequality with reliable, standardised instruments — income, landholding, occupation, educational attainment — which give comparable, repeatable numbers; then investigate what inequality means and how it is lived and reproduced through ethnography and unstructured interviewing, which no fixed schedule can reach.
- The two phases check each other: the qualitative phase tests whether the quantitative indicators are actually valid — whether caste as coded in a survey corresponds to caste as practised on the ground — while the quantitative phase tests whether what the ethnography found in one setting holds beyond it.
Reflexivity as a Corrective for the Objectifying Gaze
- Bourdieu argued that objectivity in social research is not a private mental state achieved through willpower but a collective, procedural accomplishment — and that the researcher studying a hierarchy must first “objectify the objectifying subject,” making her own social position part of the analysis rather than treating it as invisible.
- A researcher studying caste-based inequality who is herself urban, upper-caste, and formally educated shapes rapport, access, and interpretation by that very position; declaring it does not remove the effect but makes it available for a reader to weigh.
- Gunnar Myrdal’s parallel remedy — stating one’s value premises explicitly and in advance — converges with this: transparency about where bias is likely to enter is a more realistic standard than a false claim of pure neutrality.
- Reliability and validity in inequality research cannot be resolved by any single instrument, because the object of study is itself contested, multidimensional, and asymmetrically accessible to the researcher.
- The realistic resolution combines standardised, comparable measurement of distribution with immersive, meaning-sensitive fieldwork on lived experience, cross-checking each against the other rather than trusting either alone.
- India’s own recent experience with multidimensional poverty measurement illustrates the stakes concretely: which indicators are chosen, and how they are weighted, changes not just the number but the story told about who is deprived and by how much.
- The final safeguard is not technical but reflexive — a researcher who makes her own position within the hierarchy she studies explicit produces more trustworthy findings than one who claims a neutrality no one studying inequality can fully possess.