The Comparative Method and its Limitations

Comparative Politics: nature and major approaches; political economy and political sociology perspectives; limitations of the comparative method.

This article covers the comparative method itself — its place among the methods of social science, its research designs, its significance in political analysis, and the full set of limitations that constrain it.

Official Syllabus: Paper II, Section A, Unit 1

Comparison is not a subject but a method, and its problems are methodological: too many variables, too few cases, and units that are not independent of one another.

That sentence contains most of what there is to say, and the rest of this article unpacks it. The comparative method exists because political science cannot experiment. A chemist manufactures the variation needed to isolate a cause; a comparativist has to find variation that history produced for its own reasons, and then argue that it can be read as though someone had designed it.

Sometimes that argument works. Kerala and Bihar share a constitution, a currency, a colonial past and a supreme court, and differ enormously in literacy and life expectancy — which makes the comparison between them close to a controlled experiment, and the differences that remain genuinely informative. Sometimes it does not work at all: comparing India with Sweden holds nothing constant, so any of a thousand differences could be doing the causal work.

The limitations of the comparative method are therefore not a list of practical inconveniences. They are the standing conditions under which every comparative claim is made, and the honest comparativist states them alongside the finding rather than in a footnote. A comparison whose limitations are unstated is not a stronger comparison; it is a less careful one.

The correct conclusion is neither dismissal nor complacency. These limitations do not invalidate comparative politics, but they do require that comparisons be made carefully and contextually — and they mean that comparison may not give a complete answer to the problem at hand.

The Comparative Method Among Methods

PSIR Optional Notes {PDF}

Complete Notes Package (Paper I & II) | 1500+ Pages | Instant Download

Discounted Price | Buy Now @ ₹1000 →

• Lijphart’s framework is the standard one. Social science has four methods, distinguished by how they control for confounding variables.
  • The experimental method — the strongest. The researcher assigns cases to treatment and control, so that everything except the treatment is held constant by randomisation. Rarely available in politics for ethical and practical reasons, though field experiments have expanded its range.
  • The statistical method — control through mathematical manipulation of a large number of observations. Confounders are held constant by including them as variables in a model.
  • The comparative method — the same logic as the statistical method, applied where the number of cases is too small for statistical control. Control is achieved by selecting cases rather than by modelling them.
  • The case study method — a single case, in which cross-case control is impossible altogether, but within-case inference remains available.
• The crucial point is that the comparative method is not a weaker form of statistics but the same logic under a binding constraint. Where there are 190 states and a hypothesis with a dozen plausible confounders, statistical control is possible; where there are five relevant cases, it is not, and the researcher must build the control into the case selection.
• Case studies are not the opposite of comparison, and Lijphart’s typology distinguishes six kinds by their relation to theory: atheoretical, interpretative, hypothesis-generating, theory-confirming, theory-infirming, and deviant case analyses.
  • Theory-infirming and deviant case studies do the most comparative work, because a single well-chosen case can falsify a general claim. India as a poor, durable democracy is a deviant case in exactly this sense, and its value to the discipline is that it breaks a generalisation rather than that it confirms one.
• The comparative method is a method of discovering empirical relationships, not a method of measurement. This distinction matters: measurement problems and inference problems are separate, and a study can be excellent at one and fail at the other.

The Significance of the Comparative Method in Political Analysis

Why comparison is indispensable

• Comparison is the substitute for the laboratory. Natural sciences have the luxury of controlled conditions; political science does not, and comparison is what stands in for it. Wherever scientific rigour and objectivity are required, comparison is the available method.
• Meaning is established by contrast. Strong state, federal, stable, corrupt — none of these has content until something is being contrasted with something else. A political system studied entirely on its own terms yields description, not analysis.
• It is the only route to general theory. Propositions derived from one country are hypotheses about that country. Only comparison converts them into claims about politics. Theories can be verified, rejected or modified only against evidence from many settings, and a theory that holds across many countries and conditions has stronger validity.
• It makes the discipline more objective and less parochial. Earlier scholarship was criticised as ethnocentric and as cultural imperialism. Comparison is what disciplines that tendency, by forcing categories to work outside the society that produced them.
• It improves understanding of one’s own system. By comparing India with other countries, the strengths and weaknesses of Indian institutions, policies and practices become visible in a way they cannot from inside. What looks natural from within is revealed as one option among several.
• It enables learning and the avoidance of error. Every system innovates in response to its own situation, and since social science has no paradigm shifts that render past experience obsolete, others’ solutions remain usable. The Indian institution of the Lokpal is inspired by the Ombudsman institutions of Scandinavia, and comparison also shows which experiments failed, so that failures need not be repeated.
• It permits the study of political behaviour under varying conditions, allowing behavioural trends and patterns to be verified or rejected across circumstances, and showing how the same institution functions differently in different atmospheres.

A discipline that cannot manufacture variation must find it — which makes case selection in comparative politics not preparation for the argument but the argument itself.

The four tasks comparison performs

• Description — rendering an unfamiliar system in a vocabulary that also fits familiar ones, so the two can be discussed together.
• Classification — sorting cases into types. Aristotle’s six-fold scheme is the first instance, and every typology of party systems, welfare regimes and authoritarianisms since is the same operation.
• Hypothesis-testing and explanation — the core task, and the one that requires the designs discussed below.
• Prediction — the weakest of the four, and the one the discipline has failed at most consistently. No comparativist predicted the collapse of the Soviet Union, the Arab Spring, or the speed of democratic backsliding after 2010.

The three levels of comparative study

• Studies of one country or one institution — a single party system, judiciary, parliament or pressure group, examined against the background of a larger comparative literature. Formally a case study; comparative in effect when its findings are framed against general knowledge.
• Studies of two or more countries that are genuinely comparable — most often within a region such as South Asia, Europe or Latin America, where enough is shared for the differences to be interpretable.
• Global and statistical comparison — large-N studies drawing on cross-national datasets and indices. Polity, Freedom House, V-Dem, the Democracy Index and the regional barometers are what made this level possible, and the Human Development Index and Gender Inequality Index extended it to development.

Research Designs

Mill’s methods

• John Stuart Mill’s canons of induction remain the logical foundation of every comparative design, and he is also the source of the standing scepticism about them, since he doubted they could be applied to social phenomena at all.
  • The method of agreement — if two or more instances of a phenomenon share only one circumstance, that circumstance is the cause. Cases that differ in everything except the outcome and one factor.
  • The method of difference — if an instance where the phenomenon occurs and one where it does not are alike in every respect but one, that one is the cause. Cases that are alike in everything except the outcome and one factor.
  • The method of concomitant variation — where one phenomenon varies whenever another varies, the two are causally connected.
  • The method of residues — subtract from a phenomenon what is known to be the effect of known causes, and the residue is the effect of the remaining antecedent.

The two classic designs

Whether comparison should be of similar or of dissimilar cases is itself a matter of debate, and both have their purpose and rationale.

• Why compare similar situations? Like the sciences, it is a way of keeping experimental conditions the same, so that the factors that make a difference — the ones responsible for the phenomenon — can be identified.
• Why compare dissimilar situations? It strengthens the hypothesis of a theory by establishing that an assumption holds only in given circumstances, and that changing certain factors changes the outcome.

• The MSSD is the more powerful design where it is available, because holding a shared history, constitution, legal system and currency constant removes an enormous number of rival explanations at a stroke. This is why sub-national comparison has become the growth area of the discipline.
• The MDSD’s characteristic risk is that a factor common to very different cases may be common for entirely different reasons in each — the appearance of a shared cause concealing several distinct ones.
• Equifinality is the deeper problem both designs face: the same outcome may be produced by several different causal paths, in which case there is no single cause to find and Mill’s canons return no answer.

The other designs

• Process tracing works within a case rather than across cases, tracing the causal chain step by step to establish that a mechanism actually operated. It is the standard answer to the objection that correlation across a handful of cases cannot establish causal direction, and it is what allows a single case to carry inferential weight.
• Qualitative Comparative Analysis, developed by Charles Ragin, uses Boolean and fuzzy-set logic on a medium number of cases to identify necessary and sufficient conditions and to handle equifinality and conjunctural causation — the finding that a factor may produce an outcome only in combination with others. It occupies the space between case study and regression that the discipline had left empty.
• Natural experiments exploit variation produced by history, geography or administrative rule that approximates random assignment — arbitrary borders, discontinuous eligibility thresholds, lotteries. India’s rotating reservation of panchayat seats for women is among the most cited natural experiments in the world, because the rotation is genuinely arbitrary.
• Paired comparison — two carefully matched cases studied in depth. Varshney’s matched Indian cities with similar demographics and opposite records of communal violence is the model.
• Nested analysis combines a large-N statistical study with intensive case studies chosen on the basis of the statistical results — using the regression to identify which cases will be most informative and the cases to establish the mechanism.
• Counterfactual analysis asks explicitly what would have happened had the presumed cause been absent, and is the logical form every causal claim implicitly takes.

Every comparative finding is a claim about cases nobody selected, in numbers too small for the variables involved — which is why stating the limitation is part of stating the finding.

The Limitations of the Comparative Method

1. Many variables, small N

• Lijphart’s formulation is the field’s founding statement of the problem: the comparative method has too many variables and too few cases, so that the number of plausible explanations exceeds the number of observations that could discriminate between them.
• The consequence is indeterminacy. Where there are five cases and fifteen candidate explanations, more than one explanation will fit the evidence perfectly, and no amount of care in analysing those five cases can decide between them.
• The problem compounds with the number of variables that matter in politics. Society, history, climate, geography, economy, resources, religion, colonial experience, elite composition, external environment — not all of these can be quantified or even observed, and they operate to different degrees in different states.
• Lijphart’s four remedies are the standard answers, and each has a cost.
  • Increase the number of cases as far as possible — which at some point converts the study into a statistical one, and requires thinning the concepts so they apply to more cases.
  • Reduce the property space by combining variables into broader categories — which loses information.
  • Focus on comparable cases — the comparable-cases strategy, which controls for shared features but restricts the range over which findings generalise.
  • Focus on the key variables identified by theory, discarding the rest — which requires that the theory be right about which variables matter.
• Sub-national comparison is the most productive modern remedy, because it multiplies cases while holding national-level variables constant. Twenty-eight Indian states are twenty-eight cases with one constitution.

2. Galton’s problem — cases are not independent

• Statistical and comparative inference both assume that cases are independent observations. In comparative politics they are not.
• Institutions diffuse — by imitation, conquest, colonial imposition, professional networks and international conditionality — so a correlation across countries may reflect borrowing rather than a causal relation.
• The illustration is sharp: fifty countries with similar central bank statutes are not fifty independent confirmations that this design is optimal. They may be one idea copied forty-nine times.
• The effective N is therefore smaller than the nominal N, sometimes drastically, which worsens the small-N problem rather than existing beside it.
• Remedies include modelling spatial and network dependence explicitly, treating diffusion as the phenomenon to be explained rather than as noise, and process-tracing whether a particular adoption was independent or imitative.
• The post-colonial version of the problem is the most important for comparative politics: much of the world’s institutional similarity exists because institutions were imposed by a small number of colonial powers, so cross-national findings about, say, common-law systems may be findings about the British Empire.

3. Conceptual stretching

• Sartori’s warning is the classic statement: extending a concept to more cases by thinning its content produces categories that travel widely and mean very little.
• The mechanism is the ladder of abstraction. To apply party, democracy or civil society to more countries, the researcher must drop the defining attributes that some cases lack. Extension increases as intension decreases, until the concept covers everything and distinguishes nothing.
• Democracy qualified into dozens of adjectival subtypes is the standard example — illiberal, delegative, electoral, competitive-authoritarian, defective, hybrid, guided. Each qualifier is an attempt to keep the concept meaningful while extending its reach.
• The problem is directly consequential for measurement. When scholars apply concepts such as democracy, state or legitimacy across different political systems, the terms lose clarity because the historical and institutional contexts differ, and an index that scores a stretched concept is measuring different things in different places while reporting one number.
• Sartori’s remedy is disciplined movement up and down the ladder of abstraction, with explicit statement of which attributes are being dropped, and preference for taxonomic differentiation over silent stretching.

4. Ethnocentrism and the origin of categories

• The categories of comparative politics were derived overwhelmingly from Western experience — party system, civil society, secularisation, interest group, state capacity, public sphere — and they may not correspond to how politics is organised elsewhere.
• The consequence is that difference registers as deficiency. A society organised through kinship, caste association or religious authority is recorded as having weak civil society, when it may have a dense associational life the instrument cannot see.
• The post-colonial critique makes the stronger claim: the categories are not merely parochial but were formed in the encounter with the colonised, so that the standard against which the non-West is measured was constructed by measuring the non-West.
• Chatterjee’s distinction between civil society and political society is the most useful constructive response — a new category built from the material rather than an adjustment to an imported one.
• The problem also runs the other way, and this is less often noticed. Concepts derived from non-Western experience — dominant caste, sanskritisation, communalism, asabiyyah — are rarely applied to Western cases, though several would illuminate them.

5. The problem of objectivity and cultural bias

• Certain social practices may be too incomprehensible to a foreign mind, and objectivity is obscured when the researcher cannot grasp what an act means to those performing it.
• Scholars interpret political systems through the lens of their own cultural background, producing biased analysis. What appears rational or democratic in one context may be inappropriate or simply misunderstood in another.
• Bias operates in both directions. The outsider misses meaning; the insider takes it for granted. Neither position is privileged, and comparative teams combining both are the practical answer.
• The index literature illustrates the difficulty concretely. Expert-coded democracy scores depend on coders’ priors, and the coders are disproportionately drawn from a small number of academic cultures — which is one reason the leading instruments disagree about the same countries in the same years.

6. Lack of sufficient and comparable information

• Public information is often incomplete and can be filtered by state agencies. China exercises strong control over what is released, and some totalitarian states release effectively nothing — North Korea being the limiting case. Comparison then becomes genuinely difficult.
• Even where data exists, its format, frequency and quality differ across countries, which makes valid comparison harder than the existence of numbers suggests.
• Definitional non-comparability is the deeper problem. Unemployment, poverty, literacy and even population are measured on different definitions in different countries, and a cross-national table of such figures is frequently a table of different quantities.
• Governments have a direct political interest in what they report, and the measurement of politically sensitive quantities is itself political.
• The result is a systematic bias in the literature: open societies are over-studied because they are legible, and findings drawn disproportionately from them are generalised to systems where the same measurement was impossible.

7. Difficulty in establishing causality

• Isolating one variable as the cause of a political outcome is difficult because of the complex interaction of multiple factors. Correlation is easier to observe; causation remains hard to prove.
• Endogeneity is pervasive and frequently unresolvable with the available evidence. Does wealth produce democracy, or democracy wealth? Do good institutions cause prosperity, or can prosperous societies afford good institutions? The correlation is identical under both readings.
• Reverse causation, simultaneity and omitted common causes each generate the same correlation, and cross-national data alone cannot distinguish them.
• Multiple causation and equifinality compound it: several factors may be jointly necessary, and different combinations may produce the same result in different cases.
• The remedies — natural experiments, instrumental variables, regression discontinuity, difference-in-differences, and process tracing to establish mechanism — have raised the standard of proof sharply, at the cost discussed below.

8. Language and translation barriers

• Accessing primary data, laws, speeches and cultural texts requires language proficiency, and misinterpretation through poor translation or lack of local insight distorts the analysis.
• The deeper problem is equivalence of meaning rather than accuracy of translation. A survey question about trust in government or interest in politics translated faithfully into twenty languages may still be asking twenty different questions, because the concepts carry different connotations.
• Terms also evolve over time. The meaning of authoritarianism, development and rights has changed, which complicates historical comparison and trend analysis — a 1965 measurement and a 2025 measurement of the same nominal variable may not be comparable.
• Response styles differ systematically across cultures — acquiescence, extreme responding, socially desirable answering — which introduces bias into cross-national surveys that no amount of translation care removes.

9. Selection bias

• Researchers may choose countries or cases that fit their hypothesis, producing non-representative findings and undermining the generalisability of conclusions.
• The specific and most damaging form is selecting on the dependent variable — studying only cases where the outcome occurred. Studying only successful revolutions cannot establish what causes revolution, because the supposed causes may be equally present in the many societies that did not have one.
• The problem is under-recognised because it is rarely deliberate. Cases are chosen because they are interesting, and they are interesting because something happened in them — which is selection on the dependent variable arrived at innocently.
• Data availability is itself a source of selection bias: the cases that can be studied are the cases that publish data, and these are systematically different from the cases that do not.
• The remedy is deliberate inclusion of negative cases, explicit statement of the population from which cases were drawn, and — where feasible — random or theoretically stratified selection.

10. The dynamic nature of the discipline

• Politics is shaped by multiple factors and is never static. A comparison valid today may not be relevant tomorrow, which limits the shelf life of comparative findings.
• The freezing hypothesis is the cautionary example. A finding about the stability of European party systems, correct when made, described a structure that began dissolving within a decade of its statement.
• Concept validity also decays. Categories built for the Cold War world of two blocs and a third world do not fit a world of one superpower, several great powers and a large number of hybrid regimes.
• The practical implication is that comparative claims should carry their temporal scope, and the discipline’s habit of citing decades-old findings without checking whether the world they described still exists is a genuine methodological failure.

11. Value bias

• The selection of what to compare, and of what counts as development, stability or good governance, embeds a normative position.
• Development as differentiation and secularisation, modernisation as industrialisation and urbanisation, inclusive as against extractive institutions — each of these pairs is presented as a measurement and functions as a judgment.
• The claim of value-neutrality makes the problem worse rather than better, because an acknowledged standpoint can be discounted by the reader and an unacknowledged one cannot.

12. Levels-of-analysis fallacies

• The ecological fallacy — inferring individual behaviour from aggregate data. A constituency where a caste is numerous and a party wins tells you nothing certain about how members of that caste voted.
• The individualistic fallacy — the reverse error of inferring aggregate outcomes from individual-level findings.
• Methodological nationalism, or whole-nation bias, is the least examined and possibly the most consequential. The nation-state is taken as the natural unit of comparison, though many of the causally relevant processes operate below it (regions, cities, ethnic groups) or above it (empires, markets, diasporas, international organisations).
  • Treating a country as one observation when its regions differ more from each other than from foreign regions discards the most useful variation available.

Remedies, and What They Cost

The Current Methodological Turn

• The causal-inference revolution has transformed standards of evidence. Natural experiments, regression discontinuity designs, instrumental variables, difference-in-differences and field experiments have made it possible to establish causal effects with a confidence the earlier literature never had.
  • The cost is a narrowing of ambition, and it is the field’s live methodological dispute. Designs that identify a causal effect cleanly are typically available only for small, local, short-run questions, while the questions comparative politics exists to answer — why some countries are democratic, why states form, why regimes collapse — admit no such design.
  • The risk is answering trivial questions rigorously while leaving the important ones to speculation.
• Multi-method and nested research is the practical settlement: statistical analysis to establish patterns across many cases, case studies and process tracing to establish that the mechanism actually operated in specific ones. Neither alone is sufficient, and the combination addresses different limitations.
• The revival of area studies within comparative politics is a methodological correction rather than a nostalgic one. Deep contextual and linguistic knowledge is now argued to be a precondition of valid measurement, not an alternative to it — because the errors it prevents are precisely conceptual stretching, mistranslation and category error.
• Transparency and replication have become explicit disciplinary commitments — data access and research transparency standards, pre-registration of hypotheses and analysis plans, replication archives — in response to the broader crisis of reproducibility across the social sciences. Pre-registration addresses the specifically comparative danger of selecting cases and specifications after seeing the results.
• Measurement has itself become a research programme. V-Dem’s use of multiple expert coders with statistical models that estimate and report coder uncertainty is a direct methodological answer to the objectivity and comparability problems, and it makes disagreement visible rather than concealing it inside a single number.
• Text-as-data and computational methods have expanded the universe of comparable material — manifestos, legislative speech, judicial opinions, news coverage — across many languages, while introducing new versions of the equivalence problem.

The Indian Illustrations

• India is both a hard case for comparison and the best available solution to its central problem.
  • As a single case it is enormous, internally heterogeneous and genuinely deviant, which makes it difficult to place in any cross-national dataset. Treating India as one observation alongside Estonia discards nearly all the information it contains.
  • As a comparative universe it is exceptionally rich. Twenty-eight states share a constitution, a legal system, a currency, an army and a central bank, and differ enormously in literacy, welfare delivery, party competition, caste structure and industrialisation. This is the most similar systems design available anywhere at that scale, and it is why sub-national Indian comparison has become a substantial international literature.
• The measurement disputes of the last two years are unusually clean illustrations of the limitations at work.
  • The inequality dispute turns entirely on what is measured. The government’s July 2025 claim that India is the fourth most equal nation rests on the consumption-based Gini from household expenditure surveys; the World Inequality Report 2026 finding that India is among the world’s most unequal rests on income and wealth distributions built from tax and survey data. Consumption inequality is always lower than income inequality, which is always lower than wealth inequality, so two accurate measurements support opposite headlines. This is a data-comparability problem, not a dishonesty problem, and it is exactly what the sixth limitation describes.
  • The democratic classification dispute illustrates conceptual and coder problems together. V-Dem’s Democracy Report 2026 classifies India as an electoral autocracy and places it among the top ten autocratisers; the EIU Democracy Index 2025 places it within the flawed-democracy range on a different definition and a different coding procedure. Two respected instruments applying different concepts of democracy to the same country reach materially different conclusions, which is conceptual stretching and coder bias operating simultaneously, and it is a reminder that every index is a theory of what democracy is, wearing a number.
• Data infrastructure problems compound both. The decennial census, due in 2021, was postponed and is now scheduled for 2027, so a decade of Indian social and economic statistics rests on projections from a 2011 base — a data limitation that affects every comparative claim about India made in that period.
• Category non-portability is illustrated best by caste. It is not class, not ethnicity, not race and not religion, and every attempt to code it as one of these in a cross-national dataset loses what makes it politically consequential. The concepts generated to handle it — dominant caste, sanskritisation, politicisation of caste — do not appear in comparative datasets at all, which is conceptual stretching in reverse: a phenomenon too specific to travel.
• The natural-experiment literature is where Indian data has contributed most to general method. The rotating reservation of panchayat seats for women under the 73rd Amendment assigns treatment in a manner close to random, and studies exploiting it have produced some of the most credible causal evidence anywhere on what descriptive representation does to policy outcomes and to aspirations.

Conclusion

The comparative method is the only instrument political science has for turning the world’s variation into knowledge, and it is a considerably weaker instrument than the confidence of its findings usually suggests. Its limitations are not defects to be corrected but the permanent conditions of its use — the consequence of studying an object that cannot be experimented on, in units that were not designed for comparison, with concepts formed in one civilisation and applied to all of them.

None of this invalidates comparative politics. It establishes the standard of care the method demands: cases selected for reasons stated in advance, concepts specified rather than stretched, data limitations reported rather than absorbed, causal claims held to the strength of the design that produced them, and findings dated to the period they describe.

What it does invalidate is a particular style of comparison — the confident cross-national generalisation drawn from a convenient handful of cases, expressed in concepts thinned until they fit, and stated without the qualifications that would make it useful. The most valuable comparative work is the work most explicit about what its comparison cannot show, because a finding whose limits are known can be built on, and a finding whose limits are hidden cannot.

Comparison, in short, may not give a complete answer to the problem at hand. It gives a better answer than the alternative, which is to study one country and call the result a theory.