Major Steps in Psychological Research: Problem, Hypothesis, Design and Report

Major steps in Psychological research (problem statement, hypothesis formulation, research designs, sampling, tools of data collection, analysis and interpretation and report writing); Fundamental versus applied research; Research designs (ex-post facto).

Official Syllabus: Paper I · Unit 3 Research Methods

“Scientific research is systematic, controlled, empirical and critical investigation of hypothetical propositions about the presumed relations among natural phenomena.”

— Kerlinger (1973)

Every word in that definition is doing work. Systematic and controlled mean the steps run in an order that rules out rival explanations as it goes; empirical means the answer is decided by observation rather than by argument; critical means the researcher’s own conclusion is the thing most severely tested. The sequence of steps below is not a procedure for tidiness. It is the mechanism by which a question becomes an answer that someone else can check.

Why the steps are a method and not a ritual

A step skipped does not merely delay the work; it makes a specific error available that nothing later can repair.

• Define the problem loosely and the hypothesis will be untestable, because there is no statement that could turn out false.
• Fix the hypothesis after seeing the data and the significance test becomes meaningless, since the probability it reports assumes the prediction came first.
• Choose the design after collecting data and the confounds are already in the numbers, where no analysis can separate them.
• Draw the sample carelessly and the finding describes nobody, however large the sample and however small the p-value.
• Analyse without deciding the test in advance and the result is whichever of several tests happened to pass.

This is the substance of the claim that method is the backbone of research. The backbone metaphor is apt in a particular way: it carries the weight, and it also fixes the order. Content can be replaced; the sequence cannot be rearranged without the structure failing.

Each step shuts a door the next one cannot.

The major steps

Identifying and stating the problem

• The research problem is stated as a question about a relation between variables, and in a form that observation could answer. “Does massed or distributed practice produce better retention in school children?” is a problem; “the importance of study habits” is a topic.
• Sources of problems: gaps and contradictions in existing findings, a theory’s untested implication, a practical difficulty in the field, a new phenomenon, and replication failures.
• Three criteria a problem must meet: it states a relation between two or more variables; it is stated unambiguously, usually as a question; and it implies possibilities of empirical testing. A problem failing the third is a philosophical question, not a research problem.
• Feasibility is a separate test — time, cost, access to participants, availability of instruments, and ethical permissibility. A well-posed problem that cannot be investigated is still not a research problem for this investigator.

Reviewing the literature

• Establishes what is already settled, what is contested, and which measures and designs have worked, so that the study adds rather than repeats.
• Supplies the theoretical frame from which the hypothesis is deduced, and the expected effect size from which sample size is calculated.
• In exploratory and qualitative work the review is deliberately deferred rather than skipped, so that existing categories do not pre-empt the participants’ own.

Formulating the hypothesis

“A hypothesis is a conjectural statement of the relation between two or more variables.”

— Kerlinger (1973)

• A hypothesis is a tentative, testable, falsifiable proposition, stated before data are seen, specifying direction where theory permits. “Sleep-deprived participants will recall fewer words than rested participants” is directional; “recall will differ” is non-directional.
• The null hypothesis states no difference or no relation; the alternative or research hypothesis states what the investigator expects. Statistical tests are performed on the null, which is why a result is reported as rejecting or failing to reject it, never as proving the alternative.
• What a hypothesis does for the study, and why it is not an optional flourish:
  • It makes the study falsifiable, which is what distinguishes research from illustration. Popper (1959) made this the demarcation criterion of science: a proposition no observation could contradict is not scientific.
  • It determines the design — the variables to manipulate, the controls needed, the groups required.
  • It fixes the statistical test and the direction of the critical region before the numbers arrive, which is what keeps the stated error rate honest.
  • It limits the data collected, preventing the indiscriminate gathering that produces a dataset nobody can analyse.
  • It supplies the logical link between theory and observation: a theory is confirmed or embarrassed only through the hypotheses deduced from it.
• An operational definition specifies the concept by the operations used to measure or produce it — anxiety as the score on a named inventory, frustration as a blocked goal in a specified task. It is what makes a hypothesis testable, and it is also where most disputes about findings actually lie.

The three are often confused and the distinction is exact.

Three levels of claim, and only one of them is tested directly.

Must the hypothesis always come first?

No, and the honest answer distinguishes two situations rather than choosing a side.

• Where a theory exists and the question is whether it holds, the hypothesis must precede the data. The logic of significance testing assumes it: a test reports how often data this extreme would arise if the null were true, and that probability is only interpretable for a prediction made in advance. Formulating a hypothesis after inspecting the data and presenting it as prior — sometimes called hypothesising after the results are known — inflates the error rate invisibly and is a principal cause of findings that fail to replicate.
• Where the field is new, no theory exists, or the phenomenon has not been described, insisting on a prior hypothesis forces the investigator to invent one and then find it. Exploratory surveys, ethnographic work, case studies and the grounded theory approach all begin from a research question instead, and generate hypotheses as output. Developing a layperson’s account of wellbeing, or mapping how a newly displaced community organises itself, admits no sensible prior prediction.
• The resolution in practice is to keep the two labelled and separate. Exploratory analyses are legitimate and valuable; they are reported as exploratory, and the hypotheses they generate are tested on fresh data. The error is not exploring but presenting exploration as confirmation. Preregistration of hypotheses, design and analysis plan before data collection is the current procedural answer.

Choosing the research design

• The design is the arrangement that will let the hypothesis be tested with rival explanations excluded, and it is chosen before data are collected, since it dictates what is collected.
• The choice follows the question: experimental where the variable can be manipulated and participants assigned at random; quasi-experimental where intact groups must be used; non-experimental where the variable is given rather than manipulable.

Sampling

• A population is the entire set of units about which a conclusion is wanted; a sample is the subset actually studied; and the inference from one to the other is valid only if the sample was drawn by a known probability mechanism.
• Probability methods — simple random, systematic, stratified, cluster and multi-stage — each give every unit a calculable chance of selection. Non-probability methods — convenience, purposive, quota and snowball — do not, and do not support estimation.
• Two errors are routinely conflated. Sampling error is the unavoidable discrepancy between a sample statistic and the population parameter arising from chance alone, and it shrinks as sample size grows. Error in sampling, or sampling bias, is a systematic distortion from a defective procedure, and no increase in sample size reduces it at all.

Tools of data collection

• Interview, observation, questionnaire, psychological test, archival record and physiological measure. The tool is chosen for the construct and the respondent, and its reliability and validity are established before the main study, usually through a pilot.

Analysis and interpretation

• Analysis is the arithmetic: descriptive statistics to summarise, inferential tests to decide, with the test fixed by the design, the measurement scale and the assumptions the data meet.
• Interpretation is the separate act of saying what the numbers mean for the hypothesis and the theory, and it is where the characteristic errors live — treating statistical significance as practical importance, reading a correlation as a cause, accepting the null as proof of no effect, and generalising beyond the sampled population.
• Current practice requires an effect size and a confidence interval alongside any significance test, precisely because the test alone says nothing about magnitude.

Report writing

• The report’s purpose is to make the study repeatable by a stranger, so the method section carries whatever another investigator would need to run it again: participants and how they were obtained, materials, procedure in order, and the analysis as planned.
• The conventional sections are abstract, introduction with the hypothesis stated, method, results, discussion with limitations, and references. Negative and null findings belong in the record; their omission is what produces publication bias.

Fundamental versus applied research

• Fundamental research, also called basic or pure research, is undertaken to extend knowledge of principles, with no particular practical use in view. Ebbinghaus memorising nonsense syllables to establish the shape of forgetting is the type case.
• Applied research is undertaken to solve an identified practical problem in a particular setting — whether a specific training module reduces industrial accidents at this plant.

The distinction is one of purpose, not of method or rigour, and the common examination error is to treat applied work as looser. Both use the same designs and the same statistics.

How basic knowledge reaches practice is the substantive half of the question, and it happens along identifiable routes:

• The principle transfers to a new domain. Pavlovian conditioning, established on salivating dogs with no clinical intent, supplied the mechanism for systematic desensitisation and for conditioned drug tolerance. Operant principles gave behaviour modification and token economies.
• The measure transfers. Basic work on memory supplied the recall and recognition paradigms used clinically to detect dementia.
• Basic findings set the limits of what an intervention can achieve. Work on the capacity of working memory tells an interface designer how many options a menu may carry.
• The flow runs both ways. Applied problems generate basic questions: industrial studies of attention produced vigilance research, and clinical observation of amnesic patients produced the distinction between declarative and procedural memory. The relation is a cycle, not a pipeline, which is why use-inspired basic research is now treated as a category of its own rather than a compromise.

Ex-post facto research designs

An ex-post facto design examines a relation between variables after the presumed cause has already occurred, by selecting groups that differ on it. Smokers compared with non-smokers, children of divorced and intact homes, flood-affected and unaffected villages. Kerlinger (1973) treated it as the basic form of non-experimental research, and the name means “from what is done afterwards”.

The groups chose themselves, so what made them differ cannot be said.

It is often the only design available, because the variable cannot be manipulated or it would be unethical to try, and it is high in realism because nothing was arranged. Its drawbacks are severe and specific:

• No manipulation of the independent variable. The investigator takes the groups as found, so the “treatment” was self-selected or imposed by circumstance.
• No random assignment, so the groups may differ on any number of further variables. This is the central defect, and every other one follows from it.
• The risk of improper interpretation, since a difference found between the groups is readily read as caused by the variable that defined them.
• The direction of causation is open. Poor school performance may follow low self-esteem or produce it, and a cross-sectional comparison cannot tell which.
• The third-variable problem is unlimited. Any characteristic correlated with group membership is a candidate explanation, and only measured ones can be controlled for.
• Self-selection operates where people choose their own exposure, so those who enter a condition differ from those who do not, before the condition acts.
• Retrospective reconstruction. Where the data depend on recall of past events, memory is reshaped by the present outcome, and the affected group recalls more precursors because they have been looking for them.
• The result is that an ex-post facto study can establish association and its magnitude, and cannot establish causation. The honest use is to generate hypotheses, estimate the size of a relation, and narrow the field for designs that can test it.

Criticisms and limitations

• The steps are a reconstruction, not a diary. Real investigations loop back: the problem is restated after a pilot, the design is revised when access fails, measures are changed when they do not work. Presenting the sequence as what happened is a convention of reporting, and treating it as a description of the process misleads the beginner.
• The requirement of a prior hypothesis is applied selectively. It is enforced in confirmatory work and quietly suspended in much published research, where several analyses are run and the successful one is reported as predicted. The replication difficulties of the last decade are substantially a consequence.
• Operationalisation can hollow out the concept. Defining a variable by its measure makes the hypothesis testable at the cost of possibly testing something else, and a literature can accumulate consistent findings about an operation that nobody has shown corresponds to the construct named.
• Null results do not reach the record. Journals and investigators both favour positive findings, so the published literature overstates effects, and the step of report writing is where the distortion enters.
• The basic-applied distinction is institutionally loaded. It affects funding and prestige rather than describing a real divide in method, and the categories fit poorly where a study is designed from the start to answer a practical question through a fundamental mechanism.