falsifiability of research hypothesis

Karl Popper: Theory of Falsification

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

Karl Popper’s theory of falsification contends that scientific inquiry should aim not to verify hypotheses but to rigorously test and identify conditions under which they are false. For a theory to be valid according to falsification, it must produce hypotheses that have the potential to be proven incorrect by observable evidence or experimental results. Unlike verification, falsification focuses on categorically disproving theoretical predictions rather than confirming them.

• Karl Popper believed that scientific knowledge is provisional – the best we can do at the moment.
• Popper is known for his attempt to refute the classical positivist account of the scientific method by replacing induction with the falsification principle.
• The Falsification Principle, proposed by Karl Popper, is a way of demarcating science from non-science. It suggests that for a theory to be considered scientific, it must be able to be tested and conceivably proven false.
• For example, the hypothesis that “all swans are white” can be falsified by observing a black swan.
• For Popper, science should attempt to disprove a theory rather than attempt to continually support theoretical hypotheses.

Theory of Falsification

Karl Popper is prescriptive and describes what science should do (not how it actually behaves). Popper is a rationalist and contended that the central question in the philosophy of science was distinguishing science from non-science.

Karl Popper, in ‘The Logic of Scientific Discovery’ emerged as a major critic of inductivism, which he saw as an essentially old-fashioned strategy.

Popper replaced the classical observationalist-inductivist account of the scientific method with falsification (i.e., deductive logic) as the criterion for distinguishing scientific theory from non-science.

All inductive evidence is limited: we do not observe the universe at all times and in all places. We are not justified, therefore, in making a general rule from this observation of particulars.

According to Popper, scientific theory should make predictions that can be tested, and the theory should be rejected if these predictions are shown not to be correct.

He argued that science would best progress using deductive reasoning as its primary emphasis, known as critical rationalism.

Popper gives the following example:

Europeans, for thousands of years had observed millions of white swans. Using inductive evidence, we could come up with the theory that all swans are white.

However, exploration of Australasia introduced Europeans to black swans.  Poppers’ point is this: no matter how many observations are made which confirm a theory, there is always the possibility that a future observation could refute it.  Induction cannot yield certainty.

Karl Popper was also critical of the naive empiricist view that we objectively observe the world. Popper argued that all observation is from a point of view, and indeed that all observation is colored by our understanding. The world appears to us in the context of theories we already hold: it is ‘theory-laden.’

Popper proposed an alternative scientific method based on falsification.  However, many confirming instances exist for a theory; it only takes one counter-observation to falsify it. Science progresses when a theory is shown to be wrong and a new theory is introduced that better explains the phenomena.

For Popper, the scientist should attempt to disprove his/her theory rather than attempt to prove it continually. Popper does think that science can help us progressively approach the truth, but we can never be certain that we have the final explanation.

Critical Evaluation

Popper’s first major contribution to philosophy was his novel solution to the problem of the demarcation of science. According to the time-honored view, science, properly so-called, is distinguished by its inductive method – by its characteristic use of observation and experiment, as opposed to purely logical analysis, to establish its results.

The great difficulty was that no run of favorable observational data, however long and unbroken, is logically sufficient to establish the truth of an unrestricted generalization.

Popper’s astute formulations of logical procedure helped to reign in the excessive use of inductive speculation upon inductive speculation, and also helped to strengthen the conceptual foundation for today’s peer review procedures.

However, the history of science gives little indication of having followed anything like a methodological falsificationist approach.

Indeed, and as many studies have shown, scientists of the past (and still today) tended to be reluctant to give up theories that we would have to call falsified in the methodological sense, and very often, it turned out that they were correct to do so (seen from our later perspective).

The history of science shows that sometimes it is best to ’stick to one’s guns’. For example, “In the early years of its life, Newton’s gravitational theory was falsified by observations of the moon’s orbit”

Also, one observation does not falsify a theory. The experiment may have been badly designed; data could be incorrect.

Quine states that a theory is not a single statement; it is a complex network (a collection of statements). You might falsify one statement (e.g., all swans are white) in the network, but this should not mean you should reject the whole complex theory.

Critics of Karl Popper, chiefly Thomas Kuhn , Paul Feyerabend, and Imre Lakatos, rejected the idea that there exists a single method that applies to all science and could account for its progress.

Popperp, K. R. (1959). The logic of scientific discovery . University Press.

Further Information

• Thomas Kuhn – Paradigm Shift Is Psychology a Science?
• Steps of the Scientific Method
• Positivism in Sociology: Definition, Theory & Examples
• The Scientific Revolutions of Thomas Kuhn: Paradigm Shifts Explained

• Foundations
• Write Paper

Search form

• Experiments
• Anthropology
• Self-Esteem
• Social Anxiety
• Foundations >
• Reasoning >

Falsifiability

Karl popper's basic scientific principle, karl popper's basic scientific principle.

Falsifiability, according to the philosopher Karl Popper, defines the inherent testability of any scientific hypothesis.

This article is a part of the guide:

• Inductive Reasoning
• Deductive Reasoning
• Hypothetico-Deductive Method
• Scientific Reasoning
• Testability

Browse Full Outline

• 1 Scientific Reasoning
• 2.1 Falsifiability
• 2.2 Verification Error
• 2.3 Testability
• 2.4 Post Hoc Reasoning
• 3 Deductive Reasoning
• 4.1 Raven Paradox
• 5 Causal Reasoning
• 6 Abductive Reasoning
• 7 Defeasible Reasoning

Science and philosophy have always worked together to try to uncover truths about the universe we live in. Indeed, ancient philosophy can be understood as the originator of many of the separate fields of study we have today, including psychology, medicine, law, astronomy, art and even theology.

Scientists design experiments and try to obtain results verifying or disproving a hypothesis, but philosophers are interested in understanding what factors determine the validity of scientific endeavors in the first place.

Whilst most scientists work within established paradigms, philosophers question the paradigms themselves and try to explore our underlying assumptions and definitions behind the logic of how we seek knowledge. Thus there is a feedback relationship between science and philosophy - and sometimes plenty of tension!

One of the tenets behind the scientific method is that any scientific hypothesis and resultant experimental design must be inherently falsifiable. Although falsifiability is not universally accepted, it is still the foundation of the majority of scientific experiments. Most scientists accept and work with this tenet, but it has its roots in philosophy and the deeper questions of truth and our access to it.

What is Falsifiability?

Falsifiability is the assertion that for any hypothesis to have credence, it must be inherently disprovable before it can become accepted as a scientific hypothesis or theory.

For example, someone might claim "the earth is younger than many scientists state, and in fact was created to appear as though it was older through deceptive fossils etc.” This is a claim that is unfalsifiable because it is a theory that can never be shown to be false. If you were to present such a person with fossils, geological data or arguments about the nature of compounds in the ozone, they could refute the argument by saying that your evidence was fabricated to appeared that way, and isn’t valid.

Importantly, falsifiability doesn’t mean that there are currently arguments against a theory, only that it is possible to imagine some kind of argument which would invalidate it. Falsifiability says nothing about an argument's inherent validity or correctness. It is only the minimum trait required of a claim that allows it to be engaged with in a scientific manner – a dividing line between what is considered science and what isn’t. Another important point is that falsifiability is not any claim that has yet to be proven true. After all, a conjecture that hasn’t been proven yet is just a hypothesis.

The idea is that no theory is completely correct , but if it can be shown both to be falsifiable  and supported with evidence that shows it's true, it can be accepted as truth.

For example, Newton's Theory of Gravity was accepted as truth for centuries, because objects do not randomly float away from the earth. It appeared to fit the data obtained by experimentation and research , but was always subject to testing.

However, Einstein's theory makes falsifiable predictions that are different from predictions made by Newton's theory, for example concerning the precession of the orbit of Mercury, and gravitational lensing of light. In non-extreme situations Einstein's and Newton's theories make the same predictions, so they are both correct. But Einstein's theory holds true in a superset of the conditions in which Newton's theory holds, so according to the principle of Occam's Razor , Einstein's theory is preferred. On the other hand, Newtonian calculations are simpler, so Newton's theory is useful for almost any engineering project, including some space projects. But for GPS we need Einstein's theory. Scientists would not have arrived at either of these theories, or a compromise between both of them, without the use of testable, falsifiable experiments. 

Popper saw falsifiability as a black and white definition; that if a theory is falsifiable, it is scientific , and if not, then it is unscientific. Whilst some "pure" sciences do adhere to this strict criterion, many fall somewhere between the two extremes, with  pseudo-sciences  falling at the extreme end of being unfalsifiable. 

Pseudoscience

According to Popper, many branches of applied science, especially social science, are not truly scientific because they have no potential for falsification.

Anthropology and sociology, for example, often use case studies to observe people in their natural environment without actually testing any specific hypotheses or theories.

While such studies and ideas are not falsifiable, most would agree that they are scientific because they significantly advance human knowledge.

Popper had and still has his fair share of critics, and the question of how to demarcate legitimate scientific enquiry can get very convoluted. Some statements are logically falsifiable but not practically falsifiable – consider the famous example of “it will rain at this location in a million years' time.” You could absolutely conceive of a way to test this claim, but carrying it out is a different story.

Thus, falsifiability is not a simple black and white matter. The Raven Paradox shows the inherent danger of relying on falsifiability, because very few scientific experiments can measure all of the data, and necessarily rely upon generalization . Technologies change along with our aims and comprehension of the phenomena we study, and so the falsifiability criterion for good science is subject to shifting.

For many sciences, the idea of falsifiability is a useful tool for generating theories that are testable and realistic. Testability is a crucial starting point around which to design solid experiments that have a chance of telling us something useful about the phenomena in question. If a falsifiable theory is tested and the results are significant , then it can become accepted as a scientific truth.

The advantage of Popper's idea is that such truths can be falsified when more knowledge and resources are available. Even long accepted theories such as Gravity, Relativity and Evolution are increasingly challenged and adapted.

The major disadvantage of falsifiability is that it is very strict in its definitions and does not take into account the contributions of sciences that are observational and descriptive .

• Psychology 101
• Flags and Countries
• Capitals and Countries

Martyn Shuttleworth , Lyndsay T Wilson (Sep 21, 2008). Falsifiability. Retrieved Apr 11, 2024 from Explorable.com: https://explorable.com/falsifiability

You Are Allowed To Copy The Text

The text in this article is licensed under the Creative Commons-License Attribution 4.0 International (CC BY 4.0) .

This means you're free to copy, share and adapt any parts (or all) of the text in the article, as long as you give appropriate credit and provide a link/reference to this page.

That is it. You don't need our permission to copy the article; just include a link/reference back to this page. You can use it freely (with some kind of link), and we're also okay with people reprinting in publications like books, blogs, newsletters, course-material, papers, wikipedia and presentations (with clear attribution).

Want to stay up to date? Follow us!

Save this course for later.

Don't have time for it all now? No problem, save it as a course and come back to it later.

Footer bottom

• Privacy Policy

• Subscribe to our RSS Feed
• Like us on Facebook
• Follow us on Twitter

• Table of Contents
• Random Entry
• Chronological
• Editorial Information
• About the SEP
• Editorial Board
• How to Cite the SEP
• Special Characters
• Advanced Tools
• Support the SEP
• PDFs for SEP Friends
• Make a Donation
• SEPIA for Libraries
• Entry Contents

Bibliography

Academic tools.

• Friends PDF Preview
• Author and Citation Info
• Back to Top

Karl Popper

Karl Popper is generally regarded as one of the greatest philosophers of science of the twentieth century. He was also a social and political philosopher of considerable stature, a self-professed critical-rationalist, a dedicated opponent of all forms of scepticism and relativism in science and in human affairs generally and a committed advocate and staunch defender of the “Open Society”. One of the many remarkable features of Popper’s thought is the scope of his intellectual influence: he was lauded by Bertrand Russell, taught Imre Lakatos, Paul Feyerabend and philanthropist George Soros at the London School of Economics, numbered David Miller, Joseph Agassi, Alan Musgrave and Jeremy Shearmur amongst his research assistants, was counted by Thomas Szasz as “among my foremost teachers” and had close ties with the economist Friedrich Hayek and the art historian Ernst Gombrich. Additionally, Peter Medawar, John Eccles and Hermann Bondi are amongst the distinguished scientists who have acknowledged their intellectual indebtedness to his work, the latter declaring that “There is no more to science than its method, and there is no more to its method than Popper has said”.

2. Backdrop to Popper’s Thought

3. the problem of demarcation, 4. basic statements, falsifiability and convention, 5. the growth of human knowledge, 6. probability, knowledge and verisimilitude, 7. objective knowledge and the three worlds ontology, 8. social and political thought—the critique of historicism and holism, 9. scientific knowledge, history, and prediction, 10. immutable laws and contingent trends, 11. critical evaluation, primary literature: works by popper, secondary literature/other sources, other internet resources, related entries.

Karl Raimund Popper was born on 28 July 1902 in Vienna. His parents, who were of Jewish origin, brought him up in an atmosphere which he was later to describe as “decidedly bookish”. His father was a lawyer by profession, but he also took a keen interest in the classics and in philosophy, and communicated to his son an interest in social and political issues. His mother inculcated in him such a passion for music that for a time he contemplated taking it up as a career; he initially chose the history of music as a second subject for his Ph.D. examination. Subsequently, his love for music became one of the inspirational forces in the development of his thought, and manifested itself in his highly original interpretation of the relationship between dogmatic and critical thinking, in his account of the distinction between objectivity and subjectivity, and, most importantly, in the growth of his hostility towards all forms of historicism, including historicist ideas about the nature of the “progressive” in music. The young Karl attended the local Realgymnasium , where he was unhappy with the standards of the teaching, and, after an illness he left to attend the University of Vienna in 1918, matriculating four years later. In 1919 he became heavily involved in left-wing politics and became for a time a Marxist. However, he was quickly disillusioned with the doctrinaire character of the latter, and soon abandoned it entirely. He also discovered the psychoanalytic theories of Freud and Adler (he served briefly as a voluntary social worker with deprived children in one of the latter’s clinics in the 1920s), and heard Einstein lecture on relativity theory. The dominance of the critical spirit in Einstein, and its total absence in Marx, Freud and Adler, struck Popper as being of fundamental importance: the pioneers of psychoanalysis, he came to think, couched their theories in terms which made them amenable only to confirmation, while Einstein’s theory, crucially, had testable implications which, if false, would have falsified the theory itself.

Popper took some time to settle on a career; he trained as a cabinetmaker, obtained a primary school teaching diploma in 1925 and qualified to teach mathematics and physics in secondary school in 1929. He undertook a doctoral programme with the department of psychology at the University of Vienna, then under the supervision of Karl Bühler, one of the founder members of the Würzburg school of experimental psychology. Popper’s project was initially designed as a psychological investigation of human memory, on which he conducted initial research. However, the subject matter of a planned introductory chapter on methodology assumed a position of increasing pre-eminence and this resonated with Bühler, who, as a Kant scholar (a professor of philosophy and psychology), had famously addressed the issue of the contemporary “crisis in psychology”. This “crisis”, for Bühler, related to the question of the unity of psychology and had been generated by the proliferation of then competing paradigms within the discipline which had undermined the hitherto dominant associationist one and problematized the question of method. Accordingly, under Bühler’s direction, Popper switched his topic to the methodological problem of cognitive psychology and received his doctorate in 1928 for his dissertation “Zur Methodenfrage der Denkpsychologie”. In extending Bühler’s Kantian approach to the crisis in the dissertation, Popper critiqued Moritz Schlick’s neutral monist programme to make psychology scientific by transforming it into a science of brain processes. This latter ideal, Popper argued, was misconceived, but the issues raised by it ultimately had the effect of refocusing Popper’s attention away from Bühler’s question of the unity of psychology to that of its scientificity. This philosophical focus on questions of method, objectivity and claims to scientific status was to become a principal life-long concern, bringing the orientation of Popper’s thought into line with that of such contemporary “analytic” philosophers as Frege and Russell as well as that of many members of the Vienna Circle.

Popper married Josephine Anna Henninger (“Hennie”) in 1930, and she also served as his amanuensis until her death in 1985. At an early stage of their marriage they decided that they would never have children. In 1937 he took up a position teaching philosophy at the University of Canterbury in New Zealand, where he was to remain for the duration of the Second World War.

The annexation of Austria in 1938 became the catalyst which prompted Popper to refocus his writings on social and political philosophy. He published The Open Society and Its Enemies , his critique of totalitarianism, in 1945. In 1946 he moved to England to teach at the London School of Economics, and became professor of logic and scientific method at the University of London in 1949. From this point on his reputation and stature as a philosopher of science and social thinker grew, and he continued to write prolifically—a number of his works, particularly The Logic of Scientific Discovery (1959), are now widely seen as pioneering classics in the field. However, he combined a combative personality with a zeal for self-aggrandisement that did little to endear him to professional colleagues. He was ill-at-ease in the philosophical milieu of post-war Britain which was, as he saw it, fixated with trivial linguistic concerns dictated by Wittgenstein, whom he considered his nemesis. Popper’s commitment to the primacy of rational criticism was counterpointed by hostility towards anything that amounted to less than total acceptance of his own thought, and in Britain—as had been the case in Vienna—he increasingly became an isolated figure, though his ideas continued to inspire admiration.

In later years Popper came under philosophical criticism for his prescriptive approach to science and his emphasis on the logic of falsification. This was superseded in the eyes of many by the socio-historical approach taken by Thomas Kuhn in The Structure of Scientific Revolutions (1962). In that work, Kuhn, who argued for the incommensurability of rival scientific paradigms, denied that science grows linearly through the accumulation of truths.

Popper was knighted in 1965, and retired from the University of London in 1969, remaining active as a writer, broadcaster and lecturer until his death in 1994. (For more detail on Popper’s life, see his Unended Quest [1976]).

A number of biographical features may be identified as having a particular influence upon Popper’s thought. His teenage flirtation with Marxism left him thoroughly familiar with the Marxian dialectical view of economics, class-war, and history. But he was appalled by the failure of the democratic parties to stem the rising tide of fascism in Austria in the 1920s and 1930s, and the effective welcome extended to it by the Marxists, who regarded fascism as a necessary dialectical step towards the implosion of capitalism and the ultimate victory of communism. The Poverty of Historicism (1944; 1957) and The Open Society and Its Enemies (1945), Popper’s most impassioned and influential social works, are powerful defences of democratic liberalism, and strident critiques of philosophical presuppositions underpinning all forms of totalitarianism.

Popper was also profoundly impressed by the differences between the allegedly “scientific” theories of Freud and Adler and the revolution effected by Einstein’s theory of Relativity in physics in the first two decades of the twentieth century. The main difference between them, as Popper saw it, was that while Einstein’s theory was highly “risky”, in the sense that it was possible to deduce consequences from it which were, in the light of the then dominant Newtonian physics, highly improbable (e.g., that light is deflected towards solid bodies—confirmed by Eddington’s experiments in 1919), and which would, if they turned out to be false, falsify the whole theory, nothing could, even in principle , falsify psychoanalytic theories. They were, Popper argues, “simply non-testable, irrefutable. There was no conceivable human behaviour which could contradict them” (1963: 37). As such, they have more in common with myths than with genuine science; “They contain most interesting psychological suggestions, but not in a testable form” (1963: 38). What is apparently the chief source of strength of psychoanalysis, he concluded, viz. its capability to accommodate and explain every possible form of human behaviour, is in fact a critical weakness, for it entails that it is not, and could not be, genuinely predictive. To those who would respond that psychoanalytic theory is supported by clinical observations, Popper points out that

… real support can be obtained only from observations undertaken as tests (by ‘attempted refutations’); and for this purpose criteria of refutation have to be laid down beforehand: it must be agree which observable situations, if actually observed, mean that the theory is refuted. (1963: 38, footnote 3)

Popper also considers that contemporary Marxism also lacks scientific status. Unlike psychoanalysis, he argues, Marxism had been initially scientific, in that it was genuinely predictive. However, when these predictions were not in fact borne out, the theory was saved from falsification by the addition of ad hoc hypotheses which made it compatible with the facts. By this means, Popper asserts, a theory which was initially genuinely scientific degenerated into pseudo-scientific dogma. As he sees it, the Hegelian dialectic was adopted by Marxists not to oppose dogmatism but to accommodate it to their cause by eliminating the possibility of contradictory evidence. It has thus become what Popper terms “reinforced dogmatism” (1963: 334).

These factors combined to make Popper take falsifiability as his criterion for demarcating science from non-science: if a theory is incompatible with possible empirical observations it is scientific; conversely, a theory which is compatible with all such observations, either because, as in the case of Marxism, it has been modified solely to accommodate such observations, or because, as in the case of psychoanalytic theories, it is consistent with all possible observations, is unscientific. However, Popper is not a positivist and acknowledges that unscientific theories may be enlightening and that even purely mythogenic explanations have performed a valuable function in the past in expediting our understanding of the nature of reality.

For Popper the central problem in the philosophy of science is that of demarcation, i.e., of distinguishing between science and what he terms “non-science” (e.g., logic, metaphysics, psychoanalysis, and Adler’s individual psychology). Popper is unusual amongst contemporary philosophers in that he accepts the validity of the Humean critique of induction, and indeed, goes beyond it in arguing that induction is never actually used in science. However, he does not concede that this entails scepticism and argues that the Baconian/Newtonian insistence on the primacy of “pure” observation, as the initial step in the formation of theories, is completely misguided: all observation is selective and theory-laden and there are no pure or theory-free observations. In this way he destabilises the traditional view that science can be distinguished from non-science on the basis of its inductive methodology. In contradistinction to this, Popper holds that there is no unique methodology specific to science; rather, science, like virtually every other organic activity, consists largely of problem-solving.

Popper accordingly rejects the view that induction is the characteristic method of scientific investigation and inference, substituting falsifiability in its place. It is easy, he argues, to obtain evidence in favour of virtually any theory, and he consequently holds that such “corroboration”, as he terms it, should count scientifically only if it is the positive result of a genuinely “risky” prediction, which might conceivably have been false. In a critical sense, Popper’s theory of demarcation is based upon his perception of the asymmetry which, at the level of logic, holds between verification and falsification: it is logically impossible to verify a universal proposition by reference to experience (as Hume saw clearly), but a single genuine counter-instance falsifies the corresponding universal law. In a word, an exception, far from “proving” a rule, conclusively refutes it.

Every genuine scientific theory then, in Popper’s view, is prohibitive , because the theories of natural science take the form of universal statements. “All A s are X ” is equivalent to “No A is not- X ” which is falsified if “Some A is not- X ” turns out to be true. For example, the law of the conservation of energy can be expressed as “There is no perpetual motion machine”.

However, the universality of such laws, he argues, does rule out the possibility of their verification. Thus, a theory that has withstood rigorous testing should be deemed to have received a high measure of corroboration. and may be retained provisionally as the best available theory until it is finally falsified and/or is superseded by a better theory.

Popper stresses in particular that there is no unique way, no single method such as induction, which functions as the route to scientific theory, and approvingly cites Einstein on that point:

There is no logical path leading to [the highly universal laws of science]. They can only be reached by intuition, based upon something like an intellectual love of the objects of experience. (2002: 8–9)

Science, in Popper’s view, starts with problems rather than with observations—it is, indeed, precisely in the context of grappling with a problem that the scientist makes observations in the first instance: his observations are selectively designed to test the extent to which a given theory functions as a satisfactory solution to a given problem.

On this criterion of demarcation physics, chemistry, and (non-introspective) psychology, amongst others, are classified as sciences, psychoanalysis is a pre-science and astrology and phrenology are pseudo-sciences.

Popper draws a clear distinction between the logic of falsifiability and its applied methodology . The logic of his theory is utterly simple: a universal statement is falsified by a single genuine counter-instance. Methodologically, however, the situation is complex: decisions about whether to accept an apparently falsifying observation as an actual falsification can be problematic, as observational bias and measurement error, for example, can yield results which are only apparently incompatible with the theory under scrutiny.

Thus, while advocating falsifiability as the criterion of demarcation for science, Popper explicitly allows for the fact that in practice a single conflicting or counter-instance is never sufficient methodologically for falsification, and that scientific theories are often retained even though much of the available evidence conflicts with them, or is anomalous with respect to them.

In this connection, in the Logic of Scientific Discovery Popper introduces the technical concept of a “basic statement” or “basic proposition”, which he defines as a statement which can serve as a premise in an empirical falsification and which takes the singular existential form “There is an X at Y ”. Basic statements are important because they can formally contradict universal statements, and accordingly play the role of potential falsifiers. To take an example, the (putative) basic statement “In space-time region k there is an apparatus which is a perpetual motion machine” contradicts the law of the conservation of energy, and would, if true, falsify it (2002: 48). Accordingly, Popper holds that basic statements are objective and are governed by two requirements: (a) the formal, that they must be both singular and existential and (b) the material, that they must be intersubjectively testable.

In essence, basic statements are for Popper logical constructs which embrace and include ‘observation statements’, but for methodological reasons he seeks to avoid that terminology, as it suggests that they are derived directly from, and known by, experience (2002: 12, footnote 2), which would conflate them with the “protocol” statements of logical positivism and reintroduce the empiricist idea that certain kinds of experiential reports are incorrigible. The “objectivity” requirement in Popper’s account of basic statements, by contrast, amounts to a rejection of the view that the truth of scientific statements can ever be reduced to individual or collective human experience. (2002: 25).

Popper therefore argues that there are no statements in science which cannot be interrogated: basic statements, which are used to test the universal theories of science, must themselves be inter-subjectively testable and are therefore open to the possibility of refutation. He acknowledges that this seems to present a practical difficulty, in that it appears to suggest that testability must occur ad infinitum , which he acknowledges is an operational absurdity: sooner or later all testing must come to an end. Where testing ends, he argues, is in a convention-based decision to accept a basic statement or statements; it is at that point that convention and intersubjective human agreement play an indispensable role in science:

Every test of a theory, whether resulting in its corroboration or falsification, must stop at some basic statement or other which we decide to accept . If we do not come to any decision, and do not accept some basic statement or other, then the test will have led nowhere. (2002: 86)

However, Popper contends that while such a decision is usually causally related to perceptual experience, it is not and cannot be justified by such experience; basic statements are experientially underdetermined.

Experiences can motivate a decision, and hence an acceptance or a rejection of a statement, but a basic statement cannot be justified by them—no more than by thumping the table. (2002: 87–88)

Statements can be justified only by other statements, and therefore testing comes to an end, not in the establishment of a correlation between propositional content and observable reality, as empiricism would hold, but by means of the conventional, inter-subjective acceptance of the truth of certain basic statements by the research community.

The acceptance of basic statements is compared by Popper to trial by jury: the verdict of the jury will be an agreement in accordance with the prevailing legal code and on the basis of the evidence presented, and is analogous to the acceptance of a basic statement by the research community:

By its decision, the jury accepts, by agreement, a statement about a factual occurrence—a basic statement, as it were. (2002: 92)

The jury’s verdict is conventional in arising out of a procedure governed by clear rules, and is an application of the legal system as a whole as it applies to the case in question. The verdict is accordingly represented as a true statement of fact, but, as miscarriages of justice demonstrate all too clearly,

the statement need not be true merely because the jury has accepted it. This … is acknowledged in the rule allowing a verdict to be quashed or revised. (2002: 92)

This is comparable, he argues, to the case of basic statements: their acceptance-as-true is also by agreement and, as such, it also constitutes an application of a theoretical system, and

it is only this application which makes any further applications of the theoretical system possible. (2002: 93)

However, the agreed acceptance of basic statements, like that of judicial verdicts, remain perennially susceptible to the requirement for further interrogation. Popper terms this “the relativity of basic statements” (2002: 86), which is reflective of the provisional nature of the entire corpus of scientific knowledge itself. Science does not, he maintains, rest upon any foundational bedrock. Rather, the theoretical systems of science are akin to buildings in swampy ground constructed with the support of piles:

The piles are driven down from above into the swamp, but not down to any natural or “given” base; and if we stop driving the piles deeper, it is not because we have reached firm ground. We simply stop when we are satisfied that the piles are firm enough to carry the structure, at least for the time being. (2002: 94)

For Popper, the growth of human knowledge proceeds from our problems and from our attempts to solve them. These attempts involve the formulation of theories which must go beyond existing knowledge and therefore require a leap of the imagination. For this reason, he places special emphasis on the role played by the creative imagination in theory formulation. The priority of problems in Popper’s account of science is paramount, and it is this which leads him to characterise scientists as “problem-solvers”. Further, since the scientist begins with problems rather than with observations or “bare facts”, he argues that the only logical technique which is an integral part of scientific method is that of the deductive testing of theories which are not themselves the product of any logical operation. In this deductive procedure conclusions are inferred from a tentative hypothesis and are then compared with one another and with other relevant statements to determine whether they falsify or corroborate the hypothesis. Such conclusions are not directly compared with the facts, Popper stresses, simply because there are no “pure” facts available; all observation-statements are theory-laden, and are as much a function of purely subjective factors (interests, expectations, wishes, etc.) as they are a function of what is objectively real.

How then does the deductive procedure work? Popper specifies four steps (2002: 9):

• The first is formal , a testing of the internal consistency of the theoretical system to see if it involves any contradictions.
• The second step is semi-formal , “the investigation of the logical form of the theory, with the object of determining whether it has the character of an empirical or scientific theory, or whether it is, for example, tautological” (2002: 9).
• The third step is the comparing of the new theory with existing ones to determine whether it constitutes an advance upon them. If its explanatory success matches that of the existing theories, and it additionally explains some hitherto anomalous phenomenon or solves some hitherto unsolvable problems, it will be adopted as constituting an advance upon the existing theories. In that sense, science involves theoretical progress: on this account, a theory X is better than a “rival” theory Y if X has greater empirical content , and hence greater predictive power , than Y .
• The fourth and final step is the testing of a theory by the empirical application of the conclusions derived from it. If such conclusions are shown to be true, the theory is corroborated (but never verified). If the conclusion is shown to be false, then this is taken as a signal that the theory cannot be completely correct (logically the theory is falsified), and the scientist begins his quest for a better theory. He does not, however, abandon the present theory until such time as he has a better one to substitute for it.

The general picture of Popper’s philosophy of science, then is this: Hume’s philosophy demonstrates that there is a contradiction implicit in traditional empiricism, which holds that universal scientific laws are in some way finally confirmable by experience, despite the open-ended nature of the latter being acknowledged. Popper eliminates the contradiction by removing the demand for empirical verification in favour of empirical falsification or corroboration. Scientific theories, for him, are not inductively inferred from experience, nor is scientific experimentation carried out with a view to verifying or finally establishing the truth of theories; rather, all knowledge is provisional, conjectural, hypothetical —the universal theories of science can never be conclusively established. Hence Popper’s emphasis on the importance of the critical spirit to science—for him critical thinking is the very essence of rationality. For it is only by critical thought that we can eliminate false theories and determine which of the remaining theories is the best available one, in the sense of possessing the highest level of explanatory force and predictive power.

In the view of many social scientists, the more probable a theory is, the better it is, and if we have to choose between two theories which differ only in that one is probable and the other is improbable, then we should choose the former. Popper rejects this. Science values theories with a high informative content, because they possess a high predictive power and are consequently highly testable. For that reason, the more improbable a theory is the better it is scientifically, because the probability and informative content of a theory vary inversely—the higher the informative content of a theory the lower will be its probability. Thus, the statements which are of special interest to science are those with a high informative content and (consequentially) a low probability, which nevertheless come close to the truth . Informative content, which is in inverse proportion to probability, is in direct proportion to testability. As a result, the severity of the test to which a theory can be subjected, and by means of which it is falsified or corroborated, is of fundamental importance.

Popper also argues that all scientific criticism must be piecemeal, i.e., he holds that it is not possible to question every aspect of a theory at once, as certain items of what he terms “background knowledge” must be taken for granted. But that is not knowledge in the sense of being conclusively established; it may be challenged at any time, especially if it is suspected that its uncritical acceptance may be responsible for difficulties which are subsequently encountered.

How then can one be certain that one is questioning the right thing? The Popperian answer is that we cannot have absolute certainty here, but repeated tests usually show where the trouble lies. As we saw, for Popper even observation statements are corrigible and open to review, and science in his view is not a quest for certain knowledge, but an evolutionary process in which hypotheses or conjectures are imaginatively proposed and tested in order to explain facts or to solve problems. For that reason, he emphasises both the importance of questioning the background knowledge when the need arises, and the significance of the fact that observation-statements are theory-laden and corrigible.

Popper was initially uneasy with the concept of truth, and in his earliest writings he avoided asserting that a theory which is corroborated is true—for clearly if every theory is an open-ended hypothesis, then ipso facto it has to be at least potentially false. However, he came to accept Tarski’s reformulation of the correspondence theory of truth, and in Conjectures and Refutations (1963) he integrates the concepts of truth and content to frame the metalogical concept of “truthlikeness” or “ verisimilitude ”. A “good” scientific theory, Popper argues in that work, has a higher level of verisimilitude than its rivals, and he explicates this concept by reference to the logical consequences of theories. A theory’s content is the totality of its logical consequences, which can be divided into two classes:

• the “ truth-content ”, which is the class of true propositions which may be derived from it, and
• the “ falsity-content ” of a theory, which is the class of the theory’s false consequences (which may be empty, and in the case of a theory which is true is necessarily empty).

Popper offers two accounts of how rival theories can be compared in terms of their levels of verisimilitude; these are the qualitative and quantitative definitions. On the qualitative account, verisimilitude is defined in terms of subclass relationships: a theory \(t_2\) has a higher level of verisimilitude than \(t_1\) if and only if their truth- and falsity-contents are comparable through subclass relationships, and either

• \(t_2\)’s truth-content includes \(t_1\)’s and \(t_2\)’s falsity-content, if it exists, is included in, or is the same as, \(t_1\)’s, or
• \(t_2\)’s truth-content includes or is the same as \(t_1\)’s and \(t_2\)’s falsity-content, if it exists, is included in \(t_1\)’s.

On the quantitative account, verisimilitude is defined by assigning quantities to contents, where the index of the content of a given theory is its logical improbability, given again that content and probability vary inversely (1963: 233–4). The utilisation of either method of computing verisimilitude shows, Popper argues, that even if a theory \(t_2\) with a higher content than a rival theory \(t_1\) is subsequently falsified, it can still legitimately be regarded as a better theory than \(t_1\), and “better” is here now understood to mean \(t_2\) is closer to the truth than \(t_1\) (1963: 235).

Thus, scientific progress involves, on this view, the abandonment of partially true, but falsified, theories, for theories with a higher level of verisimilitude, i.e., which approach more closely to the truth. In this way, verisimilitude allowed Popper to mitigate what some saw as the pessimism of an anti-inductivist philosophy of science. With the introduction of the concept, Popper was able to represent his account as an essentially realistic position in terms of which scientific progress can be seen as progress towards the truth, and experimental corroboration can be viewed as an indicator of verisimilitude.

However, subsequent research revealed defects in Popper’s formal definitions of verisimilitude. The concept is most important in his system because of its application to theories which are approximations (which are common in the social sciences) and thus known to be false . In this connection, Popper had written:

Ultimately, the idea of verisimilitude is most important in cases where we know that we have to work with theories which are at best approximations—that is to say, theories of which we know that they cannot be true… In these cases we can still speak of better or worse approximations to the truth (and we therefore do not need to interpret these cases in an instrumentalist sense). (1963: 235)

In 1974, David Miller and Pavel Tichý, working independently, demonstrated that the conditions specified by Popper in his accounts of both qualitative and quantitative verisimilitude for comparing the truth- and falsity-contents of theories can be satisfied only when the theories are true . In the crucially important case of false theories, Popper’s definitions are formally defective: for with regard to a false theory \(t_2\) which has excess content over a rival theory \(t_1\), both the truth-content and the falsity-content of \(t_2\) will exceed that of \(t_1\). With respect to theories which are false then, Popper’s conditions for comparing levels of verisimilitude, whether in quantitative and qualitative terms, can never be met.

Popper’s response was two-fold. In the first place, he acknowledges the deficiencies in his own formal account:

“…my main mistake was the failure to see at once that my ’A Theorem on Truth-Content’ [1966], could be extended to falsity content: if the content of a false statement a exceeds that of a statement b , then the truth content of a exceeds the truth content of b , and the same holds of their falsity contents.” (1979, 371)

But he argues that

“I do think that we should not conclude from the failure of my attempts to solve the problem that the problem cannot be solved. Perhaps it cannot be solved by purely logical means but only by a relativization to relevant problems or even by bringing in the historical problem situation” (1979, 372).

This suggestion was to precipitate a great deal of important technical research in the field. He additionally moves the task of formally defining the concept from centre-stage in his philosophy of science by protesting that he had never intended to imply

that degrees of verisimilitude … can ever be numerically determined, except in certain limiting cases. (1979: 59)

Instead, he argues, the chief value of the concept is heuristic, in which the absence of an adequate formal definition is not an insuperable impediment to its utilisation in the actual appraisal of theories relativised to problems in which we have an interest.

Many see the thrust of this latter strategy as genuinely reflecting the significance of the concept of verisimilitude in Popper’s system, but it has not satisfied all of his critics.

Popper’s hostility to psychologistic approaches to epistemology is evident from his earliest works. Questions relating to the origins of convictions, feelings of certainty and the like, he argues, are properly considered the province of psychology; their attempted use in epistemology, which has been characteristic in particular of some schools of empiricism, can lead only to confusion and ultimately to scepticism. Against it, he repeatedly insists on the objectivity of scientific knowledge and sees it as the principal task of epistemology to engage with the questions of justification and validity in that connection (2002: 7).

In some of his later works in the field, particularly his “Epistemology Without a Knowing Subject” (1967, republished in Objective Knowledge [1972]) and in his lecture “Three Worlds” in the Tanner Lectures on Human Values delivered in 1978 (published 1980), Popper develops the notion of objectivity further in a novel but controversial way by seeking to free it completely from all psychological constraints. What is central to epistemology, he reaffirms, is the concept of objectivity, which he seeks to show requires neither the notions of subjective mental states nor even that of a subject “possessing” knowledge: knowledge in its full objective sense, he argues, is knowledge “without a knowing subject”.

Popper’s angle of approach here is to situate the development of knowledge in the context of animal and human evolution. It is characteristic of evolutionary processes, he points out, that they come to take place in an environment which is itself in part fashioned by the species in question. Examples of this abound, such as the reefs built by corals, the hives built by bees, the dams built by beavers and the atmospheric effects yielded by plants. This, Popper contends, is also true of human beings: we too have created new kinds of products, “intellectual products”, which shape our environment. These are our myths, our ideas, our art works and our scientific theories about the world in which we live. When placed in an evolutionary context, he suggests, such products must be viewed instrumentally, as exosomatic artefacts. Chief amongst them is knowledge

in the objective or impersonal sense, in which it may be said to be contained in a book; or stored in a library; or taught in a university. (1979: 286)

On this view, termed “evolutionary epistemology”, the growth of human knowledge is an objective evolutionary process which involves the creation and promulgation of new problem-solving theories, which are then subjected to the challenge of criticism, modification, elimination and replacement. These latter activities are seen by Popper as growth-promoting in the evolution of knowledge, which he represents by means of a tetradic schema (1979: 287):

Here “\(P_1\)” stands for the “initial problem”; “ TT ” stands for “tentative theory” designed to solve it, “ EE ” stands for “attempts at error-elimination”, and “\(P_2\)” represents further problems that arise out of the critical process.

This kind of knowledge development, Popper argues, cannot be explained either by physicalism, which seeks to reduce all mental processes and states to material ones, or by dualism, which usually seeks to explicate knowledge by means of psychological categories such as thought, perception and belief. Consequently, he proposes a novel form of pluralistic realism, a “Three Worlds” ontology, which, while accommodating both the world of physical states and processes (world 1) and the mental world of psychological processes (world 2), represents knowledge in its objective sense as belonging to world 3, a third, objectively real ontological category. That world is the world

of the products of the human mind, such as languages; tales and stories and religious myths; scientific conjectures or theories, and mathematical constructions; songs and symphonies; paintings and sculptures. (1980: 144)

In short, world 3 is the world of human cultural artifacts, which are products of world 2 mental processes, usually instantiated in the physical world 1 environment.

Popper proceeds to explicate his distinction between the subjective and objective senses of knowledge by reference to this ontology. The subjective sense of knowledge relates to psychological processes and states, mental dispositions, beliefs and expectations, which may generically be termed “thought processes” and which belong to world 2. Knowledge in the objective sense, by contrast, consists not of thought processes but of thought contents , that is to say, the content of propositionalised theories: it is

the content which can be, at least approximately, translated from one language into another. The objective thought content is that which remains invariant in a reasonably good translation. (1980: 156)

And it is that thought content, when linguistically codified in texts, works of art, log tables, mathematical formulae, which constitutes world 3, to which objective knowledge relates.

For those who would suggest that such objects are mere abstractions from world 2 thought processes, Popper counters that world 3 objects are necessarily more than the thought processes which have led to their creation. Theories, for example, usually have logical implications beyond anything considered by their original author, as instanced in the case of Einstein’s Special Theory of Relativity. Moreover, what is most characteristic about such objects is that, unlike world 2 mental processes, they can stand in logical relationships to each other, such as equivalence, deducibility and compatibility, which makes them amenable to the kind of critical rational analysis and development that is one of the hallmarks of science. As he puts it,

Criticism of world 3 objects is of the greatest importance, both in art and especially in science. Science can be said to be largely the result of the critical examination and selection of conjectures, of thought contents. (1980: 160)

Popper takes Michelangelo’s sculpture The Dying Slave as an illustrative example of a world 3 object, embodied in a world 1 block of marble. Other examples given include memory engrams in the brain, the American Constitution, Shakespeare’s tragedies, Beethoven’s Fifth Symphony and Newton’s theory of gravitation. Each one of these, he contends, is a world 3 object that transcends both its physical, world 1 embodiments and its world 2 cognitive origins (1980: 145).

Popper was aware that he would be accused of hypostatising abstractions in asserting the reality and objectivity of world 3 objects. In response, he indicates strongly that he has no interest in what he regards as pointless terminological disputes over the meaning of the term “world” or “real”. He is therefore content, if required, to express his account of objective knowledge in more familiar and perhaps more mundane terms: world 3 objects are abstract objects while their physical embodiments are concrete objects. But that should not be allowed to disguise the fact that he saw the relationships between the three categories of his ontology as of enormous importance in understanding the role of science as an element of culture:

my thesis is that our world 3 theories and our world 3 plans causally influence the physical objects of world 1; that they have a causal action upon world 1. (1980: 164)

In the final analysis it is the causal interaction between the worlds that ultimately matters in Popper’s objectivist epistemology: it allows him to represent the growth of human knowledge as an evolutionary process of exosomatic adaptations, which is ultimately a function of the interplay of relations between the physical and mental worlds and the world of objective knowledge or thought content.

Given Popper’s personal history and background, it is hardly surprising that he developed a deep and abiding interest in social and political philosophy. He understood holism as the view that human social groupings are greater than the sum of their members, that they act on their human members and shape their destinies and that they are subject to their own independent laws of development. Historicism he identified as the belief that history develops inexorably and necessarily according to certain principles or rules towards a determinate end (as for example in the Marx’s dialectical materialism). The link between them is that holism holds that individuals are essentially formed by the social groupings to which they belong, while historicism suggests that we can understand such a social grouping only in terms of the internal principles which determine its development.

These lead to what Popper calls “The Historicist Doctrine of the Social Sciences”, the views (a) that the principal task of the social sciences is to make predictions about the social and political development of man, and (b) that the task of politics, once the key predictions have been made, is, in Marx’s words, to lessen the “birth pangs” of future social and political developments. Popper thinks that this view of the social sciences is both theoretically misconceived and socially dangerous, as it can give rise to totalitarianism and authoritarianism—to centralised governmental control of the individual and the attempted imposition of large-scale social planning. Against this, he advances the view that any human social grouping is no more (or less) than the sum of its individual members, that what happens in history is the (largely unforeseeable) result of the actions of such individuals, and that large scale social planning to an antecedently conceived blueprint is inherently misconceived—and inevitably disastrous—precisely because human actions have consequences which cannot be foreseen. Popper, then, is an historical indeterminist , insofar as he holds that history does not evolve in accordance with intrinsic laws or principles, that in the absence of such laws and principles unconditional prediction in the social sciences is an impossibility, and that there is no such thing as historical necessity.

The link between Popper’s theory of knowledge and his social philosophy is his fallibilism. We make theoretical progress in science by subjecting our theories to critical scrutiny, and abandoning those which have been falsified. So too in an open society the rights of the individual to criticise administrative policies will be safeguarded and upheld, undesirable policies will be eliminated in a manner analogous to the elimination of falsified scientific theories, and political differences will be resolved by critical discussion and argument rather than by coercion. The open society as thus conceived of by Popper may be defined as

an association of free individuals respecting each other’s rights within the framework of mutual protection supplied by the state, and achieving, through the making of responsible, rational decisions, a growing measure of humane and enlightened life. (R. B. Levinson 1953: 17)

Such as society is not a utopian ideal, Popper argues, but an empirically realised form of social organisation which is in every respect superior to its (real or potential) totalitarian rivals. His strategy, however, is not merely to engage in a moral defence of the ideology of liberalism, but rather to show that totalitarianism is typically based upon historicist and holist presuppositions, and of demonstrating that these presuppositions are fundamentally incoherent.

Historicism and holism, Popper argues, have their origins in what he terms

one of the oldest dreams of mankind—the dream of prophecy, the idea that we can know what the future has in store for us, and that we can profit from such knowledge by adjusting our policy to it. (1963: 338)

This dream was given impetus, he suggests, by the emergence of a genuine predictive capability regarding solar and lunar eclipses at an early stage in human civilisation, which became refined with the development of the natural sciences. Historicism derives a superficial plausibility from the suggestion that, just as the application of the laws of the natural sciences can lead to the prediction of such events as eclipses, knowledge of “the laws of history” as yielded by a social science or sciences can and should lead to the prediction of future social phenomena. Why should we not conceive of a social science which would function as the theoretical natural sciences function and yield precise unconditional predictions in the appropriate sphere of application? Popper seeks to show that this idea is based upon a series of misconceptions about the nature of science, and about the relationship between scientific laws and scientific prediction.

In relation to the critically important concept of prediction, Popper makes a distinction between what he terms “conditional scientific predictions”, which have the form “If X takes place, then Y will take place”, and “unconditional scientific prophecies”, which have the form “ Y will take place”. Contrary to popular belief, it is the former rather than the latter which are typical of the natural sciences, which means that typically prediction in natural science is conditional and limited in scope—it takes the form of hypothetical assertions stating that certain specified changes will come about if and only if particular specified events antecedently take place. This is not to deny that “unconditional scientific prophecies”, such as the prediction of eclipses, for example, do take place in science, and that the theoretical natural sciences make them possible. However, Popper argues that (a) these unconditional prophecies are not characteristic of the natural sciences, and (b) that the mechanism whereby they occur, in the very limited way in which they do, is not understood by the historicist.

What is the mechanism which makes “unconditional scientific prophecies” possible? Popper’s answer is that they are possible because they are derived from a combination of conditional predictions (themselves derived from scientific laws) and existential statements specifying that the conditions in relation to the system being investigated are fulfilled.

Given that this is the mechanism which generates unconditional scientific prophecies, Popper makes two related claims about historicism:

The first is that the historicist does not, as a matter of fact, derive his historical prophecies from conditional scientific predictions. The second … is that he cannot possibly do so because long term prophecies can be derived from scientific conditional predictions only if they apply to systems which can be described as well isolated, stationary, and recurrent. These systems are very rare in nature; and modern society is surely not one of them. (1963: 339)

Popper accordingly argues that it is a fundamental mistake for the historicist to take the unconditional scientific prophecies of eclipses as being typical and characteristic of the predictions of natural science; they are possible only because our solar system is a stationary and repetitive system which is isolated from other such systems by immense expanses of empty space. Human society and human history are not isolated systems and are continually undergoing rapid, non-repetitive development. In the most fundamental sense possible, every event in human history is discrete, novel, quite unique, and ontologically distinct from every other historical event. For this reason, it is impossible in principle that unconditional scientific prophecies could be made in relation to human history—the idea that the successful unconditional prediction of eclipses provides us with reasonable grounds for the hope of successful unconditional prediction regarding the evolution of human history turns out to be based upon a gross misconception. As Popper succinctly concludes, “The fact that we predict eclipses does not, therefore, provide a valid reason for expecting that we can predict revolutions” (1963: 340).

An additional mistake which Popper discerns in historicism is the failure of the historicist to distinguish between scientific laws and trends . This makes him think it possible to explain change by discovering trends running through past history, and to anticipate and predict future occurrences on the basis of such observations. Here Popper points out that there is a critical difference between a trend and a scientific law: the latter is universal in form, while a trend can be expressed only as a singular existential statement. This logical difference is crucial: neither conditional nor unconditional predictions can be based upon trends, because trends may change or be reversed with a change in the conditions which gave rise to them in the first instance. As Popper puts it, there can be no doubt that

the habit of confusing trends with laws, together with the intuitive observation of trends such as technical progress, inspired the central doctrines of … historicism. (1957: 106)

He does not, of course, dispute the existence of trends or deny that observing them can be of practical utility value. But the essential point is that a trend is something which itself ultimately stands in need of scientific explanation, and it cannot therefore function as the frame of reference in terms of which an unconditional prediction can be based.

A point which connects with this has to do with the role which the evolution of human knowledge has played in the historical development of human society. Human history has, Popper points out, been strongly influenced by the growth of human knowledge , and it is extremely likely that this will continue to be the case—all the empirical evidence suggests that the link between the two is progressively consolidating. However, this gives rise to a further problem for the historicist: no scientific predictor, human or otherwise, can possibly predict its own future results. From this it follows, he holds, that no society can predict, scientifically, its own future states of knowledge. Thus, while the future evolution of human history is extremely likely to be influenced by new developments in human knowledge, we cannot now scientifically determine what such knowledge will be.

From this it follows that if the future holds any new discoveries or any new developments in the growth of our knowledge, then it is impossible for us to predict them now, and it is therefore impossible for us to predict the future development of human history now, given that the latter will, at least in part, be determined by the future growth of our knowledge. Thus, once again historicism collapses—the dream of a theoretical, predictive science of history is unrealisable, because it is an impossible dream.

Popper’s argues against the propriety of large-scale planning of social structures on the basis of this demonstration of the logical shortcomings of historicism. It is, he argues, theoretically as well as practically misguided, because, again, part of what we are planning for is our future knowledge, and our future knowledge is not something which we can in principle now possess—we cannot adequately plan for unexpected advances in our future knowledge, or for the effects which such advances will have upon society as a whole. For him, this necessitates the acceptance of historical indeterminism as the only philosophy of history which is commensurate with a proper understanding of the provisional and incomplete nature of human knowledge.

This critique of historicism and holism is balanced, on the positive side, by Popper’s affirmation of the ideals of individualism and market economics and his strong defence of the open society—the view that a society is equivalent to the sum of its members, that the actions of the members of society serve to fashion and to shape it, and that the social consequences of intentional actions are very often, and very largely, unintentional. This part of his social philosophy was influenced by the economist Friedrich Hayek, who worked with him at the London School of Economics and who was a life-long friend. Popper advocates what he (rather unfortunately) terms “piecemeal social engineering” as the central mechanism for social planning: in utilising this mechanism, intentional actions are directed to the achievement of one specific goal at a time, which makes it possible to determine whether adverse unintended effects of intentional actions occur, in order to correct and readjust when this proves necessary. This, of course, parallels precisely the critical testing of theories in scientific investigation. This approach to social planning (which is explicitly based upon the premise that we do not, because we cannot, know what the future will be like) encourages attempts to put right what is problematic in society—generally-acknowledged social ills—rather than attempts to impose some preconceived idea of the “good” upon society as a whole. For Popper, in a genuinely open society piecemeal social engineering goes hand-in-hand with negative utilitarianism, the attempt to minimise the amount of suffering and misery, rather than, as with positive utilitarianism, the attempt to maximise the amount of happiness. The state, he holds, should concern itself with the task of progressively formulating and implementing policies designed to deal with the social problems which actually confront it, with the goal of mitigating human misery and suffering to the greatest possible degree. The positive task of increasing social and personal happiness, by contrast, can and should be left to individual citizens, who may, of course, act collectively to that end. “My thesis”, Popper states, is that

human misery is the most urgent problem of a rational public policy and that happiness is not such a problem. The attainment of happiness should be left to our private endeavours. (1963: 361)

Thus, for Popper, in the final analysis the activity of problem-solving is as definitive of our humanity at the level of social and political organisation as it is at the level of science, and it is this key insight which unifies and integrates the broad spectrum of his thought.

While it cannot be said that Popper was modest, he took criticism of his theories very seriously, and spent much of his time in his later years in addressing them. The following is a summary of some of the main ones which he had to address. (For Popper’s responses to critical commentary, see his “Replies to My Critics” (1974) and his Realism and the Aim of Science (1983).

First, Popper claims to be a realist and rejects conventionalist and instrumentalist accounts of science. But his account in the Logic of Scientific Discovery of the role played by basic statements in the methodology of falsification seems to sit uneasily with that. As we have seen, he follows Kant in rejecting the positivist/empiricist view that observation statements are incorrigible, and argues that they are descriptions of what is observed as interpreted by the observer with reference to a determinate conceptual and theoretical framework. He accordingly asserts that while basic statements may have a causal relationship to experience, they are neither determined nor justified by it.

However, this would seem to pose a difficulty: if a theory is to be genuinely testable, it must be possible to determine, at least in principle, whether the basic statements which are its potential falsifiers are actually true or false. But how can this be known, if basic statements cannot be justified by experience? As we have seen, Popper’s answer is that the acceptance or rejection of basic statements depends upon a convention-based decision on the part of the scientific community.

From a logical point of view, the testing of a theory depends upon basic statements whose acceptance or rejection, in its turn, depends upon our decisions. Thus it is decisions which settle the fate of theories. (2002: 91)

Traditional conventionalism, as exemplified in the work of Mach, Poincaré and Milhaud amongst others, holds that a “successful” science is one in which universal theories have assumed such explanatory critical mass that a convention emerges to pre-empt the possibility of their empirical refutation. This is strongly rejected by Popper, who differentiates his position from it by arguing that it is the acceptance of basic statements, rather than that universal theory, which is determined by convention and intersubjective agreement. For him, the acceptance or rejection of theory occurs only indirectly and at a higher investigative level, through critical tests made possible by the conventional acceptance of basic statements. As he puts it, “I differ from the conventionalist in holding that the statements decided by agreement are not universal but singular” (2002: 92). Simultaneously, however, he rejects any suggestion that basic statements are justifiable by direct experience:

I differ from the positivist in holding that basic statements are not justifiable by our immediate experiences, but are, from the logical point of view, accepted by an act, by a free decision. (2002: 92)

It is thus evident that Popper saw his account of basic statements as steering a course between the Scylla of orthodox conventionalism and the Charybdis of positivism/empiricism. However, while it is both coherent and consistent in that regard, there can be little doubt but that it constitutes a form of conventionalism in its own right. And it is not clear that it is compatible with scientific realism, understood as the view that scientific theories give true or approximately true descriptions of elements of a mind-independent world. As Lakatos puts it,

If a theory is falsified, it is proven false; if it is “falsified” [in Popper’s conventionalist sense], it may still be true. If we follow up this sort of “falsification” by the actual “elimination” of a theory, we may well end up by eliminating a true, and accepting a false, theory. (Lakatos 1978: 24)

Second, Popper’s theory of demarcation hinges quite fundamentally on the assumption that there are such things as critical tests, which either falsify a theory, or give it a strong measure of corroboration. Popper himself is fond of citing, as an example of such a critical test, the resolution, by Adams and Leverrier, of the problem which the anomalous orbit of Uranus posed for nineteenth century astronomers. They independently came to the conclusion that, assuming Newtonian mechanics to be precisely correct, the observed divergence in the elliptical orbit of Uranus could be explained if the existence of a seventh, as yet unobserved outer planet was posited. Further, they were able, again within the framework of Newtonian mechanics, to calculate the precise position of the “new” planet. Thus when subsequent research by Galle at the Berlin observatory revealed that such a planet (Neptune) did in fact exist, and was situated precisely where Adams and Leverrier had calculated, this was hailed as by all and sundry as a magnificent triumph for Newtonian physics: in Popperian terms, Newton’s theory had been subjected to a critical test, and had passed with flying colours.

Yet Lakatos flatly denies that there are critical tests, in the Popperian sense, in science, and argues the point convincingly by turning the above example of an alleged critical test on its head. What, he asks, would have happened if Galle had not found the planet Neptune? Would Newtonian physics have been abandoned, or would Newton’s theory have been falsified? The answer is clearly not, for Galle’s failure could have been attributed to any number of causes other than the falsity of Newtonian physics (e.g., the interference of the earth’s atmosphere with the telescope, the existence of an asteroid belt which hides the new planet from the earth, etc). The suggestion is that the “falsification/corroboration” disjunction offered by Popper is unjustifiable binary: non-corroboration is not necessarily falsification, and falsification of a high-level scientific theory is never brought about by an isolated observation or set of observations. Such theories are, it is now widely accepted, highly resistant to falsification; they are “tenaciously protected from refutation by a vast ‘protective belt’ of auxiliary hypotheses” (Lakatos 1978: 4) and so are falsified, if at all, not by Popperian critical tests, but rather within the elaborate context of the research programmes associated with them gradually grinding to a halt. Popper’s distinction between the logic of falsifiability and its applied methodology does not in the end do full justice to the fact that all high-level theories grow and live despite the existence of anomalies (i.e., events/phenomena which are incompatible with them). These, Lakatos suggests, are not usually taken by the working scientist as an indication that the theory in question is false. On the contrary, in the context of a progressive research programme he or she will necessarily assume that the auxiliary hypotheses which are associated with the theory can in time be modified to incorporate, and thereby explain, recalcitrant phenomena.

Third, Popper’s critique of Marxism has not, of course, gone unchallenged. The debate arising from, however, has in many cases tended to revolve around ideological rather than philosophical issues, which will be passed over here. However, there have also been some trenchant philosophical responses. Cornforth sees Marxism as a rational scientific discipline and Marxian thinking as “a rational and scientific approach to social problems” (Cornforth 1968: 6) of the kind which both Marx and Popper consider important. Consequently, he takes Popper to task for representing Marxism as a system of dogmas designed to close minds or pre-empt the operation of our rational faculties in addressing social issues. Against that view, he argues that it constitutes a way of thinking designed to open minds to the real possibilities of human life, and sees it as the philosophy best calculated to promote the ideals of the open society to which he, like Popper, subscribes. Hudelson (1980) argues that Marxian economics survives the Popperian critique of historicism and that, in any case, Marx did not hold many of the tenets of historicism identified by Popper. He also contends that Popper fails to show that there cannot be, and that we cannot know, laws of social development and that Marx did not in fact confuse trends and laws in the way that Popper suggests.

Fourth, In the case of Freudian psychoanalysis, the adequacy of Popper’s critique has been challenged on philosophical grounds by a number of commentators, particularly Adolf Grünbaum (1984). Grünbaum is highly critical, indeed scornful, of Popper’s dismissal of the claims of psychoanalysis to be scientific and argues that Freud showed “a keen appreciation of methodological pitfalls that are commonly laid at his door by critics” (Grünbaum 1984: 168) such as Popper. He argues that Freud was sensitive to the question of the logic of the confirmation and disconfirmation of psychoanalytic interpretations and cites Freud’s use of the concept of consilience, the convergence of evidence from disparate sources, as a serious and explicit attempt to meet the requirements of that logic. For Grünbaum, Popper’s critique of Freud amounts a veritable parody of the pioneering thinker who established the discipline of psychoanalysis, and he attributes that to Popper’s “obliviousness to Freud’s actual writings” (1984: 124). He points out, for example, that the case of the drowning child which Popper uses in Conjectures and Refutations (Popper 1963: 35), upon which he rests part of his case against psychoanalysis, is contrived and not in any way derived from Freud’s actual clinical texts.

Grünbaum contends that there are instances in Freud’s writings where he shows himself to be “a sophisticated scientific methodologist” (Grünbaum 1984: 128), keenly aware of the need for his theoretical system to meet the requirement of testability. One such occurs when Freud, in his assessment that anxiety neuroses are due to disturbances in sexual life, explicitly refers to the notion of falsifiability: “My theory can only be refuted when I have been shown phobias where sexual life is normal” (Freud 1895 [1962: 134]). Another occurs in Freud’s 1915 paper “A Case of Paranoia Running Counter to the Psycho-Analytic Theory of the Disease”, in which, as the title suggests, he saw the patient’s collection of symptoms as potentially falsifying the theory. Moreover, Freud’s entire account of paranoia as being due to an underlying repressed homosexuality is open to empirical refutation, Grünbaum argues, since it has the testable implication that a reduction in rates of paranoia should result from a removal or loosening of social sanctions against same-sex relationships (1984: 111).

Grünbaum accordingly holds that Freudian theory should be considered falsifiable and therefore genuinely scientific—albeit, in his view, ultimately unsuccessful, because the clinical evidence offered in its favour is contaminated by suggestion on the part of the analyst and cannot therefore support its conceptual weight. That is a verdict very different to what Grünbaum sees as the reductive dismissal of Freud offered by Popper: “the inability of certain philosophers of science to have discerned any testable consequences of Freud’s theory”, he concludes, “betokens their insufficient command or scrutiny of its logical content rather than a scientific liability of psychoanalysis” (1984: 113).

There can be little doubt of the seriousness of this challenge to Popper’s critique of the claims of psychoanalytic theory to scientific status. Further, the unparalleled cultural impact of Freudianism upon contemporary life can scarcely be denied, and even a cursory examination of the vast corpus of Freud’s works reveals a thinker of quite extraordinary theoretical power and imagination whose claims to scientific validity cannot be dismissed lightly.

However, while the detail of this psychoanalytic response to Popper is contestable, what is perhaps most striking and important about it is that, as articulated by Grünbaum, it is itself couched in broad terms of an acceptance of a Popperian account of science. That is to say, in rejecting the claim that psychoanalysis fails to meet the standard of falsifiability specified by Popper, Grünbaum—who also rejects the hermeneutic interpretation of Freud offered by thinkers such as Paul Ricoeur and Jurgen Habermas—nonetheless implicitly accepts that very standard, and with it the broad sweep of Popper’s theory of demarcation. For that reason alone, it seems clear that Popper’s work will continue to function as a critical reference point in the ongoing debate regarding the scientific status of Freudian thought.

Fifth, scientific laws are usually expressed by universal statements (i.e., they take the logical form “All A s are X ”, or some equivalent) which are therefore concealed conditionals—they have to be understood as hypothetical statements asserting what would be the case under certain ideal conditions. In themselves they are not existential in nature. Thus “All A s are X ” means “If anything is an A , then it is X ”. Since scientific laws are non-existential in nature, they logically cannot in themselves imply any basic statements, since the latter are explicitly existential. The question arises, then, as to how any basic statement can falsify a scientific law, given that basic statements are not deducible from scientific laws in themselves? Popper answers that scientific laws are always taken in conjunction with statements outlining the “initial conditions” of the system under investigation; these latter, which are singular existential statements, yield hard and fast implications when combined with the scientific law.

This reply is adequate only if it is true, as Popper assumes, that singular existential statements will always do the work of bridging the gap between a universal theory and a prediction. Hilary Putnam in particular has argued that this assumption is false, in that in some cases at least the statements required to bridge this gap (which he calls “auxiliary hypotheses”) are general rather than particular, and consequently that when the prediction turns out to be false we have no way of knowing whether this is due to the falsity of the scientific law or the falsity of the auxiliary hypotheses. The working scientist, Putnam argues (Putnam 1974; see also the 1991 reprinting with its retrospective note), always initially assumes that it is the latter, which shows not only that, but also why, scientific laws are, contra Popper, highly resistant to falsification, as Kuhn (1962) and Lakatos (1970, 1978) have also argued.

Popper’s final position is that he acknowledges that it is impossible to discriminate science from non-science on the basis of the falsifiability of the scientific statements alone; he recognises that scientific theories are predictive, and consequently prohibitive, only when taken in conjunction with auxiliary hypotheses, and he also recognises that readjustment or modification of the latter is an integral part of scientific practice. Hence his final concern is to outline conditions which indicate when such modification is genuinely scientific, and when it is merely ad hoc . This is itself clearly a major alteration in his position, and arguably represents a substantial retraction on his part: Marxism can no longer be dismissed as “unscientific” simply because its advocates preserved the theory from falsification by modifying it—for in general terms, such a procedure, it transpires, is perfectly respectable scientific practice. It is now condemned as unscientific by Popper because the only rationale for the modifications which were made to the original theory was to ensure that it evaded falsification, and so such modifications were ad hoc , rather than scientific. This contention—though not at all implausible—has, to hostile eyes, a somewhat contrived air about it, and is unlikely to worry the convinced Marxist. On the other hand, the shift in Popper’s own basic position is taken by some critics as an indicator that falsificationism, for all its apparent merits, fares no better in the final analysis than verificationism.

• 1928, Zur Methodenfrage der Denkpsychologie , Ph.D. Dissertation, University of Vienna, unpublished.
• 1935, Logik der Forschung , Vienna: Julius Springer Verlag.
• 1944–45, “The Poverty of Historicism”, Economica , Part 1, 11(42): 86–103; Part II, 11(43): 119–137; Part III, 12(46): 69–89. doi:10.2307/2549642 doi:10.2307/2550285 doi:10.2307/2549898
• 1945, The Open Society and Its Enemies , 2 volumes, London: Routledge. Republished 1966. Princeton, NJ: Princeton University Press.
• 1957, The Poverty of Historicism , London: Routledge. Revised version of Popper 1944–5.
• 1959 [2002], The Logic of Scientific Discovery , translation by the author of Logik der Forschung (1935), London: Hutchinson. Republished 2002, London & New York: Routledge Classics.
• 1963, Conjectures and Refutations: The Growth of Scientific Knowledge , London: Routledge.
• 1966, Of Clouds and Clocks: An Approach to the Problem of Rationality and the Freedom of Man . Washington, DC: Washington University Press.
• 1967 [1972], “Epistemology Without a Knowing Subject”, in Logic, Methodology and Philosophy of Science III , B. Van Rootselaar and J. F. Staal (eds.), (Studies in Logic and the Foundations of Mathematics 52), Amsterdam: Elsevier, 333–373. Reprinted in Popper 1972a: 106–152. doi:10.1016/S0049-237X(08)71204-7
• 1968 [1972], “On the Theory of the Objective Mind”, Akten des XIV. Internationalen Kongresses für Philosophie , vol. i, Vienna, 25–53. Translated and expanded in 1972a: 153–190.
• 1970 [1972], “A Realist View of Logic, Physics, and History”, in Physics, Logic and History , W. Yougrau and A. D. Breck (eds), Plenum Press, 1–30. Reprinted in Popper 1972a: 285–318.
• 1972 [1979], Objective Knowledge: An Evolutionary Approach , Oxford: Clarendon Press.
• 1972b, “Two Faces of Common Sense: An Argument for Commonsense Realism and Against the commonsense Theory of Knowledge”, in 1972a: 32–105.
• 1974, “Replies to My Critics”, in Schilpp 1974: vol. 2, 961–1197.
• 1976, Unended Quest; An Intellectual Autobiography , London: Fontana.
• 1976, “A Note on Verisimilitude”, The British Journal for the Philosophy of Science , 27: 147–159.
• 1977, The Self and Its Brain: An Argument for Interactionism , with John C. Eccles, London: Springer International. doi:10.4324/9780203537480
• 1979 [2007], Die Beiden Grundprobleme der Erkenntnistheorie , Tübingen: Routledge. Based on manuscripts written in the early 1930s. Translated as The Two Fundamental Problems of the Theory of Knowledge , Troels Eggers Hansen (ed.), Andreas Pickel (trans.), London: Routledge, 2007.
• 1980, “Three Worlds”, in The Tanner Lectures on Human Values , volume 1, S.M. McMurrin (ed.), Salt Lake City, UT: University of Utah Press, 141–167.
• 1982, The Open Universe: An Argument for Indeterminism , W.W. Bartley III (ed.), London: Hutchinson.
• 1983, Realism and the Aim of Science , W.W. Bartley III (ed.), London: Hutchinson.
• 1994, The Myth of the Framework: In Defence of Science and Rationality , M.A. Notturno, (ed.), London: Routledge.
• 1994, Knowledge and the Mind-Body Problem: In Defence of Interactionism , M.A. Notturno (ed.), London: Routledge.
• 2012, After the Open Society: Selected Social and Political Writings , Jeremy Shearmur and Piers Norris Turner (eds.), London and New York: Routledge.
• 2020, “The Feyerabend-Popper Correspondence (1948–1967)”, in Feyerabend’s Formative Years. Volume 1. Feyerabend and Popper , Matteo Collodel and Eric Oberheim (eds.), (Vienna Circle Institute Library 5), Cham: Springer International Publishing, 59–420. doi:10.1007/978-3-030-00961-8_4
• 2022, Binder, D., Piecha, T., Schroeder-Heister, P. (editors) (2022), The Logical Writings of Karl Popper . Cham: Springer. [ The Logical Writings of Karl Popper available online ]
• Ackermann, Robert, 1976, The Philosophy of Karl Popper , Amherst, MA: University of Massachusetts Press.
• Agassi, Joseph, 2014, Popper and His Popular Critics: Thomas Kuhn, Paul Feyerabend and Imre Lakatos (SpringerBriefs in Philosophy), Cham: Springer International Publishing. doi:10.1007/978-3-319-06587-8.
• Akinci, Semiha, 2004, “Popper’ s Conventionalism”, in Catton and Macdonald 2004: 28–49.
• Alamuiti, M. M., 2021, Critical Rationalism and the Theory of Society: Critical Rationalism and the Open Society (Volume 1), London: Routledge.
• Bambrough, Renford (ed.), 1967, Plato, Popper, and Politics: Some Contributions to a Modern Controversy , New York: Barnes and Noble.
• Baudoin, Jean, 1989, Karl Popper , Paris: Presses Universitaires de France.
• Brink, Chris, 1989, “Verisimilitude: Views and Reviews”, History and Philosophy of Logic , 10(2): 181–201. doi:10.1080/014453408908837149
• Brink, Chris and Johannes Heidema, 1987, “A Verisimilar Ordering of Theories Phrased in a Propositional Language”, The British Journal for the Philosophy of Science , 38(4): 533–549. doi:10.1093/bjps/38.4.533
• Briskman, Laurence Barry, 2020, A Sceptical Theory of Scientific Inquiry: Problems and Their Progress , Jeremy Shearmur (ed.), Leiden: Brill.
• Britz, Katarina and Chris Brink, 1995, “Computing Verisimilitude”, Notre Dame Journal of Formal Logic , 36(1): 30–43. doi:10.1305/ndjfl/1040308827
• Bunge, Mario (ed.), 1964, The Critical Approach to Science and Philosophy , London & New York: The Free Press.
• Burke, T.E., 1983, The Philosophy of Popper , Manchester: Manchester University Press.
• Carnap, Rudolf, 1966 [1995], Philosophical Foundations of Physics: An Introduction to the Philosophy of Science , New York: Basic Books. New edition entitled An Introduction to the Philosophy of Science , New York: Dover, 1995.
• Catton, Philip and Graham MacDonald (eds.), 2004, Karl Popper: Critical Appraisals , New York: Routledge. doi:10.4324/9780203326886
• Cornforth, Maurice, 1968., The Open Philosophy and the Open Society: A Reply to Dr. Popper’s Refutations of Marxism , London: Lawrence & Wishart.
• Corvi, Roberta, 1993 [1997], Invito al pensiero di Karl Popper , Milan: Gruppo Ugo Mursia S.P.A. Translated as An Introduction to the Thought of Karl Popper , Patrick Camiller (trans), London & New York: Routledge, 1997.
• Currie, Gregory and Alan Musgrave (eds.), 1985, Popper and the Human Sciences , Dordrecht: Nijhoff.
• Del Santo, F. 2022. ‘Beyond Method: The Diatribe Between Feyerabend and Popper Over the Foundations of Quantum Mechanics’, International Studies in the Philosophy of Science , Vol. 35, 2022, 5–22. Taylor&Francis Online. doi:10.1080/02698595.2022.2031430
• Edmonds, David and John Eidinow, 2001, Wittgenstein’s Poker: The Story of a Ten-Minute Argument Between Two Great Philosophers , New York: Harper & Collins.
• Feyerabend, Paul, 1975, Against Method , London: New Left Books.
• Freud, Sigmund, 1895 [1962], “Zur Kritik Der ‘Angstneurose’”, Wiener klinische Wochenschrift , 9(27): 417–19, 9(28): 435–7, and 9(29): 451–2; translated as “A Reply to Criticisms of My Paper on Anxiety Neurosis”, in the Standard Edition of the Complete Psychological Works of Sigmund Freud , James Strackey (ed.), London: Hogarth, 1962, 3: 121–140.
• –––, 1915 [1957], “Mitteilung eines der psychoanalytischen Theorie widersprechenden Falles von Paranoia”, Internationale Zeitschrift für Psychoanalyse , 3(6): 321–329; translated as “A Case of Paranoia Running Counter to the Psycho-Analytic Theory of the Disease”, in the Standard Edition of the Complete Psychological Works of Sigmund Freud , James Strackey (ed.), London: Hogarth, 1957, 14: 263–272.
• Fuller, Steve, 2004, Kuhn vs. Popper: The Struggle for the Soul of Science , New York: Columbia University Press.
• Galbraith, D., 2022, “Pigden Revisited, or In Defence of Popper’s Critique of the Conspiracy Theory of Society”, in Philosophy of the Social Sciences , 52(4): 235–257.
• Gattei, Stefano, 2010, Karl Popper’s Philosophy of Science: Rationality without Foundations , New York: Routledge.
• Grünbaum, Adolf, 1976, “Is the Method of Bold Conjectures and Attempted Refutations Justifiably the Method of Science?”, The British Journal for the Philosophy of Science , 27(2): 105–136. doi:10.1093/bjps/27.2.105
• –––, 1984, The Foundations of Psycho-analysis: A Philosophical Critique , Berkeley, CA: University of California Press.
• Hacohen, Malachi Haim, 2000, Karl Popper—The Formative Years, 1902–1945: Politics and Philosophy in Interwar Vienna , Cambridge: Cambridge University Press.
• Harris, John H., 1974, “Popper’s Definitions of ‘Verisimilitude’”, The British Journal for the Philosophy of Science , 25(2): 160–166. doi:10.1093/bjps/25.2.160
• Howson, Colin, 1984, “Popper’s Solution to the Problem of Induction”, The Philosophical Quarterly , 34(135): 143–147. doi:10.2307/2219507
• Hudelson, Richard, 1980, “Popper’s Critique of Marx”, Philosophical Studies: An International Journal for Philosophy in the Analytic Tradition , 37(3): 259–70.
• Hume, David, 1739–40, A Treatise of Human Nature , London. Reprinted in his The Philosophical Works , T.H. Green & T.H. Grose (eds), 4 vols., Darmstadt: Scientia Verlag Aalen, 1964 (reprint of 1886 edition).
• Jacobs, Struan, 1991, Science and British Liberalism: Locke, Bentham, Mill and Popper , Aldershot: Avebury.
• James, Roger, 1980, Return to Reason: Popper’s Thought in Public Life , Shepton Mallet: Open Books.
• Jarvie, Ian C. 2001, The Republic of Science: The Emergence of Popper’s Social View of Science 1935–1945 , Amsterdam: Brill | Rodopi
• Johansson, Ingvar, 1975, A Critique of Karl Popper’s Methodology , Stockholm: Scandinavian University Books.
• Kekes, John, 1977, “Popper in Perspective”, Metaphilosophy , 8(1): 36–61. doi:10.1111/j.1467-9973.1977.tb00261.x
• Keuth, Herbert, 1976, “Verisimilitude or the Approach to the Whole Truth”, Philosophy of Science , 43(3): 311–336. doi:10.1086/288691
• Keuth, Herbert, 2000 [2004], Die Philosophie Karl Popper , Tübingen: Mohr Siebeck. Translated by the author as The Philosophy of Karl Popper , New York: Cambridge University Press, 2004.
• Kuhn, Thomas S., 1962, The Structure of Scientific Revolutions , Chicago and London: University of Chicago Press.
• Kuipers, Theo A.F., 1982, “Approaching Descriptive and Theoretical Truth”, Erkenntnis , 18(3): 343–378. doi:10.1007/BF00205277
• ––– (ed.), 1987, What is Closer-to-the-Truth? , Amsterdam: Rodopi.
• Lakatos, Imre, 1970, “Falsification and the Methodology of Scientific Research Programmes”, in Lakatos and Musgrave 1970: 91–196. doi:10.1017/CBO9781139171434.009
• –––, 1978, The Methodology of Scientific Research Programmes , J. Worrel and G. Currie (eds.), Cambridge: Cambridge University Press.
• Lakatos, Imre and Alan Musgrave (eds.), 1970, Criticism and the Growth of Knowledge: Proceedings of the International Colloquium in the Philosophy of Science, London , 1965, Cambridge: Cambridge University Press. doi:10.1017/CBO9781139171434
• Laudan, Larry, 1977, Progress and Its Problems: Towards a Theory of Scientific Growth , Berkeley, CA: University of California Press.
• Leplin, Jarrett, 2007, “Enlisting Popper in the Case for Scientific Realism”, Philosophia Scientae , 11(1): 71–97. doi:10.4000/philosophiascientiae.323 [ Leplin 2007 available online ]
• Levinson, Paul (ed.), 1982, In Pursuit of Truth: Essays in Honour of Karl Popper on the Occasion of his 80th Birthday , Atlantic Highlands, NJ: Humanities Press.
• Levinson, Ronald B., 1953, In Defense of Plato , Cambridge, MA: Harvard University Press.
• Magee, Bryan, 1973, Karl Popper , London: Penguin.
• Maxwell, N., 2002, “Karl Raimund Popper”, in British Philosophers, 1800–2000 ( Dictionary of Literary Biography : 262), P. Dematteis, P. Fosl and L. McHenry (eds.), Detroit: Gale Research Inc., pp. 176–194
• –––, 2017, Karl Popper, Science and Enlightenment , London: University College London Press.
• –––, 2020, “Aim-Oriented Empiricism and the Metaphysics of Science”, Philosophia , 48: 347–364.
• Mellor, D. H., 1977, “The Popper Phenomenon”, Philosophy , 52(200): 195–202. doi:10.1017/S0031819100023135
• Milkov, Nikolay, 2012, “Karl Popper’s Debt to Leonard Nelson”, Grazer Philosophische Studien , 86(1): 137–156. doi:10.1163/9789401209182_009
• Miller, David, 1974a, “On the Comparison of False Theories by Their Bases”, The British Journal for the Philosophy of Science , 25(2): 178–188. doi:10.1093/bjps/25.2.178
• –––, 1974b, “Popper’s Qualitative Theory of Verisimilitude”, The British Journal for the Philosophy of Science , 25(2): 166–177. doi:10.1093/bjps/25.2.166
• –––, 1994, Critical Rationalism: A Restatement and Defence , Chicago: Open Court.
• Mulkay, Michael and G. Nigel Gilbert, 1981, “Putting Philosophy to Work: Karl Popper’s Influence on Scientific Practice”, Philosophy of the Social Sciences , 11(3): 389–407. doi:10.1177/004839318101100306
• Munz, Peter, 1985, Our Knowledge of the Growth of Knowledge: Popper or Wittgenstein? , London: Routledge.
• Naydler, Jeremy, 1982, “The Poverty of Popperism”, The Thomist: A Speculative Quarterly Review , 46(1): 92–107. doi:10.1353/tho.1982.0048
• Niiniluoto, Ilkka, 1987, Truthlikeness , Dordrecht: D. Reidel.
• Oddie, Graham, 1986, Likeness to Truth , Dordrecht: D. Reidel.
• O’Hear, Anthony, 1980, Karl Popper , London: Routledge.
• ––– (ed.), 1995, Karl Popper: Philosophy and Problems , Cambridge & New York: Cambridge University Press. doi:10.1017/CBO9780511563751
• Parusniková, Z. & Merritt, D. (eds.), 2021, Karl Popper’s Science and Philosophy , Cham: Springer, ebook. doi:10.1007/978-3-030-67036-8
• Putnam, Hilary, 1974 [1991], “The ‘Corroboration’ of Theories”, in Schilpp 1974: vol. 1: 221–240. Republished with Retrospective Note in The Philosophy of Science , Richard Boyd, Philip Gasper, and J. D. Trout (eds.), Cambridge, MA: MIT Press, 1991, 121–138.
• Quinton, Anthony, 1967, “Popper, Karl Raimund”, in Encyclopedia of Philosophy (Volume 6), Paul Edwards (ed.), New York: Collier Macmillan: 398–401.
• Radnitzky, Gerard and Gunnar Andersson (eds.), 1978, Progress and Rationality in Science , Dordrecht: D. Reidel.
• Radnitzky, Gerard and W. W. Bartley (eds.), 1987, Evolutionary Epistemology, Rationality, and the Sociology of Knowledge , La Salle, IL: Open Court.
• Richmond, Sheldon, 1994, Aesthetic Criteria: Gombrich and the Philosophies of Science of Popper and Polanyi , Amsterdam/Atlanta, GA: Rodopi.
• Rowbottom, Darrell P., 2010, Popper’s Critical Rationalism: A Philosophical Investigation , New York: Routledge. doi:10.4324/9780203836187
• Salmon, Wesley C., 1967, The Foundations of Scientific Inference , Pittsburgh: University of Pittsburgh Press.
• Royer, C., Matei, L. (eds.), 2023, Open Society Unresolved. The Contemporary Relevance of a Contested Idea. Budapest, Vienna, New York: Central European University Press.
• Sassower, Raphael and Nathaniel Laor (eds.), 2019, The Impact of Critical Rationalism: Expanding the Popperian Legacy through the Works of Ian C. Jarvie , Cham: Springer International Publishing. doi:10.1007/978-3-319-90826-7
• Schilpp, Paul Arthur (ed.), 1974, The Philosophy of Karl Popper , 2 volumes, La Salle, IL: Open Court Press.
• Shearmur, Jeremy, 1996, Political Thought of Karl Popper , London & New York: Routledge.
• Simkin, C. G. F., 1993, Popper’s Views on Natural and Social Science , Leiden: Brill.
• Stokes, Geoffrey, 1998, Popper: Philosophy, Politics and Scientific Method , New York: Wiley & Sons.
• Stove, D. C., 1982, Popper and After: Four Modern Irrationalists , Oxford: Pergamon Press.
• Sturm, Thomas, 2012, “Bühler and Popper: Kantian Therapies for the Crisis in Psychology”, Studies in History and Philosophy of Science Part C: Studies in History and Philosophy of Biological and Biomedical Sciences , 43(2): 462–472. doi:10.1016/j.shpsc.2011.11.006
• Tichý, Pavel, 1974, “On Popper’s Definitions of Verisimilitude”, The British Journal for the Philosophy of Science , 25(2): 155–160. doi:10.1093/bjps/25.2.155
• –––, 1978, “Verisimilitude Revisited”, Synthese , 38(2): 175–196. doi:10.1007/BF00486149
• Thomas, Allan James, 2019, “Thought Without a Thinking Subject: or Karl Popper as Film-Philosopher”, Screen Thought Journal , 3(1): 1–13.
• Vetter, Hermann, 1977, “A New Concept of Verisimilitude”, Theory and Decision , 8(4): 369–375. doi:10.1007/BF00141535
• Watkins, John W. N., 1984, Science and Scepticism , Princeton, NJ: Princeton University Press and London: Hutchinson.
• –––, 1997. “Popperian Ideas on Progress and Rationality in Science”, The Critical Rationalist , 2(2). [ Watkins 1997 available online ].
• Wilkins, Burleigh Taylor, 1978, Has History Any Meaning? A Critique of Popper’s Philosophy of History , Ithaca, NY: Cornell University Press.
• Williams, Douglas E., 1989, Truth, Hope and Power: The Thought of Karl Popper , Toronto: University of Toronto Press.
• Wuketits, Franz M., 1984, Concepts and Approaches in Evolutionary Epistemology: Towards an Evolutionary Theory of Knowledge , Dordrecht: D. Reidel.
• Xiao, H., 2023, ‘On Popper’s Shifting Appraisal of Scientific Status of Darwinian Theory’,  Academic Journal of Humanities & Social Sciences , 6(3): 23–30.

How to cite this entry . Preview the PDF version of this entry at the Friends of the SEP Society . Look up topics and thinkers related to this entry at the Internet Philosophy Ontology Project (InPhO). Enhanced bibliography for this entry at PhilPapers , with links to its database.

• The Karl Popper Web .
• Open Universe of the Japan Popper Society .
• The Karl Popper Archives – University Library Klagenfurt, AAU
• Karl Popper Foundation – Universität Klagenfurt (aau.at)

confirmation | convention | Feyerabend, Paul | Hume, David | induction: problem of | Kuhn, Thomas | Lakatos, Imre | science: and pseudo-science | scientific method | scientific progress | scientific realism | truthlikeness | Vienna Circle

Copyright © 2022 by Stephen Thornton < sfthornton4 @ gmail . com >

• Accessibility

Support SEP

Mirror sites.

View this site from another server:

• Info about mirror sites

The Stanford Encyclopedia of Philosophy is copyright © 2023 by The Metaphysics Research Lab , Department of Philosophy, Stanford University

Library of Congress Catalog Data: ISSN 1095-5054

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

How failure to falsify in high-volume science contributes to the replication crisis

Sarah m rajtmajer.

1 College of Information Sciences and Technology, The Pennsylvania State University University Park, United States

Timothy M Errington

2 Center for Open Science Charlottesville, United States

Frank G Hillary

3 Department of Psychology and the Social Life and Engineering Sciences Imaging Center, The Pennsylvania State University University Park, United States

Associated Data

There are no data associated with this article.

The number of scientific papers published every year continues to increase, but scientific knowledge is not progressing at the same rate. Here we argue that a greater emphasis on falsification – the direct testing of strong hypotheses – would lead to faster progress by allowing well-specified hypotheses to be eliminated. We describe an example from neuroscience where there has been little work to directly test two prominent but incompatible hypotheses related to traumatic brain injury. Based on this example, we discuss how building strong hypotheses and then setting out to falsify them can bring greater precision to the clinical neurosciences, and argue that this approach could be beneficial to all areas of science.

Background and motivation

The “replication crisis” in various areas of research has been widely discussed in journals over the past decade [see, for example, Gilbert et al., 2016 ; Baker, 2016 ; Open Science Collaboration, 2015 ; Munafò et al., 2017 ]. At the center of this crisis is the concern that any given scientific result may not be reliable; in this way, the crisis is ultimately a question about the collective confidence we have in our methods and results ( Alipourfard et al., 2012 ). The past decade has also witnessed many advances in data science, and “big data” has both contributed to concerns about scientific reliability ( Bollier and Firestone, 2010 ; Calude and Longo, 2017 ) and also offered the possibility of improving reliability in some fields ( Rodgers and Shrout, 2018 ).

In this article we discuss scientific progress in the clinical neurosciences, and focus on an example related to traumatic brain injury (TBI). Using this example, we argue that the rapid pace of work in this field, coupled with a failure to directly test and eliminate (falsify) hypotheses, has resulted in an expansive literature that lacks the precision necessary to advance science. Instead, we suggest that falsification – where one develops a strong hypothesis, along with methods that can test and refute this hypothesis – should be used more widely by researchers. The strength of a hypothesis refers to how specific and how refutable it is ( Popper, 1963 ; see Table 1 for examples). We also argue for greater emphasis on testing and refuting strong hypotheses through a “team science” framework that allows us to address the heterogeneity in samples and/or methods that makes so many published findings tentative ( Cwiek et al., 2021 ; Bryan et al., 2021 ).

Exploratory research does not generally involve testing a hypothesis. A Testable Association is a weak hypothesis as it is difficult to refute. A Testable/Falsifiable Position is stronger, and a hypothesis that is Testable/Falsifiable with Alternative Finding is stronger still.

Hyperconnectivity hypothesis in brain connectomics

To provide a specific example for the concerns outlined in this critique, we draw from the literature using resting-state fMRI methods and network analysis (typically graph theory, see Caeyenberghs et al., 2017 to examine systems-level plasticity in TBI). Beginning with one of the first papers combining functional neuroimaging and graph theory to examine network topology ( Nakamura et al., 2009 ), an early observation in the study of TBI was that physical disruption of pathways due to focal and diffuse injury results in regional expansion (increase) in strength or number of functional connections. This initial finding was observed in a small longitudinal sample, but then similar effects were observed in other samples ( Mayer et al., 2011 ; Bharath et al., 2015 ; Hillary et al., 2015 ; Johnson et al., 2012 ; Sharp et al., 2011 ; Iraji et al., 2016 ) and animal models of TBI ( Harris et al., 2016 ). These findings were summarized in a paper by one of the current authors (FGH) outlining potential mechanisms for hyperconnectivity and its possible long-term consequences, including elevated metabolic demand, abnormal protein aggregation and, ultimately, increased risk for neurodegeneration (see Hillary and Grafman, 2017 ). The “hyperconnectivity response” to neurological insult was proposed as a possible biomarker for injury/recovery in a review summarizing findings in TBI brain connectomics ( Caeyenberghs et al., 2017 ).

Nearly simultaneously, other researchers offered a distinct – in fact, nearly the opposite – set of findings. Several studies of moderate to severe brain injury (as examined above) found that white matter disruption during injury resulted in structural and functional disconnection of networks. The authors in these papers outline a “disconnection” hypothesis: the physical degradation of white matter secondary to traumatic axonal injury results in reduced connectivity of brain networks, which is visible both structurally in diffusion imaging studies ( Fagerholm et al., 2015 ) and functionally using resting-state fMRI approaches ( Bonnelle et al., 2011 ). These findings were summarized in a high-profile review ( Sharp et al., 2014 ) where the authors argue that TBI “substantially disrupts [connectivity], and that this disruption predicts cognitive impairment …”.

When juxtaposed, these two hypotheses hold distinct explanations for the same phenomenon with the first proposing that axonal injury results in a paradoxically enhanced functional network response and the second that the same pathophysiology results in reduced functional connectivity. Both cannot be true as they have been proposed, so which is correct? Even with two apparently contradictory hypotheses in place, there has been no direct testing of these positions against one another to determine the scenarios where either may have merit. Instead, each of these hypotheses remained unconditionally intact and served to support distinct sets of outcomes.

The most important point to be made from this example is not that competing theories in this literature exist. To the contrary, having competing theories for understanding a phenomenon places science in a strong position; the theories can be tested against one another to qualify (or even eliminate) one position. The point is that there have been no attempts to falsify either a hyperconnectivity or disconnection hypothesis, allowing researchers to evoke one or the other depending upon the finding for a given dataset (i.e., disconnection due to white matter loss, or functional “compensation” in the case of hyperconnectivity). What has contributed to this problem is that increasingly complex computational modeling also increases the investigator degrees of freedom, both implicitly and explicitly, to support their hypotheses. In the case of the current example of neural networks, these include selection from a number of brain atlases or other methods for brain parcellation and likewise numerous approaches to neural network definition (see Hallquist and Hillary, 2019 ). Figure 1 provides a schematic representation of two distinct and simultaneously supported hypotheses in head injury.

Panel A represents the typical pattern of resting connectivity for the default mode network (DMN) and the yellow box shows a magnified area of neuronal bodies and their axonal projections. Panel B reveals three active neuronal projections (red) that are then disrupted by hemorrhagic lesion of white matter (Panel C ). In response to this injury, a hyperconnectivity response (Panel D , left) shows increased signaling to adjacent areas resulting in a pronounced DMN response (Panel D , right). By contrast a disconnection hypothesis maintains that signaling from the original neuronal assemblies is diminished due to axonal degradation and neuronal atrophy secondary to cerebral diaschisis (Panel E , left) resulting in reduced functional DMN response (Panel E , right).

To be clear, the approach taken by investigators in this TBI literature is consistent with a research agenda designed to meet the demands for high publication throughput (more on this below). Examiners publish preliminary findings but remain appropriately tentative in their conclusions given that the sample is small and unexplained factors are numerous. Indeed, a common refrain in many publications is the “need for replication in a larger sample”. As opposed to pre-registering and testing strong hypotheses, investigators are reinforced to identify significant results (any result) for publication. In brain injury work examining network plasticity, investigators have often made general claims that brain injury results in “different” or “altered” connectivity (a problem dating back to early fMRI studies in TBI; Hillary, 2008 ). While unintentional, imprecise hypotheses increase the likelihood that chance findings are published. The primary consequence is that all findings are “winners”, permitting growing support for either position without movement toward resolution.

Overall, the TBI connectomics literature presents a clear example of a failure to falsify and we argue that it is attributable, at least in part, to the publication of large numbers of papers reporting the results of studies in which small samples were used to examine under-specified hypotheses. This “science-by-volume” approach is exacerbated by the overuse of inappropriate statistical tests, which increases the probability that spurious findings will be reported as meaningful ( Button et al., 2013 ).

The challenges outlined here, where there is a general failure to test and refute strong hypotheses, are not isolated to the TBI literature. Similar issues have been expressed in preclinical studies of stroke ( Corbett et al., 2017 ) in the translational neurosciences where investigators maintain flexible theory and predictions to fit methodological limitations ( Macleod et al., 2014 ; Pound and Ritskes-Hoitinga, 2018 ; Henderson et al., 2013 ), and in cancer research where only portions of published data sets provide support for hypotheses ( Begley and Ellis, 2012 ). These factors have likely contributed to the repeated failure of clinical trials to move from animal models to successful Phase III interventions in clinical neuroscience ( Tolchin et al., 2020 ). This example in the neurosciences also mirrors the longstanding problems of co-existing yet inconsistent theories in other disciplines like social psychology (see Watts, 2017 ).

Big data and computational methods as friend and foe

The big data revolution and advancement of computational modeling powered by enhanced computing infrastructure, on the one hand, has magnified concerns about scientific reliability through unprecedented flexibility in data exploration and analysis. Sufficiently large datasets provably contain spurious correlations and the number of these coincidental regularities increases as the dataset size increases ( Calude and Longo, 2017 ; Graham and Spencer, 1990 ). Adding flexibility, predictive algorithms built on top of these large datasets typically involve a great number of investigator decisions – the combined effects of which undermine reliability of findings [for an example in connectivity modeling see Hallquist and Hillary, 2019 ]. Results of machine learning models, for example, are sensitive to model specification and parameter tuning ( Pineau, 2021 ; Bouthillier et al., 2019 ; Cwiek et al., 2021 ). Computational approaches permit systematically combing through a great number of potential variables of interest and their statistical relationships (specifically, at scales which would be manually infeasible). Consequently, the burden of reliability falls upon the existence of strong, well-founded hypotheses with sufficient power and clear pre-analysis plans. It has even been suggested that null hypothesis significance testing should only be used in the neurosciences in support of pre-registered hypotheses based on strong theory ( Szucs and Ioannidis, 2017 ).

So, while there is concern that Big Data moves too fast and without the necessary constraints of theory, there is also emerging sentiment that the tremendous computational power coupled with unparalleled data access has the potential to transform some of the most basic scientific tenets, including introduction of a “third scientific pillar” to be added to theory and experimentation (see National Science Foundation, 2010 ). While this latest position received criticism ( Andrews, 2012 ), computational methods have been reliably demonstrated to offer novel tools to address the replication crisis – an issue addressed in greater detail in “solutions” below.

Operating without anchors in a sea of high-volume science

One challenge then is to determine where the bedrock of our field (our foundational knowledge) ends, and where areas of discovery that show promise (but have yet to be established) begin. By some measure neurology is a fledgling field in the biological sciences: the publication of De humani corporis fabrica by Vesalius in 1543 is often taken to mark the start of the study of human anatomy ( Vesalius, 1555 ) Jean-Martin Charcot – often referred to as the “founder of neurology” – arrived approximately 300 years later ( Zalc, 2018 ). If we simplify our task and start with the work of Milner, Geschwind and Luria in the 1950s, it is still a challenge to determine what is definitively known and what remains conjectural in the field. This challenge is amplified by the pressure on researchers to publish or perish ( Macleod et al., 2014 ; Kiai, 2019 ; Lindner et al., 2018 ). The number of papers published per year continues to increase without asymptote ( Bornmann and Mutz, 2015 ). When considering all papers published in the clinical neurosciences since 1900, more than 50% of the entire literature has been published in the last 10 years and 35% in the last five years (see supplementary figures S1a,b in Priestley et al., 2022 ). In the most extreme examples, “hyperprolific” lab directors publish a scientific paper roughly every 5 days ( Ioannidis et al., 2018 ). It is legitimate to ask if the current proliferation of published findings has been matched by advances in scientific knowledge, or if the rate of publishing is outpacing scientific ingenuity ( Sandström and van den Besselaar, 2016 ) and impeding the emergence of new theories ( Chu and Evans, 2021 ).

We argue that a culture of science-by-volume is problematic for the reliability of science, primarily when paired with research agendas not designed to test/refute hypotheses. First, without pruning possible explanations through falsification, the science-by-volume approach creates an ever-expanding search space where finite human and financial resources are deployed to maximize breadth in published findings as opposed to depth of understanding ( Figure 2A ). Second, and as an extension of the last point, failure to falsify in a high-volume environment challenges our capacity to know which hypotheses represent foundational theory, which hypotheses are encouraging but require further confirmation, and which hypotheses should be rejected. Finally, in the case of the least-publishable-unit ( Broad, 1981 ) a single data set may be carved into several smaller papers resulting in circles of self-citation and the illusion of reliable support for a hypothesis (or hypotheses) ( Gleeson and Biddle, 2000 ).

Panels A and B illustrate the conceptual steps in theory progression from exploration through confirmation and finally application. The x-axis is theoretical progression (time) and the y-axis is the number of active theories. Panel A depicts progression in the absence of falsification with continued branching of theories in the absence of pruning (theory reduction through falsification). By contrast the “Confirmatory Stage” in Panel B includes direct testing and refutation of theories/explanations resulting in only the fittest theories to choose from during application. Note: both Panels A and B include replication, but falsification during the “confirmation” phase results in a linear pathway and fewer choices from the “fittest” theories at the applied stage.

There have even been efforts internationally to make science more deliberate through de-emphasis of publication rates in academic circles ( Dijstelbloem et al., 2013 ). Executing this type of systemic change in publication rate poses significant challenges and may ultimately be counterproductive because it fails to acknowledge the advancements in data aggregation and analysis afforded by high performance computing and rapid scientific communication through technology. So, while an argument can be made that our rate of publishing is not commensurate with our scientific progress, a path backward to a lower annual publication rate seems an unlikely solution and ignores the advantages of modernity. Instead, we should work toward establishing scientific foundation by testing and refuting strong hypotheses and these efforts may hold the greatest benefit when used to prune theories to determine the fittest prior to replication ( Figure 2B ). This effort maximizes resources and makes the goals for replication, as a confrontation of theoretical expectations, very clear ( Nosek and Errington, 2020a ). The remainder of the paper outlines how this can be achieved with focus on several contributors to the replication crisis.

Accelerating science by falsifying strong hypotheses

In praise of strong hypotheses.

Successful refutation of hypotheses ultimately depends upon a number of factors, not the least of which is the specificity of the hypothesis ( Earp and Trafimow, 2015 ). A simple, but well-specified, hypothesis, brings greater leverage to science than a hypothesis that is far reaching with broad implications but cannot be directly tested or refuted. Even Popper wrote about concerns in the behavioral sciences regarding the rather general nature of hypotheses ( Bartley, 1978 ), a sentiment that has recently been described as a “crisis” in psychological theory advancement ( Rzhetsky et al., 2015 ). As discussed in the TBI connectomics example, hypotheses may have been broad and "exploratory" because authors remained conservative in their claims and conclusions because studies have been systematically under-powered (one report estimating power at 8%; Button et al., 2013 ). While exploration is a vital part of science ( Figure 2 ), it must be recognized as scientific exploration as opposed to an empirical test of a hypothesis. Under-developed hypotheses have been argued to be at least one contributor to repeated failure of clinical trials in acute neurological interventions ( Schwamm, 2014 ) yet, paradoxically, strong hypotheses may offer increased sensitivity to subtle effects even in small samples ( Lazic, 2018 ).

If we appeal to Popper, the strongest hypotheses make “risky predictions”, therefore prohibiting alternative explanations (see Popper, 1963 ). Moreover, the strongest hypotheses make clear at the outset the findings that would support the prediction, and also those that would not. Practically speaking this could take the form of teams of scientists developing opposing sets of hypotheses and then agreeing on both the experiments and the outcomes that would falsify one or both positions (what Nosek and Errington refer to as precommitment; Nosek and Errington, 2020b ). This creates scenarios a priori where strong hypotheses are matched with methods that can provide clear tests. This approach is currently being applied in the “accelerating research on consciousness” programme funded by the Templeton World Charity Foundation . Strong hypotheses must be matched with methods that can provide clear tests, a coupling that cannot be overstated. In the brain imaging literature alone, there are poignant examples where flawed methods (or misunderstanding of their applications) has resulted in the repeated substantiation of spurious results (in structural covariance analysis see Carmon et al., 2020 in resting-state fMRI see Satterthwaite et al., 2012 ; Van Dijk et al., 2012 ).

Addressing heterogeneity to create strong hypotheses

One approach to strengthen hypotheses is to address sample and methodological heterogeneity which plagues the clinical neurosciences ( Benedict and Zivadinov, 2011 ; Bennett et al., 2019 ; Schrag et al., 2019 ; Zucchella et al., 2020 ; Yeates et al., 2019 ). To echo a recent review of work in the social sciences, the neurosciences require a “heterogeneity revolution” ( Bryan et al., 2021 ). Returning again to the TBI connectomics example, investigators relied upon small datasets heterogeneous with respect to age of injury, time post injury, injury severity, and other factors that could critically influence the response of the neural system to injury. Strong hypotheses determine the influence of sample characteristics by directly modeling the effects of demographic and clinical factors ( Bryan et al., 2021 ) as opposed to statistically manipulating the variance accounted for by them – including the widespread and longstanding misapplication of covariance statistics to “equilibrate” groups in case-control designs ( Miller and Chapman, 2001 ; Zinbarg et al., 2010 ; Storandt and Hudson, 1975 ). Finally, strong hypotheses leverage the pace of our current science as an ally, where studies designed specifically to address sample heterogeneity can test the role of clinical and demographic predictors in brain plasticity and outcome.

Open science and sharing to bolster falsification efforts

Addressing sample heterogeneity requires large diverse samples, and one way to achieve this is via data sharing. While data-sharing practices and availability differ across scientific disciplines ( Tedersoo et al., 2021 ), there are enormous opportunities for sharing data in the clinical neurosciences (see, for example the Alzheimer’s Disease Neuroimaging Initiative (ADNI) and the Transforming Research and Clinical Knowledge in Traumatic Brain Injury (TRACK-TBI) initiative), even in cases where data were not collected with identical methods (such as the Enhancing NeuroImaging Genetics through Meta-Analysis (ENIGMA) Consortium; see Olsen et al., 2021 for more on severe brain injury, and Thompson et al., 2020 for a broad summary of work in clinical neuroscience). However, data aggregation and harmonization approaches remain largely untested as a solution to science-by-volume problems in the neurosciences.

It should be stressed that data sharing as a practice is not a panacea to poor study design and/or an absence of theory. The benefits of data combination do not eliminate any existing issues related to instrumentation and data collection occurring at individual sites; it is crucial to understand that data sharing permits faster accumulation of data while retaining any existing methodological concerns (e.g., harmonization). If unaddressed, these concerns introduce magnified noise or systematic bias masquerading as high-powered findings ( Maikusa et al., 2021 ). However, well-designed data sharing efforts with rigorous harmonization approaches (e.g., Fortin et al., 2017 ; Tate et al., 2021 ) hold opportunities for falsification through meta-analyses, mega-analyses, and between site data comparisons ( Thompson et al., 2022 ). Data sharing and team science also provide realistic opportunities to address sample heterogeneity and site-level idiosyncrasies in method.

Returning to the TBI connectomics example above, data sharing could play a central role in resolving this literature. The neural network response to injury most likely depends upon where one looks (specific neural networks), time post injury, and perhaps a range of clinical and demographic factors such as age of injury, current age, sex, and premorbid status. Clinically and demographically heterogeneous samples of n~40–50 subjects do not have the resolution necessary to determine when hyperconnectivity occurs and when it may give way to disconnection (see Caeyenberghs et al., 2017 ; Hillary and Grafman, 2017 ). Data sharing and team science organized to test strong hypotheses can provide clarity to this literature.

Harnessing big data to advance metascience

Metascience ( Peterson and Panofsky, 2014 ) has become central to many of the issues raised here. Metascience uses the tools of science to describe and evaluate science on a macro scale and to motivate reforms in scientific practice ( Munafò et al., 2017 ; Ioannidis et al., 2015 ; Gurevitch et al., 2018 ). The emergence of metascience is at least partially attributable to advances in web search and indexing, network science, natural language processing, and computational modeling. Amongst other aims, work under this umbrella has sought to diagnose biases in research practice ( Larivière et al., 2013 ; Clauset et al., 2015 ; Huang et al., 2020 ), understand how researchers select new work to pursue ( Rzhetsky et al., 2015 ; Jia et al., 2020 ), identify factors contributing to academic productivity ( Liu et al., 2018 ; Li et al., 2018 ; Pluchino et al., 2019 ; Janosov et al., 2020 ), and forecast the emergence of new areas of research ( Prabhakaran et al., 1959 ; Asooja et al., 2016 ; Salatino et al., 2018 ; Chen et al., 2017 ; Krenn and Zeilinger, 2020 ; Behrouzi et al., 2020 ).

A newer thread of ongoing efforts within the metascience community is working to build and promote infrastructure for reproducible and transparent scholarly communication (see Konkol et al., 2020 for a recent review, Wilkinson et al., 2016 ; Nosek et al., 2015 ). As part of this vision, primary deliverables of research processes include machine-readable outputs that can be queried by researchers for meta-analyses and theory development ( Priem, 2013 ; Lakens and DeBruine, 2021 ; Brinckman et al., 2019 ). These efforts are coupled with recent major investments in approaches to further automate research synthesis and hypothesis generation. The Big Mechanism program, for example, was set up by the Defense Advanced Research Projects Agency (DARPA) to fund the development of technologies to read the cancer biology literature, extract fragments of causal mechanisms from publications, assemble these mechanisms into executable models, and use these models to explain and predict new findings, and even test these predictions ( Cohen, 2015 ).

Lines of research have also emerged using creative assembly of experts (e.g., prediction markets; Dreber et al., 2015 ; Camerer et al., 2016 ; Camerer et al., 2018 ; Gordon et al., 2020 and AI-driven approaches Altmejd et al., 2019 ; Pawel and Held, 2020 ; Yang et al., 2020 ) to estimate confidence in specific research hypotheses and findings. These too have been facilitated by advances in information extraction, natural language processing, machine learning, and larger training datasets. The DARPA-funded Systematizing Confidence in Open Research and Evidence (SCORE) program, for example, is nearing the end of coordinated three-year long effort to develop technologies to predict and explain replicability, generalizability and robustness of published claims in the social and behavioral sciences literatures ( Alipourfard et al., 2012 ). As it continues to advance, the metascience community may serve to revolutionize the research process resulting in a literature that is readily interrogated and upon which strong hypotheses can be built.

Falsification for scaffolding convergence research

Advances in computing hold the promise of richer datasets, AI-driven meta-analyses, and even automated hypothesis generation. However, thus far, efforts to harness big data and emerging technologies for falsification and replication have been relatively uncoordinated, with the aforementioned Big Mechanism and SCORE programs amongst a few notable exceptions.

The need to refine theories becomes increasingly apparent when confronted with resource, ethical, and practical constraints that limit what can be further pursued empirically. At the same time, addressing pressing societal needs requires innovation and convergence research. An example are calls for “grand challenges”, a family of initiatives focused on tackling daunting unsolved problems with large investments intended to make an applied impact. These targeted investments tend to lead to a proliferation of science; however, these mechanisms could also incorporate processes to refine and interrogate theories as they progress towards addressing a specific and compelling issue. A benefit of incorporating falsification into this pipeline is that it encourages differing points of view, a desired feature of grand challenges ( Helbing, 2012 ) and other translational research programs. For example, including clinical researchers in the design of experiments being conducted at the preclinical stage can strengthen the quality of hypotheses before testing them to potentially increase the utility of the result, regardless of the outcome ( Seyhan, 2019 ). To realize the full potential, investment in developing and maturing computational models is also needed to leverage the sea of scientific data to help identify the level of confidence in the fitness and replicability of each theory, and where best to deploy resources. This could lead to more rapid theory refinements and greater feedback for what new data to collect than would be possible using hypothesis-driven or data-intensive approaches in isolation ( Peters et al., 2014 ).

Practical challenges to falsification

We have proposed that falsification of strong hypothesis provides a mechanism to increase study reliability. High volume science should ideally function to eliminate possible explanations, otherwise productivity obfuscates progress. But can falsification ultimately achieve this goal? A strict Popperian approach, that every observation represents either a confirmation or refutation of a hypothesis, is challenging to implement in day-to-day scientific practice ( Lakatos, 1970 ; Kuhn, 1970 ). What’s more, one cannot, with complete certainty, disprove a hypothesis any more than one can hope to prove a hypothesis (see Lakatos, 1970 ). It was Popper who emphasized that truth is ephemeral and even when it can be accessed, it remains provisional ( Popper, 1959 ).

The philosophical dilemma in establishing the “true” nature of a scientific finding is reflected in the pragmatic challenges facing replication science. Even after an effort to replicate a finding, when investigators are presented with the results and asked if the replication was a success, the outcome is often disagreement resulting in “intellectual gridlock” ( Nosek and Errington, 2020b ). So, if the goal to falsify a hypothesis is both practically and philosophically flawed, why the emphasis? The answer is that, while falsification cannot remove the foibles of human nature, systematic methodological error, and noise from the scientific process, by setting our sights on testing and refuting strong a priori hypotheses we may uncover the shortcomings to our explanations. Attempts to falsify through refutation cannot be definitive but the outcome of multiple efforts can critically inform the direction of a science ( Earp and Trafimow, 2015 ) when formally integrated into the scientific process (as depicted in Figure 2 ).

Finally, falsification alone serves as an incomplete response to problems of scientific reliability but becomes a powerful tool when combined with efforts that maximize transparency in method, make null results available, facilitate data/code sharing, and increase the incentive structures for investigators to refocus on open and transparent science.

Due to several factors including a high-volume science culture and previously unavailable computational resources, the empirical sciences have never been more productive. This unparalleled productivity invites questions about the rigor and direction of science and, ultimately, how these efforts translate to scientific advancement. We have proposed that it should be a primary goal to identify the “ground truths” that can serve as a foundation for more deliberate study and, to do so, there must be greater emphasis on testing and refuting strong hypotheses. The falsification of strong hypotheses enhances the power of replication first by pruning options and identifying the most promising hypotheses including possible mechanisms. When conducted through a team science framework, the endeavor leverages shared datasets that allow us to address heterogeneity that makes so many findings tentative. We must take steps toward more transparent and open science including – and most importantly – study pre-registration of strong hypotheses. The ultimate goal is to harness the rapid advancements in big data, computational power, and strong, well-defined theory with the goal to accelerate science.

Biographies

Sarah M Rajtmajer is in the College of Information Sciences and Technology, The Pennsylvania State University, University Park, United States

Timothy M Errington is at the Center for Open Science, Charlottesville, United States

Frank G Hillary is in the Department of Psychology and the Social Life and Engineering Sciences Imaging Center, The Pennsylvania State University, University Park, United States

Funding Statement

No external funding was received for this work.

Contributor Information

Peter Rodgers, eLife, United Kingdom .

Additional information

No competing interests declared.

Writing – original draft, Writing – review and editing.

Writing – review and editing.

Conceptualization, Writing – original draft, Writing – review and editing.

Data availability

• Alipourfard N, Arendt B, Benjamin DM, Benkler N, Bishop MM, Burstein M, Bush M, Caverlee J, Chen Y, Clark C, Dreber A, Errington TM, Fidler F, Fox NW, Frank A, Fraser H, Friedman S, Gelman B, Gentile J, Giles CL, Gordon MB, Gordon-Sarney R, Griffin C, Gulden T, Hahn K, Hartman R, Holzmeister F, Hu XB, Johannesson M, Kezar L, Kline Struhl M, Kuter U, Kwasnica AM, Lee DH, Lerman K, Liu Y, Loomas Z, Luis B, Magnusson I, Miske O, Mody F, Morstatter F, Nosek BA, Parsons ES, Pennock D, Pfeiffer T, Pujara J, Rajtmajer S, Ren X, Salinas A, Selvam RK, Shipman F, Silverstein P, Sprenger A, Squicciarini AM, Stratman S, Sun K, Tikoo S, Twardy CR, Tyner A, Viganola D, Wang J, Wilkinson DP, Wintle B, Wu J. Systematizing Confidence in Open Research and Evidence (SCORE) SocArXiv. 2012 https://osf.io/preprints/socarxiv/46mnb
• Altmejd A, Dreber A, Forsell E, Huber J, Imai T, Johannesson M, Kirchler M, Nave G, Camerer C, Wicherts JM. Predicting the replicability of social science lab experiments. PLOS ONE. 2019; 14 :e0225826. doi: 10.1371/journal.pone.0225826. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Andrews GE. Drowning in the data deluge. Notices of the American Mathematical Society. 2012; 59 :933. doi: 10.1090/noti871. [ CrossRef ] [ Google Scholar ]
• Asooja K, Bordea G, Vulcu G, Buitelaar P. Forecasting emerging trends from scientific literature. Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016.2016. [ Google Scholar ]
• Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 2016; 533 :452–454. doi: 10.1038/533452a. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bartley WW. The philosophy of Karl Popper. Philosophia. 1978; 7 :675–716. doi: 10.1007/BF02378843. [ CrossRef ] [ Google Scholar ]
• Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature. 2012; 483 :531–533. doi: 10.1038/483531a. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Behrouzi S, Shafaeipour Sarmoor Z, Hajsadeghi K, Kavousi K. Predicting scientific research trends based on link prediction in keyword networks. Journal of Informetrics. 2020; 14 :101079. doi: 10.1016/j.joi.2020.101079. [ CrossRef ] [ Google Scholar ]
• Benedict RHB, Zivadinov R. Risk factors for and management of cognitive dysfunction in multiple sclerosis. Nature Reviews Neurology. 2011; 7 :332–342. doi: 10.1038/nrneurol.2011.61. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bennett SD, Cuijpers P, Ebert DD, McKenzie Smith M, Coughtrey AE, Heyman I, Manzotti G, Shafran R. Practitioner review: unguided and guided self-help interventions for common mental health disorders in children and adolescents: A systematic review and meta-analysis. Journal of Child Psychology and Psychiatry, and Allied Disciplines. 2019; 60 :828–847. doi: 10.1111/jcpp.13010. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bharath RD, Munivenkatappa A, Gohel S, Panda R, Saini J, Rajeswaran J, Shukla D, Bhagavatula ID, Biswal BB. Recovery of resting brain connectivity ensuing mild traumatic brain injury. Frontiers in Human Neuroscience. 2015; 9 :513. doi: 10.3389/fnhum.2015.00513. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bollier D, Firestone CM. The promise and peril of big data Aspen Institute, Communications and Society Program. 2010. [August 2, 2022]. https://www.aspeninstitute.org/publications/promise-peril-big-data/
• Bonnelle V, Leech R, Kinnunen KM, Ham TE, Beckmann CF, De Boissezon X, Greenwood RJ, Sharp DJ. Default mode network connectivity predicts sustained attention deficits after traumatic brain injury. Journal of Neuroscience. 2011; 31 :13442–13451. doi: 10.1523/JNEUROSCI.1163-11.2011. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bornmann L, Mutz R. Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references. Journal of the Association for Information Science and Technology. 2015; 66 :2215–2222. doi: 10.1002/asi.23329. [ CrossRef ] [ Google Scholar ]
• Bouthillier X, Laurent C, Vincent P. Unreproducible research is reproducible. International Conference on Machine Learning PMLR.2019. [ Google Scholar ]
• Brinckman A, Chard K, Gaffney N, Hategan M, Jones MB, Kowalik K, Kulasekaran S, Ludäscher B, Mecum BD, Nabrzyski J, Stodden V, Taylor IJ, Turk MJ, Turner K. Computing environments for reproducibility: capturing the “whole tale.” Future Generation Computer Systems. 2019; 94 :854–867. doi: 10.1016/j.future.2017.12.029. [ CrossRef ] [ Google Scholar ]
• Broad WJ. The publishing game: Getting more for less. Science. 1981; 211 :1137–1139. doi: 10.1126/science.7008199. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bryan CJ, Tipton E, Yeager DS. Behavioural science is unlikely to change the world without a heterogeneity revolution. Nature Human Behaviour. 2021; 5 :980–989. doi: 10.1038/s41562-021-01143-3. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, Munafò MR. Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience. 2013; 14 :365–376. doi: 10.1038/nrn3475. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Caeyenberghs K, Verhelst H, Clemente A, Wilson PH. Mapping the functional connectome in traumatic brain injury: What can graph metrics tell us? NeuroImage. 2017; 160 :113–123. doi: 10.1016/j.neuroimage.2016.12.003. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Calude CS, Longo G. The deluge of spurious correlations in big data. Foundations of Science. 2017; 22 :595–612. doi: 10.1007/s10699-016-9489-4. [ CrossRef ] [ Google Scholar ]
• Camerer CF, Dreber A, Forsell E, Ho TH, Huber J, Johannesson M, Kirchler M, Almenberg J, Altmejd A, Chan T, Heikensten E, Holzmeister F, Imai T, Isaksson S, Nave G, Pfeiffer T, Razen M, Wu H. Evaluating replicability of laboratory experiments in economics. Science. 2016; 351 :1433–1436. doi: 10.1126/science.aaf0918. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Camerer CF, Dreber A, Holzmeister F, Ho TH, Huber J, Johannesson M, Kirchler M, Nave G, Nosek BA, Pfeiffer T, Altmejd A, Buttrick N, Chan T, Chen Y, Forsell E, Gampa A, Heikensten E, Hummer L, Imai T, Isaksson S, Manfredi D, Rose J, Wagenmakers EJ, Wu H. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour. 2018; 2 :637–644. doi: 10.1038/s41562-018-0399-z. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Carmon J, Heege J, Necus JH, Owen TW, Pipa G, Kaiser M, Taylor PN, Wang Y. Reliability and comparability of human brain structural covariance networks. NeuroImage. 2020; 220 :117104. doi: 10.1016/j.neuroimage.2020.117104. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Chen C, Wang Z, Li W, Sun X. Modeling scientific influence for research trending topic prediction. Proceedings of the AAAI Conference on Artificial Intelligence; 2017. [ CrossRef ] [ Google Scholar ]
• Chu JSG, Evans JA. Slowed canonical progress in large fields of science. PNAS. 2021; 118 :e2021636118. doi: 10.1073/pnas.2021636118. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Clauset A, Arbesman S, Larremore DB. Systematic inequality and hierarchy in faculty hiring networks. Science Advances. 2015; 1 :e1400005. doi: 10.1126/sciadv.1400005. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cohen PR. DARPA’s Big Mechanism program. Physical Biology. 2015; 12 :045008. doi: 10.1088/1478-3975/12/4/045008. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Corbett D, Carmichael ST, Murphy TH, Jones TA, Schwab ME, Jolkkonen J, Clarkson AN, Dancause N, Weiloch T, Johansen-Berg H, Nilsson M, McCullough LD, Joy MT. Enhancing the alignment of the preclinical and clinical stroke recovery research pipeline: Consensus-based core recommendations from the Stroke Recovery and Rehabilitation Roundtable Translational Working Group. Neurorehabilitation and Neural Repair. 2017; 31 :699–707. doi: 10.1177/1545968317724285. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cwiek A, Rajtmajer SM, Wyble B, Honavar V, Grossner E, Hillary FG. Feeding the machine: challenges to reproducible predictive modeling in resting-state connectomics. Network Neuroscience. 2021; 1 :1–20. doi: 10.1162/netn_a_00212. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Dijstelbloem H, Miedema F, Huisman F, Mijnhardt W. Position paper: Why science does not work as it should and what to do about it. 2013. [August 2, 2022]. http://scienceintransition.nl/app/uploads/2013/10/Science-in-Transition-Position-Paper-final.pdf
• Dreber A, Pfeiffer T, Almenberg J, Isaksson S, Wilson B, Chen Y, Nosek BA, Johannesson M, Wachter KW. Using prediction markets to estimate the reproducibility of scientific research. PNAS. 2015; 112 :15343–15347. doi: 10.1073/pnas.1516179112. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Earp BD, Trafimow D. Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology. 2015; 6 :621. doi: 10.3389/fpsyg.2015.00621. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Fagerholm ED, Hellyer PJ, Scott G, Leech R, Sharp DJ. Disconnection of network hubs and cognitive impairment after traumatic brain injury. Brain: A Journal of Neurology. 2015; 138 :1696–1709. doi: 10.1093/brain/awv075. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Fortin JP, Parker D, Tunç B, Watanabe T, Elliott MA, Ruparel K, Roalf DR, Satterthwaite TD, Gur RC, Gur RE, Schultz RT, Verma R, Shinohara RT. Harmonization of multi-site diffusion tensor imaging data. NeuroImage. 2017; 161 :149–170. doi: 10.1016/j.neuroimage.2017.08.047. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gilbert DT, King G, Pettigrew S, Wilson TD. Comment on “Estimating the reproducibility of psychological science.” Science. 2016; 351 :1037. doi: 10.1126/science.aad7243. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gleeson M, Biddle S. Duplicate publishing and the least publishable unit. Journal of Sports Sciences. 2000; 18 :227–228. doi: 10.1080/026404100364956. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gordon M, Viganola D, Bishop M, Chen Y, Dreber A, Goldfedder B, Holzmeister F, Johannesson M, Liu Y, Twardy C, Wang J, Pfeiffer T. Are replication rates the same across academic fields? Community forecasts from the DARPA SCORE programme. Royal Society Open Science. 2020; 7 :200566. doi: 10.1098/rsos.200566. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Graham RL, Spencer JH. Ramsey theory. Scientific American. 1990; 263 :112–117. doi: 10.1038/scientificamerican0790-112. [ CrossRef ] [ Google Scholar ]
• Gurevitch J, Koricheva J, Nakagawa S, Stewart G. Meta-analysis and the science of research synthesis. Nature. 2018; 555 :175–182. doi: 10.1038/nature25753. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hallquist MN, Hillary FG. Graph theory approaches to functional network organization in brain disorders: A critique for brave new small-world. Network Neuroscience. 2019; 3 :1–26. doi: 10.1162/netn_a_00054. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Harris NG, Verley DR, Gutman BA, Thompson PM, Yeh HJ, Brown JA. Disconnection and hyper-connectivity underlie reorganization after TBI: A rodent functional connectomic analysis. Experimental Neurology. 2016; 277 :124–138. doi: 10.1016/j.expneurol.2015.12.020. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Helbing D. Accelerating scientific discovery by formulating grand scientific challenges. The European Physical Journal Special Topics. 2012; 214 :41–48. doi: 10.1140/epjst/e2012-01687-x. [ CrossRef ] [ Google Scholar ]
• Henderson VC, Kimmelman J, Fergusson D, Grimshaw JM, Hackam DG. Threats to validity in the design and conduct of preclinical efficacy studies: A systematic review of guidelines for in vivo animal experiments. PLOS Medicine. 2013; 10 :e1001489. doi: 10.1371/journal.pmed.1001489. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hillary FG. Neuroimaging of working memory dysfunction and the dilemma with brain reorganization hypotheses. Journal of the International Neuropsychological Society. 2008; 14 :526–534. doi: 10.1017/S1355617708080788. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hillary FG, Roman CA, Venkatesan U, Rajtmajer SM, Bajo R, Castellanos ND. Hyperconnectivity is a fundamental response to neurological disruption. Neuropsychology. 2015; 29 :59–75. doi: 10.1037/neu0000110. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hillary FG, Grafman JH. Injured brains and adaptive networks: the benefits and costs of hyperconnectivity. Trends in Cognitive Sciences. 2017; 21 :385–401. doi: 10.1016/j.tics.2017.03.003. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Huang J, Gates AJ, Sinatra R, Barabási AL. Historical comparison of gender inequality in scientific careers across countries and disciplines. PNAS. 2020; 117 :4609–4616. doi: 10.1073/pnas.1914221117. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ioannidis JPA, Fanelli D, Dunne DD, Goodman SN. Meta-research: Evaluation and improvement of research methods and practices. PLOoS Biology. 2015; 13 :e1002264. doi: 10.1371/journal.pbio.1002264. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ioannidis JPA, Klavans R, Boyack KW. Thousands of scientists publish a paper every five days. Nature. 2018; 561 :167–169. doi: 10.1038/d41586-018-06185-8. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Iraji A, Chen H, Wiseman N, Welch RD, O’Neil BJ, Haacke EM, Liu T, Kou Z. Compensation through functional hyperconnectivity: A longitudinal connectome assessment of mild traumatic brain injury. Neural Plasticity. 2016; 2016 :4072402. doi: 10.1155/2016/4072402. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Janosov M, Battiston F, Sinatra R. Success and luck in creative careers. EPJ Data Science. 2020; 9 :9. doi: 10.1140/epjds/s13688-020-00227-w. [ CrossRef ] [ Google Scholar ]
• Jia T, Wang D, Szymanski BK. Quantifying patterns of research-interest evolution. Nature Human Behaviour. 2020; 1 :0078. doi: 10.1038/s41562-017-0078. [ CrossRef ] [ Google Scholar ]
• Johnson B, Zhang K, Gay M, Horovitz S, Hallett M, Sebastianelli W, Slobounov S. Alteration of brain default network in subacute phase of injury in concussed individuals: Resting-state fMRI study. NeuroImage. 2012; 59 :511–518. doi: 10.1016/j.neuroimage.2011.07.081. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kiai A. To protect credibility in science, banish “publish or perish.” Nature Human Behaviour. 2019; 3 :1017–1018. doi: 10.1038/s41562-019-0741-0. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Konkol M, Nüst D, Goulier L. Publishing computational research - a review of infrastructures for reproducible and transparent scholarly communication. Research Integrity and Peer Review. 2020; 5 :10. doi: 10.1186/s41073-020-00095-y. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Krenn M, Zeilinger A. Predicting research trends with semantic and neural networks with an application in quantum physics. PNAS. 2020; 117 :1910–1916. doi: 10.1073/pnas.1914370116. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kuhn TS. In: Criticism and the Growth of Knowledge. Lakatos I, Musgrave A, editors. Cambridge: Cambridge University Press; 1970. Logic of discovery or psychology of research; pp. 1–23. [ CrossRef ] [ Google Scholar ]
• Lakatos I. History of science and its rational reconstructions. PSA. 1970; 1970 :91–136. doi: 10.1086/psaprocbienmeetp.1970.495757. [ CrossRef ] [ Google Scholar ]
• Lakens D, DeBruine LM. Improving transparency, falsifiability, and rigor by making hypothesis tests machine-readable. Advances in Methods and Practices in Psychological Science. 2021; 4 :251524592097094. doi: 10.1177/2515245920970949. [ CrossRef ] [ Google Scholar ]
• Larivière V, Ni C, Gingras Y, Cronin B, Sugimoto CR. Bibliometrics: Global gender disparities in science. Nature. 2013; 504 :211–213. doi: 10.1038/504211a. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lazic SE. Four simple ways to increase power without increasing the sample size. Laboratory Animals. 2018; 52 :621–629. doi: 10.1177/0023677218767478. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Li W, Aste T, Caccioli F, Livan G. Early coauthorship with top scientists predicts success in academic careers. Nature Communications. 2018; 10 :5170. doi: 10.1038/s41467-019-13130-4. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lindner MD, Torralba KD, Khan NA, Ouzounis CA. Scientific productivity: An exploratory study of metrics and incentives. PLOS ONE. 2018; 13 :e0195321. doi: 10.1371/journal.pone.0195321. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Liu L, Wang Y, Sinatra R, Giles CL, Song C, Wang D. Hot streaks in artistic, cultural, and scientific careers. Nature. 2018; 559 :396–399. doi: 10.1038/s41586-018-0315-8. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JPA, Al-Shahi Salman R, Chan AW, Glasziou P. Biomedical research: increasing value, reducing waste. Lancet. 2014; 383 :101–104. doi: 10.1016/S0140-6736(13)62329-6. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Maikusa N, Zhu Y, Uematsu A, Yamashita A, Saotome K, Okada N, Kasai K, Okanoya K, Yamashita O, Tanaka SC, Koike S. Comparison of traveling-subject and combat harmonization methods for assessing structural brain characteristics. Human Brain Mapping. 2021; 1 :5278–5287. doi: 10.1002/hbm.25615. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mayer AR, Mannell MV, Ling J, Gasparovic C, Yeo RA. Functional connectivity in mild traumatic brain injury. Human Brain Mapping. 2011; 32 :1825–1835. doi: 10.1002/hbm.21151. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Miller GA, Chapman JP. Misunderstanding analysis of covariance. Journal of Abnormal Psychology. 2001; 110 :40–48. doi: 10.1037//0021-843x.110.1.40. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Munafò MR, Nosek BA, Bishop DVM, Button KS, Chambers CD, du Sert NP, Simonsohn U, Wagenmakers EJ, Ware JJ, Ioannidis JPA. A manifesto for reproducible science. Nature Human Behaviour. 2017; 1 :0021. doi: 10.1038/s41562-016-0021. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nakamura T, Hillary FG, Biswal BB. Resting network plasticity following brain injury. PLOS ONE. 2009; 4 :e8220. doi: 10.1371/journal.pone.0008220. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• National Science Foundation Computational and Data-enabled Science and Engineering. 2010. [August 2, 2022]. http://www.nsf.gov/mps/cds-e/
• Nosek BA, Alter G, Banks GC, Borsboom D, Bowman SD, Breckler SJ, Buck S, Chambers CD, Chin G, Christensen G, Contestabile M, Dafoe A, Eich E, Freese J, Glennerster R, Goroff D, Green DP, Hesse B, Humphreys M, Ishiyama J, Karlan D, Kraut A, Lupia A, Mabry P, Madon T, Malhotra N, Mayo-Wilson E, McNutt M, Miguel E, Paluck EL, Simonsohn U, Soderberg C, Spellman BA, Turitto J, VandenBos G, Vazire S, Wagenmakers EJ, Wilson R, Yarkoni T. Promoting an open research culture. Science. 2015; 348 :1422–1425. doi: 10.1126/science.aab2374. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nosek BA, Errington TM. What is replication? PLOS Biology. 2020a; 18 :e3000691. doi: 10.1371/journal.pbio.3000691. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nosek BA, Errington TM. The best time to argue about what a replication means? Before you do it. Nature. 2020b; 583 :518–520. doi: 10.1038/d41586-020-02142-6. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Olsen A, Babikian T, Bigler ED, Caeyenberghs K, Conde V, Dams-O’Connor K, Dobryakova E, Genova H, Grafman J, Håberg AK, Heggland I, Hellstrøm T, Hodges CB, Irimia A, Jha RM, Johnson PK, Koliatsos VE, Levin H, Li LM, Lindsey HM, Livny A, Løvstad M, Medaglia J, Menon DK, Mondello S, Monti MM, Newcombe VFJ, Petroni A, Ponsford J, Sharp D, Spitz G, Westlye LT, Thompson PM, Dennis EL, Tate DF, Wilde EA, Hillary FG. Toward a global and reproducible science for brain imaging in neurotrauma: the ENIGMA adult moderate/severe traumatic brain injury working group. Brain Imaging and Behavior. 2021; 15 :526–554. doi: 10.1007/s11682-020-00313-7. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Open Science Collaboration Estimating the reproducibility of psychological science. Science. 2015; 349 :aac4716. doi: 10.1126/science.aac4716. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Pawel S, Held L. Probabilistic forecasting of replication studies. PLOS ONE. 2020; 15 :e0231416. doi: 10.1371/journal.pone.0231416. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Peters DPC, Havstad KM, Cushing J, Tweedie C, Fuentes O, Villanueva-Rosales N. Harnessing the power of big data: infusing the scientific method with machine learning to transform ecology. Ecosphere. 2014; 5 :art67. doi: 10.1890/ES13-00359.1. [ CrossRef ] [ Google Scholar ]
• Peterson D, Panofsky DPA. Metascience as a Scientific Social Movement. SocArXiv. 2014 https://osf.io/preprints/socarxiv/4dsqa/
• Pineau J. Improving reproducibility in machine learning research: a report from the neurips 2019 reproducibility program. Journal of Machine Learning Research. 2021; 22 :1–20. [ Google Scholar ]
• Pluchino A, Burgio G, Rapisarda A, Biondo AE, Pulvirenti A, Ferro A, Giorgino T. Exploring the role of interdisciplinarity in physics: success, talent and luck. PLOS ONE. 2019; 14 :e0218793. doi: 10.1371/journal.pone.0218793. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Popper KR. The Logic of Scientific Discovery. Julius Springer, Hutchinson & Co; 1959. [ Google Scholar ]
• Popper K. Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge; 1963. [ Google Scholar ]
• Pound P, Ritskes-Hoitinga M. Is it possible to overcome issues of external validity in preclinical animal research? Why most animal models are bound to fail. Journal of Translational Medicine. 2018; 16 :304. doi: 10.1186/s12967-018-1678-1. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Prabhakaran V, Hamilton WL, McFarland D, Jurafsky D. Predicting the Rise and Fall of Scientific Topics from Trends in their Rhetorical Framing. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics; 1959. [ CrossRef ] [ Google Scholar ]
• Priem J. Beyond the paper. Nature. 2013; 495 :437–440. doi: 10.1038/495437a. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Priestley D, Staph J, Koneru S, Rajtmajer S, Hillary F. Establishing Ground Truth in the Clinical Neurosciences: If Replication Is the Answer, Then What Are the Questions? PsyArXiv. 2022 https://psyarxiv.com/rb32d/
• Rodgers JL, Shrout PE. Psychology’s replication crisis as scientific opportunity: A précis for policymakers. Policy Insights from the Behavioral and Brain Sciences. 2018; 5 :134–141. doi: 10.1177/2372732217749254. [ CrossRef ] [ Google Scholar ]
• Rzhetsky A, Foster JG, Foster IT, Evans JA. Choosing experiments to accelerate collective discovery. PNAS. 2015; 112 :14569–14574. doi: 10.1073/pnas.1509757112. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Salatino AA, Osborne F, Motta E. AUGUR: Forecasting the Emergence of New Research Topics. Proceedings of the ACM/IEEE Joint Conference on Digital Libraries; 2018. [ CrossRef ] [ Google Scholar ]
• Sandström U, van den Besselaar P. Quantity and/or quality? The importance of publishing many papers. PLOS ONE. 2016; 11 :e0166149. doi: 10.1371/journal.pone.0166149. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Satterthwaite TD, Wolf DH, Loughead J, Ruparel K, Elliott MA, Hakonarson H, Gur RC, Gur RE. Impact of in-scanner head motion on multiple measures of functional connectivity: relevance for studies of neurodevelopment in youth. NeuroImage. 2012; 60 :623–632. doi: 10.1016/j.neuroimage.2011.12.063. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schrag A, Zhelev SS, Hotham S, Merritt RD, Khan K, Graham L. Heterogeneity in progression of prodromal features in Parkinson’s disease. Parkinsonism & Related Disorders. 2019; 64 :275–279. doi: 10.1016/j.parkreldis.2019.05.013. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schwamm LH. Progesterone for traumatic brain injury--resisting the sirens’ song. The New England Journal of Medicine. 2014; 371 :2522–2523. doi: 10.1056/NEJMe1412951. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Seyhan AA. Lost in translation: the valley of death across preclinical and clinical divide – identification of problems and overcoming obstacles. Translational Medicine Communications. 2019; 4 :18. doi: 10.1186/s41231-019-0050-7. [ CrossRef ] [ Google Scholar ]
• Sharp DJ, Beckmann CF, Greenwood R, Kinnunen KM, Bonnelle V, De Boissezon X, Powell JH, Counsell SJ, Patel MC, Leech R. Default mode network functional and structural connectivity after traumatic brain injury. Brain: A Journal of Neurology. 2011; 134 :2233–2247. doi: 10.1093/brain/awr175. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sharp DJ, Scott G, Leech R. Network dysfunction after traumatic brain injury. Nature Reviews Neurology. 2014; 10 :156–166. doi: 10.1038/nrneurol.2014.15. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Storandt M, Hudson W. Misuse of analysis of covariance in aging research and some partial solutions. Experimental Aging Research. 1975; 1 :121–125. doi: 10.1080/03610737508257953. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Szucs D, Ioannidis JPA. When null hypothesis significance testing is unsuitable for research: A reassessment. Frontiers in Human Neuroscience. 2017; 11 :390. doi: 10.3389/fnhum.2017.00390. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Tate DF, Dennis EL, Adams JT, Adamson MM, Belanger HG, Bigler ED, Bouchard HC, Clark AL, Delano-Wood LM, Disner SG, Eapen BC, Franz CE, Geuze E, Goodrich-Hunsaker NJ, Han K, Hayes JP, Hinds SR, Hodges CB, Hovenden ES, Irimia A, Kenney K, Koerte IK, Kremen WS, Levin HS, Lindsey HM, Morey RA, Newsome MR, Ollinger J, Pugh MJ, Scheibel RS, Shenton ME, Sullivan DR, Taylor BA, Troyanskaya M, Velez C, Wade BS, Wang X, Ware AL, Zafonte R, Thompson PM, Wilde EA. Coordinating global multi-site studies of military-relevant traumatic brain injury: opportunities, challenges, and harmonization guidelines. Brain Imaging and Behavior. 2021; 15 :585–613. doi: 10.1007/s11682-020-00423-2. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Tedersoo L, Küngas R, Oras E, Köster K, Eenmaa H, Leijen Ä, Pedaste M, Raju M, Astapova A, Lukner H, Kogermann K, Sepp T. Data sharing practices and data availability upon request differ across scientific disciplines. Scientific Data. 2021; 8 :192. doi: 10.1038/s41597-021-00981-0. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Thompson PM, Jahanshad N, Ching CRK, Salminen LE, Thomopoulos SI, Bright J, Baune BT, Bertolín S, Bralten J, Bruin WB, Bülow R, Chen J, Chye Y, Dannlowski U, de Kovel CGF, Donohoe G, Eyler LT, Faraone SV, Favre P, Filippi CA, Frodl T, Garijo D, Gil Y, Grabe HJ, Grasby KL, Hajek T, Han LKM, Hatton SN, Hilbert K, Ho TC, Holleran L, Homuth G, Hosten N, Houenou J, Ivanov I, Jia T, Kelly S, Klein M, Kwon JS, Laansma MA, Leerssen J, Lueken U, Nunes A, Neill JO, Opel N, Piras F, Piras F, Postema MC, Pozzi E, Shatokhina N, Soriano-Mas C, Spalletta G, Sun D, Teumer A, Tilot AK, Tozzi L, van der Merwe C, Van Someren EJW, van Wingen GA, Völzke H, Walton E, Wang L, Winkler AM, Wittfeld K, Wright MJ, Yun JY, Zhang G, Zhang-James Y, Adhikari BM, Agartz I, Aghajani M, Aleman A, Althoff RR, Altmann A, Andreassen OA, Baron DA, Bartnik-Olson BL, Marie Bas-Hoogendam J, Baskin-Sommers AR, Bearden CE, Berner LA, Boedhoe PSW, Brouwer RM, Buitelaar JK, Caeyenberghs K, Cecil CAM, Cohen RA, Cole JH, Conrod PJ, De Brito SA, de Zwarte SMC, Dennis EL, Desrivieres S, Dima D, Ehrlich S, Esopenko C, Fairchild G, Fisher SE, Fouche JP, Francks C, Frangou S, Franke B, Garavan HP, Glahn DC, Groenewold NA, Gurholt TP, Gutman BA, Hahn T, Harding IH, Hernaus D, Hibar DP, Hillary FG, Hoogman M, Hulshoff Pol HE, Jalbrzikowski M, Karkashadze GA, Klapwijk ET, Knickmeyer RC, Kochunov P, Koerte IK, Kong XZ, Liew SL, Lin AP, Logue MW, Luders E, Macciardi F, Mackey S, Mayer AR, McDonald CR, McMahon AB, Medland SE, Modinos G, Morey RA, Mueller SC, Mukherjee P, Namazova-Baranova L, Nir TM, Olsen A, Paschou P, Pine DS, Pizzagalli F, Rentería ME, Rohrer JD, Sämann PG, Schmaal L, Schumann G, Shiroishi MS, Sisodiya SM, Smit DJA, Sønderby IE, Stein DJ, Stein JL, Tahmasian M, Tate DF, Turner JA, van den Heuvel OA, van der Wee NJA, van der Werf YD, van Erp TGM, van Haren NEM, van Rooij D, van Velzen LS, Veer IM, Veltman DJ, Villalon-Reina JE, Walter H, Whelan CD, Wilde EA, Zarei M, Zelman V, ENIGMA Consortium ENIGMA and global neuroscience: A decade of large-scale studies of the brain in health and disease across more than 40 countries. Translational Psychiatry. 2020; 10 :100. doi: 10.1038/s41398-020-0705-1. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Thompson PM, Jahanshad N, Schmaal L, Turner JA, Winkler AM, Thomopoulos SI, Egan GF, Kochunov P. The Enhancing NeuroImaging Genetics through Meta-Analysis Consortium: 10 years of global collaborations in human brain mapping. Human Brain Mapping. 2022; 43 :15–22. doi: 10.1002/hbm.25672. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Tolchin B, Conwit R, Epstein LG, Russell JA, on behalf of the Ethics, Law, and Humanities Committee a joint committee of the American Academy of Neurology, American Neurological Association, and Child Neurology Society AAN position statement: ethical issues in clinical research in neurology. Neurology. 2020; 94 :661–669. doi: 10.1212/WNL.0000000000009241. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Van Dijk KRA, Sabuncu MR, Buckner RL. The influence of head motion on intrinsic functional connectivity MRI. NeuroImage. 2012; 59 :431–438. doi: 10.1016/j.neuroimage.2011.07.044. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Vesalius A. De Humani Corporis Fabrica (Of the Structure of the Human Body) Basel: Johann Oporinus; 1555. [ Google Scholar ]
• Watts DJ. Should social science be more solution-oriented? Nature Human Behaviour. 2017; 1 :0015. doi: 10.1038/s41562-016-0015. [ CrossRef ] [ Google Scholar ]
• Wilkinson MD, Dumontier M, Aalbersberg IJJ, Appleton G, Axton M, Baak A, Blomberg N, Boiten JW, da Silva Santos LB, Bourne PE, Bouwman J, Brookes AJ, Clark T, Crosas M, Dillo I, Dumon O, Edmunds S, Evelo CT, Finkers R, Gonzalez-Beltran A, Gray AJG, Groth P, Goble C, Grethe JS, Heringa J, ’t Hoen PAC, Hooft R, Kuhn T, Kok R, Kok J, Lusher SJ, Martone ME, Mons A, Packer AL, Persson B, Rocca-Serra P, Roos M, van Schaik R, Sansone SA, Schultes E, Sengstag T, Slater T, Strawn G, Swertz MA, Thompson M, van der Lei J, van Mulligen E, Velterop J, Waagmeester A, Wittenburg P, Wolstencroft K, Zhao J, Mons B. The FAIR guiding principles for scientific data management and stewardship. Scientific Data. 2016; 3 :160018. doi: 10.1038/sdata.2016.18. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Yang Y, Youyou W, Uzzi B. Estimating the deep replicability of scientific findings using human and artificial intelligence. PNAS. 2020; 117 :10762–10768. doi: 10.1073/pnas.1909046117. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Yeates KO, Tang K, Barrowman N, Freedman SB, Gravel J, Gagnon I, Sangha G, Boutis K, Beer D, Craig W, Burns E, Farion KJ, Mikrogianakis A, Barlow K, Dubrovsky AS, Meeuwisse W, Gioia G, Meehan WP, Beauchamp MH, Kamil Y, Grool AM, Hoshizaki B, Anderson P, Brooks BL, Vassilyadi M, Klassen T, Keightley M, Richer L, DeMatteo C, Osmond MH, Zemek R, Pediatric Emergency Research Canada (PERC) Predicting Persistent Postconcussive Problems in Pediatrics (5P) Concussion Team Derivation and initial validation of clinical phenotypes of children presenting with concussion acutely in the emergency department: Latent class analysis of a multi-center, prospective cohort, observational study. Journal of Neurotrauma. 2019; 36 :1758–1767. doi: 10.1089/neu.2018.6009. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Zalc B. One hundred and fifty years ago Charcot reported multiple sclerosis as a new neurological disease. Brain: A Journal of Neurology. 2018; 141 :3482–3488. doi: 10.1093/brain/awy287. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Zinbarg RE, Suzuki S, Uliaszek AA, Lewis AR. Biased parameter estimates and inflated type I error rates in analysis of covariance (and analysis of partial variance) arising from unreliability: Alternatives and remedial strategies. Journal of Abnormal Psychology. 2010; 119 :307–319. doi: 10.1037/a0017552. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Zucchella C, Mantovani E, Federico A, Lugoboni F, Tamburin S. Non-invasive brain stimulation for gambling disorder: A systematic review. Frontiers in Neuroscience. 2020; 14 :729. doi: 10.3389/fnins.2020.00729. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• eLife. 2022; 11: e78830.

Decision letter

In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.

Thank you for submitting your article "How Failure to Falsify in High Volume Science contributes to the Replication Crisis" to eLife for consideration as a Feature Article. Your article has been reviewed by four peer reviewers, and the evaluation has been overseen by a member of the eLife Features Team (Peter Rodgers). The following individuals involved in review of your submission have agreed to reveal their identity: Robert Thibault; Jean-Baptiste Poline; Nishant Sinha; and Yujiang Wang.

The reviewers and editors have discussed the reviews and we have drafted this decision letter to help you prepare a revised submission.

This manuscript is of broad interest to the scientific community and particularly engaging for the readers in neuroscience. As the number of scientific articles exploring new hypotheses grows exponentially every year, there have not been many attempts to select the most competent hypotheses while falsifying the others. The authors juxtapose two prevalent hypotheses in network neuroscience literature on brain injury as an example. The manuscript offers suggestions on selecting the most compelling hypothesis and directions for designing studies to falsify them. However, there are a number of points that need to be addressed to make the article suitable for publication.

Essential revisions:

1. Lines 55-62: Please give examples of strong and weak hypotheses. Please also explain why "hypotheses that are supported by a range of findings should be considered weak". You might also want to consider linking strong/weak hypotheses to the idea of "research questions", which can be broad and vague, as compared to "hypotheses" which should be more precise.

2. Lines 120-122: Are there any systematic reviews on this topic that use one (or several) strong hypotheses and then check if the various studies support the stronger hypothesis? (e.g., does area X change within timeframe Y, in Z types of patients)? Also, I don't think the issue here is that they lack power to refute hypotheses, but instead that this lack of power is coupled with many statistical tests, leading to false positives that are used to support weak hypotheses (e.g. that brain activity "changes").

3. Lines 230-232: This paragraph could benefit from having a concrete example of a strong hypothesis and a matching weak hypothesis. This will likely help the reader grasp the issue more concretely.

4. Line 261: I am confused by the statement "sample heterogeneity requires large diverse samples". What part of heterogeneity are you trying to address? If testing a treatment that will be deployed to a diverse population of TBIs, I get that you would want a diverse sample, but a diverse sample will increase heterogeneity.

5. Lines 280-282: Or…it might be almost all noise? If we take Button's 2013 finding of 8% power in neuroimaging studies, then differences in findings are more likely due to noise than to demographics or other characteristics.

6. One issue to consider is that the hypotheses considered in the TBI example seem to be too vague or formulated in a too general manner.

7. The interaction between publication incentivization and weak hypotheses is alluded to but leaves the reader unclear on the topic; please say more about this.

8. As noted in the last part of the paper, one of the common tool to limit false discovery is pre-registration. It is unclear why this is not emphasized in the core of the text, rather than lately, as a tool to *review* hypotheses.

9. The social mechanisms to lead us to "team science" are unclear – if this article is meant to help the research community to move in this specific direction, a practical path should be proposed, as "modification of mindset" is certainly more a goal than a practical approach.

10. Specifically, framing all of scientific research in neuroscience in terms of hypotheses that can be confirmed or refuted is limiting; and as the authors acknowledged, the answer might be not a clear yes/no. Please consider discussing how it is sometimes more productive to revise aspects of a particular hypothesis. Nevertheless, I agree with the underlying sentiment that a key to replication is being able to receive recognition for the effort and rigour rather than the outcome.

11. As a computational researcher I would also like to see a stronger emphasis on both:

i) how data, code, and informatics are creating some of the replication crisis;

ii) how stronger informatics frameworks may be part of the solution.

Just an anecdote from personal experience and years of headache: Our lab has been trying to apply a specific networks neuroscience approach called "structural covariance networks". We initially applied it to test the local hyperconnection hypothesis in ASD. It took us years to realise and acknowledge that the computational method itself is very problematic in terms of being replicable even in data from the same scanner, site etc., and despite controlling for every biological variable imaginable. It took us another year, and a very talented student, to understand that it is the method itself that enhances the noise in the data in a very "unuseful" way thus drowning any biological effect. We finally could publish this insight here: https://pubmed.ncbi.nlm.nih.gov/32621973/ I use this example to highlight that our "problem" could not really be framed as a hypothesis refuting exercise, as the real insight for us, and I hope also for the community, was not whether the original hypothesis was right or wrong, but that our tool was flawed.

12. The caption for Figure 2 is confusing, and the content of (Priestley et al., under review) is not clear; please delete this figure and add one or two sentences to the text to say that the number of papers in [subject] has increased from about XX per year in 1970 to YYY per year in 2020.

Author response

Essential revisions: 1. Lines 55-62: Please give examples of strong and weak hypotheses. Please also explain why "hypotheses that are supported by a range of findings should be considered weak". You might also want to consider linking strong/weak hypotheses to the idea of "research questions", which can be broad and vague, as compared to "hypotheses" which should be more precise.

This is an important point and based upon this feedback we work to address this issue in text, we now have ~line 57 and we now include a Table with examples.

“In the work of falsification, the more specific and more refutable a hypothesis is, the stronger it is, and hypotheses that can be supported by different sets of findings should be considered weak (Popper, 1963; see Table 1 for example of hypotheses).”

2. Lines 120-122: Are there any systematic reviews on this topic that use one (or several) strong hypotheses and then check if the various studies support the stronger hypothesis? (e.g., does area X change within timeframe Y, in Z types of patients)?

The lack of direct examination of this problem in the hyperconnectivity literature was a primary impetus for the current review. There has been no systematic effort to hold these positions (hyperconnectivity v. disconnection) side-by-side to test them.

Also, I don't think the issue here is that they lack power to refute hypotheses, but instead that this lack of power is coupled with many statistical tests, leading to false positives that are used to support weak hypotheses (e.g. that brain activity "changes").

This is an outstanding point and we agree that the sheer volume of statistical tests in a number of studies increases the probability that findings by chance are published as significant and important. On line 121, we have modified this statement to reflect this point to read:

“Overall, the TBI connectomics literature presents a clear example of a failure to falsify and we argue that it is attributable at least in part by science-by-volume, where small samples are used to examine non-specific hypotheses. This scenario is further worsened using a number of statistical tests which increases the probability that spurious findings are cast as meaningful [40,95].”

This is an important point and one we discussed as a group prior to the initial submission. We agree completely that concrete examples help to understand the problem and have added several to the text and to Table 1 (line 63).

This is a good point and we have worked to clarify this statement. The point is that small samples do not allow investigation of the effects of sex, education, age, and other factors. Larger, more diverse samples permit direct modeling of these effects. This statement now reads (line 276):

“Addressing sample heterogeneity requires large diverse samples for direct modeling of influencing factors and one avenue to make this possible is data sharing.”

This is an interesting point, but there does appear to be a there, there. A number of higher powered studies do track changes in connectivity that appear to be directly related to pathophysiology and, importantly, correlate with behavior. However, one cannot deny that at least a subset of these studies presents results that capitalize upon spurious signal or noise.

The point made here by the Referee is not entirely clear. If this is a statement about the need for stronger hypotheses, we agree that greater specificity is needed and we hope to add context for this point by adding example hypotheses in Table 1. Alternatively, if the Reviewer aims to indicate that this is a TBI-specific phenomenon, that is also possible, though it is unclear why this would occur only in TBI within the clinical neurosciences.

This relationship is made more explicit with the passage on line 115:

“As opposed to pre-registering and testing strong hypotheses, investigators are compelled to identify significant results (any result) for publication. In brain injury work examining network plasticity, investigators have often made general claims that brain injury results in “different” or “altered” connectivity (a problem dating back to early fMRI studies in TBI; [Hillary, 2008]). While it is unlikely the intention, under-specified hypotheses increase the likelihood that chance findings are published. The primary consequence is that all findings are “winners”, permitting growing support for either position without movement toward resolution.”

This is an excellent point, and we now make clear the importance of study preregistration at the outset of the paper (line 55) so that when we return to it, there is context.

We appreciate this point and agree that this statement is confusing. We have removed this statement.

This is an important point. Part of the scientific process clearly requires revisions of our theory. As this reviewer alludes to, however, when we revise our hypotheses to fit our outcomes, we risk advancing hypotheses supported by spurious data. We see preregistration as part of the solution and based upon comments elsewhere in this critique, we have refocused on how preregistration can help not only in the development of strong hypotheses, but also in their modification. We also now include Table 1 to provide modern context for what might be considered a “falsifiable” hypothesis. We also include a section titled “Practical Challenges to Falsification” to make clear that falsification of strong hypotheses is one tool of many to improve our science.

11. As a computational researcher I would also like to see a stronger emphasis on both: i) how data, code, and informatics are creating some of the replication crisis; ii) how stronger informatics frameworks may be part of the solution.

We agree and now add a section to outline the parameters of this natural tension (see “Big Data as Friend and Foe”) line 146

We appreciate the reviewer sharing this illustrative example. It does add a dimension (weak hypothesis v. weak method) that requires recognition in this manuscript. I might additionally argue though that stronger hypothesis (including alternative hypotheses) place the investigator in a better position to detect flawed methodology. That is, truly spurious results may stand-out against sanity checks offered by strong hypotheses, but the point still stands that faulty methods contribute to problems of scientific reliability (something we allude to briefly at the outset with reference to Alipourfard et al., 2021). We now add comment on this on and references to examples for how methods/stats can lead to systematically flawed results (line 238). We now write:

“Strong hypotheses must be matched with methods that can provide clear tests, a coupling that cannot be overstated. In the brain imaging literature alone, there are poignant examples where flawed methods (or misunderstanding of their applications) has resulted in the repeated substantiation of spurious results (in structural covariance analysis see Carmen et al., 2021 and in resting-state fMRI see Satterthwaite et al., 2016; Van Dijk et al., 2012).”

We have accepted this recommendation and have deleted the figure and replaced it with statistics highlighting the annual increase in publication numbers.

Is Falsification Falsifiable?

• Published: 28 January 2015
• Volume 21 , pages 461–475, ( 2016 )

Cite this article

• Ulf Persson 1  

1968 Accesses

4 Citations

10 Altmetric

Explore all metrics

This is a response to a claim by Sven Ove Hansson to the effect that Poppers dictum that falsification lies at the heart of all pursuit of science has once and for all been falsified by his study of articles published in Nature during the year 2000. We claim that this is based on a misunderstanding of Poppers philosophy of science interpreting it too literally, and that alternative readings of those papers are fully compliant with falsification. We scrutinize Hansson’s arguments as well as giving an overview of Poppers falsification theory.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Falsificationism: In Defence of the Scientific Status of Marxism

Karl Popper: His Philosophy and Science

Falsificationism

In which he confuses the falsification of a fact with that of a theory!

Stating: The weakness of Popper’s argument is obvious. For scientists are not only interested in showing that certain theories are false...her arch-rival’s theory perhaps...But much more likely she is trying to convince people that her own theory is true. And goes on to claim that Popper purports to reject induction, which Okasha seems to confuse with verification.

Typically \(x\) which will ‘occur’ in the future and thus cannot be tested (verified) in the present. This accords a way of evading actual infinities.

Being rejected as scientific should only be seen as negative, when such claims are being central. Popper emphasizes that not being scientific does not mean being nonsense. In particular metaphysics could be valuable as an inspiration for a nascent science.

For Popper the content of a scientific theory is central, while Kuhn’s vision of paradigms, emphasizes the sociological aspects.

Hume ( 1739 ) relied on induction as much as anyone else, he only claimed that it was not rationally founded but based on passion in the form of habit.

Taken to the extreme, and as such potentially misleading, we may refer to the observation of William James to the effect that we have a passion for belief. If possible we would believe everything (James 1890 , (II) p. 299).

It is different in mathematics, where a strong mathematician can in principle check anything he uses by himself. Otherwise this leads to specialization and the concomitant risks of alienation, as there is nothing more satisfying than to have complete control of your knowledge, and when large parts of that has to be jettisoned, the activity may no longer seem as meaningful.

Caesar was bald but that does not mean that he was entirely devoid so that makes it easier to count. Of course we may have to specify the question further to make it logically unambiguous.

Incidentally if there is no free will, unlimited time travel does not entail logical contradictions. If there is no free will one may argue that time has a reduced meaning, as Popper pointed out as to the role of time in Einstein’s special relativity. An example of the naivety of (Popper ( 2002c ), p. 149).

Supposedly the Catholic church had no objection to the heliocentric worldview as long as it was merely an aid in computation.

Of course in recent decades distances to close celestial bodies such as the Moon has been done in direct ways assuming the velocity of light, which, however, originally was based on triangulated astronomical distances.

At the time no stellar parallax had been established, which indicated that the geometry was Euclidean (if that was the alternative choice) to a very high degree.

Our visual world is a sphere. The lower half of it is sensed as flat, as we can explore it tangibly moving around it, the upper half is sensed as a vault, only accessible by the eye. This made a split into the terrestrial world and the celestial. The first gave rise to flat Euclidean geometry, the second to the spherical one of astronomy. Ontologically the stars could be literally infinitely far away, making up an unbridgeable split between the two realms, but this extreme position seems never to have taken root, instead there seems to have been assumed that the celestial bodies were part of space, and hence had finite distances.

The latter discovered by Bradley can be given an elegant and suggestive formulation (Penrose, p. 428) as the parallax of the hyperbolic geometry of world-lines in Einstein’s theory of special relativity.

The most vocal critic—the astronomer Fred Hoyle, who incidentally coined the name with a disparaging intent, for a long time proposed an alternative theory involving the spontaneous creation of matter.

This is of course very commendable, there is a longstanding contention in philosophy that would only the concepts be clearly defined and the reasoning sufficiently strict the controversies that plague the subject would evaporate. In short, philosophy would be scientific and philosophical argument would turn into the form of calculations. We can compare with the remarks of Socrates in the dialogue Euthyphro (Plato), or with the vision of Leibniz pushing for a more precise language.

One may not take infinite literally of course, it can be argued that the infinite cannot be physically manifested, but is only an ideal construct of human thought. What is meant is that the statement holds for such large number of cases that it is not feasible to test them one by one, if for no other reasons that some of them are supposed to hold in the future.

One often says that history is no science, because it deals in particulars, not generalities. That might be true to some extent, but one can reason about the past, using more or less hidden universal statements, such as there were no airplanes in the seventeenth century. The same thing goes for natural history, we may with great confidence assert that no humans walked the earth during the era of dinosaurs. However, would human fossils be found together with those of dinosaurs, we would in principle be forced to reconsider the actual course of evolution, but using the general principle of Occam’s razor, we would be rather convinced it was a hoax. Collingwood makes an important distinction between natural history, as an unfolding of a spectacle, and human history, in which the reconstruction of thought plays a crucial role, just as motive does in forensic investigations. This adds another element akin to universal statements into the picture.

Although it is of course not as straightforward as non-mathematicians may suppose.

We may also think of the statement every human has a mother as of being in that form. Is it unscientific, because not being formally falsifiable? It hinges of course what we mean by the statement ‘x’ has a mother? Is it something that cannot be falsified, as it would in principle involve an infinite search? If so yes, it is not falsifiable, it means it is actually of the logical form \(P(x,y)\) where you for each \(x,y\) have to ascertain whether \(y\) is a mother of \(x\) . But if you take a more pragmatic view? Also, note that the sharpening every human has a human mother is in contradiction with the Darwinian interpretation of the evolutionary record. Anyway the statement in its vaguer form is something we all believe in, and do not need science to inform us of.

When the movement of Uranus did not adhere to Newtonian theory, this was not seen as a falsification, instead it was explained by an ad hoc assumption of a previously unknown planet, later vindicated by the discovery of Neptune (Popper 2002c , p. 45).

Of course we may make as our hypothesis that a universal law is not true, then of course it is possible to verify it, falsifying the law. But this is just a trivial play with words, signifying nothing.

This may be thought of as a universal statement, and thus contradicting everything I am maintaining. But in logic as well as in mathematics, universal statements are allowed based on intuition. You may of course question intuition, but then logic as we know it disintegrates and we are in strange territory indeed. Less drastically I can simple challenge him to come up with such statements.

But of course in any empirical undertaking, there is trial and error, and in the applications of the general laws, there are bound to be small guesses, to be tested and occasionally rejected, too technical to survive in the write-up. Thus there is indeed a kind of fractal structure, trial and error used at all levels.

Although that this does not contradict the fact that individual scientists may not be concerned with the big picture, but caught up in their technicalities. Popper is concerned about science as a whole, not the individual practitioners.

It is not my intention to dismiss those examples in bulk, psychoanalysis and marxism do contain very interesting ideas.

But not as hard to make sense of as I presume a paper in mathematics would be to the layman.

Here I am afraid I do not differ much from members at hiring committees, reduced to rely on formal authority.

The present Polar Star is actually more accurate as such than it was during antiquity.

Standard caveats are to be supplied. We assume that the time-period is short enough to assume that the proper motions of stars are uniform and that the speed of the precession as well as its axis, do not change significantly, something that can be safely assumed for historical periods.

The hypothesis that every parent of a human being is a human being has been confirmed literally billions of times and not a single concrete counterexample has ever been found. Yet the conclusion would in one blow destroy the theory of Evolution, the key being that the notion of Human being is not specific enough for the larger question but of course sufficient in everyday life.

The setting of those confidence intervals are more or less arbitrary. It is symptomatic that those are set to be much stricter in medicine than in the social sciences. Often those are set to acknowledge uncertainty, but that uncertainty is quantified. We claim that with such and such probability this is true. Popper, wisely refrains from making such quantitative statements as in a sense they tend to be circular.

It is far from obvious which theories will be at fault, cf. the case of debugging a program!

Collingwood, R. G. (1961). The idea of history . Oxford: Oxford University Press.

Google Scholar  

Collingwood, R. G. (1999). The principles of history . Oxford: Oxford University Press.

Dawkins, R. (2003). A devil’s chaplain . London: Phoenix.

Hansson, S.-O. (2006). Falsificationism Falsified. Foundations of Science , 11 , 275–286.

Horgan, J. (1996). The end of science . Reading, MA: Addison Wesley.

Hume, D. (1739, 1985). A treatise of human nature , Penguin.

Kuhn, T. (1962). The structure of scientific revolutions . Chicago: University of Chicago Press (3rd ed. 1996).

James, W. (1890, 1950). Principles of Psychology I, II . NY: Dover.

Lee, K. J., Dietrich, P., & Jessell, T. M. (2000) Genetic ablation reveals that roof plate is essential for dorsal interneuron specification. Nature , 403 , 734–740.

Nagel, E. (1979). Teleology revisited and other essays in the philosophy and history of science . New York: Columbia University Press.

Okasha, S. (2002). A very short introduction to the philosophy of science . Oxford: Oxford University Press.

Book   Google Scholar  

Penrose, R. (2004). The road to reality . London: Jonathan Cape.

Persson, U. (2014). Karl Popper, Falsifieringens profet [Swedish] . Sweden: CKM-förlag.

Plato. Plato’s dialogues . Any edition in the language of your choice.

Poincaré, H. (1905, 1952). Science and the hypothesis [English translation] . London: Dover.

Popper, K. (2002). The logic of scientific discovery . London: Routledge.

Popper, K. (2002). Conjectures and refutations . London: Routledge.

Popper, K. (2002). Unended quest . London: Routledge.

Popper, K. (1994). The myth of the framework. In M. A. Notturno (Ed.), Defense of science and rationality . London: Routledge.

Popper, K. (1979). Objective knowledge—an evolutionary approach . Oxford: Oxford University Press.

Spence, K. (2000). Ancient Egyptian chronology and the astronomical orientation of the pyramids. Nature , 408 , 320–324.

Article   Google Scholar  

Download references

Acknowledgments

I thank an anonymous referee for a valuable admonishment, leading to a major revision.

Author information

Authors and affiliations.

Department of Mathematics, Chalmers University of Technology, Göteborg, Sweden

Ulf Persson

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ulf Persson .

Rights and permissions

Reprints and permissions

About this article

Persson, U. Is Falsification Falsifiable?. Found Sci 21 , 461–475 (2016). https://doi.org/10.1007/s10699-015-9420-4

Download citation

Published : 28 January 2015

Issue Date : August 2016

DOI : https://doi.org/10.1007/s10699-015-9420-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Falsification
• Observation
• Explanatory theories
• Universal statements
• Find a journal
• Publish with us
• Track your research

5 Falsifiability

Textbook chapters (or similar texts).

• Deductive Logic
• Persuasive Reasoning and Fallacies
• The Falsifiability Criterion of Science
• Understanding Science

Journal articles

• Why a Confirmation Strategy Dominates Psychological Science

*******************************************************

Inquiry-based Activity:  Popular media and falsifiability

Introduction : Falsifiability, or the ability for a statement/theory to be shown to be false, was noted by Karl Popper to be the clearest way to distinguish science from pseudoscience. While incredibly important to scientific inquiry, it is also important for students to understand how this criterion can be applied to the news and information they interact with in their day-to-day lives. In this activity, students will apply the logic of falsifiability to rumors and news they have heard of in the popular media, demonstrating the applicability of scientific thinking to the world beyond the classroom.

Question to pose to students : Think about the latest celebrity rumor you have heard about in the news or through social media. If you cannot think of one, some examples might include, “the CIA killed Marilyn Monroe” and “Tupac is alive.” Have students get into groups, discuss their rumors, and select one to work with.

Note to instructors: Please modify/update these examples if needed to work for the students in your course. Snopes is a good source for recent examples.

Students form a hypothesis : Thinking about that rumor, decide what evidence would be necessary to prove that it was correct. That is, imagine you were a skeptic and automatically did not believe the rumor – what would someone need to tell or show you to convince you that it was true?

Students test their hypotheses : Each group (A) should then pair up with one other group (B) and try to convince them their rumor is true, providing them with the evidence from above. Members of group B should then come up with any reasons they can think of why the rumor may still be false. For example – if “Tupac is alive” is the rumor and “show the death certificate” is a piece of evidence provided by group A, group B could posit that the death certificate was forged by whoever kidnapped Tupac. Once group B has evaluated all of group A’s evidence, have the groups switch such that group B is now trying to convince group A about their rumor.

Do the students’ hypotheses hold up? : Together, have the groups work out whether the rumors they discussed are falsifiable. That is, can it be “proven?” Remember, a claim is non-falsifiable if there can always be an explanation for the absence of evidence and/or an exhaustive search for evidence would be required. Depending on the length of your class, students can repeat the previous step with multiple groups.

Share This Book

• Increase Font Size