In the cult 1992 film Sneakers,��Robert Redford’s Martin Bishop leads a team of renegades, with expert skills at hacking or otherwise frustrating security systems. They are hired to try their hardest to break into clients’ systems, to reveal vulnerabilities which can then be fixed. Their determined approach actually serves the purpose of protection.����

My approach to PhD supervision is similar. Alongside suggesting literature and possible avenues of enquiry, I regularly go through students’ work actively trying to find faulty reasoning, neglected evidence, overdrawn conclusions, rhetoric masquerading as argument, and rival explanations they have not considered. I make clear what I am doing: I am not trying to dismantle them, but helping them close loopholes in their thesis before an examiner opens them up in a viva.����

The analogy has limits: the Sneakers team is invited in, whereas scholars do not choose their critics. Nonetheless, publication or submission of scholarship makes this available for examination by others, although not for personal abuse or harassment.

Parts of academia have become increasingly articulate about the harm criticism can cause, but less so about the harm caused when work is not criticised severely enough. An insufficiently tested bridge is not safer because its engineers feel better about it; neither is an argument. Without sufficient scrutiny, questionable claims can enter the literature and be cited as established knowledge.����

ɫ��ֱ��

In some contexts the principle is uncontroversial: cybersecurity, aviation safety, drug trials and central-bank stress scenarios. All of these deliberately create opportunities for weaknesses to reveal themselves.����

Philosopher of science Deborah G. Mayo a “severe test”, whereby evidence is valuable to the extent that a procedure had a serious chance of uncovering errors had they been present. This does not require every scholarly claim to become a laboratory hypothesis; severity simply requires exposing claims to evidence or arguments capable of weakening them. Sociologist Robert Merton’s of “organised scepticism” makes the complementary institutional point: scrutiny should not depend on individual temperament but be built into the production of knowledge.����

ɫ��ֱ��

Humanities and social sciences are different in many ways, but the underlying principles of scrutiny are still applicable. Historical explanations should face rival causes; literary interpretations should engage inconvenient textual evidence and competing readings; qualitative sociology should seek negative cases and limits to generalisation. In each case, the stress test needs to fit the claim being made.����

One should not however confuse stress-testing with intellectual aggression. A found women disproportionately subjected to hostile or patronising questioning, while remaining agnostic on whether greater engagement was ultimately beneficial or harmful. Some interventions may provide useful feedback; others merely disrupt. Severity concerns the diagnostic capacity of criticism, not its interpersonal unpleasantness.

The harder case arises when criticism of a claim is experienced as criticism of the individual making it. This is especially likely when scholarship rests heavily on the author’s own experience, as in autoethnography, or where epistemic authority is closely tied to race, gender or other identities. Criticism can then be interpreted or framed as an attack even when never intended as such.��

Some resistance to adversarial criticism has been expressed through the language of “paranoid” scholarship. used the term “paranoid reading” for a particular “hermeneutics of suspicion”: criticism organised around an anticipation, exposure and discovery of concealed (and usually reactionary) ideology or domination. Her “reparative” alternative sought to recover other relationships to the objects of study, including attachment, pleasure and surprise. described a “reparative turn” in which correction, rejection and anger give way to affection, gratitude, solidarity and love, with critique correspondingly “downgraded”.����

Sedgwick was describing relations to objects of study, not diminished rigour. However, others extended the vocabulary to refer to scholarly relationships themselves. Musicologist William Cheng’s explicitly contrasts professional aggression, the desire to prevail and forceful judgement with an academic environment organised around empathy, compassion and care. Lilith Whiley has described the ��of peer review, which she advocated replacing with “emotionality and empathy”, while have proposed “scholarly harassment” as a category encompassing attacks on women’s work as “unscientific” or lacking in rigour.��The difficult question in this context is when severe criticism constitutes legitimate scrutiny, and when its manner or pattern turns into harassment?

Mona M. Abo-Zena and colleagues argue for ; work on epistemic exclusion argues that disciplinary conceptions of can marginalise intersectional research; have argued that assessments of rigour may perpetuate racism or white supremacy. Yet points in another direction: diversity can strengthen rather than temper scrutiny by exposing assumptions overlooked by homogeneous communities. The issue is not diversity versus rigour, but whether criticism itself becomes morally suspect.��

The problem becomes sharper in research intended to recover previously marginalised voices. Miranda Fricker’s work on ��rightly identifies the wrong of discounting testimony because of prejudice. But others have urged , sometimes , while Eve Tuck has forms of research “refusal” where enquiry itself is said to risk representational harm.��

Taking testimony seriously is however not the same as treating it as self-validating. A person’s account may be strong evidence of what they experienced and how they understood it without independently establishing what happened, why it happened, or how representative the experience is. Steven Lubet this phenomenon in ethnography, including , arguing that highly dubious informant stories taken from gang members in Philadelphia were accepted without seeking appropriate corroboration.����

ɫ��ֱ��

ɫ��ֱ��

Scholarship concerned with marginalised voices therefore needs mechanisms for corroboration, comparison and the possibility of concluding that testimony is mistaken, misleading or unrepresentative.�� Christopher Browning’s provides a particularly demanding example. He rejects both extremes: the dismissal of survivor testimony because memory is fallible, and the claim that such testimony should enjoy privileged immunity from normal rules of evidence. When reconstructing the Starachowice slave-labour camps from hundreds of survivor accounts, Browning repeatedly compares witnesses’ accounts with each other, with contemporary documents and with what later evidence makes possible. Many later witnesses remembered undergoing a selection on arrival at Birkenau, often conducted by Josef Mengele, but Browning argues no such selection took place and links the recollections to later selections, telescoping of memory and familiarity with other Holocaust imagery.

Nor did scrutiny merely discredit testimony: Browning expected memories to diverge over time, but found them more stable than anticipated. He does not conclude that survivor testimony is unreliable; indeed much of his historical reconstruction would be impossible without it. Respect for witnesses, according to this model, involves taking their testimony seriously enough to test it. If that is possible in this highly sensitive context, it should be possible in most others.

Nor is this standard peculiar to research based on testimony or marginalised experience. Tia DeNora, in , argues that the composer’s exceptional reputation arose not simply from recognition of his musical qualities, but was substantially enabled and constructed through the social resources and patronage networks of the Viennese aristocracy. DeNora argues that Beethoven’s talent was a necessary though insufficient cause of this acclaim. The stress test is therefore comparative: how much explanatory weight should be accorded to those social mechanisms as against musical features which distinguished Beethoven from other composers working within comparable patronage systems? An account of the conditions enabling reputation is not necessarily an explanation of the reputation itself.

The general question is whether a research design has the potential to weaken its preferred explanation. That question becomes especially consequential in politically charged research based on personal testimony. Jason Arday’s studies on race in higher education provide a useful test case. A draws upon 18 participants across��10 UK universities, recruited through networks and recommendations, with deductive thematic analysis explicitly informed by critical race theory. Participants all reported in an initial questionnaire an experience of racism connected to employment or career status, and subsequent interviews asked what role race or racism had played in their experiences, and how these differed from those of their white counterparts. The relevant stress-test is not whether CRT should be used; it is whether the evidence remains capable of modifying the interpretation with which the inquiry began.

This design is well suited for exploring how staff of colour understand racialised precarity; it is much less capable of isolating how much experiences are caused by race rather than precarity more generally. This matters when overwork and exploitation – also documented in Arday’s sources among wider precariously employed academics – are interpreted as directly connected to racism. Arday does acknowledge important limitations, including the need for perspectives from other institutional actors, and quantitative investigation of causality. Such limitations are hardly peculiar to Arday’s work, nor do they invalidate small-sample qualitative research. The issue is whether structural conclusions exceed what the evidence and caveats can support.

Source:��

Universal/Melinda Sue Gordon Archival Cinema/Shutterstock

The same issue is even clearer in , whose interview questions were themselves explicitly organised around experiences of racism. The question for an academic Sneakers team is therefore: what could this design have discovered that would have qualified or complicated its organising racial interpretation? One participant speaks of becoming “paranoid” because “everything you do is unfairly scrutinized” which informs the statement that “Higher education is racist”. The reported experience of scrutiny is evidence, but requires some independent means by which the racial attribution could be tested. Similar care is needed��when moving to sector-wide causal claims or policy prescriptions, as Arday does.������

The point is not that all criticism should be of maximum severity, but that the standards applied to a claim and the fairness with which a scholar is treated are separate questions.

Criticism can certainly damage confidence without thereby being improper. Academics often invest years of effort and identity in their work, so severe criticism can produce an ordinary narcissistic injury. Hurt deserves personal support, but it is not evidence that a wrong has been committed.��Equally, exacting standards applied selectively to particular individuals or groups can constitute genuine injustice, while criticism can also be used for status games and bullying. But “is this criticism correct?” and “was this person selected or treated unfairly?” remain different questions; criticism can be intellectually justified and unfairly targeted at the same time.��

The Arday affair now puts that distinction under unusual pressure. At the time of writing, he has resigned from Cambridge amid continuing controversy over his academic record and wider claims, while institutions associated with his career . Cambridge academics have simultaneously condemned alleged racist commentary surrounding the affair and . There is no contradiction in doing both. Whatever the investigations find, the institutional question remains: where were the stress tests supposed to occur – in doctoral examination, peer review, recruitment and promotion? Who, in other words, was supposed to perform the Sneakers function before any vulnerabilities became public? Failures of scrutiny can of course result from a variety of causes: insufficient time available for reviewers, hurried appointment processes, prestige effects or simple institutional incompetence. But these are reasons to strengthen rather than abandon stress-testing procedures.

Stress-testing works best before failure becomes public. What I have elsewhere called ��spares the evaluator the discomfort of delivering difficult criticism but ultimately outsources this to an examiner, reviewer, appointments panel or public controversy. The Sneakers team protected their clients by trying seriously to break into their systems. Before we call scholarship robust, somebody should have tried to do the same.����

ɫ��ֱ��

Ian Pace is professor of music, culture and society at City St Georges, University of London, where he is a university adviser on interdisciplinarity working with the department of sociology and criminology and the School of Policy and Global Affairs. He is writing in a personal capacity.