The failure arrived as a sentence no survey researcher wants to read: “I don’t experience confusion in the same way humans do.” Ladies and gentlemen of the jury, that is not a quirky respondent. That is a chatbot walking into a social-science study wearing a fake mustache and somehow getting past the clipboard.
Exhibit A: The Witness May Be Synthetic
Social science depends on asking people things. What do you believe? Who do you trust? How did that policy affect your life? The whole enterprise gets awkward when some “people” are actually large language models answering surveys for money, bots, convenience, or chaos.
The evidence shows this is no longer theoretical. Adam’s article opens with psychologist Raluca Rilla catching an AI-like response in an online behavioral study. Rilla and colleagues argue in a 2025 preprint that online behavioral research now faces “LLM pollution,” where generated responses can slip into datasets and distort findings before anyone notices arXiv:2508.01390. That is the methodological equivalent of realizing the jury pool includes three interns, one toaster, and a LinkedIn growth bot.
Why is this so damaging? Because surveys often turn messy human experience into numbers. If those numbers include fake respondents, the math still runs. The regression does not cough politely and say, “Counselor, half your dataset is cosplay.” It produces coefficients with the calm confidence of your uncle explaining geopolitics after one podcast.
Exhibit B: The Same Tool Can Help the Case
Now, I submit to you that AI is not only the defendant. It may also be a useful expert witness, provided someone checks its credentials.
Generative AI can help social scientists classify text, summarize large document sets, generate experimental materials, compare coding schemes, and explore simulations. Thomas Davidson’s 2024 review in Socius argues that these tools can support computational, qualitative, and experimental sociology, while warning about reliability, reproducibility, privacy, opacity, and bias DOI:10.1177/23780231241259651. That is a fair bargain: the robot can help sort the evidence, but it does not get to be the judge, stenographer, and surprise key witness.
This matters because social data has become enormous. Computational social science already uses machine learning, network analysis, social simulation, and large-scale digital traces to study behavior. Large language models add a new layer: they can read and generate language at industrial speed. The attention mechanism inside transformer models is basically the one person in the office who reads the whole email thread before replying, except it also needs a small power plant and still occasionally invents a meeting that never happened.
A 2025 review by Thapa and colleagues describes LLMs in computational social science as useful for sentiment analysis, hate-speech detection, misinformation studies, event understanding, social network analysis, and content generation, while emphasizing bias, privacy, legal risk, and integration challenges DOI:10.1007/s13278-025-01428-9. Translation for the jury: powerful tool, sharp edges, no running in the courthouse.
Exhibit C: The Literature Itself Is Getting Noisy
Consider the following: AI can also generate papers, references, and polished paragraphs faster than reviewers can swat them away. Adam notes concern from researchers such as David Lazer that AI-assisted analysis could flood journals with thin but plausible studies. Nature also reported that an audit of 111 million references found 146,932 hallucinated citations in 2025 materials, with social-science repositories especially exposed arXiv:2605.07723.
That number should make everyone sit up straighter. Fake citations are not harmless typos. Citations are the legal precedent of science. If half the precedent turns out to be imaginary, the argument collapses wearing a very nice suit.
This is where practical hygiene matters. Researchers need disclosure rules, verified references, audit trails, preregistration, bot detection, stronger survey panels, and better metadata about how responses were collected. Even mundane tooling helps. If you are checking manuscripts or cleaning research PDFs before review, browser-based private tools like pdfb2.io are useful precisely because social-science data often contains sensitive material that should not be casually lobbed into mystery cloud services like a sandwich into a canyon.
The Verdict: Guilty of Risk, Not of Destiny
The evidence shows AI can damage social science in two main ways: by polluting the data going in and by producing persuasive junk coming out. But I submit to you that the same technology can also make research more rigorous when used as an assistant with supervision, not as an oracle with a lab coat.
The sane verdict is conditional. AI should not be trusted blindly. Neither should humans, frankly. Humans gave us p-hacking, unread appendices, and survey respondents who click “strongly agree” just to escape. The goal is not to keep social science pure from machines. The goal is to build methods tough enough to survive a world where machines can imitate respondents, draft articles, and fake citations with unnerving confidence.
So will AI ruin the social sciences? Only if researchers let the chatbot take the stand without cross-examination.
References
- Rilla, R., Werner, T., Yakura, H., Rahwan, I. & Nussberger, A.-M. “Recognising, anticipating, and mitigating LLM pollution of online behavioural research.” arXiv (2025). arXiv:2508.01390
- Davidson, T. “Start Generating: Harnessing Generative Artificial Intelligence for Sociological Research.” Socius (2024). DOI:10.1177/23780231241259651
- Thapa, S. et al. “Large language models (LLM) in computational social science: prospects, current state, and challenges.” Social Network Analysis and Mining 15, 4 (2025). DOI:10.1007/s13278-025-01428-9
- Panizza, F., Kyrychenko, Y. & Roozenbeek, J. “How to stop the survey-taking AI chatbots that threaten to upend social science.” Nature 650, 293-295 (2026). DOI:10.1038/d41586-026-00386-2
- “LLM hallucinations in the wild: Large-scale evidence from non-existent citations.” arXiv (2026). arXiv:2605.07723
Disclaimer: This blog post is a simplified summary of published research for educational purposes. The accompanying illustration is artistic and does not depict actual model architectures, data, or experimental results. Always refer to the original paper for technical details.