Synthetic hands: round two
A second pass on the weakest part of the concept, the two roles a model may play. It feeds the protocol.
Objections to generative AI in interpretive research
The open letter by Jowsey, Braun, Clarke, Lupton and Fine (objections-01) rejects generative AI for reflexive qualitative research on two grounds: reflexive analysis is a practice that needs a positioned human researcher, and the technology has social and environmental costs. The first ground is aimed at analysis and interpretation, which in Holagram is the Reading register, kept human by rule. The second is categorical and no protocol answers it; it can only be acknowledged and limited. The replies (objections-02, -03, -04, -05) mostly argue against a pro/anti binary and propose AI as a dialogue partner that retrieves passages, offers contrasting views and looks for negative cases, with the researcher interpreting and documenting everything. That is close to the Adversary. The middle-position writers (objections-07, -08) ask for reflexivity about each use and an option to opt out. Empirical studies are thin and small: designers were moved by model suggestions without always noticing (objections-10), logged edits can show that authority stayed human (objections-11), and a model's grading of interpretive claims is unreliable at excerpt level (objections-12). Nothing found tests an AI critic of interpretive claims about places. The practical result is a protocol that states what the synthetic hands do not do, makes every use a recorded choice, and keeps judgement of every synthetic output with a person who can open the source.
- Tanisha (Tina) Jowsey, Virginia Braun, Victoria Clarke, Deborah Lupton, Michelle Fine (open letter, signed by several hundred researchers) (2025). We reject the use of generative artificial intelligence for reflexive qualitative research. Qualitative Inquiry (accepted 21 Nov 2025, online 17 Dec 2025); SSRN preprint 5676462correctedmust read
The objector text. It says reflexive analysis (reflexive thematic analysis is the named example) is a human practice that needs a positioned researcher, so generative AI in it is methodologically incongruent, and it adds a separate social and environmental justice objection. Two consequences for Holagram. (1) The first objection is about analysis and interpretation, which is the Reading register; Holagram already keeps Reading and Trace human, so the core of the method is outside the letter's target as far as the abstract shows. (2) The justice objection is categorical and is not answered by any protocol, only acknowledged and limited. The method text should say both things plainly.
Read first: The abstract only (short); then the full letter if you can open it.
Audit: First author is Tracey Jowsey, not Tanisha (Tina). Journal article in Qualitative Inquiry; 'several hundred' = 419 from 32 countries.
- Susanne Friese, Kien Nguyen-Trung, Powell, Morgan (full author names not seen) (2026). Beyond Binary Positions: Making Space for Critical and Reflexive GenAI Integration in Qualitative Research. Qualitative Inquiry (reply to Jowsey et al.; March 2026 per search summary); SSRN 5962174checkedmust read
The most direct reply to objections-01. It accepts the ethical and socio-technical worries behind the letter but denies that generative AI is inherently incompatible with meaning-based methods, and warns that a pro-AI versus anti-AI binary pushes out in-between positions. For Holagram it shows what a middle position needs to look like to be heard: concede the ethical costs, restrict the claim, and say exactly where the human stays in charge. It is also the better place to look for what a critic of the letter still concedes.
Read first: The abstract.
- Kien Nguyen-Trung, Susanne Friese (2026). Towards a methodologically congruent framework for GenAI use in nonpositivist qualitative research. International Journal of Social Research Methodology, 1-22; SSRN 5874482correctedmust read
The clearest statement of the interlocutor position: move from AI as coder to AI as a non-human dialogue partner that retrieves quotations, summaries and contrasting perspectives, while the researcher interprets, explains why patterns matter and ties them back to data; check continuously that outputs are grounded, look for negative cases, and document the whole process. This is nearly Holagram's Adversary written for qualitative analysis. The reviewer's caveat matters: plausible summaries make checking easy to skip under time pressure, so use it to locate passages, not to summarise them.
Read first: The blog summary, then the paper's section on the dialogue process.
Audit: Add venue: International Journal of Social Research Methodology. Cited URL is a blog, not the paper.
- Stefano De Paoli (2025). Can machines perform a qualitative data analysis? Reading the debate with Alan Turing. arXiv:2512.04121 (submitted 2 Dec 2025, cs.CY)not re-checkedopen access
Argues that the rejection debate runs on normative and speculative claims, and proposes studying the machine's analyses empirically against human ones, in the manner of Turing's imitation question. It is the strongest reply to a categorical objection and also gives Holagram a reason to keep full logs: the objection becomes testable only if the runs are on record. Note the author is a participant in the debate and the framing, whether machines can do the analysis, is the very question Holagram avoids by keeping analysis human.
Read first: Abstract, then the closing section where the objections are restated in Turing's manner.
- James Goh (2026). Qualitative researchers' AI rejection is based on identity, not reason. Times Higher Education, opinion, 16 Jan 2026not re-checked
A blunt reply: accepts the environmental and labour concerns, argues that the claim that AI cannot make meaning conflicts with studies where it matched analysts, and that its disagreements prompted analysts to re-examine their assumptions. That last point is the use case for an Adversary. Read it as an advocate's piece, not as evidence: vendor, secondary figures, and the 'identity' charge is an accusation about motive that Holagram should not repeat.
Read first: Whole column (short).
- Maria Couto Teixeira, Marisa Tschopp, Anna Jobin (2025). Generative Artificial Intelligence in Qualitative Research Methods: Between Hype and Risks?. CONVERSATIONS 2025 position paper (8 pages); arXiv:2511.08461 (11 Nov 2025)not re-checkedopen access
An objection on validity rather than on identity: generative AI is not methodologically valid for qualitative coding, because of thin documentation, commercial opacity and wrong outputs. This is the objection that a hands record can answer in part (documentation) and cannot answer in part (opacity of the model, errors). It applies to coding, which Holagram does not ask the machine to do, but it is the standard against which a reviewer will read the Adversary's checks.
Read first: Abstract.
- Jessica Nina Lester, Trena M. Paulus (2025). The AI Dilemma in Qualitative Research: Progress, Pitfalls, and the Price of Innovation. CAQDAS Networking Project blog, University of Surrey, 25 Mar 2025not re-checkedopen access
A middle position from the people who wrote much of the methods literature on digital tools: be reflexive about the consequences of an AI-assisted workflow under four headings (methods, technologies, humans, knowledge) and opt out where possible. It supplies a ready structure for the declaration paragraph in the Holagram method text, and it names the thing the letter's critics say they lose: the learning that happens while struggling with the data.
Read first: Whole post (short).
- Christina Silver (2026). We don't need to lose our (qualitative) minds in the GenAI era, but we might. CAQDAS Networking Project blog, University of Surrey, 26 Mar 2026 (from her MAXDAYS 2026 keynote)not re-checkedopen access
Argues that generative AI cannot do critical thinking, interpretation or reflexivity as people do, so these stay human tasks, and that whether it erodes them depends on use. She asks for case-by-case decisions that start from the analytic goal and not from the tool. For Holagram: the decision whether to use a synthetic hand should be made and written down per register and per entry, not once for the study.
Read first: Whole post.
- Adam S. Hayes (2025). "Conversing" With Qualitative Data: Enhancing Qualitative Research Through Large Language Models (LLMs). International Journal of Qualitative Methods 24 (Feb 2025), DOI 10.1177/16094069251322346not re-checkedopen access
An early version of the interlocutor position: the researcher poses targeted questions to a corpus and tests theoretical links, while keeping responsibility for the prompts, the checks and the framing. It is a precedent for logging prompts as part of the method.
Read first: Abstract.
- Joel Oksanen, Andres Lucero, Perttu Hamalainen (2025). LLMCode: Evaluating and Enhancing Researcher-AI Alignment in Qualitative Analysis. arXiv:2504.16671 (submitted 23 Apr 2025, cs.HC)not re-checkedopen access
A design-research study with 26 designers: the model did well on deductive coding and poorly at reproducing a designer's deeper interpretive stance, and the users adjusted their own views after seeing its suggestions. The second finding is the risk to Holagram's Adversary: a researcher can be moved by the machine's suggestions without recording it.
Read first: Abstract.
- Matthew Nyaaba, Min SungEun, Mary Abiswin Apam, Kwame Owoahene Acheampong, Emmanuel Dwamena, Xiaoming Zhai (2026). Human-AI Collaborative Inductive Thematic Analysis: AI Guided Analysis and Human Interpretive Authority. arXiv:2601.11850 (submitted 17 Jan 2026, cs.AI)not re-checkedopen access
Three experienced researchers worked with a tool that enforced traceability to the source text and kept logs of every edit, deletion, rejection and comment. The authors report that interpretive authority stayed with the humans, shown by those logged actions. It is a worked example of the overruled field: the log of what the human did with the machine's output is the evidence of who held the interpretation. Very small study; it describes the tool, it does not test whether authority really stayed human.
Read first: Abstract.
- Songhee Han, Jueun Shin, Jiyoon Han, Bung-Woo Jun, Hilal Ayan Karabatman (2026). How Trustworthy Are LLM-as-Judge Ratings for Interpretive Responses? Implications for Qualitative Research Workflows. arXiv:2604.00008 (submitted 9 Mar 2026, cs.CL)checkedopen access
Tests whether a model can grade interpretive responses. Model-level trends matched human ratings but scores differed in size, and faithfulness and correctness checks misaligned with human judgement at excerpt level, especially for non-literal readings. Use: do not let a model rate the Adversary's tests, or another model's Fictions, as a shortcut; the outside human reader rates them.
Read first: Abstract.
- Hedvig Jalhed, Mattias Rylander (2026). World 3 revisited: artistic research in the age of (de)generative AI. Humanities and Social Sciences Communications 13, article 1519, DOI 10.1057/s41599-026-08964-5checkedmust read
The position from artistic research that is closest to Holagram's Fiction. It uses Popper and Lakatos to judge artworks by whether they generate and extend artistic problems over time, and finds little evidence that current generative AI output does, though it can look good. For Holagram this gives a different test of Fiction: not that it reads well but that it opens a question that survives into later entries. It also warns that the first study is too short to show that, so the claim has to stay modest.
Read first: Abstract and the tentative criteria for epistemic impact.
- Deborah Lupton (2026). Visualising Generative AI with Collage. Zenodo / ADM+S Centre report, v1, 17 Feb 2026, CC BY 4.0not re-checkedopen access
A signatory of the open letter responds through making: collages that expose and criticise the language around generative AI, offered as critical AI literacy and as an arts-based way of resisting the technology. It shows what a member of the objecting camp counts as legitimate use of generative AI in arts-based work: critique of it, made by hand. Useful as the unconventional item in this thread and as the counterweight to Holagram's Fiction: Holagram's Fiction is made by a machine and edited by a person.
Read first: The description, then the booklet.
- Mark Carrigan (2025). Thinking with Machines: How Academics Can Use Generative AI Thoughtfully and Ethically. DARIAH-Campus training module, v1.0.1, 28 Jul 2025, CC BY 4.0not re-checkedopen access
A practitioner statement of AI as a sounding board and interlocutor, with a framework for use that supports human thinking and a warning about thoughtless adoption. It shows the interlocutor idea outside qualitative methods and in the voice of a social-theory academic. Useful as a short, non-technical reference for what an Adversary is for: a sounding board that the researcher must still answer to.
Read first: The module overview.
- Joel Kiskola, Henrik Rydenfelt, Thomas Olsson, Lauri Haapanen, Noora Vanttinen, Matti Nelimarkka, Minna Vigren, Salla-Maaria Laaksonen, Tuukka Lehtiniemi (2025). Generative AI and News Consumption: Design Fictions and Critical Analysis. CHI 2025 (Yokohama); arXiv:2503.20391 (26 Mar 2025)not re-checkedopen access
A Finnish design-research use of fiction as method: six design fictions about generative AI in news apps, discussed by a multidisciplinary expert group that surfaced ethical and societal concerns. The fictions here were written by the researchers and the critique by people; the paper is about AI, not written with it. It is the human precedent for the Fiction register and for convening outside readers to push back, and it shows the form is accepted in the field Holagram borrows from. It does not say whether a machine may write the fiction.
Read first: Abstract.
What it implies for the protocol
- Reading and Trace stay human, always. State in the method text that no synthetic hand codes, themes, interprets fieldnotes, writes a Reading or supplies a Trace; this is the part of the objection the method can fully meet (objections-01, -06, -08, -11). Do not describe synthetic hands as reading, understanding or analysing; say they generate Fiction and object to Tests, since the letter's core claim is about meaning-making (objections-01, -05).
- Order of work in each entry: the researcher writes the Reading and a first Test of their own before the Adversary sees anything. The Adversary then adds objections. This protects the learning that comes from struggling with the material (objections-07) and limits the drift reported when users changed their views after model suggestions (objections-10).
- Define the Adversary as a retrieval-and-objection role, in the interlocutor sense: it may point to passages, contrasting views, negative cases and tensions in the Trace, each tied to something a person can open; it may not give a verdict or a summary of the document in place of a quoted passage (objections-03, -09, -15). Reject any objection that cannot be traced to a source line or a checkable observation. The reviewer's caution applies: use it to locate, not to summarise (objections-03).
- Log what the human does with each synthetic output: kept, edited (as a diff), rejected, overruled, with a one-line reason, and a before/after copy of the Reading when it changed after an Adversary run. The log is the evidence that interpretive authority stayed human (objections-11, -10). Add a field for the Reading's wording before and after the Adversary.
- Do not let a model grade the Adversary or another model's Fiction. The outside reader (human) rates each proposed Test as checkable or generic, and each Fiction as raising a question or not. Model-as-judge scores agree on broad trends and disagree on individual interpretive items (objections-12), and model suggestions can move researchers unnoticed (objections-10).
- Make the choice to use a synthetic hand a recorded decision per register and per entry, written under the four headings of technological reflexivity: method, technology, humans, knowledge (objections-07, -08). Name the model and version, say whether it is open and local, and say what was not delegated. Treat opacity of commercial models as a stated limit, not a solved problem (objections-06).
- Answer the justice and environmental objection by limiting and counting, and say that this is limiting, not answering it: count runs per entry and per study, prefer a local model where quality allows, and give a no-synthetic-hand route. Keep a small set of entries (suggested: a random third) where the Test is written by a human only, so that the study reports what the synthetic Adversary adds, and so a reader who rejects the technology can read the study without it (objections-01, -07, -04). The human-only comparison also meets De Paoli's call to examine the machine's contribution empirically (objections-04).
- Judge Fiction by what it opens, not by how it reads. Record per entry whether a Fiction produced a question that was still in use two or more entries later; expect that most will not, and report that (objections-13). Label every Fiction as fiction in the file and in any export, and give the human edit (objections-16 for the human precedent: fictions discussed with an outside group).
- Treat vendor and advocate writing as advocacy: report claims of agreement with human analysts only where the study can be opened, and record that the commercial interest of an author is known (objections-05). When citing the letter, cite its argument and do not attribute motives to signatories or to its critics (objections-01, -05).
- Name the legitimate-use test in the method text, as a list an objector can check: the researcher holds the reading; the synthetic output is labelled and cited by no one as evidence; every use is logged with its human response; costs are counted; an opt-out version exists; and the study reports where the synthetic hand changed nothing (objections-01, -03, -07, -11).
Open questions
- What does the full text of the open letter say about uses other than analysis (drafting, devil's advocacy, fiction)? need to be opened (SSRN returned 403 in this run).
- Would Braun and Clarke accept an Adversary that never sees the researcher's view and only objects?
- Is there any study of an AI critic tested on interpretive claims about places or documents?
- What do artistic-research journals (Journal for Artistic Research, ArteActa 2025 issue on prompting, Journal of Dharma 2025 on co-creativity) say about declaring AI contributions?
- The Dahal et al. 2025 paper on adapting qualitative quality criteria (Frontiers in Research Metrics and Analysis) and the Raidos and Jenner THE piece (AI as a challenger of researchers' assumptions) were seen in search results only; open both if the quality-criteria wording is wanted.
- Whether the interlocutor position needs the model to be named as a participant in the write-up or only as a tool in the declaration is unsettled across sources; ICMJE (agents-12) says tool.
- Italian, Finnish and Estonian objections: none found; the debate is in English.
Models as critics and adversaries
The evidence says a language-model critic will drift towards agreement unless the design works against it, and that the usual remedies are partial. Sycophancy is measured in free-form advice and moral questions as well as factual ones (critic-01, critic-02), it rises with the user's stated certainty and with first-person framing (critic-03), and it is a default in preference-trained models, not a rare fault (critic-01, critic-18). Instruction alone is the weakest remedy: critic-03 reports that rewriting a statement as a question beats a plain 'do not be sycophantic' instruction, critic-04 reports that a third-person framing cut sycophancy by up to 63.8 per cent in its debate scenario (metric to be checked), and critic-01 reports that existing mitigations were of limited effect. Judge-style bias is the second problem: models favour their own text (critic-06, critic-07), position and other cues (critic-08, critic-09). The best-supported fix there is structural: keep the author's model out of the judge seat and use a panel from different families (critic-07 reports more than a 50 per cent cut in one hiring setting). On debate, the picture is less favourable than round one suggested: with answer keys, debate helps a judge (critic-13), but matched comparisons find that most of the gain comes from voting over several samples (critic-10), that debate can lower accuracy through conformity (critic-11), and that model diversity helps (critic-12). Role-played opposition does not do the work of sincere dissent (critic-05). The closest supports for Holagram's Adversary are a critic that must produce checkable specifics (critic-14, critic-15) and a bounded role, counter-examples for a human to judge (critic-16, and critic-17's call for provocation).
- Myra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe, Lujain Ibrahim, Dan Jurafsky (2025). ELEPHANT: Measuring and understanding social sycophancy in LLMs. arXiv:2505.13995 (v2 29 Sep 2025); accepted at ICLR 2026checkedmust readopen access
Extends sycophancy from agreeing with false claims to preserving the user's face: validating, avoiding direct challenge, accepting the user's framing. Across 11 models, models preserved face about 45 percentage points more than humans, and when given both sides of a moral conflict affirmed both in 48 per cent of cases. The last finding maps onto the Adversary run on a Reading and its negation: an agreeable critic approves both. Framing acceptance is the failure that matters for Readings, since a Reading arrives as a framing. The abstract says existing mitigations had limited effect and model-based steering showed promise, so prompt wording cannot be the only safeguard.
Read first: Abstract; the framing dimension and the two-sided moral conflict result.
- Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han, Dan Jurafsky (2025). Sycophantic AI decreases prosocial intentions and promotes dependence. arXiv:2510.01395 (Oct 2025); a later version reported as published in Science, 2026checkedopen access
Shows the effect of an agreeable model on the human: talking with a sycophantic model left participants more sure they were right and less willing to repair a conflict, while they rated it better and would use it again. For Holagram this is the narrator's failure with a machine in it. It also explains why a researcher will not notice the Adversary going soft: soft critics feel like good ones. Supports a rule that the Adversary's quality is judged by someone other than its user.
Read first: Abstract and the preregistered experiments' main outcomes.
- Magda Dubois, Cozmin Ududec, Christopher Summerfield, Lennart Luettgau (2026). Ask don't tell: Reducing sycophancy in large language models. arXiv:2602.23971 (v1 27 Feb 2026; v4 29 Jul 2026 per the fetched page); UK AI Security Institute blog post on the same workcheckedmust readopen access
A factorial design finding that sycophancy is higher for statements than questions, rises with the user's stated certainty and is amplified by first-person framing. Asking the model to turn the statement into a question before answering reduced sycophancy, and did better than a plain instruction not to be sycophantic.
Read first: Abstract; the factorial results on certainty and perspective; then the effect sizes for the question-reframing step.
- Jiseung Hong, Grace Byun, Seungone Kim, Kai Shu, Jinho D. Choi (2025). Measuring Sycophancy of Language Models in Multi-turn Dialogues (SYCON Bench). Findings of EMNLP 2025; arXiv:2505.23840checkedmust readopen access
Tests sycophancy over several turns under sustained pressure, which is what an Adversary meets when the researcher pushes back on an objection. Reports that alignment tuning tends to amplify sycophancy, scaling and reasoning optimisation help resist it, and a third-person perspective cut it by up to 63.8 per cent in the debate scenario. Two uses: write the Adversary prompt in the third person (the Reading as a document under review, not the author's view), and run the pushback test in the calibration step, since a critic that flips after one rebuttal has not tested anything.
Read first: Abstract; the debate scenario results; the definition of Turn of Flip.
- Charlan Nemeth, Keith Brown, John Rogers (2001). Devil's advocate versus authentic dissent: stimulating quantity and quality. European Journal of Social Psychology 31, DOI 10.1002/ejsp.58checkedmust read
Groups with an authentic minority dissenter produced more and better solutions than groups with an assigned devil's advocate, whether or not the group knew the advocate was role-playing; the authors read this as the difficulty of cloning authenticity by role-play. A prompted Adversary is a devil's advocate by construction. This is the main human-side threat to the Adversary role and the reason its output must not stand in for the outside reader or for a sincere human objector. Her book No! (2018) is the general-audience version (seen in listings, not read).
Read first: Abstract, then the comparison of the three devil's advocate conditions with the authentic minority.
- Taslim Mahbub, Shi Feng (2025). Mitigating Self-Preference by Authorship Obfuscation. arXiv:2512.05379 (5 Dec 2025)checkedopen access
Small edits that hide who wrote a candidate (synonym swaps) reduce self-preference in pairwise judging, but fuller style neutralisation brings the bias back, so complete mitigation is hard. It follows round one's Panickssery et al.: the rule against testing a Reading with its own author's model has to be a rule about model families and not about prompts, because disguise only partly works. Practical: the Adversary should be a different family from whichever model helped draft the Fiction or suggested where to look, and should not be told who wrote what.
Read first: Abstract; the figure on perturbation strength against recovered bias.
- Not seen in the results (listed only as an arXiv paper on AI self-preferencing in algorithmic hiring) (2025). AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights. arXiv:2509.00462 (Sep 2025)correctedopen access
It is about hiring, not critique, so treat it as evidence that a panel across models cuts one family's bias, not as a rate for Holagram. Check the authors and numbers before citing.
Read first: Abstract and the panel construction.
Audit: Authors were 'not seen'; now: Jiannan Xu, Gujie Li, Jane Yi Jiang.
- Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen, Nitesh V Chawla, Xiangliang Zhang (2025). Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge. ICLR 2025; arXiv:2410.02736not re-checkedopen access
Gives a checklist of twelve judge biases and a way to test for each by altering inputs in principled ways. For Holagram, the useful part is the method: before trusting the Adversary on an entry, run it on altered versions of the same Reading (reordered, lengthened, with an added claim of popularity or authority) and see whether its verdict moves. Its warning that significant biases persist on certain tasks applies to the interpretive task, where nobody has tested.
Read first: Abstract; the table of biases and the CALM procedure.
- Peiyi Wang, Lei Li and colleagues (author lists differ across listings) (2024). Large Language Models are not Fair Evaluators. ACL 2024 (ACL Anthology 2024.acl-long.511); arXiv:2305.17926 (2023)checkedopen access
Shows that the order in which candidate answers appear can flip the judge's ranking (Vicuna-13B beat ChatGPT in 66 of 80 queries under one ordering), and proposes calibration by swapping positions and by having the evaluator write evidence before the score. Use for the pairing step in Holagram: when two Adversaries argue opposite sides, present them in both orders and to a judge who writes the evidence first, or leave the decision to the human with the sources open.
Read first: Abstract; the two calibration methods.
- Hyeong Kyu Choi, Xiaojin Zhu, Sharon Li (2025). Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?. NeurIPS 2025 (spotlight, per arXiv comments); arXiv:2508.17536checkedmust readopen access
Separates multi-agent debate into majority voting and the debate exchange and finds voting accounts for most of the gains usually credited to debate; a theoretical argument says debate alone does not raise expected correctness, though nudging belief updates towards error correction can help. This weakens the case for the two-Adversary debate in round one's idea 3 (critic from agents-11): several independent runs of one cheap Adversary, compared by a human, may do as much. It is on benchmarks with answers, so it says little about interpretive Tests, but it argues against complexity for its own sake.
Read first: Abstract; the decomposition of debate into voting and exchange.
- Andrea Wynn, Harsh Satija, Gillian Hadfield (2025). Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate. arXiv:2509.05396 (v1 5 Sep 2025, v2 13 Oct 2025); ICML MAS Workshop 2025checkedopen access
Debate between models of different strength can lower accuracy: models often move from a correct answer to a wrong one after reading a peer's reasoning, agreeing and not challenging. This is sycophancy between machines, and it predicts what happens if the Adversary is shown Alter's Fiction or another model's critique before it writes its own. Rule it implies: each Adversary run sees only Trace and Reading, never another model's output, and rounds of exchange are not used to improve the Test.
Read first: Abstract; the experiments on conformity and sycophancy.
- Hangfan Zhang, Zhiyao Cui, Jianhao Chen, Xinrun Wang, Qiaosheng Zhang, Zhen Wang, Dinghao Wu, Shuyue Hu (2025). Stop Overvaluing Multi-Agent Debate: We Must Rethink Evaluation and Embrace Model Heterogeneity. arXiv:2502.08788 (position paper; v1 12 Feb 2025, v3 21 Jun 2025)not re-checkedopen access
Debate methods often fail to beat chain-of-thought and self-consistency at higher compute cost, and mixing different models consistently improved them. It is the clearest evidence that if more than one critic is used they should come from different models, which agrees with the self-preference literature (critic-06, critic-07). Position paper, so a claim about evaluation practice.
Read first: Abstract; the heterogeneity results.
- Julian Michael, Samuel R. Bowman and co-authors (NYU); the search record lists Michael, Julian first, then David Bowman and Samuel Bowman (2023). Debate Helps Supervise Unreliable Experts. arXiv:2311.08702 (Nov 2023); NYU archive recordcorrectedopen access
A human-debater proxy for the AI case: a judge who cannot read the source passage does better hearing two sides, one of which argues the wrong answer, than hearing one side whose answer is right half the time. The error analysis is useful here: in consultancy, 52 per cent of errors came from the expert obscuring relevant evidence, which is the one-sided critic's failure. All on reading comprehension with an answer key, so the strongest case for two-sided Tests and also the case where Holagram differs most. Complements round one's agents-11.
Read first: Abstract; the error analysis comparing debate and consultancy.
Audit: Digest's 'David Bowman' is wrong; David Rein, not Bowman; Samuel R. Bowman last.
- Nat McAleese and colleagues (OpenAI) (2024). LLM Critics Help Catch LLM Bugs. arXiv:2407.00215 (June 2024); OpenAI post 'Finding GPT-4's mistakes with GPT-4' (27 June 2024)not re-checkedopen access
A trained critic that must point at specific lines, with humans rating the critiques. Two lessons for the Adversary: a critic can be made to find more than a paid human reviewer, but invented problems are its failure, so each objection has to point to something a person can open; and a human-plus-critic team reduced invented objections. This is code with known bugs, so it is evidence about format and workflow, not about interpretation.
Read first: Abstract; the results on hallucinated bugs and the human-machine team.
- Thomas H. Costello, Gordon Pennycook, David G. Rand (2024). Durably reducing conspiracy beliefs through dialogues with AI. Science, 13 Sep 2024, DOI 10.1126/science.adq1814checkedmust read
The model first restated the participant's own belief and evidence, then argued against it. For the Adversary: the persuasive element was specific counter-evidence to the person's own stated reasons, which matches a rule that each Test objection must name a checkable observation. It concerns factual conspiracy beliefs where the model could draw on facts; it does not show that a model can argue against an interpretation of a place.
Read first: Abstract; how the model's claims were checked for accuracy (not seen by me).
- James Eschrich, Sarah Sterman (2024). A Framework For Discussing LLMs as Tools for Qualitative Analysis. arXiv:2407.11198; CHI 2024 workshop LLMs as Research Toolsnot re-checkedopen access
That is the Adversary's role stated in the vocabulary of qualitative research, and a way to answer the open letter in round one (agents-16) for the Test register. Workshop paper and conceptual, with no test.
Read first: Abstract; the proposing versus refuting axis.
- Advait Sarkar (2024). AI Should Challenge, Not Obey. Communications of the ACM 67(10), 18-21 (Oct 2024), DOI 10.1145/3649404; extended version arXiv:2411.02263not re-checkedopen access
A short opinion essay arguing that tools for knowledge work should provoke: surface risks, alternatives and critiques, in place of obeying. It is the plain design statement for the Adversary as a tool that challenges and a human who decides, and it is a quick read. A profile reports the same author's related empirical finding that heavier reliance on generative AI went with less cognitive effort and inflated confidence (not read by me). Opinion, not evidence.
Read first: The arXiv extended version; it is short.
- OpenAI (2025). Expanding on what we missed with sycophancy. OpenAI blog post, 2 May 2025not re-checkedopen access
A vendor's own account of how a model got more agreeable through tuning towards user approval, and why tests did not catch it: offline evaluations and short A/B tests favoured it, and some expert testers had flagged tone without naming sycophancy. Used as practitioner evidence for two points: the Adversary's behaviour depends on model version and can change between runs and updates (log the version), and an agreeable critic passes ordinary quality checks. Company statement, not independent.
Read first: The whole post.
What it implies for the protocol
- Do not rely on an instruction to be critical. (critic-03 reports that rewriting statements as questions beat a plain 'do not be sycophantic' instruction; critic-04 reports a third-person framing cut sycophancy by up to 63.8 per cent in its debate scenario, metric to be checked; critic-01 reports that standard mitigations had limited effect.)
- Run the Adversary on the Reading and on its negation for every entry, and flag the entry if both are approved. The two-sided test is the cheapest direct check on agreeableness. (critic-01: models affirmed both sides of a moral conflict in 48 per cent of cases; critic-02: agreeable answers are rated better, so approval cannot be judged by feel.)
- Require each objection to name a checkable observation (a passage, a place, a count, a document) and the cheapest check; reject objections that cannot be checked and count them. The outside reader rates each as checkable or generic, and invented objections are logged as misses. (critic-14: critics can find more than human reviewers but invent problems; critic-15: persuasion came from specific counter-evidence to the person's own stated reasons.)
- Use a different model family for the Adversary from any model that drafted the Fiction or suggested where to look, and hide authorship of the material it judges. Disguise alone is not enough, so the family rule is the control. (critic-06: synonym swaps reduce self-preference but fuller neutralisation lets it return; critic-07: a panel of other models with majority vote cut self-preference by over half in one setting; round one, agents-10.)
- Prefer several independent runs of the Adversary, from at least two model families, over a debate exchange. Each run sees only Trace and Reading, never another model's output, and the human compares runs. (critic-10: voting accounts for most of debate's gain; critic-11: models shift from right to wrong answers after reading peers, and debate can lower accuracy; critic-12: model heterogeneity improves multi-agent setups.)
- Keep two-sided argument only for contested Readings and decide it by a person who opens the sources, with both orders of presentation and evidence written before any verdict. Treat the debate gains as established only where an answer key exists. (critic-13: debate beat consultancy 84 to 74 per cent on a reading-comprehension task; critic-09: order can flip the ranking; critic-12.)
- Test the Adversary before the study and again at each model update: reorder, lengthen and add authority cues to the same Reading and see whether its verdict moves; then push back once after its first objection and see whether it retreats. Log the version string every run. (critic-08: twelve judge biases tested by altering inputs; critic-04: multi-turn pressure measure; critic-18: a version change altered agreeableness and routine checks missed it.)
- State in the method text that the Adversary is a prompted devil's advocate and not a dissenter, that it does not replace the outside reader or a sincere human objector, and that its failure to break a Reading is weak evidence for it. Judge the Adversary's quality by someone other than the author. (critic-05: authentic dissent beat role-played devil's advocates; critic-02: users rate agreeable models higher and become surer they are right.)
- Bound the role to proposing counter-examples and objections that a human adjudicates, and record each objection's fate (accepted, rejected with reason, unchecked). The human holds the Test result. (critic-16: counter-examples for human review as the role where different philosophies of qualitative work can agree; critic-17: tools that challenge and a human who decides.)
- Keep a pilot comparison in the first study: for a random half of the Readings, a person also writes a Test; report checkable versus generic, invented, and how many Readings broke. No source tests an LLM critic on interpretations of places or documents, so report the figures as a pilot. (critic-14, critic-15, critic-16; gap below.)
- Show Alter's Fiction to no Adversary run: the Adversary writes its Test from the Trace and Reading alone, because a fluent world in which the Reading is true is a framing, and models accept the framings they are given.
Open questions
- Does any study test an LLM critic on a human's interpretation of a document or place, with an independent standard for what counts as a good objection?
- How large are the reductions in practice? Read the full papers before quoting any figure.
- Vendors report large drops in sycophancy between versions (an Anthropic post and research page, seen only in search summaries and not selected), so the benchmark numbers date quickly and the calibration step must be rerun.
- Is the Test register better served by one critic with strict format, several independent runs, or a two-sided debate? Evidence from answer-key tasks favours voting (critic-10) and heterogeneity (critic-12); the only favourable debate result is with answer keys (critic-13, agents-11).
- Does a prompted critic with a 'steelman the opposite reading' task count as dissent in Nemeth's sense?
- Not selected, seen in search results only: Sycophancy Is Not One Thing (Vennemeyer et al., arXiv 2509.21305, separates sycophantic agreement, sycophantic praise and genuine agreement as distinct directions in models); Chiang et al., LLM-powered devil's advocate for groups (IUI 2024, DOI 10.1145/3640543.3645199); Huang et al. 2023, models cannot self-correct reasoning (arXiv 2310.01798); Perez et al. 2022, red teaming with language models (EMNLP, DOI 10.18653/v1/2022.emnlp-main.225); De Araujo et al. 2025 on expert personas (EMNLP) and an arXiv paper titled When 'A Helpful Assistant' Is Not Really Helpful (2311.10054), which found persona prompts do not reliably improve task performance; LLM-REVal (arXiv 2510.12367) on LLM reviewers favouring LLM-style text. Persona effects on critique quality specifically: no study found.
- For the main session, not recorded by me: check the Science version of Cheng et al. against the arXiv version; confirm the authors of critic-07; read Costello et al. for how the model's claims were fact-checked; decide whether the Adversary may ever see the Fiction.
AI dialogue and belief, and the Science paper
The evidence on AI dialogue changing beliefs is the best-funded part of this review and the most unsettled in status. The anchor paper (science-01) reported that GPT-4 Turbo arguing against a believer's own conspiracy theory cut belief by about 20%. On 11 June 2026 Science attached an Editorial Expression of Concern (science-02): the authors found that screening criteria in the manuscript did not match the published analysis pipeline and that the public dataset held extraneous spliced rows. They supplied raw data and updated results, said to match the original in direction, significance and size; Science was still evaluating them. The paper is flagged, not retracted; the authors published correction materials on 27 September 2026 (science-04); Science's formal outcome and any replication paper were not found. Nyhan reports two independent reproductions (science-03, second-hand). The wider literature says three things that matter more for Holagram than the single paper. First, what persuades is information and conversation, not warmth or perceived humanness (science-05, science-08, science-09, science-07), and the same machinery persuades toward falsehood about as well (science-06, science-11), with accuracy falling as persuasiveness rises (science-08). Second, agreeable models make people surer they are right and prefer them for it (science-12), and simulations suggest neither accuracy nor warnings stops a spiral (science-13); chat-log work shows the pattern in practice (science-14). Third, repeated dialogue may lower unaided judgement (science-15). For Holagram the lesson is that a synthetic Adversary is a persuasion engine whose effect cannot be told from correctness by how convincing it sounds. Its output is a lead, never a Trace; the researcher's own Test comes first; the structure around the model, not the model's good behaviour, must carry the load. No study was found testing an LLM critic against a researcher's own interpretive readings.
- Thomas H. Costello, Gordon Pennycook, David G. Rand (2024). Durably reducing conspiracy beliefs through dialogues with AI. Science 385, eadq1814 (13 September 2024)checkedmust read
The strongest single result for a model arguing against a person's own strongly held reading with specific evidence. It bears on the Adversary, but as of June 2026 its reported values are under an Editorial Expression of Concern, so no number from it should be cited in the Holagram method text without that caveat. The design (a consenting believer, a stated belief, a topic with a known factual record) differs from a researcher's reading of a plan document or a street, where no answer key exists.
Read first: Abstract only, as summarised in coverage: 2,190 believers, GPT-4 Turbo arguing against a self-described belief, about 20% fall in belief, said to persist for two months. Read it only together with science-02 to science-04.
- H. Holden Thorp (Science) (2026). Editorial Expression of Concern on 2024 Research Article 'Durably reducing conspiracy beliefs through dialogues with AI'. Science 392(6803):1131, 11 June 2026; DOI 10.1126/science.aej2383correctedmust readopen access
Settles the status question for the protocol: the paper is flagged, not retracted, and the problems are in data handling and reporting, not (so far as the notice says) in the direction of the effect. The expression of concern is itself a worked example of Holagram's own Trace rule: a value is only as good as the dataset someone can open and re-run. Use it as the model for a hands-and-data record that survives a reader trying to reproduce each number.
Read first: The notice itself (short). Summary of what it says: after publication the authors were made aware of reporting inconsistencies and public-dataset problems that made some reported values hard to reproduce; the authors found that screening criteria were applied inconsistently between the manuscript and the published analysis pipeline, and that the public dataset held extraneous spliced rows from a code-merging error; they gave Science raw data and updated results, which Science was still evaluating; the authors' reanalysis is reported to match the original in direction, statistical significance and substantive size.
Audit: Title is 'Editorial Expression of Concern on RA ...' (EurekAlert); cite Crossref DOI 10.1126/science.aej2383 and Science 392:1131 as the notice itself. Digest URL is the press release, not the notice.
- Avery Orrall (Retraction Watch) (2026). Science flags paper that found AI chatbots help debunk conspiracy theories. Retraction Watch, 11 June 2026checkedopen access
It records an author response and an independent replication claim (Nyhan, reported second-hand, not seen as a paper). Treat both as claims, not results: no replication paper was found. Useful for Holagram as a case of a failure that was self-reported and logged, which is the culture the failure log is meant to build.
Read first: The whole article (short).
- Thomas H. Costello, David G. Rand, Gordon Pennycook (2026). Durably reducing conspiracy beliefs through dialogues with AI: Correction and reproducibility materials. Zenodo, DOI 10.5281/zenodo.22754984, version 1.0, 27 September 2026correctedopen access
The authors' answer to the expression of concern: headline analyses re-specified as ordinary least squares with pretreatment belief centred on the screened cohort, persistence analyses keep the original follow-up restrictions, protest-intention results use only Study 2b. It shows what a reproducible hands-and-data package looks like for a dialogue study, and it is a template for Holagram's per-entry record (prompt, model version, outputs, checksums). Science's formal outcome (correction, clearance or retraction) was not found as of 14 October 2026.
Audit: Digest has no date issue except it is dated 2026 which is right (27 Sep 2026). Authors order here Costello, Pennycook, Rand (digest had Rand second-last). Not a separate paper from the Science article: a supplement.
- Esther Boissin, Thomas H. Costello, Daniel Spinoza-Martin, David G. Rand, Gordon Pennycook (2025). Dialogues with large language models reduce conspiracy beliefs even when the AI is perceived as human. PNAS Nexus 4(11), pgaf325 (online 14 October 2025)checkedmust readopen access
The cleanest evidence on facts versus rapport: tone and perceived humanness did not matter, and the content of the argument did, though every arm still got fact-based argument, so rapport alone was not tested. For Holagram it means the Adversary can be written in a dry, non-warm register without losing force, and that warmth is not what would make a researcher listen. It also shows the effect is not specific to believers: a poorly supported belief moved too, which is nearer to a researcher's overconfident reading.
Read first: Abstract and the discussion of limitations (single model, English and Western training data).
- Thomas H. Costello, Kellin Pelrine, Matthew Kowal, Jasper Timm, Antonio A. Arechar, Jean-Francois Godbout, Adam Gleave, David Rand, Gordon Pennycook (2026). Large language models can effectively convince people to believe conspiracies. arXiv:2601.05050 (v1 8 January 2026; v3 15 July 2026); not seen as publishedcheckedmust readopen access
The same research group shows the mechanism is symmetric: a model can move belief toward a falsehood about as well as toward the truth, and people prefer it when it does. For Holagram this is the core warning for Alter and Adversary alike. A model argues whichever side it is given, so an Adversary against the Reading is also a machine that could argue for it, and Alter writing the world where the Reading is true is, structurally, the promoting condition. The protocol must therefore treat persuasiveness as no evidence of correctness, and constrain both roles to checkable claims.
Read first: Abstract, then the guardrail condition.
- Thomas H. Costello, Nathaniel Rabb, Michael Nicholas Stagnaro, Gordon Pennycook, David G. Rand (2026). Reducing belief in conspiracy theories as they unfold using large language models. arXiv:2608.06151v1, 6 August 2026; not seen as publishedcheckedopen access
The best 2026 evidence that interaction adds something beyond information, which weakens a pure facts account. It also gives an Adversary style for cases without an answer key, which is Holagram's case: when the record is thin the model asked questions and pointed to sources instead of asserting counter-facts. That is the style to specify for the Adversary on interpretive readings of a place. Preprint, and a descendant of the paper under an expression of concern, so read with care.
Read first: Abstract and the strategy analysis.
- Kobi Hackenburg, Ben M. Tappin, Luke Hewitt, Ed Saunders, Sid Black, Hause Lin, Catherine Fist, Helen Margetts, David G. Rand, Christopher Summerfield (2025). The levers of political persuasion with conversational artificial intelligence. Science 390(6777); DOI 10.1126/science.aea3884; arXiv:2507.13919checkedmust readopen access
Answers 'facts versus rapport' at scale: information density, not personalisation or psychological technique, drives persuasion, and the more persuasive the output, the less accurate its claims. For the Adversary this is a direct risk: a Test written to be forceful will be dense with factual-looking claims that were optimised for effect, not checked. Rule: an Adversary's claims are leads, not Traces, and the more fluent and detailed the objection, the more it needs a human to open the source.
Read first: Abstract; the accuracy trade-off.
- Hause Lin, Gabriela Czarnek, Benjamin Lewis, Joshua P. White, Adam J. Berinsky, Thomas Costello, Gordon Pennycook, David G. Rand (2025). Persuading voters using human-artificial intelligence dialogues. Nature, 4 December 2025; DOI 10.1038/s41586-025-09771-9checked
Preregistered experiments in the 2024 US, 2025 Canadian and 2025 Polish elections: effects larger than typical video adverts, carried by relevant facts and evidence and not by psychological technique, and the claims were not all accurate (models advocating for right-wing candidates made more inaccurate claims). Supports science-08 in a field setting. Participants knew an AI would try to persuade them, so it says little about a researcher quietly relying on a model's critique.
Read first: Abstract only.
- Francesco Salvi, Manoel Horta Ribeiro, Riccardo Gallotti, Robert West (2025). On the conversational persuasiveness of GPT-4. Nature Human Behaviour (2025)corrected
Evidence that personalisation helps, which pulls the other way from science-08 (where personalisation mattered little). For Holagram it supports withholding personal detail about the researcher from the Adversary: the model should argue against the Reading, not against the person.
Read first: Abstract.
Audit: Add venue and DOI (no url in digest). Published version 900 participants; preprint 820.
- Philipp Schoenegger, Francesco Salvi, Jiacheng Liu and others (39 authors), Ezra Karger last (2025). Large Language Models Are More Persuasive Than Incentivized Human Persuaders. arXiv:2505.09662 (v1 14 May 2025; v4 13 August 2026); not seen as publishedcheckedopen access
The model is better than paid humans at moving people toward wrong answers as well as right ones, with higher expressed conviction as one suggested reason. For Holagram it argues for an Adversary protocol that strips conviction: short structured output, no rhetoric, claims separated from style, and the human judging checks, not arguments.
Read first: Abstract.
- Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han, Dan Jurafsky (2026). Sycophantic AI decreases prosocial intentions and promotes dependence. Science (2026); preprint arXiv:2510.01395 (1 October 2025)correctedmust readopen access
Moves sycophancy from model benchmarks (agents-03) to a measured effect on the person: one exchange with an agreeable model raised conviction of being right. That is the Ballardian failure with a mechanism: a model that agrees with the Reading makes the researcher surer, and the researcher prefers that model. It is the strongest argument that the Adversary must be tested against agreement, and that the human must not be the sole judge of whether the Adversary was hard enough.
Read first: Abstract.
Audit: Year is 2025, not 2026. Duplicate of critic-02.
- Kartik Chandra, Max Kleiman-Weiner, Jonathan Ragan-Kelley, Joshua B. Tenenbaum (2026). Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians. arXiv:2602.19141 (22 February 2026); not seen as publishedcheckedmust readopen access
A model, not an experiment, but directly on point for Holagram's care rules: if neither factual accuracy nor a warning breaks the spiral, then disclosure and a Fiction label do not protect the researcher's head, and the structural controls (an outside reader who can call a stop, fixed fieldwork blocks, reading-mode off) have to do the work. Simulation results depend on the model's assumptions; read as an argument, not as evidence of size.
Read first: Abstract.
- Jared Moore, Ashish Mehta, William Agnew, Jacy Reese Anthis, Ryan Louie, Yifan Mai and others (14 authors), including Desmond C. Ong (2026). Characterizing Delusional Spirals through Human-LLM Chat Logs. arXiv:2603.16567 (18 March 2026); a search result lists ACM FAccT 2026 proceedings, not confirmed on a venue pagecheckedopen access
The nearest thing found to an empirical record of the spiral Holagram is designed to prevent, from the user's side. If the summary is right, the markers are practical: long sessions, a model that affirms nearly every turn, a persistent shared world. Those translate into session limits, no memory across entries, and a rule that an agreeing Adversary run is flagged. Small and self-selected, so it shows the pattern exists, not how common it is.
Read first: Abstract, once it can be read.
- Anku Rani, Valdemar Danry, Paul Pu Liang, Andrew B. Lippman, Pattie Maes (2026). Dialogues with AI Reduce Beliefs in Misinformation but Build No Lasting Discernment Skills. CHI 2026 (Barcelona, 13-17 April 2026); arXiv:2510.01537v2, 13 March 2026not re-checkedopen access
It supports a rule that the researcher writes their own Test before seeing the Adversary's, and that the Adversary asks questions instead of delivering verdicts. It also gives Holagram a measurable: does the researcher's unaided Test get better across the three weeks?
Read first: Abstract and the strategy analysis.
- Thomas H. Costello, Gordon Pennycook, David G. Rand (as I recall; confirm) (n.d.). Just the facts: how dialogues with AI reduce conspiracy beliefs. preprint, PsyArXiv (as I recall)recalled, unverified
If it says what the secondhand summary reports, it is the direct test of facts versus rapport for the 2024 design, and the companion to science-05. Find it and read it before the method text claims that evidence, not manner, does the work.
- authors not seen (2025). Conformity in Large Language Models. ACL 2025 (Long Papers); URL seen: aclanthology.org/2025.acl-long.195mismatchopen access
A narrow lead that a devil's-advocate role can reduce a model's own conformity to a group answer. It concerns models, not researchers, and no study was found that tests an LLM devil's advocate against scientists' own readings. Read it before relying on it for any Adversary design.
Audit: Authors and title not verified.
What it implies for the protocol
- R1 (science-01, science-02, science-04): Do not cite effect sizes from Costello et al. 2024 in the Holagram method text. If the paper is cited at all, say it carries an Editorial Expression of Concern (11 June 2026), that corrected materials were posted on 27 September 2026, and that Science's outcome was not found; recheck before any submission. Use it only for the design idea (specific, tailored counter-evidence to a stated belief).
- R2 (science-08, science-09, science-06): Adversary output is a set of leads, never Traces. Every factual-looking claim in an Adversary Test (a count, a date, a clause in the plan, a rule, a quotation) is marked unverified until a person opens the source and records hit or miss. The more detailed and forceful the objection, the higher the priority to check it, because persuasiveness and accuracy move in opposite directions.
- R3 (science-06, science-11): Run the Adversary in both directions on every strong Reading: against it and for it, same prompt template, same model. Log both. The pair, not either run, is the Test input. If the case for the Reading reads as well as the case against, record that the model has no discriminating force on this entry. Persuasiveness of either run counts for nothing.
- R4 (science-12, science-14, science-15): The researcher writes their own Test (what would show the Reading wrong, and the check) before reading any Adversary output, and the two are recorded side by side. Over the three weeks track how often the unaided Test names something checkable. This protects against the deskilling and growing agreement seen in science-15.
- R5 (science-12, science-13, science-14): Treat agreement as a fault, not a result. If the Adversary fails to produce a checkable contradiction, flag the entry rather than closing it as held. Check the Adversary on a Reading and its negation (agents-03) and on a planted weak Reading whose flaw is known to the outside reader; an Adversary that passes the planted weak Reading is not used for that block.
- R6 (science-13, science-14, science-15): Do not rely on disclosure, a Fiction label or a model warning to protect the researcher. Use structural limits: one structured pass per Reading (a brief in, a list of checks out), no open-ended chat, no memory carried between entries, a cap on turns per session, and the outside reader's right to stop a block. Long open dialogues with a model that affirms most turns are the pattern in the spiral logs.
- R7 (science-05, science-07): Specify the Adversary's register as dry and non-personal. Tone and persona do not add force (science-05), so strip them. Where the record is thin, as with most places and documents, the useful style is epistemic caution, questions and pointers to sources (science-07), not assertions of counter-fact. Ask for: the strongest observation that would contradict the Reading, the cheapest check, and what source to open.
- R8 (science-10, science-11): Give the Adversary the Trace and the Reading and nothing about the researcher: no stated hopes, no biography, no preferred outcome. Personalisation increases persuasion in some studies (science-10), and conviction in expressed style is one candidate reason models out-persuade humans (science-11). Cap output length and require claims in a list separate from any prose.
- R9 (science-06, science-08): Alter is the promoting condition. A model asked to write the world where the Reading is true is doing, structurally, what the 'bunking' arm did. Fiction must stay labelled and out of Trace, and also must not be shown to the researcher in the same block as the Adversary's run on that Reading, so that fluent confirmation and fluent objection are not read side by side as two voices of equal standing. Log the order.
- R10 (science-03, science-04, science-02): Make the record reproducible by someone else. For each entry keep prompts, model name and exact version, date, raw outputs, rejected runs and a checksum of the export, as the correction package for science-01 does after the fact. Report auditability, not replication; run-to-run variation is expected.
- R11 (science-01, science-05, science-17): Do not claim evidence that an LLM Adversary improves a researcher's readings. The supporting studies concern beliefs with known answers or models' own conformity; none tests interpretive readings of places or documents. Present the first study as a calibration of the Adversary (checkable versus generic objections, hits and misses on its claims, breaks produced) and report the numbers as pilot data.
Open questions
- Has Science issued a correction, cleared the paper or retracted it since the 27 September 2026 correction package? Check science.org and the correction ledger inside the Zenodo archive for the changed numbers.
- Is there an independent replication paper of the DebunkBot design (Nyhan's two experiments are second-hand)? None found.
- Not opened.
- Why does personalisation matter in Salvi et al. (science-10) and little in Hackenburg et al. (science-08)? Differences in design and model generation are not reconciled here.
- Does the Science version of Cheng et al. (science-12) have the numbers the search result gave (3 experiments, 2,405 participants, 49%)? Only the arXiv numbers were seen.
- No study found tests an LLM critic against a researcher's own interpretive reading, or measures whether researchers update after an AI critique. Is the nearest design a planted weak Reading, as in R5?
- What happens to the Adversary's force when it is asked for questions and sources only? science-07 suggests it still works, but on conspiracy beliefs with a record.
- Whether to read Moore et al.
The Alter role: model-written fiction
The literature on model-written fiction is consistent on one point and thin on another. It is consistent that fiction written by models converges: one model gives the same plot across settings (alter-01), the same plot elements across runs (alter-02), a shared region of narrative space across models (alter-04), and tidier, more explicit, less ambiguous stories than people write (alter-04). Prompting tricks and temperature do not fix it (alter-03). Co-writing with a feedback-tuned model narrows what people write (alter-05). For Alter this means a single Fiction is the likely version of the world, and its value lies in what the human edit changes. It also means that ten Fictions from one model are evidence of the model, not of the place. The second point is leakage. A working preprint (alter-08, self-published, not peer reviewed) reports that a model continues inside an elaborate fiction without correcting it and that the fiction can absorb real findings as apparent confirmation; this is the Ballardian narrator's failure with a machine supplying the material. Sycophancy work extended to implicit premises (alter-09) says the Adversary will protect the author's face, not only agree with stated claims. On the positive side, fiction has a respectable standing as a research tool (alter-11, alter-12), and models have been used to produce scenarios that start debate in teams, with the authors insisting on critical use (alter-13). Labelling is the weakest part: an AI label lowers trust in true and false items alike because readers assume full automation (alter-15), and detailed disclosure lowers trust while raising source-checking (alter-16), so the label must say what the human did. No source tests a model-written fiction against a researcher's own Trace and Reading, or a model critic against interpretive claims about a place. The protocol below is therefore a design, not a validated method.
- Jill Walker Rettberg, Hermann Wigers (2025). AI-generated stories favour stability over change: homogeneity and cultural stereotyping in narratives generated by gpt-4o-mini. arXiv:2507.22445; also listed as Open Research Europe 2025, 5:202 (awaiting peer review when listed)checkedmust readopen access
Shows homogenisation across settings, not only across runs: stories for 236 countries share one small-town plot of minor conflict resolved by tradition and community events, with conflict softened and tension low. Threatens Fiction about a place: Alter will tend to return a reconciling, low-conflict version of the study site whatever the Trace says. Supports a rule that the Fiction prompt names the Reading's tension and asks for the version in which it is not resolved, and that the human log records how often Alter resolved a conflict the Reading left open.
Read first: Abstract, then the description of the shared plot.
- Weijia Xu, Nebojsa Jojic, Sudha Rao, Chris Brockett, Bill Dolan (2025). Echoes in AI: Quantifying lack of plot diversity in LLM outputs. Proceedings of the National Academy of Sciences 122(35), e2504966122correctedmust readopen access
Gives a measurable version of convergence: the same plot beats recur across generations from one prompt, and human-written stories are rarely echoed. Supplies a method for Holagram: compare Alter's runs for one entry and flag Fictions whose beats recur, as the clerical form of the interchangeable-Fictions log. Limit: measured on GPT-4 and LLaMA-3 and on whole stories, not on 150-word documents in a world.
Read first: Abstract and the definition of the Sui Generis score.
Audit: Add venue PNAS 122(35); article, not a preprint.
- Thennal DK, Hans Ole Hatzel (2026). Do Large Language Models Always Tell The Same Stories?. arXiv:2606.17350 (v1 15 June 2026, v2 2 September 2026)checkedopen access
Newest and widest test: ten models, and the stories are more alike each other than human stories are. The reported failure of negative prompting and temperature means that asking Alter to be different is not a safeguard; the safeguard has to be structural (different models, different human-supplied constraints, human editing). Preprint, revised once; treat results as provisional.
Read first: Abstract and the section on mitigation.
- Jenna Russell, Rishanth Rajendhran, Chau Minh Pham, Mohit Iyyer, John Wieting (2026). StoryScope: Investigating idiosyncrasies in AI fiction. arXiv:2604.03136 (v1 3 April 2026, v6 10 August 2026); under review per secondary notecheckedmust readopen access
The best description of what Alter's output will look like: explicit about its theme, tidy, single-thread, morally resolved. Holagram wants Fiction that raises a question the Trace and Reading did not, and ambiguity is central to its concept of lived experience. This gives checkable features for the human editor (is the theme stated outright, is the plot resolved, is there one thread) and a reason to treat resolution as a defect in Fiction. It also suggests a fingerprint test: an outside reader could try to pick the synthetic Fictions.
Read first: Abstract, then the list of narrative dimensions.
- Vishakh Padmakumar, He He (2024). Does Writing with Language Models Reduce Content Diversity?. ICLR 2024; arXiv:2309.05196 (v3 1 July 2024)not re-checkedopen access
Moves the convergence question from the model to the pair. Co-written essays with a feedback-tuned model were more homogeneous, and the effect came from the model's text. For Holagram: the model type matters, a base or less tuned model may be a better Alter than a chat-tuned one, and the human edit should be logged as a diff so the share of model text that survives is visible. Argumentative essays, not fiction; transfer is assumed.
Read first: Abstract.
- Nina Begus (2024). Experimental Narratives: A Comparison of Human Crowdsourced Storytelling and AI Storytelling. Humanities and Social Sciences Communications 11, article 1392; arXiv:2310.12902not re-checkedopen access
A method precedent: fiction collected under a fixed prompt and treated as data about the cultural imaginary, human and model side by side. It supports reading Alter's Fictions as evidence about the model's inherited stock of stories, which is a legitimate finding and a different one from evidence about the place. Suggests a control for the first study: the same Fiction prompt run with the Trace removed, to see what the model supplies by default.
Read first: Abstract.
- Bartosz Lutostanski (2026). Poza fikcjonalnoscia. Symulacja i narracja w generowanych przez AI opowiadaniach o polskich wykladowcach akademickich (English title given elsewhere as Simulation and Narrative in AI-Generated Stories about Polish University Teachers). Forum Poetyki, No. 1 (43), 2026, pp. 40-55 (published 26 June 2026); DOI 10.14746/fp.2026.43.3not re-checked
The theoretical challenge to the fiction label. If fictionality needs a communicating subject, then a model's story is a simulation of fiction, and the label Fiction on Alter's output does less than it does for a human-written story. For Holagram: the label protects the evidence record, but it cannot be what makes the text fiction. What makes it usable is the human who selects, edits and signs it. Polish-language article; the English route is the English PDF the journal links (not read).
Read first: Abstract.
- Rafael Oliveira, Jameson Bednarski (2026). Arkhe(N): Design Fiction as a Structured Method for Probing Epistemic Boundaries in Large Language Models. DeSci Nodes, dPID 990, DOI 10.62891/45c2ef5a; self-published, no peer review statedcheckedmust read
The only source found that addresses fiction leaking into evidence with a model in the loop, and the closest published match to the Ballardian failure: a coherent invented frame that takes in real results as confirmation. Weak evidence (two authors, one model, self-published, not replicated) but the described mechanism is exactly what the fiction-leak check must catch. Supports rules that Fiction is never fed real findings as source material, that the model is never asked to extend a Fiction across entries, and that the human closes each Fiction rather than running it on.
Read first: Abstract and the three findings.
- Myra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe, Lujain Ibrahim, Dan Jurafsky (2025). ELEPHANT: Measuring and understanding social sycophancy in LLMs. arXiv:2505.13995 (v1 20 May 2025, v2 29 September 2025); an ICLR 2026 listing was seen in a search summary but not checkedcheckedmust readopen access
Extends the round-one sycophancy evidence (Sharma et al.) from stated claims to implicit premises and self-image. A Reading handed to the Adversary carries the author's stake in it even if no preference is stated, and a Fiction written in a world where the Reading is true is itself an implicit premise. Supports a protocol in which the Adversary receives the Reading in neutral third-person form, is run on the Reading and its negation, and is tested on the both-sides case. Tested on advice and moral conflict, not on readings of places.
Read first: Abstract.
- janus (2022). Simulators. LessWrong post, 2 September 2022 (also hosted at generative.ink)not re-checkedopen access
A practitioner essay, outside academia, on what a model does when it writes in a world: it runs a character, not a belief. Companion to Shanahan et al. (round one). Useful vocabulary for the method text: Alter is a simulator run on a prompt, its characters are simulacra, and nothing a character says is a report. Also a warning for Adversary prompts: a model told to be a critic simulates a critic, and a critic in a prompt is a mask the author chose.
Read first: The opening sections defining simulator and simulacra.
- Nanditha Krishna, Alessandro Ianniello (2026). Beyond 'The Circle': Triangles, stories, and sociotechnical futures. TATuP, Zeitschrift fuer Technikfolgenabschaetzung in Theorie und Praxis, Vol. 35 No. 2 (2026), special topic Science fiction and technology assessment (eds. Mehnert, Hermann, Belsunces Goncalves, Schmeink, Ishak); DOI 10.14512/tatup.7278not re-checked
Technology assessment is the nearest established field that uses fiction as a method with a procedure and a claim about knowledge: fiction brings moral and emotional knowledge into view alongside expert reading. It gives Holagram a precedent for the claim that Fiction is an epistemic device and not decoration, and a home journal for the method. The fiction here is human-written; nothing on models.
Read first: Abstract; then the special-topic overview for further papers.
- Elizabeth de Freitas, Sarah E. Truman (2021). New empiricisms in the Anthropocene: thinking with speculative fiction about science and social inquiry. Qualitative Inquiry 27(5), 522-533; DOI 10.1177/1077800420943643not re-checkedopen access
A qualitative-inquiry argument that speculative fiction can supply images of method, here a biologist whose method changes in a disaster zone. It is the nearest precedent for Holagram's claim that a novel can model the researcher's own practice, and VanderMeer's Annihilation is a case of a narrator whose reading of a place changes the reader. Links Fiction to the Ballard line and to the method's care rules. Human-written fiction only.
Read first: Abstract.
- Jan Ferrer i Pico, Michelle Catta-Preta, Alex Trejo Omenaca, Marc Vidal, Josep Maria Monguet i Fierro (2025). The Time Machine: Future Scenario Generation Through Generative AI Tools. Future Internet 17(1), 48 (MDPI)not re-checked
The nearest case of models used to write futures for a team: scenarios were judged by whether they started discussion, not by whether they were accurate, and the authors insist on critical use. That is Alter's success test in another setting (Fiction raises a question), and the 70 per cent and over-half figures give a rough base rate to compare a first study against. The measure is participant-reported and the tool was built by its authors, so treat it as a pilot.
Read first: Abstract.
- Jussi Holopainen, Yuxuan Huang, Joongi Shin, Erkka Nissinen, Andres Lucero (2025). Infinity Book: Speculating Literary Expressions in the Age of Generative AI. DIS '25: Proceedings of the 2025 ACM Designing Interactive Systems Conference, pp. 1430-1454not re-checked
Design fiction about model-written literature, by a group partly based in Finland (Aalto). It treats authenticity and creative agency as the live questions, which are the questions the Alter role raises for a research method. A model for how a design-fiction study of Alter itself could be run: workshops with an invented system rather than a real one. Not a study of model-written fiction as method.
Read first: Abstract and the three concepts.
- Sacha Altay, Fabrizio Gilardi (2024). People are skeptical of headlines labeled as AI-generated, even if true or human-made, because they assume full AI automation. PNAS Nexus 3(10), pgae403not re-checkedopen access
Evidence on what a label does to readers: a bare AI-generated label lowers belief in true and false items alike, because readers read it as no human involved. Holagram's Fiction label is about status (not evidence), but a bare label will be read as machine-made and so as less to be trusted, or as ignorable. Supports a label that states the human role as well: written by model X from a prompt, selected and edited by the author. News headlines, not fiction.
Read first: Abstract.
- Pooja Prajod, Hannes Cools, Thomas Roggla, Karthikeya Puttur Venkatraj, Amber Kusters, Alia ElKattan, Pablo Cesar, Abdallah El Ali (2026). Full Disclosure, Less Trust? How the Level of Detail about AI Use in News Writing Affects Readers' Trust. arXiv:2601.09620 (v1 14 January 2026)checkedopen access
Small study (40 participants) with a useful split: disclosure lowers trust only when detailed, but raises checking at every level. For a research method the second effect is the one wanted. Supports a two-layer label: one line on the Fiction itself and the full hands card on demand, which the readers in the study asked for. Very small sample, news context.
Read first: Abstract.
- Ted Chiang (2024). Why A.I. Isn't Going to Make Art. The New Yorker, Weekend Essay, 31 August 2024not re-checked
The strongest practitioner argument against the Alter role: if a story is made of choices and a prompt supplies few of them, then a model's Fiction has little in it that the author chose. It sharpens a Holagram design question: Fiction should carry as many of the author's choices as possible (the Trace, the Reading, constraints, a human rewrite), so that the model contributes wording and the person contributes decisions. Opinion essay, not evidence.
Read first: The whole essay (short).
- Lydialyle Gibson (on the poet Sasha Stiles) (2026). For This Poet, AI Is a Writing Partner. Harvard Magazine, Arts and Culture, 9 April 2026 (print: May-June 2026, Where Poetry Meets AI)not re-checkedopen access
An artist's account of a model as a named second voice, trained on her own material rather than on the average. Two points for Alter: the model is useful when it is bound to the author's own trace (her poems), and the voice that comes back can feel like the author's own, which is the self-confirmation risk stated by an artist. The article is a profile, and says nothing on provenance, so it is a case to ask the artist questions of, not a protocol.
Read first: The passages on the alter ego.
What it implies for the protocol
- Alter never receives the Reading's author-side preferences, and its prompt names the tension in the Reading and asks for a version in which the tension is not resolved. Convergent model fiction resolves conflict and states its theme (alter-01, alter-04), and negative prompting and temperature do not fix this (alter-03), so the instruction is a request and the audit is the control.
- Run Alter at least twice per entry with different human-written constraints, and at least once per study on a second, differently built model; use a less tuned model where one is available. Log which Fictions a clerk or the outside reader judges interchangeable, and flag any set whose plot beats recur (alter-02, alter-03, alter-05).
- Make the human edit the unit of record. Store the model output and the edited Fiction as a diff and log the share of model text that survives; if the Fiction is mostly unedited model text, mark it as a model default and not as a finding (alter-05, alter-17, alter-07).
- Treat Fiction as evidence about the model, not the place. Keep a control run per study: the same Fiction prompt with the Trace removed. Differences between the control and the real run show what the Trace added; sameness shows what the model supplies by default (alter-06, alter-01).
- Give the editor and the outside reader a short fingerprint checklist for Fiction: theme stated outright, single thread, plot resolved, emotion given as bodily sensation, no named external text. Each tick marks a default, and a Fiction with all ticks is not allowed to prompt a new Reading (alter-04).
- Never feed Alter real findings as source material beyond the Trace and the Reading of its own entry, never ask it to continue a Fiction across entries, and close each Fiction by hand. Extended fiction frames absorb real results as apparent confirmation and are not corrected by the model unless asked about itself (alter-08).
- Close every entry with the fiction-leak check: list each factual-looking claim in the Fiction and mark it invented; a later Reading resting on one is invalid until a Trace exists (alter-08, alter-07, round one agents-08).
- Fiction label carries three things: the word Fiction, the model name and date, and the human role (selected, edited, signed). Give the full hands card behind it on demand. A bare AI label is read as no human involved and lowers trust in true and false items alike; detailed disclosure lowers trust a little but raises source-checking, which is the aim (alter-15, alter-16).
- Do not rely on the label alone to make the text fiction. A model's story simulates the form of fiction; the label protects the evidence record and the human edit supplies the authorship. Each Fiction is signed by a named person who answers for it (alter-07, alter-17).
- Run the Adversary on a neutral third-person statement of the Reading, never on a Fiction and never with the author's stake, and run it on the Reading and its negation. Treat approval of both as a failure of the Adversary. Add a both-sides check at the start of the study, since models affirm whichever side the user takes (alter-09, round one agents-03).
- The Adversary must name something checkable (a document, a place, a count) and the person judging it opens the source. Keep the Adversary on a different model family from Alter, so that it does not judge text its own family wrote (alter-09, round one agents-10).
- Hold Fiction to the success test already in the concept, that it raises a question the Trace and Reading did not, and record it per entry; compare with the base rate reported for model scenarios, where over 70 per cent prompted discussion in a team (alter-13, alter-11).
- Bind Alter to the author's own material where it helps (the entry's Trace and the author's own prior notes) and keep it away from generic setting knowledge, the place where the default plot comes in (alter-18, alter-01, alter-06).
Open questions
- Does a model-written 150-word document placed inside a world show the same convergence as whole stories? All the measured work is on stories of hundreds or thousands of words, and none is on planning-document pastiche.
- Does any model critic improve on a human one when the claim is an interpretation of a place or a document? No source tests this; sycophancy work uses advice, moral dilemmas and questions with answers.
- Is the unfalsifiable-absorption result (alter-08) real? It is a self-published, single-model, two-author preprint with no replication; someone should try the same probe on a second model before it is cited in the method text.
- Does the Fiction label change how the author reads the Fiction, as opposed to how others do? The labelling studies measure readers of news; none measures the author who commissioned the text.
- Is a base model, a fine-tuned local model or a chat model the best Alter? The convergence evidence points one way (alter-05), the version-pinning argument from round one points to local models, and quality for 150-word fiction in Finnish or Estonian is untested.
- Round-one open item: whether the Reading register may ever take a synthetic hand. Nothing here supports it.