Synthetic agents and collective intelligence

Evidence on models as research instruments, round one.

Most of the evidence on synthetic agents in research concerns two uses Holagram avoids and one it adopts. The avoided uses are standing in for respondents and standing in for evidence. On respondents, the positive claim (Argyle et al.) is met by findings on unreliability and flattening (Bisbee et al.; Wang et al.), and the one Finnish planning pilot found (Jauhiainen et al.) uses synthetic residents to test tools, not to find things out. On evidence, journal policy (ICMJE) already refuses AI output as a primary source, and Hicks et al. give the mechanism: the model is indifferent to truth. That supports the Trace bar as written. Alter is the best supported role: the role-play account (Shanahan et al.) describes what a model does when given a world to perform. The creativity evidence warns that its output converges (Doshi and Hauser), and Steyerl's mean images say why. The Adversary is the weakest. Sycophancy (Sharma et al.) and self-preference (Panickssery et al.) both predict an agreeable critic, and the favourable debate results (Khan et al.) come from questions with an answer key. Documentation is where the literature helps most. Prompt sensitivity and drift (Bisbee; Barrie et al.) show that model named, prompt kept is not enough; CRediT, AI Usage Cards and ICMJE supply formats. The cybernetic line (Negroponte, Pask, Price) gives the dialogue a lineage; Suchman and Houston et al. ask for the grounds of the human/synthetic split.

Sources

Ideas to take

Replace 'model named, prompt kept' with a hands card per entry, built on existing vocabularies.

Changes: The rule that every register carries its hands is extended. Each synthetic contribution gets a card: model name and exact version string, provider, route (chat, API, local), date and time of each run, system and user prompt verbatim, any settings, number of runs and the rule for choosing one, the full output including rejected runs, and the human edit as a diff. Registers map to CRediT roles so the card reads as a standard author contribution statement. Where quality allows, the Adversary runs on a version-pinned local model; otherwise the choice of a proprietary model is justified in one line. The study claims auditability, not replication.

Risk: Volume and effort in a three-week study. Prompts that quote planning documents may carry text under licence, and a card that stores whole outputs may hold personal data from photographs. Local models may be weaker critics. Nothing here has been tested on a method like Holagram's.

Write the Trace bar as explicit cases, including the two the first study currently crosses, and add a fiction-leak check at closing.

Changes: Add to the rules: (a) an AI summary of the planning document is not a Trace, only a quoted passage with page is; (b) the synthetic companion in week 2 describes photographs, but the description is not a Trace and is logged as a prompt or a Fiction, the photograph being the Trace; (c) an agent may suggest where to look, but nothing it outputs is pasted into Trace, quotation marks included, and each suggestion is logged as hit or miss after a person opens the source. The hit rate becomes data about the agent. (d) When an entry closes, list every factual-looking claim in the Fiction (opening hours, counts, who lives where) and mark each as invented; a later Reading that rests on one is invalid until a Trace is found. In the concept's worked entry the opening hours are invented and the count of lit windows is a separate check. (Finnish planners already use AI to summarise plan documents: Raymond et al., 2025, Discover Cities, checked but not listed.)

Risk: The line between suggesting where to look and supplying blurs when a suggestion contains a quotation. The leak check is clerical and will not catch a Reading that has absorbed a fictional detail without citing it. The rule costs time and may feel pedantic on walks; some Traces from photographs will be described better by a model than by the walker, and the rule gives that up on purpose.

Build the Adversary as a small pipeline with sycophancy controls, and test the controls.

Changes: The Test register gets sub-fields. The Adversary is given the Trace and the Reading but not the author's preference, and is asked for the strongest observation that would contradict the Reading and the cheapest check, not for a verdict; each objection must point to something checkable (a document, a place, a count), not to a rhetorical figure. The same Adversary is also run on the negation of the Reading; if both get uncritical approval, the entry is flagged. The Adversary comes from a different model family than Alter. For contested Readings two Adversaries argue opposite sides and a person who can open the sources decides. The outside reader rates each proposed test as checkable or generic.

Risk: The debate evidence is on closed-text questions with answers, so it may not transfer. Triple runs add cost. A person judging two persuasive arguments can still be persuaded by the better writer.

Calibrate the Adversary inside the first study and report the numbers as method evidence.

Changes: In week 3, for a random half of the Readings, the outside reader or another person also writes a Test. Compare how many of the Adversary's tests are checkable, how many are generic (ask the stakeholders), and how many broke the Reading. The concept's own success criterion (some readings break; if none do, the tests are too soft) becomes a measured comparison between the human and the synthetic Adversary. Report it in the write-up as a result about the method.

Risk: About ten entries is a tiny sample, and the comparison is made by the same person who ran the study. Treat it as a pilot, not a finding.

Treat Fiction as the average of its sources and design against convergence.

Changes: Alter runs at least twice per entry with different prompts, and from time to time with a different model; the human edits and logs which of the ten Fictions could swap places. Fiction is given a genre: the short document that would exist in the world where the Reading is true (a leaflet, a notice, a review of a report that does not exist), after Lem and in the line of Ballard's invisible literature. The success test (Fiction raised a question the Trace and Reading had not) is recorded per entry, not once for the study.

Risk: Documents-in-genre look like sources, and published fakes can be mistaken for evidence or cited; each carries a label and none enters Trace. Fluent fiction can also make a Reading feel confirmed, which is the narrator's problem.

Keep identity out of Alter: fiction about acts and situations, not about kinds of resident.

Changes: Add a rule to the Fiction register: Alter prompts carry no demographic labels, and the voices in Fiction are described by what they do (the people upstairs, as in the concept's worked entry). Any statement about what real residents think needs a Trace, such as a quoted sentence or a count. Synthetic personas, if used at all, are used only to pilot an instrument, as in the Turku study, and never to answer a question about the place.

Risk: Fiction without identity may lose the specificity that makes lived experience legible. The rule cannot stop a reader from imagining a stereotype; it only avoids prompting one.

Use the cybernetic line as the method's own argument: partners with defined rights, and a human who holds the last move.

Changes: Describe Alter and Adversary as conversation partners in Pask's sense: they may propose and object, and may be overruled, and the person records each overruling and why (Generator's crane operator is the model: the machine proposes, a human executes). Add an overruled field to the log. A Test may close as held, broke or unchecked, or as a recorded agreement or disagreement between Reading and Adversary. The concept gains a lineage from Negroponte through Pask and Price, and a reason for the dialogue wording.

Risk: Lineage is not evidence. Generator was never built, Negroponte's machines mostly did not exist, and Pask's text is hard to cite correctly. Halpern's history also warns that this tradition made data look neutral.

Say in the method text what the synthetic hands do not do, and answer the objections from reflexive qualitative research.

Changes: A short paragraph states: no synthetic hand codes data, writes Readings, analyses fieldnotes or supplies Traces; synthetic hands produce Fiction and Tests, and a person judges both. The paragraph names the justice and environmental objection from the open letter and records what is done about it (run counts per study, local models where possible). It also makes the Reading rule explicit, since the concept already keeps it human.

Risk: Counting runs addresses visibility, not the objection.

Counter-arguments

Gaps