Loading prices…
〽️NEUTRAL

Claude Identifies Authors of 56% of Private Reviews

A small study highlights how unsigned peer reviews can reveal their authors, though its six-person sample leaves the wider risk uncertain.

Claude Identifies Authors of 56% of Private Reviews
Claude Identifies Authors of 56% of Private Reviews

Anthropic's Claude Fable 5.2 correctly identified the authors of 56% of 398 unsigned reviews and meta-reviews written by six volunteers. The model received one report at a time, with no web access, tools, candidate list or other writing samples.

In a separate test of 1,559 public reviews, Claude identified the writer correctly in 21.4% of the 1,530 cases it answered. It was more accurate on public reviews containing self-citations: 80.2% for 106 such reports, compared with 17.1% for 1,453 without them.

The private-review result gave more weight to volunteers who supplied more reports. Weighting each reviewer equally produced an average accuracy of 49.1%. The researchers said a larger, more varied sample is needed to estimate the broader risk to confidential peer review, and a model's identification would still require independent verification.

Frequently asked questions

  1. What information did Claude receive for the private-review test?

    Claude saw one unsigned report at a time, without web access, tools, a list of possible authors or other writing samples.

  2. How did weighting each reviewer equally change the private-test result?

    Average accuracy was 49.1% when each of the six reviewers received equal weight, compared with 56% when each report was weighted equally.

  3. How did Claude perform on public reviews?

    Claude correctly identified the writer in 21.4% of the 1,530 public-review reports it answered, from a set of 1,559.

  4. How did self-citations relate to identification accuracy?

    Claude was correct on 80.2% of 106 public reviews containing self-citations, compared with 17.1% of 1,453 without them.

  5. Can a model's answer establish who wrote an unsigned review?

    No. The model made mistakes, and its identification would require independent verification. The researchers also said a larger, more varied sample is needed to estimate the broader risk.

Source attribution
Aggregated from CryptoSlate · Verified · Last refreshed 39m ago
Open original →