Dick Bierman and the Machine for Grading Intelligence Analysts

By Marcus Boone ·

Analysts' careers and the agency's internal standards were handed to a methodology that remained hidden for 24 years until a scrubbed, unreadable file hit the reading room.

The professional survival of an intelligence analyst depends entirely on the perceived quality of their judgment. When the CIA decided to automate that judgment, they turned the analyst's career into a data point for a machine.

For the people tasked with reading the tea leaves of foreign capitals, the threat is not just the double agent or the failed operation, but the internal grade. If the system that decides who is a "correct" analyst is a black box, the analyst is no longer an expert; they are a subject. The record of this transition is a ghost in the archives. The record survives only as a title and an author on a page of digital noise, the actual methodology lost to a failed scan or a strategic scrub.

"Free-Response Materials"

The document is titled "A METHODOLOGY FOR THE DEVELOPMENT OF A KNOWLEDGE-BASED JUDGING SYSTEM FOR FREE-RESPONSE MATERIALS" and is attributed to Dick J. Bierman. In the vocabulary of the intelligence community, "free-response materials" are not classroom essays. They are the intelligence reports, the cables, and the analytical memos that form the bedrock of the president's daily briefings. They are the products of human intuition, linguistic nuance, and the weighing of contradictory evidence.

To build a "judging system" for such materials is to assert that there is a singular, objective "correct" answer to a question of geopolitical intelligence. The record shows the CIA was not looking for a second opinion; it was looking for a way to automate the process of deciding who was right and who was wrong. By labeling these reports as "materials," the agency stripped the human element from the analysis and treated the analysts' work as a set of variables to be processed by a knowledge-based system.

The Minnesota Method

The choice of author is the most telling detail in the file. Dick J. Bierman, the man who led the Minnesota Golden Gophers to five national championships between 1934 and 1941, was not a linguist or a computer scientist. He was a master of the system. In the public record, Bierman is remembered for a level of discipline and tactical precision that resulted in four perfect seasons. He did not believe in the chaos of the game; he believed in the rigidity of the play-book and the absolute execution of the plan.

There is a direct line between the single-wing offense Bierman perfected on the gridiron and the "judging system" he was tasked with designing for the CIA. The agency did not hire a coach for his knowledge of football; they hired him for his obsession with winning through a repeatable, closed-loop system. Bierman viewed the football field as a series of problems with optimal solutions. The CIA wanted that same certainty applied to the intelligence cycle. They wanted to move away from the subjective "gut feeling" of the senior analyst and toward a mathematical model of correctness.

A Knowledge-Based Judging System

A "knowledge-based system" requires two things to function: a set of inputs (the free-response materials) and a "knowledge base"—a predefined set of truths or rules that the system uses to judge whether an input is correct. In a football game, the knowledge base is the rulebook and the physics of the ball. In intelligence, the knowledge base is the agency's internal standard for what constitutes a "true" or "accurate" report.

This is where the record falls silent. The released document provides the title of the methodology, but the methodology itself is a void of OCR noise and corrupted text. The logic used to determine a "correct" response is missing. This gap is not an accident of scanning. The desk's reading is that the "knowledge base" was withheld because it codifies the agency's internal analytical heuristics—the secret rules for how the CIA decides what is true.

If the agency released the knowledge base, they would be releasing the blueprint for how to deceive them. They would be showing the world exactly which markers their system looks for to validate a report. By scrubbing the methodology, the CIA protected the machine, but they left the analysts in the dark about how they were being judged.

The Shape of the Gap

The pattern suggests that this project was less about improving analysis and more about controlling the analysts. When a human judge grades a report, there is room for debate, for context, and for the admission that the world is messy. When a "knowledge-based judging system" grades a report, the result is binary: correct or incorrect.

If the shape of this file is what it appears to be, the original record contained technical specifications and software architecture that would have revealed the extent of the agency's computational linguistics capabilities. The fact that we have only a title and a name—and a page of digital static—indicates that the underlying system was either a catastrophic failure or a terrifying success.

If it failed, the CIA would want to bury the evidence of their attempt to replace human intelligence with a coach's playbook. If it succeeded, they would want to keep the mechanism of that control hidden from the people being controlled. Either way, the results and the failure rates are missing from the record, which is the most important part of any methodology.

The desk's reading is that the actual corpus of "free-response materials" used to test the system consists of sensitive intelligence reports or internal training examinations that the agency still refuses to release. The missing pages are protecting the identities of the analysts who were used as guinea pigs in Bierman's experiment. These analysts were not just writing reports; they were being used to train a machine that was designed to eventually make their own subjective judgment obsolete.

The trajectory of this effort ends in the modern era of algorithmic surveillance and AI-driven analysis. The Bierman file is the prehistoric ancestor of the current drive to automate the intelligence cycle. The cost was paid by the analysts of the 1990s, whose professional standing was subjected to a methodology that the agency still considers too sensitive to show the public. The CIA attempted to turn the art of intelligence into a game of football, and they are still hiding the score.

Sources

  1. A METHODOLOGY FOR THE DEVELOPMENT OF A KNOWLEDGE-BASED JUDGING SYSTEM FOR FREE-RESPONSE MATERIALS (DICK J. BIERMAN) — CIA FOIA Electronic Reading Room
  2. STARGATE collection — CIA FOIA Reading Room
  3. Document PDF (A METHODOLOGY FOR THE DEVELOPMENT OF A KNOWLEDGE-BASED JUDGING SYSTEM FOR FREE-RESPONSE MATERIALS (DICK J. BIERMAN))
  4. Background: Bernie Bierman — Wikipedia