The CIA's 1966 Big Data Problem: When Spies Drowned in Paper

2026-05-15

Book: CIA Reading Room cia-rdp79b00873a001800010046-5: PROJECT OBJECTIVE - MATERIALS HANDLING STUDY (NON-DIGITALLY STORED DATA) by CIA Reading Room (1966)

Read it: Internet Archive

Tucked inside a declassified 1966 CIA proposal lies a snapshot of an agency suffocating under its own paperwork. The document, prepared for the National Photographic Interpretation Center (NPIC) — the unit that analyzed U-2 and satellite imagery during the Cold War — describes what we'd now recognize as an unmistakable big data crisis, two decades before the term existed.

The study's authors lay out the scale with startling specificity:

"It is currently estimated that there are between 1,000,000 and 2,000,000 separate items on hand, e.g., 1,250,000 maps and charts; 75,000 reports; 20,000 books and magazines; 50,000 to 100,000 other miscellaneous indexes and files; and in excess of 150,000 photographs and undeterminate number of which exist in random 'chip' form."

The problem statement reads like a memo that could have been written in any modern enterprise IT department:

"Current and anticipated increases in the volume of imagery and collateral data inputs to NPIC require more rapid and accurate handling, updating, and accessing... The manual methods used to reproduce, store..."

What's striking is the framing. The CIA explicitly carves out the difficult problem — the non-digitally stored data: maps, briefing aids, photo "chips," reports. They knew digital indexing was coming. They had computers. But the actual stuff being indexed was paper, film, and chemical emulsion. The metadata was tractable; the substrate was not.

This is the lost lesson. We tend to remember the 1960s as the dawn of mainframes and orderly card catalogs. But the CIA — arguably the most resource-rich information consumer on the planet — was openly admitting that human analysts were drowning. Their proposed solution wasn't more storage; it was a study to figure out how to even study the problem. The very first task was to admit they didn't know how their own information moved.

Modern parallels are everywhere:

The CIA was, in effect, the first organization to publicly admit (in classified form) that information abundance is a logistics problem disguised as a knowledge problem. The U-2 program could photograph the Soviet Union. Nobody had figured out how to find any specific photograph six months later.

The forgotten claim: By 1966, the CIA was already grappling with a million-item information overload — proving that "big data" was a human-systems crisis decades before it became a computing buzzword.

All newsletters