HistoryLab AI: Democratizing Access to Declassified History

The Problem: Buried History

Most declassified government documents are technically public, but practically inaccessible. The CIA CREST database, Foreign Relations of the United States (FRUS) series, and various White House and State Department files sit buried in awful formats, riddled with OCR errors, and tangled in metadata messes. Researchers spend weeks manually digging through thousands of pages to find relevant information.

Over the past few months, I collaborated with Professor Matthew Connelly at Columbia University to build HistoryLab AI — a fast, clean AI-powered search engine that transforms how people interact with declassified documents.

Technical Architecture

The backend leverages years of work by the History Lab team at Columbia, who had collected and cleaned millions of declassified documents. What I built on top of that foundation:

  • Vector database of embedded text chunks for semantic search
  • Gemini API wrapper for intelligent summarization and citation
  • React frontend with natural language query interface
  • Real-time document retrieval with source PDF linking

Natural Language Queries

Instead of hunting through document databases with keyword searches, users can ask natural questions:

  • “How do these documents help us think about the longstanding debate over the extent of U.S. responsibility for the coup in Chile?”
  • “What was the U.S. position on the Formosa Question between 1947-1952?”
  • “Show me documents about CIA operations in Latin America during the Cold War”

The system pulls relevant passages from documents, provides citations, and links back to source PDFs — all in seconds.

Early Impact & User Response

The initial launch exceeded all expectations:

  • 165 unique users in the first week
  • 362 conversations generated
  • 1,000+ document searches performed
  • 27-minute average session duration

The platform attracted a remarkable roster of academics from prestigious institutions worldwide, including leading Cold War historians, diplomatic experts, and intelligence studies researchers from Oxford, Cambridge, Columbia, UC Berkeley, King’s College London, and government agencies like the U.S. Department of State.

User Feedback

“So cool” — First-time user discovering the platform’s capabilities

The response from the academic community has been overwhelmingly positive, with users praising the tool’s ability to surface relevant historical information that would otherwise require weeks of manual research.

Current Development

I’m continuing to expand the platform by:

  • Adding new datasets from additional government archives
  • Improving entity linking for better document connections
  • Enhancing citation accuracy for academic research standards
  • Optimizing search algorithms based on user interaction data

The Bigger Picture

HistoryLab AI represents more than just a technical achievement — it’s about democratizing access to historical knowledge. By making declassified documents searchable and understandable, we’re empowering students, researchers, and journalists to uncover historical truths that shape our understanding of the present.

The tool is live at history-lab.ramus.network and actively being used by the research community. It’s still a work in progress, but it’s already changing how people interact with historical archives.

This project combines my passion for both history and computer science, demonstrating how modern AI can be applied to preserve and democratize access to our collective historical memory.