Over 94% Classification Accuracy. Over 50% Less Research Time.
A 150-person defense contractor headquartered in Arlington, VA.
A 150-person defense contractor headquartered in Arlington, VA.
What They Were Facing
A 150-person defense contractor headquartered in Arlington, VA had a knowledge management problem that was costing them contracts. Their analysts were spending the majority of their working hours on document intake, classification, and retrieval rather than the analytical work they were hired to do. The firm's institutional knowledge lived in a sprawling SharePoint environment with over 140,000 documents, most of them poorly tagged and effectively unsearchable. The classification problem was the most acute pain point. Documents entering the system needed to be tagged with the correct security classification level, program association, and subject taxonomy. Analysts were doing this manually, and the error rate had climbed to nearly 1 in 5. In defense contracting, a misclassified document isn't just an inconvenience, it's a compliance violation that can jeopardize facility clearances and contract eligibility. Previous attempts to fix the problem hadn't stuck. The firm had invested in an enterprise search tool two years prior, but adoption was low because the results were unreliable. Analysts had reverted to maintaining personal document collections on their workstations, creating exactly the kind of fragmented knowledge silos that the search tool was supposed to eliminate. The technical constraints made this harder than a typical knowledge management project. Everything had to run on-premises in an air-gapped environment with no internet connectivity. The system needed to handle documents across multiple classification levels. And it had to be simple enough that analysts without technical backgrounds could use it without training.
Analysts spending the majority of working hours on document intake, classification, and retrieval
140,000+ documents in SharePoint, most poorly tagged and effectively unsearchable
Nearly 1-in-5 classification error rate creating compliance violations risking clearances and contracts
Previous enterprise search tool had low adoption due to unreliable results
Air-gapped environment with no internet connectivity and multi-level classification requirements
How We Solved It
We spent the first three weeks on-site in Arlington, working directly with analysts, program managers, and the IT security team. We audited the existing SharePoint taxonomy, catalogued the most common classification errors, and documented the actual workflows analysts used to find information (as opposed to the workflows they were supposed to use). That gap turned out to be significant. The core of the system is a RAG (retrieval-augmented generation) architecture running entirely on-premises. We deployed it on the firm's existing classified infrastructure with no external network dependencies. The system indexes documents at ingestion, extracts key entities and concepts, and makes the entire corpus searchable through natural language queries. An analyst can ask "What were the delivery milestones for the Phase II SIGINT contract?" and get a direct answer with source citations, rather than scrolling through 40 search results. For classification, we built a validation pipeline that runs on every document at intake. The system analyzes document content, cross-references it against classification guides and program security plans, and either confirms the analyst's classification or flags discrepancies for review. This caught the most dangerous category of errors: documents classified too low for their actual content. We also built a quality assurance layer that continuously monitors the system's own performance. Classification decisions are sampled and audited weekly, and the QA module tracks accuracy metrics, response quality, and retrieval relevance over time. When performance drifts below defined thresholds on any metric, the system alerts administrators rather than silently degrading.
Three weeks on-site auditing SharePoint taxonomy, classification errors, and actual analyst workflows
RAG architecture running entirely on-premises on classified infrastructure with zero external dependencies
Natural language search across 140,000+ documents returning direct answers with source citations
Classification validation pipeline cross-referencing content against classification guides at intake
Quality assurance layer with weekly sampling, accuracy tracking, and threshold-based alerting
Measurable Outcomes
Quantifiable improvements delivered within the project timeline
Improved from around 80% to over 94% accuracy on document classification
Research time reduced by over 50% across the analyst team
Average retrieval time reduced from 20-45 minutes of manual searching
Zero classification-related findings in the first post-deployment audit
Analysts using the system as their primary research tool within 60 days
Existing documents re-indexed and searchable for the first time
The over-50% reduction in research time translated directly to billable capacity. Analysts who had been spending three or more hours a day on document retrieval were now completing the same research in under 90 minutes. For a firm billing analyst time at federal contract rates, that freed well over a million dollars in annual productive capacity across the team. The classification accuracy improvement had an outsized impact on the firm's risk posture. During their next Defense Counterintelligence and Security Agency review, the assessor specifically noted the improvement in document handling procedures. The firm's facility security officer told us it was the first review in four years where classification wasn't flagged as an area of concern.
Implementation Timeline
A structured approach from discovery to deployment
Security architecture review with analysts, program managers, and IT security
Weeks 1-3Security architecture review with analysts, program managers, and IT security
Deployed on classified infrastructure with natural language search
Weeks 4-7Deployed on classified infrastructure with natural language search
Automated intake validation against classification guides
Weeks 8-9Automated intake validation against classification guides
Performance tracking, sampling, and threshold alerting
Weeks 10-11Performance tracking, sampling, and threshold alerting
Hands-on training for non-technical analysts
Week 12Hands-on training for non-technical analysts
Re-indexed 140,000+ existing documents
Weeks 13-14Re-indexed 140,000+ existing documents
System-wide rollout with performance optimization
Weeks 15-16System-wide rollout with performance optimization
Frequently Asked Questions
How does the system operate in an air-gapped environment without cloud AI services?
The entire system runs on locally deployed models and infrastructure. We use open-source language models that run on the firm's on-premises GPU hardware, with no internet connectivity required. The vector database, embedding models, and inference engine all operate within the classified enclave. This was a non-negotiable architectural decision made during the first week of scoping.
What happens when the system flags a potential classification error?
Flagged documents enter a review queue managed by the facility security officer's team. The system provides its reasoning, citing the specific classification guide references and content elements that triggered the flag. A human reviewer makes the final determination. The system never changes a classification on its own, it only recommends and flags.
Can the system handle documents across multiple classification levels?
Yes. The architecture supports multi-level document handling with access controls that mirror the firm's existing security infrastructure. Analysts only see results they're cleared to access. The system enforces these boundaries at the query level, not just at the document storage level, so there's no risk of inadvertent spillage in search results.
How do you maintain and update the system without internet access?
We established an update protocol that works within the firm's existing cross-domain transfer procedures. Model updates, security patches, and system improvements are packaged, reviewed, and transferred through the firm's approved data transfer process. We also trained two members of their IT team to handle routine maintenance and monitoring independently.
Services Used in This Project
Ready for Similar Results?
Schedule a discovery call to discuss your specific challenges and learn how we can deliver measurable outcomes for your organization.
Schedule Discovery Call