AI Development Company in Washington DC

Federal Government

A federal agency processing 8.2 million benefits applications annually (employment-based benefits requiring eligibility determination) was drowning in manual document processing. Each application contained 12-45 pages of supporting documents — identity verification, employment records, financial statements, medical documentation, and legal declarations. The agency's 2,400 adjudicators spent 65% of their time on document intake (reading, classifying, and extracting information from supporting documents) and only 35% on actual eligibility determination (the cognitive work that requires human judgment). Processing backlog: 1.4 million pending applications with an average processing time of 8.2 months — more than double the agency's 4-month service standard. The backlog generated 3.2 million inquiries annually from applicants checking status, further consuming adjudicator time. Previous automation attempts had failed: a 2019 RPA project automated form filling but couldn't handle the variety of supporting documents (handwritten forms, scanned letters, foreign-language documents, varied layouts). An OCR pilot achieved only 71% character accuracy on the diverse document set — insufficient for benefits determination where errors have legal consequences.

We built a document intelligence pipeline that automates document classification, information extraction, and quality verification — handling the intake work so adjudicators can focus on determination. (1) Document classification: a computer vision model (fine-tuned vision transformer) that classifies incoming documents into 34 categories (birth certificate, passport, employment letter, tax return, medical report, etc.) with 96.8% accuracy. The model handles: scanned documents (varying quality — from high-resolution scans to photos taken with smartphones), handwritten documents (approximately 12% of submissions include handwritten components), multi-page documents (correctly identifying that pages 3-7 of a submission are a single employment letter, not five separate documents), and foreign-language documents (documents in 42 languages — the model classifies by document type regardless of language, with language identification as a secondary output). Documents that the model classifies with <90% confidence are flagged for human review (approximately 8% of documents). (2) Information extraction: specialised extraction models for each document category. For structured documents (tax returns, government forms): template-based extraction with ML-assisted field location (handling variations in form versions, state-specific formats, and scan alignment issues). For semi-structured documents (employment letters, bank statements): NLP-based extraction that identifies key entities (employer name, dates of employment, salary, job title) from varied formats. For unstructured documents (personal statements, medical narratives): NLP summarisation that extracts relevant facts and flags sections requiring adjudicator attention. Extraction accuracy: 94.2% across all document types (measured as exact match for structured fields, semantic equivalence for extracted entities). (3) Cross-document validation: the system cross-references extracted information across documents within each application. Consistency checks: does the name on the birth certificate match the passport? Does the employment period on the employer letter match the tax returns? Does the medical documentation support the claimed condition? Inconsistencies are flagged for adjudicator review with specific discrepancies highlighted — "Employment letter states start date March 2022; tax return shows no income from this employer until July 2022." (4) Adjudicator workspace: a redesigned interface where adjudicators see: pre-processed application (all documents classified, key information extracted and organised by eligibility criterion), validation results (consistency checks with flagged discrepancies), confidence indicators (which extractions are high-confidence vs. which need human verification), and original documents (always accessible — the AI augments but never replaces the adjudicator's ability to review source documents). The workspace reduces the adjudicator's intake task from 35 minutes average to 8 minutes — they verify AI extractions rather than reading every document from scratch. (5) NIST AI RMF compliance: the system affects individuals' access to benefits — a rights-impacting AI. We implemented full AI RMF documentation: Govern (established governance with the agency's Chief AI Officer, defined risk tolerance — the system must not deny benefits based on AI errors), Map (identified risks: incorrect classification leading to missing documents, extraction errors leading to incorrect eligibility data, bias in OCR/NLP performance across document languages and handwriting styles), Measure (continuous monitoring of accuracy across document types, languages, and demographic indicators), and Manage (human-in-the-loop for all determinations, automatic flagging below confidence thresholds, regular fairness audits). (6) Responsible AI safeguards: bias testing across document languages (ensuring extraction accuracy is consistent across English, Spanish, Chinese, Vietnamese, Korean, and other frequently submitted languages), demographic fairness (ensuring processing speed and accuracy don't vary by applicant demographics), transparency (applicants are informed that AI assists in processing, with the right to request fully human review), and human oversight (adjudicators make all eligibility decisions — the AI processes documents, not applicants).

Document classification accuracy: 96.8% (8% flagged for human review — virtually all correctly classified upon review, but the conservative threshold ensures no documents are miscategorised). Information extraction accuracy: 94.2% across all document types — for high-confidence extractions (78% of fields), accuracy is 98.6%. Adjudicator intake time: reduced from 35 minutes to 8 minutes per application (77% reduction). Adjudicators now spend 75% of their time on eligibility determination (from 35%) — the work that requires human judgment. Processing throughput: the agency now processes 12,400 applications per day (from 6,800) with the same 2,400 adjudicators — 82% throughput improvement without hiring additional staff. Backlog reduction: the 1.4 million backlog was eliminated in 11 months. Average processing time decreased from 8.2 months to 3.4 months (below the 4-month service standard for the first time in 6 years). Inquiry reduction: status inquiries decreased 58% (from 3.2 million to 1.34 million annually) — faster processing means fewer anxious applicants calling to check status. Fairness: extraction accuracy across the top 10 submission languages varies by less than 2 percentage points (94.8% for English, 93.1% for Vietnamese, 92.9% for Chinese) — no significant language-based disparity. Processing speed is consistent across applicant demographics. Congressional reporting: the agency presented the results to its appropriations committee — the improved processing times resolved a longstanding Congressional concern. Annual value: $48 million in productivity improvement (82% throughput increase × adjudicator costs), $12 million in inquiry handling reduction, and unquantified value of timely benefits delivery to 1.4 million previously backlogged applicants. Development investment: $620,000 over 24 weeks.