We built a multi-agent document processing system over 12 weeks: (1) Agent architecture — 5 specialised agents orchestrated by LangGraph: Intake Agent: receiving documents from all channels (email, scan, portal, fax). Performing OCR on scanned documents (Azure Document Intelligence — handling German handwriting, which is notoriously difficult for OCR). Normalising document format (PDF/images → structured text). Output: clean text + document metadata (source channel, receipt timestamp, any identifiable reference numbers). Classification Agent: analysing the document text to determine: document type (42 categories — trained on 100,000 historical classified documents), department routing (Schaden, Vertrag, Recht, Compliance, Leistung), priority level (detecting urgency indicators — court deadlines, regulatory references, medical urgency), and policy number extraction (identifying the relevant policy using regex patterns and contextual understanding — German insurance policy numbers follow specific formats per line of business).
The classification agent: using a fine-tuned model (Mistral 7B fine-tuned on the company's historical classification data — running on local GPU infrastructure in the Berlin data centre for GDPR compliance, no data leaving Germany). Accuracy: 96.4% on document type, 97.8% on department routing. Extraction Agent: specialised per document type — different extraction logic for Schadenmeldungen vs. Vertragsänderungen vs. Arztberichte. For claims (Schadenmeldung): extracting date of loss (Schadentag), location, description (Schadenbeschreibung), estimated amount (Schadenhöhe), involved parties (Versicherungsnehmer, Geschädigter, Zeuge), and police report reference if applicable. For medical reports (Arztbericht): extracting diagnosis (ICD-10 codes), treatment (OPS codes), prognosis, and work incapacity period (Arbeitsunfähigkeit). Extraction: using GPT-4 with structured output (JSON schema defining required fields per document type) — the model extracting data points and providing confidence scores for each extraction.
Validation Agent: cross-referencing extracted data against the policy administration system (SAP FS-PM): verifying the policy number exists, checking that the claimant is the policyholder or an authorised party, validating that the claimed event falls within the policy period, and confirming coverage for the claimed event type. Discrepancies: flagged for human review rather than auto-rejected (the document might be legitimate with a data entry error — the agent distinguishes between "definitely wrong" and "possibly wrong, needs human check"). Routing Agent: based on classification and validation results, routing the document to the correct handler — considering: department and line of business, handler workload (distributing documents evenly), handler expertise (complex legal documents routed to senior handlers), and geographic assignment (certain handlers responsible for specific regions). The routing agent: integrating with the company's workflow system (JIRA Service Management) — creating tickets with extracted data pre-populated, document attached, and classification/validation results included.
(2) Orchestrator: LangGraph managing the agent pipeline — sequential processing (Intake → Classification → Extraction → Validation → Routing) with error handling at each stage. If any agent fails: the document is flagged for manual processing (no document is lost or stuck). If extraction confidence is below threshold (85%): the document is routed with a "low confidence" flag — the handler reviewing the AI extraction rather than starting from scratch. (3) Human review interface: a React dashboard for the document management team — showing: documents processed by the AI (with classification, extraction results, and confidence scores), documents flagged for review (low confidence, validation discrepancies), and system performance metrics (accuracy, volume, processing time). The team: shifting from processing every document to reviewing AI decisions — a supervisory role rather than a manual processing role.
(4) Betriebsrat agreement: worked with the company's Betriebsrat to document: which employee data the system processes (none — the system processes customer documents, not employee performance data), how the system affects employee roles (role change from manual processing to AI supervision — no job losses, skill upgrade), and training programme for affected employees (3-day training on the review dashboard and AI supervision).