We built a bilingual AI voice agent system handling all three service domains over 7 months. Conversational AI engine (months 1-3). Natural language understanding: voice agent understanding citizen enquiries in natural speech across all three domains — "I got a letter saying my property tax went up and I want to know why" (property tax assessment enquiry), "My registration is expired and I need to renew it but I have an emissions issue" (vehicle registration with complication), and "I need to renew my nursing licence but I'm 4 CE hours short" (professional licensing with deficiency). Intent recognition: 156 distinct intents identified across the three service domains — from simple lookups ("What's my property tax amount?") to complex multi-step processes ("I want to file a property tax exemption for my disabled veteran status"). Recognition accuracy: 93.4% intent accuracy on real citizen speech patterns captured from 10,000 anonymised call recordings.
Bilingual AI: native Spanish voice agent — not translated English but Spanish-first conversational design with Mexican Spanish language patterns reflecting Texas's predominantly Mexican-American Hispanic population. Spanish intent recognition: 91.8% accuracy (slightly lower due to smaller training corpus, improving with production data). The voice agent: offering Spanish from the first interaction — "¿Prefiere continuar en español?" — rather than requiring callers to navigate English menus to reach Spanish options. Multi-domain knowledge (months 2-4). Property tax knowledge base: Texas property tax code provisions, exemption eligibility rules (homestead, over-65, disabled veteran, agricultural), assessment dispute procedures, and payment plans encoded into the voice agent's knowledge. Dynamic data: agent accessing county assessment records, tax account balances, payment history, and exemption status in real time during calls. Conversation capability: explaining assessment calculations, guiding callers through exemption applications, initiating dispute filings, and processing payments — all through voice conversation.
Vehicle registration knowledge base: registration renewal procedures, emissions compliance requirements by county, title transfer processes, fee schedules, and special plate options. Dynamic data: accessing DMV records for specific vehicles, checking emissions status, and verifying insurance compliance. Conversation capability: processing renewals, explaining emissions requirements, guiding title transfers, and calculating fees. Professional licensing knowledge base: licensing requirements for 48 professional categories (nursing, real estate, cosmetology, engineering, etc.), renewal timelines, continuing education requirements, and disciplinary procedures. Dynamic data: accessing licensing records, CE hour tracking, and renewal status. Conversation capability: processing renewals, explaining CE deficiencies, and guiding new licence applications. System integration (months 3-5).
Backend integration: the voice agent connected to 7 agency backend systems — the property tax assessment database (Oracle-based legacy system), the motor vehicle registration system (mainframe with API gateway), the professional licensing database (SQL Server), the payment processing system, the document management system, the correspondence system, and the case management system. Integration approach: API-first integration where APIs existed (3 systems), screen-scraping integration with business logic layer for legacy systems without APIs (2 systems), and database query integration for read-only data access (2 systems). The integration: voice agent accessing the same data and performing the same actions that human agents used — not a separate data layer but direct integration with agency operations. Identity verification: multi-factor voice-based identity verification — account number + date of birth + last 4 of SSN verified conversationally. Voice biometric option: offered for repeat callers who enrolled, reducing verification to a 3-second voice sample.
The verification: secure identity confirmation that met state security requirements while keeping the conversation flowing. Telephony and operations (months 4-7). Call routing: voice agent replacing the IVR entirely — callers greeted by the AI voice agent that determined their need through natural conversation rather than menu navigation. Routing intelligence: the voice agent handling calls it could resolve autonomously and seamlessly transferring complex cases to human agents with full context (issue identified, caller verified, relevant data pulled up on the agent's screen). The routing: zero-menu citizen experience replacing the 47-option IVR tree. Escalation protocol: voice agent detecting when calls required human intervention — emotionally distressed callers, complex legal questions, multi-party disputes, and situations where the caller explicitly requested a human. The detection: seamless handoff with context, maintaining caller confidence.
Continuous improvement: every voice agent interaction feeding into weekly model improvement cycles — new citizen phrasings learned, edge cases identified and addressed, and accuracy metrics tracked across all 156 intents. The improvement: voice agent quality increasing continuously with real-world conversation data.