NLP Development in Sydney, Australia

Healthcare / Clinical Documentation / Health Informatics

A Sydney hospital network (3 hospitals, 1,800 beds, 8,200 employees, 420,000 annual inpatient admissions, part of a major NSW Local Health District) needed NLP to extract structured information from clinical documentation. (1) Unstructured clinical text: 85% of clinical information stored in free-text — progress notes, discharge summaries, operative reports, and radiology reports. Volume: 2.3 million clinical documents generated annually across 3 hospitals. Challenge: critical clinical information buried in narrative text — diagnoses, medications, allergies, procedures, and outcomes not in structured fields. Impact: clinicians spending 25 minutes per patient reviewing notes to find relevant information — for a hospital processing 1,150 admissions daily across the network, this represents massive clinical time spent on information retrieval. (2) Clinical coding: ICD-10-AM coding from discharge summaries — 12 clinical coders processing 420,000 separations annually. Manual: coders reading discharge summaries and assigning diagnosis and procedure codes. Accuracy: coding accuracy at 88% — incorrect codes affecting hospital funding (Activity Based Funding — each separation funded based on DRG derived from codes). Backlog: coding backlog averaging 15 days — affecting funding cash flow and data quality for clinical quality reporting. Revenue: estimated A$4.2M in annual under-coding (missed secondary diagnoses reducing DRG complexity and funding). (3) Adverse events: detecting adverse events from clinical notes — medication errors, falls, and hospital-acquired complications. Current: relying on voluntary incident reporting — estimated 60-70% of adverse events not formally reported. Text indicators: adverse events often documented in clinical notes but not in incident system — "Patient found on floor" in nursing notes not always triggering a fall incident report. Impact: undetected adverse events preventing systematic quality improvement — "you can't fix what you don't know about." (4) Research: clinical research requiring structured data from notes — identifying patient cohorts, extracting clinical variables, and measuring outcomes. Current: research assistants manually reviewing charts — extracting data from hundreds of patient records for each study. Time: chart review for a 500-patient study taking 6-8 weeks of research assistant time. Bottleneck: research limited by data extraction capacity — important studies delayed or reduced in scope due to manual extraction cost. (5) Patient matching: identifying patients eligible for clinical trials, screening programmes, and care pathways. Current: relying on structured data (diagnosis codes) to identify patients — missing patients whose conditions are documented in notes but not coded. Gap: estimated 25-30% of eligible patients missed for clinical programmes — conditions documented in free-text notes but not in structured fields.

We built a clinical NLP system processing documents across the 3 hospitals over 8 weeks. (1) Clinical entity recognition: extracting medical concepts. NER model: fine-tuned clinical NER on Australian clinical text — extracting diagnoses (including severity, stage, and laterality), medications (including dose, route, and frequency), procedures (including approach and site), lab results (including values and reference ranges), and vital signs (including temporal patterns). Australian medical: trained on Australian medical terminology — PBS drug names, Medicare item descriptions, Australian hospital terminology, and Australian clinical abbreviations. Negation: handling clinical negation — "no evidence of metastasis," "denies chest pain," "unlikely to be malignant" — correctly marking negated findings. Temporal: understanding temporal context — "previous history of MI" (past), "currently experiencing" (present), "planned for" (future). Accuracy: entity extraction at 93% F1 score on Australian clinical text — tested against clinical expert annotations. (2) Clinical coding assist: AI-powered coding suggestions. Architecture: discharge summary → NLP extracts diagnoses and procedures → maps to ICD-10-AM candidate codes → presents to human coder for verification. Mapping: clinical concepts mapped to ICD-10-AM codes using SNOMED-to-ICD mapping plus Australian-specific coding rules (Australian Coding Standards). Complexity: identifying secondary diagnoses and complications that coders sometimes miss — NLP reading the entire document while coders may focus on primary diagnosis. Australian coding: understanding Australian coding standards — distinguishing between "condition present on admission" and "hospital-acquired condition" from clinical narrative. Output: suggested codes ranked by confidence — high confidence codes (>95%) presented as primary suggestions, lower confidence as alternatives for coder consideration. Coder workflow: NLP suggestions integrated into coding software — coders reviewing and accepting/modifying rather than coding from scratch. (3) Adverse event detection: finding unreported events. NLP scanning: processing clinical notes in near-real-time — identifying language patterns associated with adverse events. Detection: falls ("found on floor," "unwitnessed fall," "slip in bathroom"), medication errors ("wrong dose administered," "medication given despite allergy"), pressure injuries ("new sacral redness," "stage 2 pressure injury noted"), and hospital-acquired infections ("new fever on day 5," "wound showing signs of infection"). Alerting: detected events triggering notification to clinical governance — ensuring events are investigated and recorded even if not voluntarily reported. Sensitivity: prioritising sensitivity over specificity — better to flag a false positive for investigation than miss an actual adverse event. (4) Research extraction: structured data from notes. Pipeline: research team defining variables → NLP extracting from clinical notes → structured dataset generated → research team validates sample. Variables: custom extraction for each study — diagnosis details, treatment timelines, outcome measures, and patient characteristics. Cohort identification: finding patients meeting study criteria from free-text — "patients with Type 2 diabetes and CKD stage 3 who were on metformin" — NLP searching across millions of notes. De-identification: automated de-identification pipeline — removing names, MRN, dates (shifted), locations, and other identifiers before data leaves the hospital network. Speed: extraction for 500-patient study from 6-8 weeks (manual) to 2-3 days (NLP extraction + human validation). (5) Patient identification: finding eligible patients. Screening: NLP continuously scanning clinical notes for conditions matching care pathways, clinical trials, and screening programmes. Example: identifying undiagnosed chronic kidney disease — NLP detecting references to elevated creatinine, reduced eGFR, and proteinuria in clinical notes for patients without a CKD diagnosis code. Matching: clinical trial eligibility matching — extracting patient characteristics from notes and comparing against trial inclusion/exclusion criteria. Output: daily list of potentially eligible patients → clinical team reviews → patients offered appropriate programme or trial.

Results over 10 months. Entity extraction: processing 6,300 clinical documents daily (2.3M annually) — extracting 4.2M clinical entities per month. Accuracy: 93% F1 score maintained in production — regular quality sampling confirming accuracy. Processing: average 8 seconds per document (from 25 minutes of clinician review for equivalent extraction). Clinical coding: coder productivity increased 45% — coders reviewing AI suggestions rather than coding from scratch. Accuracy: coding accuracy from 88% to 95.5% (secondary diagnosis capture improving most — NLP finding diagnoses coders missed). Backlog: coding backlog from 15 days to 3 days — improved throughput clearing backlog. Revenue: estimated A$3.8M in recovered annual funding — previously uncaptured secondary diagnoses increasing DRG complexity. Under-coding: reduction from estimated A$4.2M to A$400K — NLP identifying the additional diagnoses human coders were missing. Adverse events: NLP-detected events: 2,800 potential adverse events flagged in 10 months. Confirmed: 1,960 confirmed adverse events (70% true positive rate). Previously unreported: 1,340 of confirmed events (68%) had not been reported through voluntary incident reporting — previously invisible to quality improvement. Falls: 380 unreported falls identified — triggering falls prevention programme review. Medication: 290 medication-related events identified — triggering medication safety review. Impact: safety quality improvements initiated from NLP-detected events — falls prevention protocol updated, medication reconciliation process improved. Research: 12 research studies using NLP extraction in 10 months. Time: average study data extraction from 6-8 weeks to 3 days (96% reduction). Studies enabled: 4 studies that wouldn't have proceeded due to extraction cost now completed — research output increased. Patient identification: 2,400 patients identified for clinical programmes through NLP (previously missed by structured data queries alone). CKD: 680 patients with undiagnosed CKD identified — referred to nephrology for early intervention. Trials: 340 patients matched to clinical trials — improving trial recruitment for the network. Financial: annual value — coding revenue recovery A$3.8M, clinical time saving A$1.2M (reduced note review), research extraction A$480K (research assistant time), adverse event detection A$650K (estimated quality improvement value), and patient identification A$380K (early intervention value). Total annual value: A$6.51M. Development: A$340,000 (8-week project — clinical NER, coding assist, adverse event detection, research extraction, and patient identification). Annual operation: A$120,000 (compute, model maintenance, and clinical validation). ROI: 19.2x first-year return.