Computer Vision Development in San Francisco

Technology / Artificial Intelligence

A San Francisco-based AI startup (Series A, $18M raised, 35 employees — building a visual AI API for document understanding used by 450 enterprise customers for automated document processing) needed to re-architect its computer vision infrastructure to reduce inference costs, improve accuracy, and build an API product that could scale to millions of daily document pages. (1) Inference cost: the company's document understanding model processed 2.5 million pages per day — running on NVIDIA A100 GPU instances costing $185K per month. Cost-per-page: $0.0024. At this cost, the company's gross margin was only 28% (below the 70%+ expected for SaaS/API businesses). The model architecture (a large vision-language model) was accurate but computationally expensive — and the team lacked the inference optimisation expertise to reduce per-page cost without sacrificing accuracy. (2) Accuracy gaps: the document model achieved 94% field-level extraction accuracy on standard documents (invoices, receipts, forms) — but dropped to 78% on complex documents (multi-page contracts, handwritten notes, poor-quality scans, and documents in non-Latin scripts). Enterprise customers needed 95%+ accuracy on their specific document types — and several customers were evaluating competitors who claimed higher accuracy through specialised models. (3) Latency: average processing time was 3.2 seconds per page — acceptable for batch processing but too slow for real-time document workflows (where users expected sub-1-second responses for single-page processing). The real-time use case represented 40% of enterprise pipeline but only 15% of current usage — latency was the adoption barrier. (4) Model customisation: enterprise customers wanted models fine-tuned on their specific document types — mortgage applications for banks, medical records for healthcare, contracts for legal. The company could fine-tune models, but the process required 3-4 weeks of ML engineering time per customer — and with 450 customers, customisation requests created an unsustainable engineering backlog. (5) API product maturity: the API lacked features expected by enterprise customers — no webhook for async processing, limited error handling, no batch processing endpoint, and no model versioning (model updates affected all customers simultaneously without warning). Developer documentation was incomplete, and there were no SDKs — customers integrated through raw HTTP calls.

We re-architected the visual AI platform over 12 weeks. (1) Inference optimisation: model quantisation (FP32 to INT8 with careful calibration — reducing model size by 75% with less than 0.3% accuracy degradation on the validation set), TensorRT compilation (optimising the model graph for NVIDIA inference hardware — 2.4x throughput improvement), dynamic batching (accumulating requests and processing in batches of 8-16 — increasing GPU utilisation from 38% to 82%), and input preprocessing optimisation (parallel image decoding and resizing on CPU while GPU processed the previous batch — eliminating preprocessing bottleneck). Result: cost-per-page from $0.0024 to $0.00068 (72% reduction). Gross margin from 28% to 72%. (2) Accuracy improvement: model architecture upgrade — replacing the monolithic vision-language model with a two-stage pipeline: fast detection model (identifying document regions, tables, and text blocks in 50ms) followed by targeted extraction models (specialised models for each region type — table extraction, form field extraction, handwritten text, and multi-script OCR). Ensemble: for complex documents, multiple extraction approaches combined — OCR-based, layout-based, and vision-language-based — with a confidence-weighted fusion that selected the most reliable extraction for each field. Result: standard document accuracy from 94% to 97.2%. Complex document accuracy from 78% to 91.5%. (3) Latency reduction: the two-stage architecture enabled sub-1-second processing for standard single-page documents — the fast detection model ran in 50ms, and targeted extraction models ran in parallel (150-300ms depending on document complexity). Total: 200-400ms for standard pages (from 3.2 seconds). Complex multi-page documents: processed with the full pipeline in 1.5-2.5 seconds per page (improved from 3.2+ seconds). Caching: document template caching — when a document matched a previously seen template (same form layout), the detection stage was skipped and extraction ran immediately on known regions. Template hit rate: 45% — further reducing average latency. (4) Self-service fine-tuning: automated fine-tuning pipeline — customers uploaded labelled examples (minimum 100 documents) through the API, and the system automatically fine-tuned a customer-specific model layer. Fine-tuning: from 3-4 weeks of ML engineering to 4 hours of automated pipeline (data validation, training, evaluation, and deployment). Customer-specific models: each customer could have up to 10 custom model variants — deployed and served through the same API with a model version parameter. Accuracy: customer-specific models typically improved accuracy by 3-5 percentage points on their document types (from the base model's already improved accuracy). (5) Enterprise API: complete API redesign — async processing (webhook-based callbacks for batch processing), batch endpoint (submitting multiple documents in a single request with progress tracking), model versioning (customers pinned to specific model versions with opt-in upgrades and 90-day deprecation), error handling (structured error responses with specific error codes, human-readable messages, and documentation links), and SDKs (Python, JavaScript/TypeScript, Java, and Go client libraries with typed models and idiomatic interfaces). Developer portal: interactive documentation, sandbox with sample documents, and webhook testing tools.

Unit economics: gross margin from 28% to 72% — transforming the business from economically unsustainable to a healthy API business. Cost-per-page: from $0.0024 to $0.00068 (72% reduction). GPU spending: from $185K/month to $68K/month (while handling 60% more volume). Accuracy: standard documents from 94% to 97.2%. Complex documents from 78% to 91.5%. Customer satisfaction: 4 customers who were evaluating competitors re-committed after seeing accuracy improvements on their specific document types. Latency: standard pages from 3.2 seconds to 200-400ms. Real-time adoption: from 15% to 45% of processing volume (real-time use cases adopting after latency improvement). Self-service fine-tuning: 85 customers created custom models within 6 months (from zero — all customisation previously required ML engineering). Customer-specific accuracy averaging 4.2 percentage points above base model. Engineering time saved: equivalent of 6 ML engineers redeployed from manual customisation to product development. API maturity: developer NPS from +22 to +61. SDK adoption: 78% of new integrations used SDKs (from 0%). Average integration time: from 3 weeks to 4 days. Enterprise wins: 12 new enterprise deals ($50K+ ACV) citing API maturity and self-service fine-tuning as decision factors. Revenue growth: 85% ARR growth in the 12 months following re-architecture (from 45% trajectory) — driven by improved unit economics enabling sales investment, accuracy improvements winning competitive evaluations, and self-service fine-tuning expanding per-customer revenue. Additional ARR above baseline trajectory: approximately $4.2M. Annual value: $4.2M (revenue acceleration — additional ARR from improved product), $1.4M (GPU cost reduction — from $185K to $68K monthly), $1.1M (engineering efficiency — 6 ML engineers redeployed from customisation to product development), $0.8M (churn reduction — retained customers who were evaluating competitors) = $7.5M. Development cost: $280,000. Payback: under 14 days.