Our RAG development for Houston follows a systematic approach that prioritizes retrieval quality and answer grounding. Phase 1 — Knowledge Base Preparation (2-4 weeks): building the foundation: document ingestion (collecting and processing the documents that will form the RAG knowledge base — for petrochemical: PSI files, P&IDs, SOPs, investigation reports, maintenance records — for energy: well files, engineering reports, best practice documents, lessons learned — for regulatory: regulation text, guidance documents, interpretive letters, enforcement actions — for healthcare: medical literature, clinical guidelines, institutional protocols — document ingestion handles multiple formats: PDF, Word, HTML, scanned images (OCR), CAD drawings (metadata extraction), and structured databases), chunking strategy (dividing documents into retrievable chunks — the chunking strategy critically affects RAG performance — chunks too large dilute relevant information with irrelevant context — chunks too small lose the context needed for accurate answers — for Houston technical documents: we implement semantic chunking that respects document structure (keeping sections, paragraphs, and tables intact), document-type-specific chunking (different strategies for regulations vs. SOPs vs. investigation reports), and hierarchical chunking (maintaining parent-child relationships between section headers and content) — for Houston petrochemical PSI: equipment data sheets are chunked by equipment tag, maintaining all specifications for a single piece of equipment together), embedding and indexing (converting document chunks into vector embeddings and indexing them for semantic search — model selection: we evaluate domain-specific embedding models against general-purpose models on the client's actual documents — for Houston energy: we typically test OpenAI embeddings, Cohere embeddings, and domain-specific models fine-tuned on energy technical text — the embedding model that produces the most relevant retrieval results for the client's typical queries is selected — indexing: we use vector databases (Pinecone, Weaviate, or Qdrant) with metadata filtering capability (filtering by document type, date, asset, regulatory agency, etc.)), and metadata enrichment (adding structured metadata to each chunk — for petrochemical: equipment tag, unit, document type, revision date, document owner — for energy: well name, field, basin, engineering discipline, document type — for regulatory: agency, regulation number, effective date, applicability — metadata enables filtered retrieval (search only within SOPs for a specific unit, or search only within PHMSA regulations) and provides context for answer generation)). Phase 2 — Retrieval Pipeline Development (3-5 weeks): building accurate retrieval: hybrid search (combining semantic search (vector similarity) with keyword search (BM25) — semantic search finds conceptually relevant content even when different terminology is used — keyword search finds exact terms that are critical in technical contexts (equipment tags, chemical names, regulation citations) — hybrid search combines both approaches, typically improving retrieval accuracy 15-25% over either approach alone — for Houston technical RAG: hybrid search is essential because technical queries often combine conceptual intent ("what is the maximum operating pressure") with specific identifiers ("for exchanger E-1402")), query processing (transforming user queries for optimal retrieval — query decomposition (breaking complex queries into sub-queries that are each retrieved independently), query expansion (adding domain-specific synonyms and related terms to improve recall), and query classification (routing queries to appropriate knowledge base segments — a regulatory question routes to the regulatory corpus, a procedure question routes to the SOP corpus) — for Houston energy: query processing must handle domain jargon, abbreviations, and the technical shorthand that engineers use ("what's the MAWP for the C-101?" should be understood as "maximum allowable working pressure for compressor C-101")), re-ranking (improving retrieval accuracy by re-ranking initial retrieval results — cross-encoder re-rankers that evaluate query-document relevance more accurately than embedding similarity — for Houston RAG: re-ranking typically improves the precision of top-5 retrieved chunks from 70-80% to 85-95%, ensuring the LLM receives highly relevant context for answer generation), and citation tracking (maintaining provenance for every retrieved chunk — document source, page number, section, and revision — citations are passed through to the generated answer so users can verify the answer against the source document — for Houston's safety-critical and compliance-sensitive applications: citation tracking is not optional — engineers and compliance professionals must be able to verify that RAG answers are grounded in authoritative sources)). Phase 3 — Generation Pipeline (2-3 weeks): producing accurate, grounded answers: prompt engineering (designing prompts that instruct the LLM to generate accurate, grounded answers — for Houston technical RAG: prompts instruct the model to: answer only based on the provided context (do not hallucinate information not present in the retrieved documents), cite specific documents and sections for each claim, indicate confidence level and flag when the retrieved information may be incomplete, and use domain-appropriate terminology and precision — prompt engineering for Houston energy requires iterative refinement with domain experts who evaluate whether the generated answers meet the accuracy and precision standards of their discipline), hallucination prevention (implementing guardrails against hallucinated content — for Houston's safety-critical applications: hallucination is not an acceptable failure mode — an engineer acting on a hallucinated equipment specification or a compliance professional relying on a fabricated regulation citation could face serious consequences — we implement: retrieval verification (checking that the generated answer is supported by the retrieved context), confidence scoring (flagging answers with low retrieval confidence for human verification), and answer grounding analysis (comparing generated claims against retrieved content to identify unsupported statements)), and answer formatting (formatting answers for the target user audience — for engineers: technical precision with equipment references and data values — for compliance professionals: regulatory citations with section numbers and effective dates — for clinicians: evidence-based formatting with study references and evidence quality ratings — formatting includes structured outputs when appropriate (tables of regulatory requirements, equipment specification summaries, drug interaction matrices))). Phase 4 — Evaluation and Deployment (2-3 weeks): measuring and monitoring quality: retrieval evaluation (measuring retrieval accuracy against golden datasets — domain experts create question-answer pairs with the expected source documents — retrieval accuracy is measured as the percentage of queries where the correct source documents are retrieved in the top-k results — for Houston technical RAG: we target >90% retrieval accuracy at top-5 and >95% at top-10), answer evaluation (measuring answer quality against expert-generated reference answers — evaluating factual accuracy, completeness, citation accuracy, and appropriate confidence calibration — for Houston: domain experts review a representative sample of RAG answers and rate them for accuracy and usefulness — we iterate on retrieval and generation until answer quality meets the domain experts' standards), and production monitoring (monitoring RAG performance in production — tracking retrieval confidence scores, answer quality metrics, user feedback, and system performance — for Houston: monitoring includes hallucination detection (automated checks for generated claims not supported by retrieved context) and retrieval drift detection (alerting when retrieval accuracy degrades, which may indicate that the knowledge base needs updating)).