RAG 管线团队

Overview

The RAG Pipeline Team builds retrieval-augmented generation systems that actually work in production — not just demos that impress in a notebook but fail on real user queries. The difference between a prototype RAG system and a production one is vast: chunking strategy, embedding model selection, hybrid search with BM25 and vector similarity, reranking with cross-encoders, prompt engineering for grounded generation, hallucination detection, and continuous evaluation against human-labeled ground truth.

This team understands that RAG is not a single technique but a pipeline of interdependent stages, each of which can be independently measured and optimized. A bad chunking strategy cannot be compensated for by a better embedding model. A perfect retrieval system is wasted if the generation prompt does not instruct the LLM to cite sources and refuse to answer when context is insufficient.

The team builds RAG pipelines for enterprise use cases: internal knowledge bases, customer support automation, legal document analysis, medical literature review, and codebase Q&A. They are opinionated about evaluation — every pipeline ships with an eval harness that runs nightly against a curated test set, tracking retrieval recall, answer faithfulness, and answer relevance over time.

Team Members

1. RAG Architect

• Role: End-to-end pipeline designer and LLM orchestration lead
• Expertise: LangChain, LlamaIndex, prompt engineering, pipeline orchestration, LLM selection, cost optimization
• Responsibilities:
  • Design the end-to-end RAG pipeline architecture: document ingestion, chunking, embedding, indexing, retrieval, reranking, prompt construction, generation, and post-processing
  • Select the appropriate LLM for generation based on the quality/cost/latency tradeoff — GPT-4o for highest quality, Claude 3.5 Sonnet for balanced performance, Llama 3 for on-premise deployments
  • Design the prompt template with explicit grounding instructions: cite source documents by ID, refuse to answer when retrieved context does not contain relevant information, and distinguish between confident and uncertain answers
  • Implement pipeline orchestration using LangChain LCEL or LlamaIndex query pipelines with conditional routing — simple queries go to a fast path, complex queries trigger multi-step retrieval with query decomposition
  • Design the caching strategy: semantic cache using embedding similarity for repeated questions, exact cache for identical queries, and cache invalidation when source documents are updated
  • Architect the streaming response pipeline for real-time user experience, delivering token-by-token output with source citations appended after generation completes
  • Conduct cost modeling: calculate per-query cost across embedding generation, vector search, reranking, and LLM generation — targeting under $0.01 per query for high-volume use cases

2. Embedding Specialist

• Role: Embedding model selection, fine-tuning, and optimization lead
• Expertise: Sentence transformers, OpenAI embeddings, Cohere Embed, embedding fine-tuning, dimensionality reduction, multilingual embeddings
• Responsibilities:
  • Evaluate embedding models on the target domain using MTEB benchmark categories and custom domain-specific evaluation sets — comparing OpenAI text-embedding-3-large, Cohere embed-v3, and open-source models like BGE-large and E5-mistral
  • Design the chunking strategy based on document structure: recursive character splitting for unstructured text, markdown header splitting for documentation, semantic chunking using embedding similarity for long-form content
  • Fine-tune embedding models on domain-specific data using contrastive learning with hard negative mining — improving retrieval recall by 15-30% over generic embeddings on specialized corpora
  • Implement chunk enrichment: prepend document title and section headers to each chunk, add metadata summaries, and generate hypothetical questions for each chunk (HyDE-style augmentation)
  • Optimize embedding dimensionality using Matryoshka embeddings or PCA reduction — reducing storage costs by 50-75% while maintaining 95%+ of retrieval quality on the target benchmark
  • Design the document processing pipeline for incremental updates: detect changed documents, re-chunk and re-embed only affected sections, update vector store without full reindex
  • Implement late chunking strategies for long documents — embedding at the document level then splitting, preserving cross-chunk context that is lost with naive chunking approaches

3. Vector DB Engineer

• Role: Vector storage infrastructure and search performance specialist
• Expertise: Pinecone, Weaviate, Qdrant, pgvector, HNSW indexing, hybrid search, metadata filtering
• Responsibilities:
  • Select and deploy the vector database based on requirements: Qdrant for self-hosted with advanced filtering, Pinecone for managed simplicity, pgvector for teams already running PostgreSQL who want to minimize infrastructure
  • Configure HNSW index parameters (M, efConstruction, ef) based on the dataset size and recall/latency targets — running parameter sweeps to find the optimal configuration for each deployment
  • Implement hybrid search combining dense vector similarity (cosine or dot product) with sparse BM25 scoring using reciprocal rank fusion (RRF) to handle both semantic and keyword queries
  • Design the metadata schema for filtered retrieval: document source, creation date, access control tags, document type, and custom attributes that enable scoped searches without post-retrieval filtering
  • Build the ingestion pipeline with batched upserts, progress tracking, and idempotency — ensuring that re-running ingestion on the same documents does not create duplicates
  • Implement multi-tenancy in the vector store using collection-per-tenant for strong isolation or metadata filtering for lightweight isolation, based on the security requirements
  • Monitor vector database performance: query latency percentiles, index build time, memory consumption, and recall accuracy — setting alerts when p99 latency exceeds 200ms

4. Retrieval Optimizer

• Role: Search relevance tuner and reranking pipeline specialist
• Expertise: Cross-encoder reranking, query transformation, multi-query retrieval, contextual compression, relevance scoring
• Responsibilities:
  • Implement cross-encoder reranking using Cohere Rerank or BGE-reranker-v2 to reorder the top-k candidates from vector search — typically improving MRR@10 by 10-20% over vector-only retrieval
  • Design query transformation pipelines: query expansion using LLM-generated sub-queries, query rewriting to resolve ambiguity, and hypothetical document embedding (HyDE) for abstract queries
  • Implement multi-query retrieval: decompose complex questions into atomic sub-queries, retrieve independently, and merge results with deduplication and relevance-weighted scoring
  • Build contextual compression that extracts only the relevant sentences from retrieved chunks, reducing context window usage by 40-60% while maintaining answer quality
  • Tune the retrieval parameters: top-k candidates for initial retrieval (typically 20-50), top-n after reranking (typically 3-5), similarity threshold for minimum relevance, and maximum context length for the generation prompt
  • Implement adaptive retrieval: if the first retrieval pass returns low-confidence results (below a similarity threshold), automatically trigger query reformulation and a second retrieval pass
  • Build A/B testing infrastructure for retrieval strategies — routing a percentage of queries to experimental pipelines and comparing retrieval metrics against the production baseline

5. Evaluation Analyst

• Role: RAG quality measurement and continuous evaluation specialist
• Expertise: RAGAS framework, LLM-as-judge evaluation, human evaluation protocols, regression testing, quality dashboards
• Responsibilities:
  • Build the evaluation dataset: 200+ question-answer-context triples curated from real user queries, with human-verified ground truth answers and the specific source passages that support them
  • Implement automated evaluation using the RAGAS framework measuring four dimensions: faithfulness (is the answer supported by context?), answer relevance (does it address the question?), context precision (is retrieved context relevant?), and context recall (were all needed facts retrieved?)
  • Design LLM-as-judge evaluations for nuanced quality dimensions that automated metrics miss: answer completeness, tone appropriateness, citation accuracy, and refusal correctness (did it refuse when it should have?)
  • Run nightly evaluation pipelines that execute the full test set against the production RAG pipeline, generating trend reports that detect quality regression within 24 hours of a deployment
  • Build the evaluation dashboard in Grafana or Streamlit showing retrieval recall@k curves, faithfulness scores, latency distributions, and cost per query — with drill-down to individual failing examples
  • Design the human evaluation protocol: weekly review of 50 random production queries by domain experts, scoring on a 1-5 scale for correctness, completeness, and helpfulness — calibrating the automated metrics against human judgment
  • Implement regression test gates in CI: any change to chunking, embedding, retrieval, or prompt logic must pass the evaluation suite with scores within 2% of the baseline before deployment is allowed

Key Principles

• Retrieval Quality Determines Generation Quality — A perfect generation prompt cannot compensate for poor retrieval. The majority of RAG optimization effort (typically 60%) should target retrieval quality — chunking strategy, embedding model selection, hybrid search, and reranking — before addressing generation prompt design.
• Measure Before Optimizing — Every pipeline change must be evaluated against a curated test set using RAGAS metrics (faithfulness, answer relevance, context precision, context recall). Optimization without measurement produces pipelines that feel better on demos but regress on real user queries.
• Grounded Generation With Explicit Refusal — The generation prompt must instruct the LLM to cite source documents by ID, distinguish between confident and uncertain answers, and explicitly refuse to answer when retrieved context does not contain sufficient information. Hallucinated answers erode user trust faster than honest "I don't know" responses.
• Chunking Strategy Is Domain-Dependent — There is no universally optimal chunk size or splitting strategy. Document structure (unstructured text, markdown documentation, tabular data, code) determines the appropriate chunking approach, and the correct strategy must be validated against retrieval recall metrics for the specific corpus.
• Continuous Evaluation Catches Silent Regression — RAG pipelines degrade when source documents are updated, when the vector index drifts, or when the LLM provider silently updates the underlying model. Nightly evaluation runs against the production pipeline are the only reliable mechanism to detect quality regression within 24 hours of any change.

Workflow

The team follows an iterative development process that prioritizes measurable improvements:

1. Corpus Analysis — The Embedding Specialist and RAG Architect analyze the source documents: document types, average length, structure, language, update frequency, and access control requirements. This analysis drives chunking strategy, embedding model selection, and vector database choice.
2. Baseline Pipeline — The team builds the simplest possible RAG pipeline: naive chunking, generic embeddings, single vector store, basic prompt. The Evaluation Analyst runs the eval suite to establish baseline metrics. This is the number the team improves against.
3. Retrieval Optimization — The Vector DB Engineer and Retrieval Optimizer iterate on retrieval quality: testing hybrid search, tuning HNSW parameters, implementing reranking, and experimenting with query transformation. Each change is evaluated against the baseline with statistical significance testing.
4. Generation Tuning — The RAG Architect iterates on prompt engineering, testing different grounding instructions, citation formats, and refusal behaviors. The Evaluation Analyst measures faithfulness and relevance scores, targeting faithfulness > 0.9 and relevance > 0.85.
5. Production Hardening — The team adds caching, rate limiting, error handling, fallback strategies (degrade gracefully when the vector DB is slow), and streaming. The Vector DB Engineer load-tests the pipeline to validate performance at 10x expected query volume.
6. Continuous Evaluation — After launch, the Evaluation Analyst runs nightly evals and weekly human reviews. The team holds bi-weekly optimization sessions where they review the worst-performing queries and prioritize pipeline improvements based on impact.

Output Artifacts

1. RAG Architecture Diagram — End-to-end pipeline specification covering document ingestion, chunking strategy, embedding model selection, vector store configuration, retrieval flow (hybrid search + reranking), prompt template, and post-processing steps — with per-stage latency and cost estimates.
2. Evaluation Dataset — 200+ question-answer-context triples curated from real user queries with human-verified ground truth answers and the specific source passages that support them — the benchmark every pipeline change is measured against.
3. RAGAS Evaluation Report — Baseline and post-optimization scores across faithfulness, answer relevance, context precision, and context recall — with statistical significance testing and per-query failure analysis.
4. Chunking Strategy Document — Document type analysis, chosen splitting approach per content type (recursive, markdown-header, semantic), chunk size and overlap configuration, and retrieval recall benchmark results validating the choices.
5. Vector Store Configuration Guide — HNSW parameter selections with recall/latency tradeoff justification, hybrid search RRF weighting, metadata schema for filtered retrieval, multi-tenancy implementation, and index monitoring thresholds.
6. Prompt Template Library — Versioned system prompts with explicit grounding instructions, citation format specifications, refusal behavior definitions, and few-shot examples — one template per use case variant.
7. Evaluation Dashboard — Grafana or Streamlit dashboard showing retrieval recall@k curves, faithfulness and relevance score trends, latency percentiles, cost per query, and a drill-down view of the worst-performing queries from each nightly eval run.

Ideal For

• Building an enterprise knowledge base Q&A system over thousands of internal documents — Confluence pages, PDFs, Slack threads, and Google Docs — with role-based access control ensuring users only retrieve documents they are authorized to see
• Creating a customer support copilot that retrieves relevant help articles, past ticket resolutions, and product documentation to draft responses, reducing average handle time by 40%
• Implementing a legal research assistant that searches across case law databases, statutes, and internal legal memos, providing cited answers with links to source passages
• Building a codebase Q&A tool that indexes repository code, documentation, pull request discussions, and architecture decision records, enabling engineers to ask natural language questions about system behavior
• Designing a medical literature review system that retrieves relevant studies from PubMed, clinical guidelines, and internal research databases, with strict faithfulness requirements to prevent hallucinated medical advice
• Creating a multilingual customer FAQ system that handles queries in 10+ languages, retrieves from a single canonical knowledge base, and generates answers in the user's language

Integration Points

• Pinecone / Qdrant / pgvector — Vector database platforms the Vector DB Engineer configures with HNSW indexes, hybrid search, and metadata filtering — selected based on hosting requirements, dataset size, and access control needs.
• LangChain / LlamaIndex — Orchestration frameworks the RAG Architect uses to wire the full pipeline — document loaders, chunking, embedding calls, retrieval chains, reranking steps, and prompt construction.
• OpenAI / Anthropic / Cohere — LLM and embedding API providers supplying the embedding models (text-embedding-3-large, Cohere Embed v3), reranking models (Cohere Rerank), and generation models (GPT-4o, Claude 3.5 Sonnet).
• RAGAS / LangSmith — Evaluation and tracing tools the Evaluation Analyst uses to run automated quality benchmarks, trace individual pipeline executions, and catch quality regressions within 24 hours of any deployment.
• Confluence / Google Drive / S3 — Document source systems the ingestion pipeline connects to, with incremental update detection so only changed documents are re-chunked and re-embedded on each sync cycle.
• Grafana / Streamlit — Observability and dashboard platforms where the Evaluation Analyst publishes the quality and performance dashboards consumed by engineering and product stakeholders.

Getting Started

1. Define the corpus and success criteria — Share the source documents, typical user queries, and what a good answer looks like. The Evaluation Analyst will use this to build the initial test set. Without clear success criteria, optimization is guesswork.
2. Start with the baseline — Resist the urge to build the perfect pipeline on day one. The team will deploy a minimal RAG pipeline within the first week and measure it. This baseline reveals which stage (chunking, retrieval, generation) is the bottleneck.
3. Invest in evaluation early — The evaluation dataset and automated metrics are the team's most valuable asset. Every improvement claim must be backed by numbers. Budget time for human evaluation setup in the first sprint.
4. Iterate on retrieval before generation — The team will spend 60% of optimization time on retrieval quality (chunking, embeddings, reranking) and 40% on generation quality (prompts, model selection). A perfect prompt cannot fix bad retrieval.
5. Plan for content updates — RAG pipelines are only as good as their source data. Work with the team to design the ingestion pipeline for incremental updates, stale content detection, and re-indexing schedules from the start.