Building Agentic RAG with Python and LangChain
Short answer
Agentic RAG adds an LLM agent on top of retrieval-augmented generation: instead
of a single retrieve-then-answer step, the agent can decide to search, re-search, call tools,
and reason over your private data. In Python, LangChain's
create_retrieval_agent or a tool-wrapped retriever is the common pattern.
1. Install
pip install langchain langchain-openai langchain-community chromadb2. Build a retriever over your data
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings
vectordb = Chroma.from_documents(docs, OpenAIEmbeddings())
retriever = vectordb.as_retriever(search_kwargs={"k": 4})3. Wrap it as a tool for an agent
from langchain.agents import AgentExecutor, create_openai_tools_agent
from langchain_core.prompts import ChatPromptTemplate
tool = RetrieverTool.from_llama_index(retriever, "search_docs", "Search private docs.")
agent = create_openai_tools_agent(llm, [tool], prompt)
executor = AgentExecutor(agent=agent, tools=[tool], verbose=True)
executor.invoke({"input": "What does our refund policy say about late returns?"})FAQ
Agentic RAG vs classic RAG?
Classic RAG does one retrieval then answers. Agentic RAG lets the model plan: it can retrieve, reflect, retrieve again, or call other tools before answering — better for multi-step or ambiguous questions.
Do I need a vector database?
For real corpora, yes — Chroma, pgvector, or Pinecone store embeddings for semantic search. For a handful of documents, in-memory splitting can work, but a vector store scales.