LlamaIndex

Python/JS framework focused on RAG and document indexing. Cleaner than LangChain for retrieval-heavy use cases.

Install via pip: pip install llama-index llama-index-embeddings-openai llama-index-llms-openai. Requires Python 3.10+. LlamaIndex is a data framework for LLM applications focused on ingestion, indexing, and retrieval over your own data. Quick start: from llama_index.core import VectorStoreIndex, SimpleDirectoryReader; documents = SimpleDirectoryReader("./data").load_data(); index = VectorStoreIndex.from_documents(documents); query_engine = index.as_query_engine(); response = query_engine.query("What is this document about?"). This creates an in-memory vector index from text files in ./data/ and queries it. For persistent storage: swap the default in-memory index for a vector store (ChromaDB, Pinecone, Qdrant, Weaviate) via their integration packages (pip install llama-index-vector-stores-chroma). Local models: pip install llama-index-llms-ollama llama-index-embeddings-huggingface then use Ollama or HuggingFaceEmbedding. LlamaIndex supports 160+ data connectors (PDF, SQL, Notion, Slack, GitHub, etc.) via llama-index-readers-* packages. First run: ~3 minutes for install, downloads model + indexes documents. Verify: the quick start example above returns an answer about your documents. Time-to-first-RAG: ~10 minutes.

Best for: RAG applications over private document collections (PDF reports, knowledge bases, manuals), data ingestion pipelines that parse and index heterogeneous document formats (160+ connectors), complex retrieval patterns (sub-question decomposition, recursive retrieval, hierarchical indexing), teams standardizing on "chat with your data" use cases, knowledge management and enterprise search built on LLM retrieval. Not suited for: general LLM application frameworks beyond data retrieval (use LangChain), simple Q&A over a few documents where direct embedding + cosine search suffices, applications where latency budget excludes the ingestion+retrieval pipeline overhead, non-Python environments (LlamaIndex has a TypeScript port but it trails the Python library), one-off queries where framework setup cost exceeds value.

Use LlamaIndex when your primary use case is RAG (retrieval-augmented generation) over your own data — document ingestion, chunking, embedding, indexing, and retrieval are LlamaIndex's core competency and the pipeline is more polished than LangChain's. Switch to LangChain when you need a broader application framework beyond RAG (chains, agents, tool orchestration) — LlamaIndex is data-centric, LangChain is application-centric. Use Haystack for an alternative RAG framework with a more opinionated pipeline API. Use direct vector DB + LLM integration for simpler RAG needs where a framework adds overhead. LlamaIndex's strengths: the ingestion pipeline (parsing → chunking → metadata extraction → indexing) handles PDFs, complex documents, and hierarchical data better than competitors; the query engine abstraction (router, sub-question, recursive) makes complex retrieval patterns straightforward. Its weakness: overlapping with LangChain's RAG features creates ecosystem confusion, and the package ecosystem (llama-index-*) has grown fragmented.

Problem: ModuleNotFoundError: No module named 'llama_index' after pip install. Fix: The package name is llama-index (with hyphen), not llama_index (underscore). The import uses underscore: from llama_index.core import .... Ensure llama-index-core is installed (it's a dependency of llama-index). For specific integrations, install separately: pip install llama-index-embeddings-huggingface etc. Problem: RAG returns "I don't have enough information" despite relevant documents. Fix: Check the chunking strategy — default chunk size (1024 tokens) may fragment key information across chunks. Reduce chunk overlap or increase chunk size: Settings.chunk_size = 2048; Settings.chunk_overlap = 200. Check the retrieval window — the default similarity_top_k=2 may be too low. Increase to 5–10. Use query_engine = index.as_query_engine(similarity_top_k=10). Problem: Document ingestion hangs on large PDFs. Fix: Some PDF readers (PyPDF2) are slow on scanned/image-heavy PDFs. Switch the reader: from llama_index.readers.file import PDFReader with parser=PyMuPDFReader(). For scanned PDFs, enable OCR: parser = PDFReader(ocr_languages=["eng"]). Split ingestion into batches with num_workers=4 for parallel processing.