Neo4j GraphRAG

What it is and how it works

Neo4j GraphRAG is the official Python package (neo4j-graphrag-python) maintained by Neo4j for building retrieval-augmented generation pipelines on top of a Neo4j graph database. It is not a standalone server or a new database engine — it is a library layer that sits between an application and an existing Neo4j instance (Community, Enterprise, or AuraDB), providing the plumbing to turn unstructured documents into a knowledge graph and then retrieve from that graph in ways a plain vector store cannot.

The architecture has two halves. The first is a knowledge-graph construction pipeline: text is chunked, passed through an LLM (or a configurable extraction component) to pull out entities and relationships, and written into Neo4j as nodes and edges alongside the original chunks. This is exposed as a SimpleKGPipeline and a set of composable pipeline components so teams can swap in their own extraction prompts, schemas, or entity-resolution logic rather than accepting a black box. The second half is retrieval: the library ships several retriever types, including pure vector search against Neo4j's native vector index, vector search followed by graph traversal (pulling in neighboring nodes for context expansion), hybrid retrieval that combines vector and full-text/keyword search, and Cypher-based retrievers where an LLM generates a Cypher query against the graph schema (text-to-Cypher). These retrievers return context that a calling application feeds into its own LLM generation step — the package handles retrieval, not the full RAG loop end to end, though it does provide convenience wrappers for common patterns.

Because it is built directly on Neo4j's driver and query language, everything the library does is ultimately expressed as Cypher under the hood. That is both the point and the constraint: you get the full expressiveness of a mature graph query language and Neo4j's indexing (vector, full-text, and native graph indexes together in one system), but you are also committing to running and operating a Neo4j deployment rather than a lightweight embedded store.

Deployment patterns

For solo developers and prototyping, the typical setup is Neo4j Desktop or a single Docker container (neo4j:latest) running Community Edition locally, with the Python package installed via pip and pointed at bolt://localhost:7687. This is enough to build and test a knowledge-graph-backed RAG pipeline end to end on a laptop with no GPU requirement — the package itself does no local inference; it calls out to whatever LLM and embedding provider you configure (OpenAI, Azure, Anthropic, Ollama for fully local embedding/generation, etc.), so GPU support is genuinely "n/a" for the library itself and depends entirely on which model backend you wire in.

For homelab or small-team use, the common pattern is a single Docker Compose service running Neo4j Community with a mounted volume for persistence, fronted by an application server that imports neo4j-graphrag-python and exposes a retrieval API. Because Community Edition lacks clustering and some enterprise security features, this tier is fine for internal tools and small-scale RAG but not for high-availability production traffic.

For team or production deployments, organizations either stand up Neo4j Enterprise (self-hosted, with clustering, role-based access control, and better multi-tenancy) or move to AuraDB, Neo4j's managed cloud offering, which removes the operational burden of running the graph database yourself. In either case the Python package's role doesn't change — it's the same retriever and pipeline code, just pointed at a different connection string with different auth. This portability (identical application code across laptop Docker, self-hosted cluster, and managed cloud) is one of the more practical benefits of committing to the Neo4j ecosystem.

How it compares

Against LangChain's or LlamaIndex's generic Neo4j integrations, this package is narrower in scope but deeper: it's purpose-built by Neo4j engineers specifically for GraphRAG patterns, with first-class support for hybrid and graph-traversal retrievers that the generic framework integrations often bolt on more loosely. If you're already committed to LangChain's abstractions for the rest of your stack, the generic integration may be less friction; if graph retrieval quality is the priority, the native package tends to expose more of Neo4j's retrieval capabilities directly.

Against Microsoft's GraphRAG (the research-originated project focused on community detection and hierarchical summarization over an extracted graph), Neo4j GraphRAG is more of an operational toolkit than a research methodology — it doesn't natively implement Microsoft's specific community-summarization algorithm, and it assumes you already have (or will build) a persistent, queryable Neo4j graph rather than an ephemeral extracted structure. Teams wanting the specific "global search over community summaries" pattern from Microsoft's paper will find that elsewhere; teams wanting a durable, queryable, production-grade graph store with RAG retrievers on top will find this a more practical fit.

Against agent-memory-focused graph tools like Mem0g or Letta, Neo4j GraphRAG is explicitly not an agent memory framework — it has no built-in concept of conversational memory decay, agent state, or tool-calling loops. It's a retrieval library you'd wire into your own agent loop, not a drop-in memory layer for one. That's a real gap for teams building agentic applications who want memory management out of the box.

Best use cases and honest limitations

This is the right choice for teams that already run Neo4j or are willing to operate it, and who need retrieval that goes beyond flat vector similarity — cases involving multi-hop relationships, entity-centric queries, or hybrid keyword-plus-semantic search. The excellent Cypher tooling and battle-tested graph engine underneath mean you inherit decades of query-engine maturity rather than a young vector-store project. Enterprise support contracts and AuraDB give a credible production and compliance story that many local-first RAG tools lack entirely.

The honest downsides: Neo4j operational overhead is real. Running, tuning, and backing up a graph database is a meaningfully bigger commitment than an embedded vector store like Chroma or a single-binary server. It also has less native integration with agent loops than purpose-built agent-memory tools, so expect to write glue code. Teams that just need simple document RAG without relational structure are almost certainly over-engineering by reaching for a graph database at all — a plain vector store will be faster to stand up and cheaper to run. This tool earns its complexity only when the data genuinely has graph structure worth exploiting.