RAGU: A Multi-Step GraphRAG Engine with a Compact Domain-Adapted LLM

Mikhail Komarov, Ivan Bondarenko, Stanislav Shtuka, Oleg Sedukhin, Roman Shuvalov, Yana Dementyeva, Matvey Solovyov, Nikolay O. Nikitin

RAGU is a modular GraphRAG engine that uses a compact 7B model to achieve high-quality knowledge graph construction on consumer hardware.

How can we improve GraphRAG performance on complex synthesis tasks while reducing the noise inherent in single-pass knowledge graph extraction?

Existing GraphRAG systems often rely on single-pass extraction, which produces noisy, duplicated knowledge graphs that struggle with complex synthesis tasks. RAGU addresses this by decoupling extraction from consolidation, using a two-stage typed extraction process followed by clustering and summarization to produce a cleaner, more connected graph. On GraphRAG-Bench, RAGU retrieves more complete context than competitors and matches larger models on synthesis tasks while running on a single consumer GPU.

Paper Primer

RAGU replaces the "single-pass" extraction common in frameworks like LightRAG with a multi-step pipeline. It uses a two-stage typed extractor to validate entities against a schema before extracting relations, then applies DBSCAN clustering and Leiden community detection to consolidate redundant information into a structured graph.

The system hinges on the "Language/World Knowledge Hypothesis": the LLM capabilities required for RAG—comprehension, extraction, and reasoning over context—scale weakly with parameter count compared to factual recall. RAGU operationalizes this with Meno-Lite-0.1, a 7B model fine-tuned specifically for these language skills, allowing it to outperform 32B-class models on knowledge graph construction tasks.

RAGU achieves superior context retrieval compared to existing GraphRAG frameworks.

Evidence Recall on GraphRAG-Bench (Medical).

Meno-Lite-0.1 provides high-performance extraction at a fraction of the compute cost.

Harmonic mean on IE benchmark (NER, RE, and definitions).

Why does RAGU separate extraction from consolidation?

Single-pass extraction often produces noisy, duplicated entities and brittle relations. By separating these stages, RAGU can validate entities against a schema and deduplicate mentions before building the graph, resulting in a structurally cleaner and more connected knowledge base.

Is RAGU intended to replace large frontier models for all RAG tasks?

No. RAGU is optimized for synthesis tasks (summarization, creative generation) and broad context retrieval. For precise, chain-traversal multi-hop fact lookup, the authors note that systems using personalized PageRank (like HippoRAG 2) may still hold a performance advantage.

Abstract

RAGU delivers a modular multi-step GraphRAG pipeline that outperforms larger models on knowledge‑graph construction.

Retrieval‑augmented Graph generation (GraphRAG) improves large language models by injecting structured knowledge, but existing pipelines perform a single‑pass extraction that yields noisy entities and brittle retrieval. RAGU replaces this with a modular, multi‑step pipeline: typed entity and relation extraction, DBSCAN‑backed deduplication, LLM summarization, and Leiden community detection. The design is motivated by the observation that the language‑skill components needed for extraction (comprehension, extraction, context handling) scale only weakly with model size, unlike factual world knowledge. Leveraging this, we train Meno‑Lite‑0.1, a 7 B model that surpasses a 32 B Qwen2.5 model on knowledge‑graph construction (+12.5% relative harmonic mean) and matches it on English GraphRAG tasks. On the medical subset of GraphRAG‑Bench, RAGU achieves the highest evidence recall (up to 0.84 versus ≤0.76 for baselines) and overtakes HippoRAG 2 on synthesis tasks; its apparent advantage on multi‑hop factoid QA is largely due to answer‑format differences. RAGU is distributed as a pip‑installable package, runs on a single GPU, and is released under an MIT license with source code publicly available.

The GraphRAG Bottleneck

We expose why existing GraphRAG pipelines falter and motivate a multi‑step extraction redesign.

Retrieval‑augmented generation (RAG) anchors large language models in external knowledge, but traditional RAG returns flat text chunks and ignores cross‑document entity relationships. GraphRAG addresses this by constructing a knowledge graph and performing graph traversal during retrieval, yet three practical obstacles remain.

Obstacle 1 is single‑pass extraction: a single LLM pass yields noisy, duplicated entities with no consolidation. Obstacle 2 is dependence on costly API‑scale models—extraction quality drives graph quality, but the required capability is language skill, not raw world knowledge. Obstacle 3 is engineering immaturity: many open‑source frameworks suffer from installation failures or unsafe code paths.

**Figure 1.** Effect of model size on world-knowledge (CheGeKa) vs. language-skill (MultiQ) tasks in the Qwen2.5-Instruct family (F1 scores on MERA (Fenogenova et al., 2024)). CheGeKa F1 grows 21.1× from 0.5 B to 72 B; MultiQ only 4×. Log-linear slopes: 0.65 vs. 0.26.

GraphRAG augments retrieval by extracting entities and relations from documents, linking them into a graph, and then traversing that graph to fetch contextually relevant facts for a query.

Retrieval‑Augmented Graph Utility (RAGU) replaces the single‑pass GraphRAG extraction with a modular, multi‑step pipeline that decouples entity and relation extraction, allowing smaller, domain‑adapted models to build more robust knowledge graphs for complex synthesis tasks.

The key shift is moving from scaling world‑knowledge via larger models to scaling language‑skill via more capable, smaller models.

The Multi-Step Extraction Engine

Method Overview explains RAGU’s multi‑step extraction pipeline and its consolidation stages.

RAGU first pulls out typed entities, then separately extracts relations that are forced to link only those validated entities—so the graph is built on a solid, self‑consistent foundation.

Stage 1 extracts entities: “Paris” (Location), “Eiffel Tower” (Structure), “France” (Location).

Each entity is checked against NEREL; all three pass because their types exist in the schema.

Stage 2 proposes a relation “`located_in`( Eiffel Tower, Paris )”. Both arguments match validated entities, so the relation is kept.

A second candidate “`located_in`( Paris, Eiffel Tower )” fails because the source entity “Paris” is a Location but the target “Eiffel Tower” is a Structure; the schema requires a Structure→Location direction, so it is discarded.

The constraint that relations must use only validated entities turns many false positives into easy rejections, dramatically cleaning the graph before any clustering.

How does this differ from the single‑pass extraction used in LightRAG?

LightRAG extracts entities and relations in one sweep, so a relation can reference a non‑existent or mis‑typed entity. RAGU’s two‑stage pipeline forces the relation extractor to see only the vetted entity set, preventing those mismatches.

The schema enumerates the allowed entity and relation types (29 entities, 49 relations) that RAGU’s extractors must obey.

EntitySummarizer groups entities by (name, type); for groups with many duplicate mentions it runs DBSCAN clustering and then an LLM summarizer to produce a canonical representation.

RelationSummarizer applies the same DBSCAN + LLM pattern to relations, merging redundant edges.

Hierarchical Leiden clustering partitions the deduplicated graph into communities.

An LLM generates a structured report for each community (title, summary, findings).

Optional pluggable refinement modules (e.g., RemoveIsolatedNodes) clean the graph further.

LocalSearch retrieves entities via dense vector similarity, then expands the result set to related relations and text chunks.

GlobalSearch first runs community detection, then lets an LLM rank and summarize whole communities for a query.

NaiveSearch performs standard vector‑based retrieval without any graph‑aware expansion.

MixSearch runs all engines in parallel and merges their results.

QueryPlanEngine decomposes a complex query into a DAG of sub‑queries, dispatching each to the most suitable engine.

All engines support cross‑encoder reranking and hybrid dense + sparse retrieval via Qdrant.

Three‑tier storage abstraction separates graph, key‑value, and vector layers, allowing backend swapping (e.g., NetworkX → Neo4j, NanoVDB → Qdrant).

Async‑first API with bounded concurrency guarantees production‑grade throughput.

LLM outputs are validated with Pydantic v2, eliminating manual JSON post‑processing.

Deterministic hash‑based IDs and merge policies enable incremental upsert/update/delete.

A consistency auditor checks cross‑store integrity after each mutation.

~374 unit tests cover the core pipeline.

A 7 B LLM fine‑tuned on the NEREL schema to excel at entity and relation extraction for RAGU.

The modular pipeline that strings together multi‑step extraction, DBSCAN‑backed summarization, and Leiden community detection into a production‑ready GraphRAG system.

Indexing and Clustering

The pipeline breaks graph building into six stages, letting lightweight models extract and organize knowledge robustly.

Single‑pass extraction often collapses under noisy domains and limited model capacity. RAGU therefore decomposes graph construction into six configurable stages, each handled by a lightweight component that can be swapped or tuned independently.

Step 1 – Chunking: split documents into overlapping or semantically coherent spans using SimpleChunker, SemanticTextChunker, or SmartSemanticChunker.

Step 2 – Entity & Relation Extraction: apply the NEREL‑based extractor to each chunk, producing typed nodes and edges.

Step 3 – Deduplication & Summarization: collapse duplicate mentions and generate concise descriptions for each entity.

Step 4 – Graph Construction: connect entities via extracted relations, forming a directed knowledge graph.

Step 5 – Community Detection (Leiden Clustering): group tightly‑connected subgraphs into communities for downstream synthesis.

Step 6 – Storage & Indexing: persist the graph, its communities, and associated summaries across a graph DB, a key‑value store, and a vector store.

Think of a social network where friends repeatedly merge into tighter circles until no further improvement is possible – Leiden clustering does the same for graph nodes, iteratively refining community assignments to maximize modularity.

Initial partition: each node forms its own community → six communities.

Local moves: nodes $A$, B, C merge because moving any of them to the other's community raises modularity; similarly D, E, F merge.

Resulting partition: two communities {A,B,C} and {D,E,F}.

Aggregation: each community becomes a super‑node; the graph now has two nodes connected by a single inter‑community edge (the former A‑D, B‑E, etc. are absent).

Since no further moves improve modularity, the algorithm stops.

The algorithm quickly collapses dense triangles into a single community, illustrating why it can reveal coherent sub‑graphs even when the raw extraction yields many overlapping mentions.

**Figure 2.** End-to-end indexing pipeline. Documents are chunked, entities and relations are extracted under the NEREL schema, deduplicated and summarized, then grouped into communities via Leiden clustering. All artifacts persist across three swappable storage tiers (graph database, key-value store, vector store). Solid arrows indicate data flow between pipeline stages; dashed arrows indicate artifacts produced at each stage.

Synthesis Performance

RAGU matches or exceeds baselines on multi‑hop QA when answers are forced terse.

Recall that RAGU replaces the single‑pass GraphRAG extraction with a modular multi‑step pipeline that separates entity and relation extraction, letting smaller domain‑adapted models build richer knowledge graphs.

RAGU closes the 2Wiki‑MultiHopQA answer‑correctness gap to HippoRAG 2 by 13.8 percentage points under the terse prompt.

Under the terse protocol, RAGU (GPT) achieves 58.0 % AC versus HippoRAG 2’s 63.5 % AC, reducing the previous –19.3 pp gap to –5.5 pp.

GraphRAG‑Bench is the evaluation suite that measures multi‑hop QA systems on answer correctness and ROUGE‑L across several datasets.

RAGU outperforms factoid‑retrieval baselines on synthesis‑heavy multi‑hop QA.

Benchmarking and Scaling

Key cross‑over results show HippoRAG 2 excels at fact retrieval while RAGU dominates creative generation.

HippoRAG 2 leads on fact‑level Answer Correctness but RAGU overtakes on Creative Generation and overall Coverage.

Fact Retrieval AC: HippoRAG 2 72.4 % vs RAGU 54.2 % ($\Delta$ −18.2 pp); Creative Generation AC: RAGU 59.0 % vs HippoRAG 2 56.9 %; Coverage: RAGU 57.4 % vs HippoRAG 2 34.7 %.

HippoRAG 2 builds a typed knowledge graph via a multi‑step extraction pipeline, then traverses it with personalized PageRank to answer queries.

**Figure 3.** Cross-over by task complexity on GraphRAG-Bench (Medical). All three systems use Meno-Lite-0.1 (7 B) as the index LLM and gpt-4o-mini for answer generation. (a) Answer Correctness: RAGU trails HippoRAG 2 on Fact Retrieval, but the gap closes and RAGU leads on Creative Generation. (b) Evidence Recall: RAGU retrieves the most complete context on all factoid levels. Task difficulty increases left to right.

Table 1 lists the full set of Answer Correctness, Coverage, and Faithfulness scores for all systems on GraphRAG‑Bench (Medical).

Knowledge Graph Construction Metrics

RAGU matches top IE performance with a 7B extractor, narrowing the gap to 1–2 pp.

RAGU (Ours) matches HippoRAG 2’s harmonic‑mean performance within 1–2 pp while using a 7B extractor instead of a 20B model.

Table 3 shows HM scores of 0.468 for RAGU (Ours) versus 0.496 for HippoRAG 2 (GPT) on the BioASQ terse prompt.

Despite using the lightweight Meno‑Lite‑0.1 (7B) extractor, RAGU’s end‑to‑end pipeline retains performance across all four IE tasks, confirming that extraction quality is not the bottleneck once graph consolidation is in place.

**Table 3.** IE benchmark (knowledge-graph construction) NER = entity recognition (F1), RE = relation extraction (F1), Def = entity definition (chrF++), RDef = relation definition (chrF++), HM = harmonic mean of all four tasks.

Case Study: Dennis Ritchie

A concrete case study shows RAGU extracting entities and relations from a biographical sentence.

We run the bundled example script on a single biographical sentence about Dennis Ritchie. The pipeline first extracts entities, then grounds relations between them, producing a complete knowledge graph for the passage.

This table compares the RAGU system against HippoRAG 2 across eight categories: Silent data loss on crash, Backend migration cost, API-bound throughput, Code execution from LLM output, Transient failure recovery, Regression detection, Incremental maintenance, Reproducible deployment, and Modularity.

**Table 5.** Relations extracted from the Ritchie passage (5 of 8 shown).

**Figure 4.** Knowledge graph built from the Ritchie passage. Nodes are typed entities; edges are typed relations. Two communities emerge: Ritchie’s professional legacy and the Bell Labs geographic cluster.

Meno‑Lite‑0.1 achieves the highest harmonic mean on the IE benchmark, beating Qwen2.5‑32B by a sizable margin.

Table 3 reports a 12.5 % relative improvement in harmonic mean for Meno‑Lite‑0.1 over Qwen2.5‑32B.

Stage 1 scans the sentence and tags nine spans: “Dennis Ritchie” (PERSON), “C programming language” (PRODUCT), “Unix operating system” (PRODUCT), “October 12, 2011” (DATE), “70” (AGE), “Alistair E. Ritchie” (PERSON), “Bell Laboratories” (ORGANIZATION), “Murray Hill” (DISTRICT), “New Jersey” (`STATE_OR_PROV`).

Stage 2 enumerates all ordered pairs of identified entities and scores each with a relation classifier; only eight pairs exceed the confidence threshold, yielding the triples shown in Table 5.

The resulting graph connects “Dennis Ritchie” to “C programming language” via `WORKS_AS`, to “Unix operating system” via `WORKS_AS`, and to “October 12, 2011” via `DATE_OF_DEATH`, among others.

The two‑stage design guarantees that every relation is anchored to a vetted entity, preventing spurious triples that would arise from unconstrained extraction.

Operational Insights

Practical limits, ethical licensing, and open‑source release details for RAGU.

RAGU surpasses HippoRAG 2 on synthesis‑oriented benchmarks—Creative Generation AC and Coverage—while HippoRAG 2 leads on pure retrieval precision, excelling at single‑fact AC and chain‑following multi‑hop reasoning on MuSiQue. The larger multi‑hop gap observed with verbose prompts stems mainly from answer‑format artifacts. Consequently, for tasks requiring broad contextual synthesis under a single‑GPU budget, RAGU is the preferred choice; for precise multi‑hop fact lookup, chain‑traversal systems remain superior.

First, our scaling evidence relies on a single model family, Qwen2.5, and a limited set of tasks. Although results are consistent across six model sizes, this constitutes a well‑supported hypothesis rather than a universal theorem.

Second, the Meno‑Lite‑0.1 model sacrifices parametric factual recall for contextual grounding and should not be used as a standalone knowledge base. Its multi‑hop reasoning performance degrades beyond 32 K tokens, a typical limit for 7 B‑class models.

Third, RAGU’s default NetworkX graph backend—implemented via a swappable BaseGraphStorage adapter—does not scale to corpora with millions of nodes, necessitating a dedicated graph‑database adapter for large deployments. Moreover, final graph quality remains sensitive to the extraction LLM; weaker base models introduce structural noise that consolidation cannot fully rectify.

The RAGU system is released under the MIT license, while the Meno‑Lite‑0.1 model is distributed under Apache 2.0. Meno‑Lite‑0.1 was trained exclusively on publicly available datasets that are fully cited in the References, including educational resources.

RAGU can be installed via pip install `graph_ragu`; comprehensive API documentation and examples are hosted at https://github.com/RaguTeam/RAGU. A demonstration video is available at https://youtu.be/bicJDMJuQfg. Both the code (MIT) and the model (Apache 2.0) are openly released.

Leiden clustering partitions the nine‑entity graph into two communities: one centered on Dennis Ritchie and his creations (five entities, four relations), and another linking Bell Laboratories, Murray Hill, New Jersey, and Alistair Ritchie through spatial and professional ties (four entities, three relations). An LLM then generates a structured summary for each community.

Using the constructed graph, RAGU’s LocalSearchEngine can answer multi‑edge questions. For example, it resolves “Where did the father of the creator of the C programming language work?” with “Alistair E. Ritchie … worked at Bell Laboratories,” and “What did the person who died on October 12, 2011, create?” with “Dennis Ritchie … created the C programming language and co‑created the Unix operating system.”

Conclusion

We reflect on data choices, environmental benefits, and responsible deployment of RAGU.

We built RAGU on diverse corpora—FineWeb‑Edu, RuLM, information‑extraction datasets, multi‑hop QA benchmarks, and synthetic GPT‑4o‑mini instructions—while ensuring no personally identifiable information entered training or evaluation. The IE benchmark is a test‑only derivative of the human‑annotated NEREL corpus, released under an MIT license and integrated into the LM Evaluation Harness for reproducible testing.

Our design prioritizes accessibility: the language‑knowledge hypothesis predicts, and our experiments confirm, that a 7 B model suffices, allowing RAGU+Meno‑Lite‑0.1 to run on a single consumer GPU. Graph construction runs at ~8 k tokens per document at ~2 k tok/s, costing about \$0.001 per document-two orders of magnitude cheaper than commercial API services, and scaling to $100 versus \$10 000 for 100 k documents. This reduced compute directly lowers energy consumption and $\mathrm{CO}_2$ emissions, while multilingual support via Settings.language lowers barriers for non‑English and under‑represented language communities.

RAGU can propagate biases from its source corpus, and Meno‑Lite‑0.1 inherits additional biases from its pretraining data. We therefore advise domain‑specific evaluation before deploying in sensitive areas such as healthcare, legal, or news. By validating all LLM outputs through Pydantic models rather than executing raw responses, RAGU mitigates a class of code‑injection attacks.

Fairness and Limitations

Discusses fairness limits, reproducibility resources, and funding acknowledgments.

The Bias NEREL schema underlying RAGU’s extraction was designed for Russian news text, so its entity and relation types reflect that domain. Applying RAGU to other languages or domains therefore often requires adapting the schema, and users should expect possible performance drops outside the original design space.

All code, model weights, and benchmark data are publicly released under open licenses, and the repository ships roughly 374 automated tests together with a deterministic mock LLM server. This enables full regression testing without API keys or GPU resources—a capability that, to our knowledge, no other open‑source GraphRAG framework provides. Moreover, every domain object carries a deterministic MD5 identifier, allowing any retrieved result to be traced back to its source text.

The IE benchmark is integrated into the public LM Evaluation Harness repository under the nerel‑bench task group, ensuring that any causal LLM can be evaluated under identical conditions. RAGU’s development was funded by GitVerse, Cloud.ru, and Habr through the “Code Without Borders” grant, as well as by Yandex’s Open Source grant; we thank Kirill Novgorodtsev for the $mol‑based web frontend and Yaroslav Svetlov for the demo backend.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers