AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis

Bing Yan, Gregory Wolfe, Stefano Martiniani, Kyunghyun Cho

AskChem replaces document-level search with a claim-centered infrastructure to enable verifiable, cross-paper chemistry synthesis.

How can we shift chemistry literature search from retrieving full documents to retrieving atomic, provenance-backed scientific claims?

Chemistry literature search currently returns ranked lists of documents, forcing scientists and AI agents to manually extract, verify, and assemble findings scattered across hundreds of papers. AskChem shifts the unit of retrieval from the paper to the atomic, provenance-carrying claim: each finding is extracted as a typed assertion grounded by a source DOI and a verbatim quote. On the AskChem-Bench evaluation, grounding an LLM in this claim store yields 100% resolvable citations and the highest citation density among five tested systems.

Paper Primer

AskChem treats scientific claims as the primary data object, organizing them through three complementary structures: a stabilized faceted taxonomy for hierarchical browsing, an evidence graph for navigating relations like "supports" or "contradicts," and an exploratory living taxonomy that situates findings within broader scientific principles.

AskChem eliminates citation hallucination in cross-paper chemistry synthesis.

Comparison of GPT-5.5 grounded in AskChem versus standalone GPT-5.5 on 30 cross-paper chemistry questions. 100% of citations in AskChem-grounded answers were resolvable DOIs, compared to 88.3% for the ungrounded model.

The system provides high-density, verifiable evidence for complex chemistry queries.

Citation density metric (distinct verified DOIs per answer) across five retrieval systems. AskChem achieved 18.1 verified DOIs per answer, the highest among all tested systems (next highest was 10.7).

Why is a claim-centered approach necessary for AI agents in chemistry?

Document-level retrieval fails to expose the specific evidence, relationships, and domain organization required for synthesis, often leading LLMs to fabricate citations when forced to rely on parametric memory.

How does AskChem ensure the reliability of its extracted claims?

Every claim is strictly grounded by a source DOI and a verbatim quote or explicit evidence locator, and the system uses schema-validated extraction to ensure traceability rather than relying on unverified semantic interpretation.

The Case for Claim-Centered Search

We expose the bottleneck of document‑level retrieval and propose a claim‑centered infrastructure to enable verifiable cross‑paper chemistry synthesis.

Current chemistry literature search returns ranked lists of whole papers, leaving scientists and AI agents to locate specific findings, verify their provenance, and manually assemble cross‑paper answers. This document‑level interface misaligns with how researchers actually need evidence, and large language models that rely on such retrieval can hallucinate citations. AskChem replaces papers with atomic, provenance‑carrying claims, turning the unit of retrieval into a verifiable, searchable assertion.

**Figure 1.** The AskChem interface returns provenance-carrying claims.

The shift from document‑level to claim‑level retrieval enables both humans and AI agents to locate, verify, and synthesize chemical knowledge directly at the atomic assertion level.

Defining the Atomic Claim

Defines the claim representation that underpins AskChem’s retrieval and linking.

A Claim is a single, provenance‑bearing scientific fact extracted from a paper, identified by its DOI and a verbatim quote.

Extracted fields: reactant = A, reactant = B, product = C, temperature = 25 °C.

Confidence score assigned by the LLM = 0.92 (high confidence).

Source record created with DOI 10.1234/example, venue “Journal of Chemistry”, year 2022.

TreeNode places the claim under taxonomy paths: reaction type = “addition”, substance class = “organic”.

Edge links this claim to a prior claim “A is synthesized via method X” with relation type “extends”.

This concrete claim illustrates how provenance (DOI) and structured fields turn an unstructured sentence into a searchable, linked entity.

**Figure 2.** AskChem replaces papers with provenance-carrying claims as the retrieval unit, then exposes complementary structure over the shared claim store: a stabilized faceted taxonomy for search and browse, an evidence graph for cross-paper relations, and an exploratory living taxonomy organized by principles.

How does a Claim differ from a traditional document‑level retrieval result?

Unlike a whole paper, a Claim is a single atomic fact with its own DOI and structured fields, so it can be indexed, filtered, and linked directly without the overhead of processing an entire document.

Extraction and Taxonomy Construction

Stable, data‑driven facets turn noisy claim extraction into a reliable navigation layer.

Document‑level retrieval stalls because claims lack a consistent categorization. Without stable facets, users cannot reliably group or filter results across papers, leading to fragmented evidence trails.

Instead of imposing a rigid ontology, the system lets the corpus suggest category paths and then locks those paths into persistent L1/L2/L3 routes.

Provisional strings are “oxidation”, “oxidation”, “reduction”.

Canonical routing assigns “oxidation” → facet Reaction Type, “reduction” → facet Reaction Type.

Synonym normalization merges “oxidation” variants, leaving a single node.

Fuzzy clustering creates sub‑categories “ethanol” and “methanol” under the “oxidation” branch.

Final stable paths: oxidation/ethanol and oxidation/methanol (L1/L2), reduction/CO₂ (L1/L2).

Stabilization turns noisy textual mentions into repeatable hierarchy entries, so future queries can reliably retrieve all “oxidation” claims regardless of the original wording.

How does this differ from a traditional static chemistry ontology?

A static ontology fixes categories before seeing any data, risking mismatch with emerging terminology. Our approach first discovers frequent terms from the corpus, then solidifies them, so the taxonomy evolves with the literature while still offering a stable navigation structure.

Assign each claim a 2–5 segment path within the stabilized hierarchy (e.g., `reaction_type`/oxidation/ethanol).

Define five primary content views: reaction type, substance class, application, technique, and mechanism topic.

Populate each view by projecting the claim’s path onto the corresponding facet.

Provide auxiliary views for claim type, extracted measurements (data), temporal dimension, and source authors.

Expose all views through the same API, allowing clients to request any combination of facets without duplicating the underlying claim store.

Hybrid /search fuses lexical FTS5 claim‑text matches, paper‑level recall, taxonomy‑node recall, and dense‑vector similarity via reciprocal rank fusion (Cormack et al., 2009). Returned claims keep their view paths, so UI clients can group results, expand related categories, or drill into the hierarchy. Browse endpoints also surface node‑level claim counts and temporal overlays, turning the taxonomy into an operational index rather than a mere visualization.

**Figure 3.** Corpus-scale coverage and automatic quality checks for the deployed index. These statistics characterize provenance and extraction depth; they do not replace expert judgments of claim semantics or taxonomy placement.

Benchmarking Retrieval and Groundedness

AskChem eliminates citation hallucination and outperforms baselines on key metrics.

Recall that AskChem replaces whole papers with atomic, provenance‑carrying claims to enable verifiable cross‑paper reasoning. We now evaluate the system against four research questions.

AskChem‑Bench is a 30‑question chemistry benchmark that probes cross‑paper synthesis by requiring a system to retrieve and cite atomic claims with verifiable DOIs.

AskChem eliminates DOI hallucination, achieving 100 % resolvable DOIs and the highest citation density among five systems.

Table 1 shows AskChem attaining 100 % DOI existence and 18.1 verified DOIs per answer, surpassing the LLM‑only baseline (88.3 % DOI existence, 9.6 citations/answer).

**Table 1.** AskChem-Bench: 30 cross-paper chemistry questions (condition aggregation, temporal tracking, contradiction surfacing), GPT-5.5 reader; all five systems cover the full 30. DOIs are verified via CrossRef. Bold marks the best value in each row; Paperclip ties AskChem only on DOI existence.

**Figure 6.** Same question (AskChem-Bench ca04, "electrocatalysts for CO$_2$ reduction to CO/formate"), answering alone, GPT-5.5 fabricates 6 of 14 DOIs; when grounded in AskChem's retrieved claims, all 22 citations resolve.

RQ1 confirms that every claim in the index is source‑grounded (100 % traceability), though semantic fidelity is not guaranteed. RQ2 reports a 97.9 % edge‑type precision in the evidence graph, indicating reliable claim‑level structure for navigation and hybrid search.

RQ4 demonstrates that the deployed service can query 2.4 M claims, 307 K taxonomy nodes, and 171 K evidence edges through a unified REST schema, confirming corpus‑scale feasibility.

AskChem significantly reduces hallucination by grounding claims in DOIs.

Limitations and Future Directions

We discuss remaining limits, ethical safeguards, and broader impact of AskChem.

The corpus used by AskChem covers only a limited slice of chemistry and relies on abstracts rather than full‑text articles, so many domain‑specific details are absent.

Because abstract extraction is shallower than full‑text extraction, the system may miss nuanced information that appears only deeper in the paper.

LLM‑generated claims, relations, and their placement in the faceted taxonomy can be erroneous, and the current benchmark (AskChem‑Bench) measures groundedness on only 30 questions, not full factual accuracy or downstream utility.

String‑based taxonomy normalization sometimes merges distinct categories or preserves near‑duplicates, and the resulting retrieval gain has not been isolated in an ablation.

Each claim includes provenance—a source Digital Object Identifier (DOI) and a verbatim quote—so users can verify the statement against the original paper.

The interface also supports community flagging, allowing users to mark claims that appear inaccurate or misleading.

AskChem is meant to assist literature search and synthesis, not to replace reading primary sources when making critical decisions.

By grounding AI‑assisted chemistry workflows in verifiable citations, the system aims to reduce citation fabrication and make generated answers easier to audit.

The system indexes papers using publicly available metadata, abstracts, and open‑access full text where possible, exposing claim text, source metadata, and provenance while respecting copyright by not redistributing paywalled full text.

We thank NYU HPC for generous computational support, the NYU Google Cloud Platform (GCP) Research Grant Program for LLM‑based extraction resources, the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Ministry of Science and ICT (MSIT) of Korea, and the NYUAD Center for Interdisciplinary Data Science & AI (CIDSAI) funded by Tamkeen under the NYUAD Research Institute Award CG016.

Related Systems and Conclusion

We compare existing chemistry databases and search tools to AskChem’s claim‑centered approach.

Traditional chemistry search tools either offer proprietary curated reaction data or open molecule listings, while AskChem restructures the problem around provenance‑carrying claims.

Commercial platforms that aggregate expert‑curated reaction and substance records, providing detailed experimental provenance for each entry.

Open repositories that store structured molecular identifiers and bioactivity annotations, freely accessible to anyone.

AskChem’s claim‑centered architecture yields searchable, verifiable results that scale to corpus size while exposing the same claim objects to both humans and AI agents.

Benchmark Details

This appendix details the benchmark composition, evaluation protocol, and reproducibility resources.

AskChem‑Bench v1.1 assembles ten questions for each of three cross‑paper tasks, yielding a total of thirty queries that probe chemical literature across conditions, evolution, and conflicts.

**Table 2.** AskChem-Bench topics (30 questions total).

GPT‑5.5 is run on all thirty questions in five distinct settings. AskChem first rewrites each question into three to four keyword subqueries, fans them out to a hybrid search backend, and then merges evidence into at most forty claims before performing grounded synthesis. Paperclip applies the same rewriter and synthesizer over its paper retrieval pipeline, Edison Scientific uses its PaperQA‑family agent, and NotebookLM relies on Deep Research.

AskChem is evaluated against the claims it surfaces, whereas paper‑level baselines are judged against cited titles and abstracts. Every extracted DOI is verified through CrossRef, ensuring that each citation points to a resolvable digital object. The Gemini 3.1 Pro relevance judge was calibrated on one hundred domain‑expert labels, achieving 93 % agreement and a Cohen’s $\\kappa$ of 0.914.

The metrics used are listed in a table: Fraction resolving in CrossRef (DOI existence), Citation density (distinct verified DOIs per answer), Grounded specificity (tokens sharing a sentence with a citation marker), Recent impact (papers from the last five years with ≥ 50 citations), and Judge score (3 = direct, 2 = on‑topic, 1 = loose, 0 = irrelevant).

All benchmark resources are publicly released: the full question set, methodology, aggregate and task‑level results, and index snapshots are available via the API at https://askchem.org/api/benchmark; prompt templates and a rerun script accompany the release, and the indexed data are hosted on HuggingFace at https://huggingface.co/datasets/bing-yan/askchem.

Claim Representation Details

This appendix details the JSON schema used to store claims, edges, and extractor metadata.

The system records each atomic claim as a self‑contained JSON object, preserving its type, provenance, and outcome details.

Example claim record (truncated for width)

Relationships between claims are captured as directed edges that encode support, contradiction, or other logical links.

Example evidence edge linking two claims

The extractor pipeline combines a lightweight title/abstract model (GPT‑5‑mini) with a full‑text model (Gemini 3.1 Pro) and enforces JSON‑object‑constrained decoding for all calls.

Extractor pipeline pseudocode

Taxonomy and API Implementation

This appendix details the taxonomy structure, evidence graph, and the REST API for accessing claims.

Appendix B provides a concise overview of the taxonomy used by AskChem and the public API that exposes it.

The stabilized faceted taxonomy groups claims into persistent L1/L2/L3 categories using canonical L1 routing, synonym normalization, and fuzzy clustering.

Five content views—reaction, substance, application, technique, and mechanism—plus claim‑type, measurement, and time facets enrich the taxonomy.

An evidence graph is built in a second pass, adding directed `derives_from` and `cites_as_evidence` edges annotated with confidence scores and provenance, which an expert audit validates.

The section defines three shorthand tags: CA for conditions, TC for evolution, and CS for conflicts, and enumerates numerous reaction and technique categories such as C–N coupling, Suzuki–Miyaura, and CO₂ reduction.

Table 2 lists the 30 AskChem‑Bench topics that form the evaluation suite.

The REST API offers endpoints like GET /api/search?q=CO2+reduction, GET /api/claims/{`claim_id`}, GET /api/claims/{`claim_id`}/neighborhood, and GET /api/sources/{doi} to retrieve claim IDs, taxonomy paths, provenance, and source papers.

From a sampled set of 148 edges across relation types, 146 were decidable and 143 matched the correct type, yielding 97.9 % precision; two edges were excluded as undecidable.

The Living Taxonomy is an exploratory 4,931‑node hierarchy of principles, theories, models, mechanisms, and phenomena, with 663 open proposed branches for future expansion.

Table 3 reports the coverage of this taxonomy: 111,714 substance‑class claims, 119,361 technique claims, 98,665 mechanism claims, and 30,806 reaction‑type claims, with total placements exceeding 1 million across papers.

Placement evidence shows nitrite reduction placed under electrocatalytic redox and diene cross‑metathesis under olefin metathesis, illustrating both successful and forced‑fit assignments.

Validation gates require extraction responses to conform to the claim schema, include provenance fields, and route faceted paths through canonical top‑level categories before lower‑level normalization; evidence edges retain confidence and provenance, and Living Taxonomy placements may abstain when no suitable host exists.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers