Spark-to-Paper: End-to-End Research Paper Generation as a Composable Skill

Zhuoyang Qian, Biao Wu, Yiran Wang, Chris D Yan, Desan Dai, Liangwei Zheng, Jin Jiang, Junsheng Zhang, Wenhao Wang

Spark-to-Paper automates end-to-end research generation as a composable skill set within existing coding assistants.

How can an autonomous agent system reliably generate research papers by decomposing the process into verifiable, skill-based stages rather than relying on single-pass text generation?

Autonomous research agents typically require standalone infrastructure, separating them from the coding environments where researchers actually work. This creates a disconnect between the tools used for ideation and the tools used for execution and verification. Spark-to-Paper implements thirteen composable skills inside existing coding assistants, using deterministic programs to verify integrity and model-based judgment to handle reasoning. It separates experiment planning from reporting, ensuring claims are grounded in measured evidence rather than generated text. The system achieves 99.5% citation validity and 96.4% figure editability across controlled research topics, while increasing fabrication detection from 14% to 92% compared to a single-pass baseline.

Paper Primer

The system functions as a collection of thirteen skills—such as planning, literature retrieval, and experiment execution—that communicate via a shared project directory. It hinges on a core division of labor: the language model handles semantic judgment, while deterministic scripts handle operations that can be explicitly verified, such as citation resolution, LaTeX compilation, and plotting.

To prevent the "Self-Refutation Loop"—where a system repeatedly fails to support its own hypothesis—Spark-to-Paper bounds the number of experiment-critique-revision cycles to seven. If the objective remains unsupported, the system terminates the trajectory, records the failure, and restarts with a new research idea rather than forcing a successful narrative.

The full integrity and review stack significantly reduces factual fabrication.

A controlled ablation study using injected probes showed that fabrication detection rose from 14% in a single-pass draft to 92% with the complete stack.

Why implement this as a set of skills inside a coding assistant rather than a standalone agent platform?

Standalone platforms require separate orchestration layers and infrastructure, which isolates the research process from the coding environments where researchers already inspect files and execute code. By using existing assistant capabilities, Spark-to-Paper integrates directly into the researcher's native workflow.

How does the system ensure that experimental results are not just "hallucinated" to fit the paper's claims?

The system separates experiment planning from reporting: the required evidence is specified before results are observed. Numerical results are only admitted if they can be traced back to specific datasets, model configurations, and recorded metric files.

Spark-to-Paper demonstrates that end-to-end research automation is viable without dedicated infrastructure, provided the system is designed to abandon failed research trajectories rather than forcing them to succeed.

The Challenge of Autonomous Research

We expose why monolithic paper generation fails and motivate a workflow turning text generation into executable research steps.

Generating a complete research paper is more than stitching together prose; it requires tightly coupling literature retrieval, experiment design, execution, claim revision, and figure creation. When these steps are collapsed into a single monolithic generation pass, inconsistencies and unsupported claims proliferate.

Monolithic generation treats the entire research pipeline as a single text‑generation problem, so any error in early stages (e.g., missing experiment) propagates unchecked to later sections, breaking experimental integrity.

Recent autonomous research agents have demonstrated end‑to‑end pipelines, but they are built as standalone applications with their own orchestration services. This architectural separation makes integration with the coding environments where most researchers already work cumbersome.

Coding assistants already provide the low‑level primitives needed for research automation—file inspection, code execution, web search, and artifact revision—yet they stop short of orchestrating a full research‑to‑paper workflow.

Spark‑to‑Paper bridges this gap by embedding thirteen reusable skills directly into the assistant, each handling a distinct research task while communicating through a shared project directory. Deterministic gates enforce that any operation that can be executed concretely (e.g., running code, generating a plot) is verified before the language model proceeds.

**Algorithm 1** Evidence-Grounded Experiment Execution **Require:** Manuscript $M$, experimental design $D$, available code $C$, data $X$ **Ensure:** Revised manuscript $M'$, experiment artifacts $A$ 1: Identify claims $Q$ in $M$ that require experimental evidence 2: Derive required experiments $\mathcal{E}$ from $D$ and $Q$ 3: **for** each experiment $e \in \mathcal{E}$ **do** 4: $\quad$ Assess whether $e$ is feasible using $C$ and $X$ 5: $\quad$ **if** $e$ is feasible **then** 6: $\quad \quad$ Execute $e$ and record configurations, seeds, logs, and measurements 7: $\quad \quad$ Verify result provenance and experimental consistency 8: $\quad \quad$ **if** the result passes verification **then** 9: $\quad \quad \quad$ Admit the measured result into $A$ 10: $\quad \quad \quad$ Update the corresponding tables and figures 11: $\quad \quad$ **else** 12: $\quad \quad \quad$ Reject the result and revise the associated claim 13: $\quad \quad$ **end if** 14: $\quad$ **else** 15: $\quad \quad$ Record the missing resource 16: $\quad \quad$ Keep the corresponding result unspecified 17: $\quad$ **end if** 18: **end for** 19: Re-evaluate manuscript claims using the admitted evidence 20: Propagate revisions across evidence-dependent sections 21: Preserve negative, null, and inconclusive results 22: Recompile and validate the revised manuscript 23: **if** the central research objective remains unsupported **then** 24: $\quad$ Pass the trajectory to the bounded Self-Refutation procedure 25: **end if** 26: **return** $M', A$

Two additional mechanisms safeguard long‑horizon reliability. First, experiment planning is decoupled from reporting so that the evidence needed by a claim is specified ahead of time. Second, the system detects a Self‑Refutation Loop—repeated cycles where experiments continually reject the original hypothesis—and bounds it by recording the failure trajectory and restarting from a new idea.

The figure pipeline further separates quantitative plots, which are generated directly from measured data, from explanatory method diagrams, which are first drafted with an image‑generation model and then reconstructed into editable vector PDFs via code.

In controlled experiments across eight topics, Spark‑to‑Paper attains 99.5 % citation validity and 96.4 % figure editability. An ablation that injects unsupported claims shows fabrication detection climbing from 14 % (single‑pass) to 92 % with the full integrity stack, while adversarial review reaches 74 % precision. The full system consumes 11.9 M tokens, costs \$8.1 per manuscript, and completes in 3.2 hours on average.

The key insight is that moving from pure text generation to a workflow‑driven execution model makes research paper generation reliable and auditable.

Prior Autonomous Research Systems

Positions Spark‑to‑Paper among prior autonomous research systems.

Prior work falls into two poles: monolithic agents that run the full research loop as a standalone service, and lightweight plugins that augment an existing coding assistant with isolated writing steps. The former family—e.g., Lu et al. [13], Yamada et al. [28], Liu et al. [12], Mitchener et al. [15], Ghareeb et al. [7]—covers ideation, experiment execution, and manuscript drafting but typically ships with its own orchestration server or graph database. The latter family—e.g., Xu et al. [27], Wu [26], Elovic [5], Karpathy [10], Chen [4], Weng et al. [25]—exposes composable skills inside a coding assistant, yet none run real experiments end‑to‑end or emit editable vector figures.

These are systems that attempt to automate the entire research pipeline as a single, self‑contained application, often requiring a dedicated backend to schedule code, store results, and manage citations.

Table 1 summarizes a qualitative capability audit of seven representative systems against Spark‑to‑Paper. The audit tracks five dimensions—End‑to‑End coverage, Experiment execution, Figure generation, Editable vector output, and Infrastructure independence. Spark‑to‑Paper uniquely scores a full checkmark on every dimension, whereas all other systems lack either editable vectors or infrastructure‑free operation, and many miss additional capabilities.

**Figure 1.** Qualitative positioning against directly related systems, on two axes derived from Table 1. Coordinates are the authors' own ordinal assessment from each system's public documentation, not a measured benchmark.

The Spark-to-Paper Skill Stack

Spark‑to‑Paper replaces a monolithic writer with thirteen focused skills that cooperate via a shared project directory.

Monolithic generation bundles all research tasks into a single open‑ended prompt, which frequently yields inconsistent citations, missing data, or broken LaTeX. Spark‑to‑Paper solves this by decomposing the workflow into thirteen narrowly scoped skills that communicate through a common project folder. The coding assistant orchestrates the skills, preserving reproducibility while retaining flexibility.

Break the end‑to‑end paper‑writing pipeline into thirteen self‑contained skills, each responsible for one well‑defined research sub‑task.

Planning reads the idea, produces a blueprint file containing a title, research question, and section outline.

Citation reads the blueprint, searches two mock papers, and writes a BibTeX file with two verified entries.

Writing reads both the blueprint and the BibTeX file, then emits a short LaTeX manuscript that cites the two entries and follows the outlined sections.

This toy run shows how each skill consumes only the artifacts it needs and leaves the rest untouched, guaranteeing that later skills see a consistent, up‑to‑date project state.

How does this skill‑based pipeline differ from a monolithic agent that generates the whole paper in one prompt?

In a monolithic prompt the model must simultaneously decide structure, locate citations, produce figures, and write prose, which leads to hidden dependencies and unrecoverable errors. By contrast, each skill focuses on a single responsibility, allowing deterministic checks after every step and making failures localizable and fixable.

Stage 0 – Input Routing: inspect the user input, decide between Proposal Mode and Data‑Aware Mode, and select the appropriate result‑integrity mode.

Stage 1 – Planning: generate a structured blueprint (question, contributions, sections, experimental design).

Stage 2 – Citation: search literature, verify references, and produce a reusable BibTeX file.

Stage 3 – Writing: render the full LaTeX manuscript from the blueprint and bibliography, respecting the chosen integrity mode.

Stage 4 – Refinement: globally polish the manuscript, remove redundancy, and re‑run deterministic checks.

Stage 5 – Review: run isolated review passes that flag technical, experimental, or evidential issues; flagged items return to refinement.

Stage 6 – Figure Generation: programmatically plot measured results or invoke an image model for diagrams, then convert rasters to editable formats.

Stage 7 – Assembly: combine manuscript, bibliography, figures, and venue template; compile LaTeX and verify no unresolved citations or errors.

Stage 8 – Experiment Execution: if the blueprint specifies feasible experiments, run them, capture measurements, regenerate data‑dependent tables/figures, and update quantitative claims.

**Figure 2.** Overview of Spark-to-Paper execution. Stage 0 selects the result-integrity mode, Stages 1–7 coordinate through persistent project artifacts, and conditional Stage 8 writes measured evidence back into the manuscript. Model judgment governs context-dependent decisions, while deterministic tools execute checkable operations.

By delegating judgment to the model and verification to deterministic tools, Spark‑to‑Paper achieves a reproducible, extensible workflow that avoids the hidden inconsistencies of monolithic generation.

Enforcing Experimental Integrity

Hard checks stop obvious errors while model‑based critique fixes semantic drift.

Long‑horizon paper generation accumulates two classes of errors: explicit, machine‑checkable violations and subtle semantic drifts that only emerge after several generation steps. Deterministic gates block the former, while adversarial self‑critique iteratively catches the latter.

Think of a security checkpoint that lets a manuscript pass only if every required item—citations, figures, LaTeX compilation, and result provenance—is present and well‑formed.

Blueprint check passes because the overall structure follows the template.

Citation check fails: the missing bibliography entry triggers a gate violation.

Figure check fails: the missing file for fig5 aborts the stage.

Result‑integrity check fails in Proposal Mode because the BLEU cell is filled without a data file.

The pipeline halts and reports three distinct gate failures.

Deterministic gates expose exactly which artifact is missing, allowing the system to generate or retrieve that artifact before proceeding.

How do deterministic gates differ from learned gating mechanisms that appear in some neural architectures?

Deterministic gates are hard‑coded program checks with explicit success/failure conditions; they do not involve any learned parameters or gradient updates. Learned gates, by contrast, produce soft decisions based on data and can be wrong, whereas deterministic gates either pass or stop the pipeline.

Imagine a panel of reviewers: a local reviewer (Self‑Review) scans the recent edit for terminology drift, while a global reviewer (Adversarial‑Review) probes the whole manuscript for deeper theoretical or experimental inconsistencies.

Self‑Review spots the redundant phrase and suggests removing two occurrences.

Self‑Review also flags the empty BLEU cell as a missing result.

Adversarial‑Review reads the claim and notes that no experimental evidence is provided, violating the theory‑experiment consistency check.

Both issues survive the refutation step and are returned to the revision module.

The revision module fills the BLEU cell with actual numbers and edits the paragraph to a concise statement.

Self‑Review catches surface‑level problems quickly, while Adversarial‑Review ensures that high‑level claims are backed by concrete evidence.

Why are both Self‑Review and Adversarial‑Review needed instead of just one comprehensive review?

Self‑Review is lightweight and can be invoked after every edit, preventing error accumulation. Adversarial‑Review looks at the manuscript holistically and can catch inconsistencies that only appear when multiple sections interact. Together they provide a tiered safety net that balances speed and depth.

Think of the classic scientific loop: propose a hypothesis, run an experiment, critique the outcome, and revise the hypothesis—repeating until the evidence satisfies the claim.

Self‑Review notes that the gain is marginal and flags the claim as partially‑supported.

Adversarial‑Review checks whether the improvement holds across other datasets; it finds no gain on Dataset Y.

The cycle revises the hypothesis to “attention helps on specific tasks” and adds a new experiment on Dataset Z.

Second experiment yields a 3 % gain on Dataset Z; both reviews now accept the revised claim.

The loop terminates after two iterations, and the claim is recorded as supported.

The cycle adapts the hypothesis based on concrete evidence rather than forcing the original claim to fit the data.

What happens if the experiment‑critique‑revision loop reaches the seven‑iteration limit without a supported claim?

The system generates a failure report that records the original hypothesis, all attempted methods, the observed results, and the reasons for insufficiency. This report is treated as a legitimate research outcome, and the pipeline starts a new hypothesis instead of forcing a weak claim into the manuscript.

**Figure 3.** Integrity and correction in Spark-to-Paper. Deterministic gates enforce verifiable properties. Self-Review and Adversarial Review challenge semantic decisions, and surviving issues trigger revision. Experiment–critique–revision cycles are capped at seven; an unresolved trajectory becomes a failure report rather than a successful manuscript.

Performance and Ablation Results

Key quality gains and their incremental costs are quantified across the full system.

Recall that Spark-to-Paper decomposes paper generation into thirteen skill‑based stages guarded by deterministic gates and adversarial self‑critique, avoiding the consistency errors of monolithic pipelines. The following results assess whether this integrity stack delivers higher quality at reasonable cost.

Spark-to-Paper attains substantially higher citation validity than the single‑pass LLM baseline while remaining cost‑effective.

99.5 % vs 81 % citation validity (Table 3); the gap corresponds to a +18.5 % improvement over the baseline.

**Table 3.** Main comparison of Spark-to-Paper with human-written preprints, prior autonomous research systems, and a single-pass LLM baseline across artifact quality and generation efficiency metrics.

**Table 4.** Ablation study of the Spark-to-Paper quality stack, showing the contribution and incremental cost of gating, self-review, and adversarial review.

**Table 1.** Comparison of research paper generation systems across metrics including citation existence, figure editability, token usage, cost in USD, and wall-clock time.

**Figure 6.** Case study on two different domain demo papers with only one short proposal as inputs. Incorrect expectations are highlighted by Bold and Underline text.

The integrity stack yields large quality jumps (up to +78 % fabrication detection) for modest incremental cost (≈ \$5 USD per added review stage).

Implementation Details and Failure Modes

Limits of deterministic gating, experiment execution, and claim admission.

Deterministic gates enforce explicit correctness criteria on each stage’s artifacts, halting the pipeline on fatal violations and merely warning on non‑blocking issues.

Checks that the venue template supplies all required structural elements before any manuscript content is generated.

Ensures the planned manuscript (the blueprint) conforms to the venue’s structural constraints.

Guarantees bibliographic integrity before the manuscript proceeds.

Validates that the assembled manuscript contains no placeholders or invalid result tables.

Verifies that each figure’s artifact matches its declared role (measured vs. explanatory).

The Figure Gate often falls back to raster when the iterative HTML‑to‑vector reconstruction cannot guarantee a faithful editable output, leading to loss of editability and potential visual artifacts.

Ensures the final LaTeX project compiles with all citations and cross‑references resolved.

Experiment execution is treated as an evidence‑grounded process: required measurements are identified, feasibility is checked, and only verified results are admitted as experimental evidence.

Algorithm 1 – Evidence‑Grounded Experiment Execution

Feasibility checks preserve the experimental design from the planning stage; unavailable resources are logged and the corresponding result left unspecified rather than substituted with a synthetic value.

The claim admission protocol re‑evaluates each major claim against the verified evidence, assigning one of five evidence labels and triggering the appropriate revision action.

This table lists various research artifacts, their associated development stages, and their specific purposes in the research workflow.

**Figure 4.** Role-aware editable figure generation. Measured results follow deterministic plotting to native vector PDF, whereas explanatory content uses a raster visual target followed by iterative HTML reconstruction and vector export; unreliable reconstruction falls back to the raster.

**Figure 5:** Page, reference, and figure counts for the end-to-end papers documented in the project's own showcase materials. Self-reported by the system's maintainers, not an independent measurement; included as existence evidence, not as a benchmark result.

Generated‑paper examples are provided solely to demonstrate the structural and visual form of Spark‑to‑Paper outputs; they are not meant as quantitative evaluations.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers