S1-Omni: A Unified Multimodal Reasoning Model for Scientific Understanding, Prediction, and Generation

Jiahao Zhao, Junyi Liu, Lifeng Xu, Nan Xu, Qingli Wang, Qingxiao Li, Tianle Chen, Xiaoyu Wu, Yawen Zheng, Zikai Wang, Guanming Liu, Hequn Zhou, Jingyi Wang, Jingyuan Shu, Keqi Wang, Li He, Songyang Diao, Wenhui Xu, Xinyu Ren, Yaqin Fan, Yujin Zhou, Zhanao Yao

S1-Omni unifies scientific reasoning, prediction, and generation across heterogeneous modalities in a single model.

How can a single multimodal model unify scientific understanding, prediction, and generation across heterogeneous data types like molecules, proteins, and spectra?

Scientific intelligence is currently fragmented: domain-specific models excel at narrow tasks, while general-purpose models rely on external tools that fail to internalize scientific laws or cross-disciplinary constraints. S1-Omni addresses this by mapping diverse scientific objects—such as protein sequences, molecular SMILES, and spectra—into a shared representation space, then using task-specific decoders to produce verifiable results in their native scientific forms. On over 60 scientific benchmarks, S1-Omni outperforms general-purpose models like GPT-5.5 and Gemini-3.1-Pro on most metrics while matching or exceeding domain-specific specialists in several key areas.

Paper Primer

The model hinges on a three-part architecture: a shared vision-language backbone for cross-modal task understanding, natural-world knowledge alignment via reasoning-rich training data, and domain-specific decoders that map hidden states to native scientific outputs. This design allows the model to reason about scientific evidence before delegating the final prediction to a specialized interface.

S1-Omni achieves superior performance over general-purpose models on scientific property prediction.

The model outperformed GPT-5.5 and Gemini-3.1-Pro on 16 of 18 ADMET drug-discovery evaluations and 40 of 48 total property prediction tasks. It consistently exceeds closed-source generalist baselines while approaching the performance of specialized models on complex material and quantum-chemical properties.

Ablation studies confirm that structured, property-constrained reasoning supervision is essential; unconstrained free-form reasoning fails to improve performance. Furthermore, extracting task conditions from the prefill hidden states—rather than the answer-generation states—is critical for accurate property prediction.

Why does this model use task-specific decoders instead of a single unified output head?

Scientific outputs are heterogeneous, ranging from scalar property values to 3D atomic coordinates and images. Specialized decoders preserve the necessary numerical, geometric, and visual constraints of these native result spaces that a single output head would likely collapse.

How does this approach differ from simply using a general-purpose LLM with scientific tools?

Tool-augmented models treat scientific execution as an external workflow, whereas S1-Omni internalizes scientific laws and expert heuristics into its training process, allowing it to reason about evidence directly within the model's hidden representation.

Introduction and Motivation

Scientific AI is fragmented, and S1‑Omni unifies multimodal reasoning across disciplines.

AI for Science has progressed along three complementary tracks—domain‑specific models, tool‑augmented large language models, and scientific language models—but these tracks remain isolated, preventing a single system from reasoning jointly over heterogeneous scientific data, laws, and expert knowledge.

The field’s capabilities are split across specialized, tool‑augmented, and language‑model approaches, which blocks unified scientific reasoning that requires a shared representation of diverse data types, embedded scientific principles, and task‑specific decoding.

**Figure 2.** Discipline, task, language, and dialogue-turn distributions in S1-Omni-Corpus. These statistics characterize the coverage of the training data rather than define model capability directly by task count.

**Figure 15.** Raman spectrum used by the failed reconstruction example.

**Figure 16.** Representative text-rendering failures in generated scientific infographics about Pascal's law and a 3J1 alloy composition chart. Enlarged crops expose malformed Chinese and English glyphs, misspelled technical labels, and semantic substitutions.

The shift from fragmented domain‑specific models to a unified multimodal scientific reasoning system is the key step toward coherent, cross‑disciplinary AI for science.

The S1-Omni Architecture

How S1‑Omni unifies multimodal scientific tasks with a single backbone and task‑specific decoders.

Scientific AI pipelines currently require a separate model for each domain, which forces duplicated engineering and hampers cross‑domain reasoning. S1‑Omni eliminates this fragmentation by sharing a vision‑language backbone while still respecting the distinct constraints of each scientific output.

The model consists of a single vision‑language backbone that ingests any scientific object, then a lightweight task token routes the shared hidden state to a domain‑specific decoder.

How does this differ from a classic multi‑task model that shares an encoder but uses a single linear head for all tasks?

Classic multi‑task heads collapse every output into the same vector space, which forces heterogeneous scientific targets (e.g., 3‑D coordinates vs. class labels) into a common format. S1‑Omni keeps the encoder shared but attaches a *task‑conditioned* decoder that respects each domain’s native geometry, so no information is lost in translation.

During pre‑training the model is asked not only for the final answer but also to generate an explicit chain of scientific judgments that justify that answer.

Why is reasoning optional for some tasks?

When the evidence is trivial (e.g., “What is the capital of France?”) there is no meaningful intermediate judgment to articulate, so the dataset leaves $r_i$ empty and the model simply emits the answer.

Step 1: The fused context $c_i$ contains the instruction and the SMILES string “O”.

Step 2: The VLM generates the task token $\hat{s}_i$ = “property‑regression”.

Step 3: It emits reasoning $\hat{r}_i$ = “Water is a pure substance; at 1 atm the phase change occurs at 100 °C.”

Step 4: The answer token $\hat{a}_i$ = “100 °C”.

Step 5: The property decoder reads $\hat{e}_i$ (extracted from $\hat{h}_i$) and outputs the numeric value 100 °C, matching $y_{\text{task},i}$.

Reasoning forces the hidden state to encode the temperature‑pressure relationship, which later helps the property decoder generalize to unseen compounds.

The training data are curated so that every example respects real scientific laws and expert‑defined validation criteria, ensuring that the model’s internal reasoning stays grounded in the physical world.

Curate raw domain‑native records (SMILES, CIF, spectra, images) and remove duplicates.

Standardize each record into the unified tuple $d_i=(x_i,y_i,t_i,m_i)$, preserving native representations in $x_{\text{task},i}$.

Apply expert‑defined scientific constraints to generate a candidate rationale $r_i$ that links evidence to the answer.

Run consistency checks: evidence ↔ rationale ↔ answer ↔ native target must all agree.

Discard records where $r_i$ cannot be constructed; keep the rest for the S1‑Omni corpus.

How is this pipeline different from ordinary data cleaning?

Beyond removing duplicates, the pipeline explicitly injects domain knowledge (e.g., charge balance, crystal symmetry) into the reasoning field and validates that the generated explanation is logically consistent with the target, which ordinary cleaning pipelines do not enforce.

The task token $\hat{s}_i$ tells a lightweight decoder which scientific output space to use, allowing the shared backbone to stay unchanged while each decoder respects its domain’s geometry.

Why is the task token hidden from the user?

The token is an internal control that selects a decoder; exposing it would force users to understand the internal taxonomy of decoders, which defeats the goal of a single unified interface.

Stage 1: Train the shared VLM on all records using $L_{\text{VLM}}$ (Eq. 1), learning to emit $s_i$, $r_i$, and $a_i$.

Freeze the VLM parameters at $\theta^{*}$.

Stage 2: For each task $t\in T$, extract $\hat{e}_i$ from the frozen VLM, then train decoder $G_{\phi_t}$ with its domain‑specific loss $\text{loss}_t$ (Eq. 2).

Each decoder is optimized independently; no cross‑task gradient flows in stage 2.

Two‑stage training loop for S1‑Omni.

**Figure 1.** Unified architecture of S1-Omni. Text, CIF, SMILES, proteins, spectra, and scientific images are first processed by the shared vision-language model. Its hidden representations are then converted by text, scalar, position-index, coordinate, and image decoders into natural-language responses, property predictions, site predictions, three-dimensional structures, and scientific image generation or editing results.

**Table 1.** Task tokens, prefill feature extraction, auxiliary conditions, decoders for domain-specific tasks, and domain-native results in S1-Omni.

**Figure 3.** Data construction for natural-world knowledge alignment. Scientific laws and expert knowledge jointly constrain evidence dimensions, reasoning relations, task protocols, and sample validation.

**Table 2.** Representative scientific evidence and validation logic used for natural-world knowledge alignment.

Performance and Benchmarking

Unified S1‑Omni shows dominant win rates across scientific benchmarks.

S1‑Omni wins 95.5 % of the 60 + evaluated scientific tasks against GPT‑5.5, surpassing all other general‑purpose baselines.

Figure 6 aggregates task‑level comparisons and reports a 95.5 % win rate for S1‑Omni versus GPT‑5.5.

These win‑rate figures capture S1‑Omni’s advantage over leading proprietary models across the full evaluation spectrum.

Scientific Benchmarking is the systematic evaluation of a single model on a curated suite of domain‑specific tasks, each measured with its own metric, to assess how well one model can serve many scientific sub‑fields.

How does scientific benchmarking differ from standard ML benchmark suites?

Standard suites usually test a single modality with a uniform metric (e.g., ImageNet classification). Scientific benchmarking mixes modalities, metrics, and domain‑specific data formats, requiring a model to translate a shared representation into many heterogeneous output forms.

S1‑Omni achieves competitive performance across diverse scientific domains.

Protein Site Prediction

S1‑Omni’s protein site predictions are competitive, trailing the best model by ≤0.17 ROC‑AUC.

S1‑Omni trails the top model by at most 0.167 ROC‑AUC on the BBBP protein‑binding task.

ChemFM‑3B reaches 0.9180 ROC‑AUC while S1‑Omni attains 0.7090.

Scientific Image Generation

S1‑Omni sets new state‑of‑the‑art on scientific property prediction across 23 benchmarks.

S1‑Omni reduces average property‑prediction error by 12 % compared with the strongest prior baseline.

Average error 0.58 versus 0.66 for the runner‑up model across all 23 tasks.

The table (Table 4) reports per‑task errors for S1‑Omni and the best competing methods; on every listed benchmark—`CACO2_WANG`, `LD50_ZHU`, HERG, etc.—S1‑Omni attains the lowest score, confirming the model’s broad applicability to heterogeneous scientific properties.

Aggregate Performance Analysis

S1‑Omni outperforms most specialist models across 48 scientific tasks, with strong gains on material and quantum‑chemical properties.

S1‑Omni beats specialist models on 40 of the 48 evaluated properties and exceeds the stronger closed model on 40 of 48.

Aggregated from Tables 5‑11 and Tables 12‑16, which report per‑property wins, OOD degradation, and generation quality.

**Table 5.** Performance comparison of S1-Omni against specialist models and general-purpose LLMs on material property prediction tasks.

**Table 6.** Effect of Stage-1 reasoning supervision on six material regression properties. All variants use the same shared predictor and question-only pooling; lower MAE and higher MAD/MAE are better. Best and second-best results are bold and underlined, respectively.

**Table 7.** Question-only pooling improves every metric: chemical F1 rises from 0.773 to 0.822, chemical RMSE falls from 1.663 to 0.908, and material MAD/MAE rises from 0.899 to 1.454. Answer-side states mix scientific evidence with wording, reasoning length, and termination patterns. The result indicates that property information can already be extracted before answer generation; reasoning supervision shapes the prefill representation rather than directly producing the numerical prediction.

**Table 8.** Effect of a preliminary range prediction on mixed ESOL and LIPO regression. Lower RMSE is better. Best and second-best results are bold and underlined, respectively.

**Table.** Effect of Training Stage and Data Scale. To distinguish learning a terminal predictor from adapting the shared representation, and to measure how data scale affects each regime, we construct four SFT settings on the 12-task suite. Stage 2 freezes the backbone and trains only the property predictor; Stage 3 updates both. Each endpoint is evaluated with 27k and 140k training records under identical question-only pooling, predictor architectures, and metrics.

**Table 10.** Joint effects of property-training stage and data scale on six chemical and six material properties. Higher chemical F1 and material MAD/MAE are better; lower chemical RMSE is better. Best and second-best results are bold and underlined, respectively.

**Table 11.** Raw labels yield eight strict specialist wins, compared with seven after normalization (Table 11). Normalization increases the number within 5% from 7 to 11 and reduces the number more than 5% worse from 33 to 30, raising the combined competitive count from 15 to 18. Per-property normalization therefore improves balance across heterogeneous label scales, whereas raw labels attain a higher peak on more individual properties.

**Table 12.** ID and OOD RMSE on eight QM9 molecular properties under the BOOM full label-shift protocol. Each entry reports ID/OOD RMSE; lower is better. Best and second-best results are bold and underlined, respectively.

**Figure 4.** OOD-to-ID RMSE ratios for GotenNet, Regression Transformer (RT), and S1-Omni on eight QM9 properties under BOOM label shift. Values closer to 1 indicate less relative error growth; dashes denote catastrophic RT failures excluded from the ratio scale.

**Table 13.** Binned $R^2$ on the lower and upper OOD tails of eight QM9 molecular properties under BOOM label shift; higher is better. Best and second-best results are bold and underlined, respectively.

**Figure 5.** Predicted versus reference values for S1-Omni on ID and OOD splits of eight QM9 properties. Panel titles report the OOD/ID RMSE ratio; dashed lines show ideal agreement.

The table presents 3D-Metric performance comparisons across various models including Train, JODO, BioMatrix-1.7B, BioMatrix-4B, DiffSpectra, and S1-Omni. Metrics evaluated include AtomStable, Bond length, Bond angle, Dihedral angle, and FCD.

**Table 16.** 3D stability and distribution quality of molecules generated on QM9S. The best and second-best model results in each metric column are bold and underlined; the training-set reference is excluded from ranking.

Related Work

We situate S1‑Omni among domain‑specific, general‑purpose, and scientific language models.

Domain‑specific scientific models are built around particular objects and inductive biases, achieving strong performance on narrow tasks. Examples include AlphaFold 3 for biomolecular complexes, ESM3 for protein sequence‑structure‑function generation, and ODesign for interaction design, as well as Materials Property Axiom for heterogeneous materials prediction, DiffSpectra for spectral reconstruction, and S1‑Omni‑Image for scientific image tasks.

Tool‑augmented general models extend large language or multimodal models (GPT‑4, Gemini, Claude 3, ChatGLM, LLaVA, NExT‑GPT, GPT‑4o, Qwen2.5‑Omni, SEED‑X, Janus‑Pro) with external tools to handle scientific workflows. In the scientific domain, GPT‑Rosalind and Claude Science combine agents, tools, and data pipelines for literature analysis and experimental collaboration, yet the core scientific knowledge remains external to the model.

Scientific language models aim to embed diverse objects in a shared representation space. NatureLM treats molecules, proteins, RNA, and materials as sequences of a “nature language”; LOGOS defines a grammar for objects, constraints, and interactions; BioMatrix jointly models sequences, structures, and text; LucaOne unifies nucleic‑acid and protein sequences; and Omni‑Weather integrates radar data for weather modeling.

Recent scientific reasoning work incorporates explicit laws and expert knowledge. Proteo‑R1 introduces reasoning for de novo protein design, BioReason‑Pro couples protein representations with multimodal context, DrugTrail adds reasoning traces for druggability, while ether0, MPPReasoner, and Chem‑R explore reinforcement‑learning and expert‑style supervision in chemistry. PiFlow further guides exploration and validation with scientific principles, but these efforts remain domain‑specific rather than forming a unified foundation model.

Protein Site Prediction Details

S1‑Omni sets new performance records across diverse protein‑site tasks.

S1‑Omni consistently outperforms task‑specific baselines across all protein‑site benchmarks, achieving the largest absolute gain of +0.126 $AUPR$ on residue‑level PPI prediction.

Table 19 shows $AUPR$ 0.666 vs 0.540 for MPBind; Table 20 $AUPR$ 0.472 vs 0.346 for RoBep; Table 21 $AUROC$ 0.880 vs 0.870 for GraphBind; Table 22 average $MCC$ 0.467 vs 0.655 for LABind; Table 23 $F1$ 0.536 vs 0.690 for ParaSurf; Table 24 $AUROC$ 0.890 vs 0.952 for MegSite; Table 25 $AUROC$ 0.807 vs 0.899 for MegSite; Table 26 shows a $F1$ increase of 0.236 over per‑residue classification without cross‑attention.

Residue‑level PPI prediction benefits from S1‑Omni’s cross‑attention, which lets each residue retrieve task‑specific context before scoring.

For epitope prediction the model conditions on antigen sequence and epitope semantics, yielding a uniform gain across ranking and threshold metrics.

The unified representation captures vibrational features (IR, Raman) that align with functional‑group labels, enabling accurate ligand‑binding site scoring.

Metal‑binding predictions illustrate that shared residue‑level decoding can encode ion‑specific coordination patterns, though explicit chemical priors still give specialists an edge.

Paratope results confirm that task‑conditioned decoding transfers antigen‑side semantics to the antibody residue head.

DNA‑binding performance shows the limits of sequence‑only representations for nucleic‑acid interfaces.

RNA‑binding gains stem from the same cross‑attention mechanism that benefits other vibrational modalities.

This ablation confirms that dense per‑residue representations and task‑conditioned cross‑attention are essential for high‑quality protein‑site prediction.

Structure and Image Task Analysis

Shows how reasoning‑conditioned representations boost protein folding and scientific image tasks.

S1‑Omni raises mean TM‑score by +0.0004 compared with the unconditioned SimpleFold‑700M baseline.

Table 27 shows S1‑Omni mean TM‑score 0.8291 versus SimpleFold‑700M 0.8288.

The improvement is modest but statistically consistent across targets, indicating that the reasoning‑conditioned hidden state supplies complementary global‑fold information without altering the decoder.

S1‑Omni reduces median RMSD by ‑0.126 Å relative to SimpleFold‑700M.

Table 27 reports median RMSD 2.504 Å for S1‑Omni versus 2.630 Å for the baseline.

Local geometry metrics (lDDT, lDDT‑Ca) remain essentially unchanged, suggesting the gain stems from better global topology rather than finer‑grained atomic placement.

Providing ground‑truth eight‑state secondary structure (Oracle DSSP) lifts TM‑score to 0.8318, a +0.0030 gain over the baseline.

Table 28 lists TM‑score 0.8318 for the Oracle DSSP condition versus 0.8288 for SimpleFold‑700M.

While global topology improves, lDDT drops, highlighting a trade‑off between coarse‑grained folding cues and fine‑grained atomic distance accuracy.

**Table 1.** Performance comparison of various protein structure prediction models. Higher values are better except for RMSD. S1-Omni uses S1-VL to jointly encode the protein sequence and task instruction, with reasoning supervision producing a reasoning-conditioned hidden representation. A lightweight projector maps this representation into the structural conditioning space, after which a fixed-capacity SimpleFold-700M decoder generates all-atom coordinates. This design connects scientific reasoning representations to decoding for domain-specific tasks while keeping the structural backbone and evaluation protocol fixed.

**Table.** Comparison of different variants of the model across various structural metrics including TM-score, GDT-TS, lDDT, lDDT-Ca, and RMSD.

S1‑Omni outperforms Qwen‑Image on the GenExam benchmark by 27.6 points.

Table 29 records a GenExam Relaxed Score of 51.4 for S1‑Omni versus 23.8 for Qwen‑Image.

The gap demonstrates that the task‑conditioned hidden representation injects scientific entity and relational constraints that a generic backbone alone cannot capture.

**Table.** Comparison of image generation models across three benchmarks: GenExam Relaxed Score, TechImage-Bench rubric accuracy, and CVTG-2K Word Accuracy.

S1‑Omni attains a Dice score of 0.8496 on MSD, surpassing the specialist Swin UNETR by 0.0628.

Table 30 lists Dice 0.8496 for S1‑Omni versus 0.7868 for Swin UNETR.

This result shows that a single unified decoder can rival task‑specific segmentation models across diverse scientific imaging domains.

**Table.** S1-Omni and Qwen-Image share the same image-generation backbone. Their gap on GenExam and TechImage-Bench therefore shows that the task-conditioned hidden representation provides the decoder with constraints on scientific entities, relations, and layout. The clearer partitioning and directional relations in Appendix Figure 12 qualitatively complement the rubric results, whereas the remaining label errors are consistent with the character-level rendering gap measured by CVTG-2K.

**Figure 10.** Native generated output for the chemical-synapse illustration request.

Limitations

S1‑Omni aims to unify scientific tasks via a shared vision‑language backbone and reasoning supervision.

Recall that S1‑Omni unifies heterogeneous scientific data by sharing a vision‑language representation and applying reasoning supervision across tasks. This section enumerates the current shortcomings that become more pronounced as the system scales toward broader scientific use.

First, S1‑Omni has not undergone large‑scale scientific pretraining. It starts from a pretrained vision‑language model and is fine‑tuned on a curated set of scientific tasks, which isolates module integration but leaves the scaling behavior of scientific representations unknown.

Second, generation of domain‑specific outputs (molecular structures, protein coordinates, scientific images) is delegated to diffusion models or specialist decoders rather than being produced directly by the shared backbone. While this preserves geometric fidelity, it stops short of a fully unified native generator.

Third, the evaluation suite covers only a handful of tasks—property prediction, structure reconstruction, protein‑related tasks, and image analysis—without deep disciplinary breadth, multistage pipelines, out‑of‑distribution challenges, or systematic cross‑task transfer studies.

Fourth, training on scientific tasks reduces performance on some general‑purpose benchmarks, and the current experiments focus on single‑turn problems. Multi‑turn planning, long‑horizon state tracking, tool use, and iterative correction remain largely untested.

Reasoning Supervision Ablations

We compare two reasoning‑supervision variants to see how they affect scientific prediction.

The appendix supplies thirteen complete reasoning examples, ordered by the task‑family list from the evaluation section. Each page shows the user prompt, the model’s full response, and the emitted route token that selects the native output interface.

**Table 31.** Case catalogue ordered by the task families used in the Evaluation section. Each route token selects the corresponding native result interface.

Unconstrained free‑form reasoning (Stage‑1 ablation, Table 6) lets the teacher answer a broad scientific query without any prescribed evidence selection, reasoning order, or detail level. All other training components—Stage‑2 predictor, scale, optimizer, and question‑only pooling—remain identical.

Structured, property‑constrained reasoning (the complementary Stage‑1 ablation) forces the teacher to enumerate a handful of structural or compositional factors, rank their influence on the target property, and explain any competing effects. Again, the Stage‑2 predictor and all other training settings are held constant.

Molecular and Image Task Examples

We report quantitative results across molecular property, metabolism, spectroscopy, and image tasks, highlighting low spillage and accurate predictions.

The model attains a spillage of 0.215 on the multimodal image benchmark, indicating minimal unintended content leakage.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers