Program-as-Weights: A Programming Paradigm for Fuzzy Functions

Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng

Program-as-Weights compiles fuzzy natural-language tasks into small, locally executable neural adapters.

How can we turn natural-language descriptions of "fuzzy" tasks into small, portable, and locally executable neural weight files?

Developers often outsource "fuzzy" tasks—like log filtering or intent-based ranking—to large language model APIs because they resist precise symbolic rules. This creates a dependency on remote, costly, and opaque services for simple logic. Program-as-Weights (PAW) solves this by compiling a natural-language specification into a compact, reusable neural adapter. A 4B-parameter compiler generates this adapter, which is then injected into a frozen, lightweight 0.6B-parameter interpreter that runs entirely offline. On the FuzzyBench dataset, this 0.6B interpreter outperforms direct prompting of a 32B-parameter model while using 50x less memory.

Paper Primer

PAW treats the foundation model as a tool builder rather than a per-input solver. The core move is a hybrid program: a discrete "pseudo-program" that cleans the user's noisy specification, paired with a continuous LoRA adapter that injects task-specific behavior into the frozen interpreter.

PAW achieves higher accuracy than direct prompting of significantly larger models.

On FuzzyBench, a 0.6B-parameter interpreter with a PAW adapter reaches 73.78% exact match, compared to 68.70% for a 32B-parameter model.

The compiler-generated pseudo-program provides robust denoising for real-world specifications.

When specifications contain heavy typos, the PAW pipeline maintains performance significantly better than feeding raw specifications directly to the interpreter. 4.5 percentage point accuracy gap on heavy-typo inputs.

Why use a hybrid (discrete + continuous) program instead of just a continuous adapter?

The discrete pseudo-program acts as a buffer that shields the interpreter from ambiguity and noise in the user's original specification, while the continuous LoRA adapter provides the fine-grained behavioral control that text alone cannot capture.

Does this approach require retraining the interpreter for every new task?

No. The interpreter is frozen and installed once; adding a new fuzzy function only requires compiling a new, small (∼23 MB) LoRA adapter that is hot-swapped at runtime.

PAW shifts the paradigm from per-input API calls to per-function compilation, enabling high-performance fuzzy logic to run locally on resource-constrained devices.

Introduction: The Fuzzy Programming Problem

Program-as-Weights compiles natural‑language specs into tiny neural programs for local execution.

Modern software often outsources ambiguous tasks—such as alerting on important log lines or repairing malformed JSON—to large language‑model APIs. This incurs latency, reproducibility risks, and high cost because each input triggers a heavyweight model call. Program-as-Weights (PAW) eliminates that gap by compiling a natural‑language specification once into a compact neural artifact that can be executed locally, turning a per‑input LLM call into a cheap, offline function call.

A fuzzy function is a task whose intended behavior is easy for humans to describe but hard to encode as exact code or rules.

**Figure 19.** A library of compiled PAW programs. Three example natural-language function specifications (“Classify message urgency”, “Fix malformed JSON”, “Remove personal information”; left) are each compiled into a separate neural program (middle): a discrete pseudo-program in a fixed format plus a continuous per-example LoRA (depicted as red, blue, green adapters). At deployment time (right), all three programs are served by a single device-resident interpreter (LM) with the appropriate LoRA hot-attached per call — the “one runtime, many programs” picture that motivates compile-once-run-locally.

`CODEBLOCK_0`

**Figure 20.** The Alien-Taboo case-study UI. The player describes the secret word (here, “moon”) in free text without using any of the listed taboo words (night, orbit, lunar, full); the alien “Zog” — a one-PAW-function compiled program — must guess the word from the description. Each player turn is served by a 0.6B Qwen3 PAW interpreter on a small server, with one PAW program (and per-program LoRA adapter) per language hot-loaded by the same interpreter; the LLM is invoked only at compile time, not at every move.

How does a “fuzzy function” differ from a conventional deterministic function?

A deterministic function has a precise, rule‑based definition that yields exactly one correct output for any input. A fuzzy function is specified by natural language or examples, tolerates ambiguity, and may admit multiple acceptable outputs; PAW captures this softness in a learned program rather than in hard code.

The core gap is that rigid, hand‑written code forces developers into heavyweight LLM calls for ambiguous tasks.

The PAW Architecture

How the PAW pipeline turns a natural‑language spec into a tiny, hot‑swappable neural program.

The core obstacle is that developers need to run fuzzy logic locally without paying cloud latency or cost, yet a naïve on‑device model would require the full weight set of a large LLM.

PAW treats a natural‑language specification as a recipe that the compiler turns into a tiny neural weight file (a LoRA adapter); the frozen interpreter then “loads” this file and runs the user’s input exactly like calling a library function.

How does PAW differ from simply prompting a large model at inference time?

Prompting asks the model to recompute the entire function on every call, incurring full model latency and cost. PAW compiles the function once into a frozen‑interpreter‑compatible LoRA adapter, so subsequent calls only run the lightweight interpreter plus the tiny adapter.

The neural compiler reads the specification together with the pseudo‑program and, in a single forward pass, emits hidden states that are later reshaped into a parameter‑efficient module.

Why not train the compiler end‑to‑end with the interpreter instead of freezing the interpreter?

Freezing the interpreter isolates the heavy language‑model computation, allowing the compiler to be trained with a simple supervised loss that only back‑propagates through the PEFT mapper. This reduces training cost and makes the compiled program portable across any compatible frozen interpreter.

The mapper treats the compiler’s hidden tensor as a palette of colors and, by mean‑pooling and a tiny MLP, mixes a shared set of LoRA bases into the exact adapter needed for the current fuzzy function.

Mean‑pool over layers and positions: 𝑏𝑎𝑟{h}= (1/4)·([1,0,0,1]+[0,1,1,0]+[1,1,0,0]+[0,0,1,1]) = [0.5, 0.5, 0.5, 0.5].

Pass 𝑏𝑎𝑟{h} through the mapper MLP $\phi$ (a single linear layer with weight w=2 and bias b=0) → scalar $s$ = w·𝑏𝑎𝑟{h}_{sum}+b = 2·2 + 0 = 4.

Linear head maps s to four mixing coefficients: $\alpha^{A}$_{1,att,1}=0.3, $\alpha^{A}$_{1,att,2}=0.7, $\alpha^{B}$_{1,att,1}=0.5, $\alpha^{B}$_{1,att,2}=0.5 (similarly for layer 2).

Shared bases for the attention module: $A^{(att)}_{1}$= [[1,0],[0,1]], $A^{(att)}_{2}$= [[0,1],[1,0]], $B^{(att)}_{1}$= [[1,1],[0,0]], $B^{(att)}_{2}$= [[0,0],[1,1]].

Compute $A^{ex}_{1,att}$=0.3·$A^{(att)}_{1}$+0.7·$A^{(att)}_{2}$= [[0.7,0.3],[0.3,0.7]]; $B^{ex}_{1,att}$=0.5·$B^{(att)}_{1}$+0.5·$B^{(att)}_{2}$= [[0.5,0.5],[0.5,0.5]].

Mean‑pooling collapses the spatial structure of H, so the mapper only needs to learn a small set of scalars; the shared bases capture generic transformation patterns that are re‑weighted per function.

Why not generate the LoRA matrices directly from the hidden states instead of using shared bases?

Direct generation would require a separate large weight matrix for every function, blowing up storage. Shared bases keep the parameter budget constant (N × r × d) and let the mapper specialize each function by only learning a handful of scalars.

The interpreter is a frozen language model that treats the injected LoRA adapter as a plug‑in; at inference it simply runs the user’s input through its unchanged layers, with the adapter modifying the linear transforms on the fly.

Does swapping a LoRA adapter require re‑initializing the interpreter?

No. The adapter is simply added to the interpreter’s weight tensors; the model’s forward pass reads the updated tensors on the next token, so switching adapters is a constant‑time operation.

Instead of producing LoRA matrices, the prefix compiler maps hidden states position‑wise into KV pairs that are prepended to the interpreter’s attention cache.

Why is LoRA preferred over prefix‑tuning in this system?

LoRA yields higher accuracy (56.5 % → 65.7 % vs. 50.4 %) while keeping the adapter size comparable; it also integrates more cleanly with the interpreter’s existing linear modules.

**Figure 1.** Overview of the Program-as-Weights paradigm. Top: compile once in the cloud. A natural-language description of a fuzzy function (here, “classify if this is urgent”) is fed to a neural compiler, which produces a neural program. Bottom: run locally. A small frozen neural interpreter loads the compiled program and runs the user’s input (“Need your signature by EOD!”) to produce the output (“urgent”). The compiled program is a single file that can be cached, version-controlled, and called offline like any other library function.

**Figure 2.** Text-to-LoRA instantiation of PAW (Section 3.2). *Left.* The trained LoRA compiler reads the function specification, the pseudo-program produced by an off-the-shelf prompted pseudo compiler $C_p$ (not depicted), and a fixed sequence of learned prefix tokens; it emits prefix-position hidden states $H$. *Middle.* The LoRA mapper mean-pools $H$, passes it through an MLP, and projects into mixing coefficients that compose LoRA matrices ($A^{\text{ex}}, B^{\text{ex}}$) over shared learnable bases (eq. (3)). *Right.* The frozen interpreter ingests $p_{\text{discrete}}$ prepended to the user input $x$, with the LoRA hot-attached, and generates the output autoregressively. The same pipeline holds for the prefix-tuning precursor (Section 3.3, with architecture in Figure 18); only the mapping from compiler hidden states to PEFT module changes (LoRA $arrow$ KV-cache mapper).

Table 1 shows that the LoRA instantiation outperforms both prompting and prefix‑tuning, reaching 65.7 % accuracy at rank r = 64.

Performance and Benchmarks

PAW delivers top‑tier accuracy with a tiny local model.

We evaluate PAW against three baseline families on identical test sets, so any compute or data‑generation advantage is absorbed in the comparison.

PAW (0.6B) matches prompting performance while enabling local execution.

Exact‑match accuracy 73.78% vs. 68.70% for the strongest prompting baseline (Qwen3‑32B).

Sending the full task description to a large language model at inference time, relying on the model’s pre‑trained knowledge without any weight changes.

How does Direct Prompting differ from PAW’s compilation approach?

Direct prompting sends the entire specification at inference time, incurring network latency and large memory footprints. PAW compiles the specification into a tiny LoRA adapter that runs locally, eliminating those costs while preserving accuracy.

**Table 3.** Table 3 reports six image-conditioned tasks: three CoSyn-400K diagram-understanding tasks (Chemical, Circuit, Music) [Yang et al., 2025, Deitke et al., 2024], the structured-output Im2LaTeX-100K [Deng et al., 2017] and Im2SMILES-20K [Deng et al., 2023] tasks, and the open-ended visual question answering TextVQA [Singh et al., 2019]; full prompts are in Appendix D.

PAW matches prompting performance while enabling local execution.

Design Choices and Ablations

Ablations reveal that each PAW component substantially contributes to accuracy.

We isolate the contribution of each PAW component by ablating it and measuring the resulting accuracy drop.

Removing the compiler drops performance by 15.4 points compared to PAW.

Full fine‑tuning reaches 0.5840 while PAW achieves 0.7378 (Table 5).

The strongest fixed LoRA falls 21.7 points short of PAW.

Fixed LoRA r=64 scores 0.5210 versus PAW’s 0.7378 (Table 5).

**Table.** Comparison of interpreter input methods on accuracy under clean and heavy typo conditions.

Feeding the pseudo‑program improves heavy‑typo accuracy by 4.5 points over raw specifications.

Heavy‑typo accuracy: 0.6108 with pseudo‑program vs. 0.5662 with raw spec (Table 7).

**Table 7.** The pseudo-program protects the interpreter from noisy specifications. On heavy-typo specifications, feeding raw spec is 4.5 points worse than feeding the pseudo-program to the interpreter.

Quantizing the base to 4‑bit plus a LoRA adapter loses only 1.3 points relative to bf16.

4‑bit `Q4_K_M` base with LoRA achieves accuracy within 1.3 points of the unquantized bf16 model (Table 6).

**Table.** Quantization findings on the 0.6B Qwen3 interpreter, validated at a 4096-example test subset: a 4-bit base (`Q4_K_M`, ~484 MB) plus a `Q4_0` LoRA adapter (~23 MB per program) loses only 1.3 points relative to bf16, and a `Q6_K` base plus `Q4_0` adapter is statistically indistinguishable from bf16.

Developer Interface and Deployment

PAW programs run locally as tiny, cacheable neural weight files.

PAW compiles natural‑language specifications into tiny neural weight files that can be executed locally without any cloud API calls.

Quantizing the 0.6 B Qwen3 interpreter to a 4‑bit $Q6_K$ base with a $Q4_0$ LoRA adapter yields accuracy indistinguishable from bf16.

Table 8 shows 0.6575 vs. 0.6580 ($\Delta$ = ‑0.0005), a statistically insignificant drop.

Typical developer workflow: compile a specification and call the resulting function.

**Figure 4.** Developer interface. Left: the compiler translates a natural-language specification into a neural program. Right: the interpreter loads this program and exposes it as a local function.

**Table.** Performance metrics across different noise axes and intensities.

PAW programs are portable, cacheable, and API‑accessible.

Related Work

We situate PAW among prior hypernetwork, PEFT, synthetic data, distillation, and neural‑program research.

Generate the weights of a target network from a compact embedding; early work for vision and language models.

Text‑conditioned hypernetwork that produces BART‑Large adapters from natural‑language task descriptions.

Generates prefix‑and‑adapter modules from natural‑language instructions to amortize per‑query encoding cost.

T5‑based hypermodel that emits soft prefixes or LoRA parameters from few‑shot examples.

Maps textual task descriptions to LoRA adapters distilled from pre‑trained adapters.

Produces task‑specific adapters from a single forward pass over contextual text.

Extends hypernetwork ideas to steer intermediate activations rather than only weights.

Compresses prompts into a few prefix tokens via attention‑mask training.

Distills demonstration examples into vectors via a two‑stage meta‑distillation process.

Scalable in‑context hypernetwork that maps natural‑language contexts to LoRA adapters.

Calibrated PEFT generation with structural coupling across layers.

Meta‑learns to internalise an entire document into a LoRA adapter for later querying.

Frames a LoRA module as a compiler that distils long context into compact portable buffer tokens.

Shares basis sets across tasks to enable efficient LoRA composition.

Insert small trainable bottlenecks into a frozen backbone to adapt it to new tasks.

Prepends learned key–value pairs to the attention mechanism, leaving the base model untouched.

Learns soft input embeddings that steer a frozen model without altering its parameters.

Learns low‑rank updates to the linear projections of attention and MLP layers.

Dynamically allocates rank budgets across layers for more efficient adaptation.

Decomposes pre‑trained weights into magnitude and direction, applying LoRA only to the direction component.

Combines 4‑bit quantization with LoRA to enable memory‑efficient fine‑tuning.

Shows that PEFT can outperform in‑context learning at lower deployment cost.

Uses a strong LLM to generate diverse instruction‑input‑output triples for fine‑tuning smaller models.

Automatically generates instruction data without human prompts.

Advocates synthetic textbook‑style data for pre‑training small models.

Self‑synthesises 4 M alignment instances from an aligned LLM without seed prompts.

Distils LLM labeling logic into Python programs that run on a standard interpreter.

Translates task inputs into SQL/Python programs with embedded LLM API calls.

Distils LLM reasoning into smaller fine‑tuned models using rationales as supervision.

Early work on representing programs inside neural networks.

Compiles formal code into network weights for execution.

Advocates replacing cloud LLM APIs with locally executable small models.

Techniques (GPTQ, AWQ, QLoRA) that reduce model size and latency via weight quantization.

Open‑source inference engines that run quantized models in browsers or on CPUs.

Scaling and Robustness

PAW turns NL specs into portable weight files, enabling local fuzzy programs without cloud latency.

This appendix revisits the PAW premise and then examines how compiler size, freezing, and noise affect exact‑match performance.

Table 12’s compiler‑scaling experiment is inconclusive: performance does not increase monotonically with model size, and unfreezing a 4 B compiler beats a frozen 32 B one.

**Table 12.** Inconclusive compiler-scaling table. EM on `test_clean` (Qwen3.5 0.8B interpreter, 0.6M training examples, epoch 1). Reported as exploratory data.

**Table 13.** Per-noise-type robustness. EM on `test_clean` across eight noise axes and three intensity levels (Qwen3 0.6B interpreter, epoch 2).

**Figure 6.** **Step 2: Interactively test the compiled program.** Users can provide test inputs and inspect the corresponding outputs, enabling rapid validation and refinement before download.

**Figure 7.** **Step 3: Execute the program locally via Python.** Once compiled, the program can be loaded and invoked through a simple Python API; subsequent execution requires no internet access.

**Figure 13.** Compiler prompt, minimal style. Used by the trained PAW compiler at inference.

Interface and Prompting Details

Appendix provides the web UI workflow, prompt templates, and image-task ablation details for PAW.

The appendix documents the hosted web interface, the JSON prompt templates used for fuzzy specifications, and the ablation tables that evaluate PAW components on image tasks.

**Table 3.** Image-conditioned fuzzy functions. The PAW rows use the same Qwen3 0.6B and Qwen 3.5 0.8B interpreters as in Table 2 (only the compiler is swapped, from Qwen3-4B-Instruct to Qwen3-VL-4B).

**Figure 5.** Step 1: Compile a program from natural language. The user specifies a fuzzy function in natural language. Image inputs are also supported.

**Table 9.** Image-task component decomposition (prefix-tuning precursor). EM/accuracy on the six image tasks. Adding the discrete pseudo-program helps on classification-style tasks (Circuit, Chemical, TextVQA) but hurts on long-form structured generation (Im2LaTeX, Im2SMILES), where “Continuous KV-cache only” is the strongest variant.

Dataset and Architecture Details

Dataset composition, architecture variants, and training setup for the PAW system.

Table 9 compares three PAW variants on six image‑to‑text tasks. The discrete pseudo‑program improves short‑answer tasks but degrades long‑form generation.

When the output is a short phrase (Circuit, Chemical, Music, TextVQA) the discrete pseudo‑program adds 5–40 EM points; for long structured outputs (Im2SMILES, Im2LaTeX) it crowds the interpreter’s context, and removing it recovers 6–8 EM points.

**Figure 3.** FuzzyBench-10M task-family distribution. 29 incremental thematic versions are mapped to 7 high-level families. “Core text processing & NLP” is the largest family because the v1 base layer (2.5M examples; 277 base categories) covers parsing, classification, NER, coreference, and sentiment; the remaining 7.5M examples spread across the other six families.

FuzzyBench‑10M is a 10‑million‑example benchmark that groups ambiguous, “fuzzy” tasks into seven high‑level families.

**Figure 18.** **Prefix-tuning precursor architecture (Section 3.3).** (a) *Compile*. The user describes a fuzzy function (e.g., “extract the final answer”); the trained prefix compiler reads the description plus a handful of representative I/O examples and produces a per-example KV prefix — the “neural binary” that constitutes the compiled program. (b) *Interpret*. A small frozen interpreter loads the compiled KV prefix into its attention cache at every layer and processes user inputs locally as a callable function, in the manner of standard prefix-tuning [Li and Liang, 2021]. This is the prefix-tuning instantiation of the same compiler-interpreter abstraction depicted in Figure 2; only the mapping from compiler hidden states to per-example PEFT module differs (KV cache here, LoRA in Section 3.2). *Note*: our prefix-tuning precursor’s exact training-time input format follows the same $[s \mid p_{\text{discrete}} \mid \text{EOS} \mid \tau_{1:T}]$ structure as the LoRA instantiation.

Table 10 lists the version timeline for FuzzyBench‑10M, showing steady growth from 2.5 M to 4.25 M examples in the first eight versions.

Iterate over three epochs of the 10 M‑example dataset.

For each batch (size 16, accumulation 3) load the pre‑generated discrete pseudo‑program from disk.

Concatenate the pseudo‑program with 64 learned prefix tokens and feed the sequence to the frozen interpreter.

Compute the negative mean‑token log‑likelihood loss on the interpreter’s output.

Back‑propagate through the LoRA compiler CL and update its parameters with AdamW (learning rate $2 \times 10^{-5}$).

Repeat until all batches are processed; no policy‑gradient term is used.

Training ran on a single B300 GPU for early stages and scaled to eight H200 GPUs for later stages, completing three epochs in roughly 72 hours on the 0.6 B model.

Quantization Performance

Full quantization results and size reductions for the models evaluated in the paper.

This appendix presents the complete quantization tables referenced in the main text and the size‑reduction impact of the `Q4_0` adapter.

Replacing the fp32 LoRA adapter (162 MB) with the 23 MB `Q4_0` adapter yields the size reductions shown below.

**Table 15.** GPT-2 124M quantization sweep (36-example handcrafted set, fp32 38 MB LoRA adapter). Smaller benchmarks are used because GPT-2's accuracy ceiling makes 4096-example differences harder to isolate.

**Table 16.** Qwen3.5 0.8B (Mamba-attention hybrid) quantization sweep (36-example handcrafted set, `Q4_0` 16 MB LoRA adapter). `Q4_K_S` and below crash with `llama_decode` failed (code -3) due to the Mamba-hybrid architecture's incompatibility with aggressive quantization.

Qualitative Analysis

Qualitative patterns in quantized models and their systematic failure modes.

The quantization sweep for GPT‑2 124M (Table 15) shows that moving from fp16 ($bpw$ 16) to aggressive 2‑bit formats reduces the base model size from 252 MB to under 200 MB, but the number of correctly solved examples peaks at the $Q5\_K\_M$ and $IQ4\_NL$ settings (24/36) before dropping sharply at $IQ2\_M$ (16/36).

For the Qwen3.5 0.8B hybrid model (Table 16), fp16 ($bpw$ 16) occupies 1.5 GB, while $Q4\_K\_M$ shrinks the model to 505 MB with a modest speed increase; however, configurations $Q4\_K\_S$ and finer ($Q3\_K\_L$ and below) crash because the Mamba‑attention architecture cannot tolerate the aggressive weight compression.

Across both models we observe four systematic failure modes: (1) exclusivity violations and paraphrase detection errors, (2) off‑by‑small‑amount numeric conversion mistakes, (3) character‑level span start/end offsets, and (4) creative reformulations that alter the intended meaning.

GPT‑2 excels at pattern matching and classification when the answer space is limited, yet it cannot chain reasoning steps (e.g., sentiment timelines) nor maintain precise token‑level position information.

The 0.6 B model inherits GPT‑2’s pattern‑matching strength but exhibits more frequent span‑offset errors and fails on multi‑step JSON construction tasks, highlighting a trade‑off between model size and fine‑grained positional fidelity.

The raw per‑example scores (first ten values 0.6580 → 0.5874, second block 0.6575 → 0.6462) illustrate modest variance across the handcrafted set, confirming that quantization does not dramatically reshape the distribution of individual example accuracies.

**Figure 14.** Compiler prompt, examples style. Used by the off-the-shelf reference compiler (Qwen3-4B-Instruct-2507) to generate the rollouts used during training.

Case Studies

We present five end‑to‑end PAW case studies and discuss current limitations and broader impacts.

This appendix showcases five concrete PAW deployments and then reflects on the approach’s current constraints and societal implications.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers