News · 2026-09-04

Compile by Training turns a language specification into a reusable local neural function

Compile by Training is a new system that converts a natural-language task description into a reusable local neural function by having teacher models generate examples and then training a compact adapter. The approach matters because it shifts repeated AI work from calling a general remote model every time to building a small, testable artifact once. Its authors report 83.6% semantic accuracy on a difficult benchmark subset where a fast compiler achieved 22.4% mean LEM, but the technique trades speed and certainty for specialization.

Key facts

• The paper, “Compile by Training: Turning Natural-Language Specifications into Local Neural Functions”, is listed as an EMNLP 2026 System Demonstrations paper.
• Its public configuration uses a quantized Qwen3-0.6B interpreter, mixed teachers, and a rank-64 LoRA adapter with alpha 16.
• On the cited FuzzyBench-Hard subset, it reports 83.6% semantic accuracy against 22.4% mean LEM for the fast PAW compiler, with 50.9 seconds versus 3.5 seconds of compilation time.
• Primary source: the paper's arXiv abstract and HTML version.

The core idea is simple enough to describe without overselling it. Suppose a team repeatedly asks a powerful model to convert product descriptions into a tightly constrained internal format. Prompting a frontier model on every request is flexible, but it costs money, sends data out, and may vary from run to run. Compile by Training instead treats the natural-language instruction as a source program. At compilation time, teacher models create examples of the intended behavior; a compact interpreter receives a task-specific adapter; later inputs run locally through the result.

The analogy is a pocket calculator. Calling a large model for each request is like asking a skilled mathematician to solve every arithmetic problem from scratch. Training a local function is like making a calculator for a single kind of calculation. The calculator cannot write a proof or answer unrelated questions, but it can perform its assigned operation cheaply and quickly once built. The researchers call the output “local neural functions,” not a replacement for all software with a language model.

The system has practical engineering around the training loop. The paper says it overlaps teacher synthesis and training, holds persistent job records, and reuses cached teacher outputs across jobs. That turns compilation into a background build rather than a request that blocks a user. The adapter is a LoRA component: a small set of trainable low-rank matrices added to a frozen base interpreter, allowing task behavior to be learned without retraining every parameter. This is the same family of efficiency ideas behind distillation, but the product framing is different: the output is a runnable specialized tool.

The central experimental result compares correctness with build time. On a FuzzyBench-Hard subset where the PAW fast compiler returned no exact matches, the authors report 83.6% semantic accuracy for Compile by Training versus 22.4% mean LEM for the fast compiler. The reported compile time rises from 3.5 seconds to 50.9 seconds. That is not an apples-to-apples claim that neural training wins every compilation task. It is evidence that spending under a minute to build a better specialized approximation may be worthwhile when the function will be used many times.

The authors also demonstrate deployed examples: a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English–Claudish translator. The avatar succeeded on 43 of 44 hand-authored validation instructions. The paper says the translation deployment handled 100,747 successful requests between August 22 and September 2, and that both translation programs can be downloaded and run locally. Those are promising product demonstrations, not a randomized user study.

The strongest reception point is visible in the paper's own limitations. Teacher-generated synthetic supervision can carry teacher errors into the compiled function. Long-tail inputs can fail in ways the generated examples never covered; requirements can drift while an old adapter keeps executing the original understanding; and a function that looks right on a benchmark may be wrong in a business edge case. The authors explicitly recommend validation or deterministic control paths for correctness-critical use. That is the right caveat.

A useful way to apply the method is to ask three questions. First, is the task narrow enough that a fixed behavior is valuable? Second, can you build an automated test set or a human review loop that detects bad outputs? Third, will you invoke the result often enough to repay the compilation cost? When the answer is yes, a compact adapter can offer privacy, latency, and predictable unit economics. When the task is broad or frequently changing, a general model may remain the better tool.

This is also a reminder that “local” does not automatically mean “correct.” A local neural function is still probabilistic. For a format conversion, categorization, or low-risk creative transformation, that may be acceptable. For a money movement, a safety control, or a legal claim, constrained decoding, deterministic validation, and human approval should surround it.

The honest caveat is reproducibility beyond the authors' demonstrations. The paper's result is strong for its selected hard subset and concrete deployment prototypes, but it does not yet establish that training is the best compiler for arbitrary natural-language specifications. Its contribution is a useful systems pattern: where a language task repeats and can be tested, compile the behavior into a small specialized artifact instead of paying a generalist to rediscover it on every call.

Primary source, verified: read the paper → (arXiv 2609.04199)

Ground Truth. (2026, September 4). Compile by Training turns a language specification into a reusable local neural function. Ground Truth. https://groundtruth.day/news/compile-by-training-turns-language-specifications-into-local-neural-functions.html

@misc{groundtruth:compile-by-training-turns-language-specifications-into-local-neural-functions,
  title  = {Compile by Training turns a language specification into a reusable local neural function},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {sep},
  url    = {https://groundtruth.day/news/compile-by-training-turns-language-specifications-into-local-neural-functions.html}
}

Topics: research · program-synthesis · fine-tuning · lora · local-inference · agents · emnlp