reproducibility
An open lab shipped six models at once, and released the checkpoints and data recipes too News
IFM released K2 Horizon as six Apache 2.0 models spanning 375 billion down to 0.9 billion parameters that share architecture, vocabulary and training methodology, publishing intermediate checkpoints, data-construction recipes, training code and logs alongside the final weights.
Why temperature zero is not deterministic Lesson
Setting temperature to zero makes a model always pick its highest-scoring next token, but it does not make the model return the same answer twice, because batching, floating-point arithmetic, and expert routing change the scores themselves between runs.
Null baselines and multiple comparisons: why an untrained model can look like it learned Lesson
A null baseline is what your measurement reports when nothing happened, and it is almost never zero. Without measuring it, and without correcting for how many things you tested at once, an improvement that is pure noise will look exactly like a real result.
NeurIPS papers average six objective mistakes each, up from four News
A study of 2,500 machine learning papers using an automated checker found that the average number of objective mistakes in a NeurIPS paper rose from 3.8 in 2021 to 5.9 in 2025, a 55 percent increase over four years.
An AI replicated 105 ICML orals, and 34 mostly held up News
The research auditing group SAI reviewed all 168 oral papers from ICML 2026, ran full execution-grounded reproductions of 105 of them, and found that only 34 reproduced more than 40 percent of the claims it attempted to check.
An agent's memory records what you said; this compiler records what you did News
A single-author paper argues computer-use agents waste frontier inference re-deriving routines their user already performed, and presents a model-free compiler that turns raw screen capture into a context block 86 times smaller that an agent reads with 98.4 percent accuracy.
Three Days On, Nobody Has Publicly Compiled OpenAI's Ten Proofs News
OpenAI's repository of Lean proofs for ten mathematics results has 434 stars and 39 forks but exactly one commit, no pull requests, and no issues, and no third party has published a build log showing the proofs check.
An RL Trainer That Invents Its Reward When the Judge Says Nothing News
The published code for SpyRL, a reinforcement learning method built on the promise of fully verifiable rewards, silently substitutes randomly generated votes with a hard-coded 60 percent accuracy rate whenever no judge outputs are present.
An open project publishes the recipe for training capable AI agents News
OpenThoughts-Agent releases its full data-curation pipeline, dataset, and experiments -- showing that what an agent learns from matters more than raw size, and letting anyone reproduce it.
SAI ICML 2026 replication results Tool
A browsable record of automated replication attempts against all 168 oral papers from ICML 2026, showing which papers shipped runnable code and how many of each paper's claims actually reproduced. Useful before you build on a result you have only read the abstract of.
Claude Science Tool
An AI workbench that unifies literature search, notebooks, statistics, and cluster compute, and keeps a reproducible record behind every figure. Beta on Mac and Linux.