Baidu presented ERNIE Evolve on November 9, 2026 as an automated research system experimenting on real code, with claimed use during ERNIE-5.5-Flash development. This does not establish unlimited autonomous self-improvement.
Research needs an external criterion
The architecture separates choosing a direction from executing an experiment. A fixed external Evaluator and a Reviewer assess results and departures from the task. These are developer-described mechanisms, not isolation audited by SIExpo.
The public repository provides result code and logs for NanoChat AutoResearch, NanoGPT Speedrun, and SOL-ExecBench. Specific tracks report 38.92 seconds across 18 measurements and 2,590 steps across eight. Those are different metrics. Reproduction instructions specify hardware conditions, while comparable SOL measurements require a fixed clock. Artifacts make inspection possible; their presence is not our experimental reproduction.
Why an agent should not rewrite its own examination
Our editorial interpretation is that automatic improvement searches require a stable acceptance criterion. If a system can reduce the checks or change the expected result, a higher reported success rate may simply describe an easier examination. This is a general evaluation risk, not misuse that we have established in ERNIE Evolve.
For example, an optimization of table processing should preserve correct values while reducing execution time. A candidate that skips difficult rows might run faster without solving the same problem. Comparing output against predetermined expectations makes that distinction visible. This is our illustrative scenario.
Measurement provenance matters too. Code, configuration, and execution logs should identify the same version. A number in a table does not establish that a subsequently published file is the one that ran. Publicly retained artifacts provide a starting point for checking that connection.
An evaluator should also distinguish a change too small to separate from measurement variation from a repeatable improvement. Otherwise, a long search can select a lucky run. That is a proposed question for a replication review, rather than an extra result that SIExpo attributes to the project.
Speed does not transfer without its conditions
Accelerators, clock settings, and budgets change what a reported number means. Training time should not be directly compared with optimization steps or a GPU-kernel score. Even an identical task name is insufficient when input data and rules differ.
For a local pilot, we would fix the task version, resource budget, and acceptable numerical error. Each change should then be checked under the same conditions and on separate examples that did not select the candidate. This is a suggested methodology, not a test we have performed.
The broader question is whether accumulated experience also helps with new tasks. Improving one selected implementation can be useful on its own, but it does not establish a general ability to make scientific discoveries. Predictions about future progress should remain separate from the available files and measurements.
The publication matters because it describes a concrete loop with inspectable artifacts. Readers should distinguish claimed results, potential reproducibility, and an actual independent rerun. SIExpo checked the availability and contents of the sources, rather than executing the expensive hardware experiments.
Sources: Baidu — ERNIE Evolve research presentation, Baidu — ERNIE Evolve code, logs and reproduction conditions. The cover is a conceptual AI illustration, not a documentary photograph.
Editorial verification: 2026-11-11T07:14:46.966546+00:00.
