AutoDS uses AI to search for scientific hypotheses
Ai2 introduced AutoDS, an open-source prototype that generates and tests scientific hypotheses without receiving an initial question from an expert. Across 21 datasets, it found between 5% and 29% more surprising results than its competitors, although its findings still require human validation and peer review.
Ai2 introduced AutoDS, an open-source prototype that uses artificial intelligence to explore scientific questions without a person first telling it what to investigate. The system generates hypotheses, runs statistical experiments on available data, and uses the results to decide what to explore next.
Most AI tools for science start with a specific question. For example, a researcher might ask them to analyze whether one variable influences another. AutoDS tries to cover the step before that: deciding which questions are worth investigating.
How AutoDS works
The system looks for results that seem unexpected and uses them as a starting point for new hypotheses. To measure that surprise, it uses a technique called Bayesian surprise, which calculates how much confidence in a hypothesis changes before and after new evidence is observed.
In simple terms, if the data substantially contradicts what the system expected, that result receives more attention. AutoDS can then investigate that lead, generate another hypothesis, and repeat the process continuously.
To choose between different lines of research, AutoDS combines this measure of surprise with Monte Carlo Tree Search (MCTS), a search method also used in systems that play complex games. It also incorporates a technique called progressive widening to explore a potentially unlimited number of hypotheses without trying to analyze them all at once.
According to Ai2, using MCTS was important: the system found up to 29% more surprising hypotheses than the other search mechanisms evaluated.
What it found in testing
The researchers tested AutoDS on 21 real-world datasets from areas including economics, biology, and finance. In these tests, the system outperformed its competitors by between 5% and 29% when finding results considered surprising by a language model.
Afterward, more than 500 hypotheses were reviewed by human evaluators with master's or doctoral degrees in STEM fields. 67% of AutoDS discoveries were also considered surprising by those specialists.
That does not mean the system demonstrated new scientific theories or that its results are automatically correct. An unexpected finding is not enough: researchers still need to check that it was not caused by chance, errors in the data, or a mistaken interpretation.
AutoDS can explore without direct supervision, but its results still require academic scrutiny and peer review.
What changes for research
AutoDS's main value lies in automating part of the work that usually takes a lot of time: reviewing results, detecting non-obvious patterns, and deciding which experiment to run next. A researcher could use it to explore large databases and receive a list of leads that must then be validated.
The system does not replace scientific judgment. It cannot determine on its own whether a hypothesis is socially relevant, whether an experiment is properly designed, or whether a correlation has a reasonable explanation. Those decisions still require specialists.
Ai2 describes AutoDS as a prototype and acknowledges that it is not always fast. Even so, the project points to a different way of using AI in science: not only as a tool that answers questions, but as a system that helps find questions no one had formulated yet.
The important question now is whether AutoDS's surprising hypotheses are also reproducible, useful, and scientifically sound. The ability to generate many leads could accelerate research, but only rigorous validation can distinguish a real discovery from an interesting coincidence.