The AI Forecasting Agenda
AIThis post was created with the assistance of artificial intelligence (AI).

TL;DR

Specialized AI forecasting systems reportedly now match or exceed top human forecasters on traditional forecasting questions, while general-purpose models are described as about six months behind. Whether that performance carries over to conditional and causal questions remains unsettled; a new Forecasting Research Institute benchmark is expected in the coming months.

Specialized AI forecasting systems are now matching or exceeding top human performance on traditional forecasting questions, according to the source material, but their ability to answer questions about policy effects remains unproven. The next test is whether these systems can make reliable conditional and causal forecasts, which could help decision-makers compare outcomes under different choices.

Age 18–24?Offer from Amazon

Prime made for students and young adults

  • Fast, free delivery for dorm and study essentials
  • Prime Video and Amazon Music included
  • Member-only deals
Try Prime for Young Adults Free trial for eligible 18–24 year olds
As an affiliate, we earn on qualifying purchases.

The source describes specialized systems built with forecasting tools, or “harnesses,” from companies including FutureSearch, Mantic and Preseen. It also says general-purpose models such as ChatGPT and Claude are about six months behind the specialized systems. The material predicts they may reach top-human performance by year-end, but gives no benchmark results or evaluation details to independently verify that forecast.

Traditional questions ask whether an event will happen, such as which party will win an election. Conditional questions ask what might happen if a particular event occurs—for example, how an election result could affect the likelihood of a market downturn. The source says forecasting companies already offer conditional questions, but says strong performance on ordinary questions does not establish accuracy on these more complex tasks.

The source recounts that a Forecasting Research Institute (FRI) test about a year earlier found AI systems of that period did significantly worse on conditional questions. It says experts consulted for the account could not say whether current AI systems outperform an average informed person on conditional forecasts, and still judged them below top superforecasters. FRI is working on a new benchmark, expected in a few months, the source says.

At a glance
reportWhen: Developing; a Forecasting Research Inst…
The developmentAI forecasters’ reported gains on standard forecasting benchmarks are shifting attention to whether they can reliably answer conditional and causal questions.
AI Forecasting · November 2026 Briefing

The AI Forecasting Agenda From event odds to cause and effect

Specialized AI systems reportedly match or exceed top human forecasters on traditional questions. Whether they can reliably answer conditional and causal questions remains unresolved.

Where performance stands

The reported gains apply to traditional forecasting, not yet to questions about what a policy or event causes.

Traditional forecasts
  • Specialized systems reportedly match or exceed top human performance.
  • Examples ask whether an event, such as an election result, will happen.
Conditional and causal
  • Current performance remains unsettled.
  • An outcome after an election does not establish that the winner’s policies caused it.

What the questions ask

More complex forecasts connect an outcome to a specified event or choice.

Event odds

Will it happen?

A traditional forecast might ask which party will win an election.

Conditional

What if it happens?

A conditional forecast asks how an election result might relate to the likelihood of a market downturn.

Causal

What did it cause?

A causal claim must distinguish policy effects from background conditions that influence both events.

Claims still need verification

The source reports progress but does not provide the details needed to independently assess the performance claims.

Claim or questionWhat the account provides
Specialized systems match top humansNo scores, dates, question counts or comparison methods
General-purpose models trail by about six monthsAn estimate, not a confirmed benchmark result
Conditional forecasting has improvedCurrent performance remains unclear; an earlier FRI test found weaker results
Causal analysis can identify policy effectsA proposed direction, with no results establishing reliability

The next test

The Forecasting Research Institute is working on a benchmark expected in the coming months; the source gives no release date or test details.

1
Separate task types

Check traditional event forecasts separately from conditional questions and causal claims.

2
Inspect the evaluation

Look for question selection, scoring, comparison groups and the assumptions behind each forecast.

3
Assess decision value

Check whether answers account for relevant variables and communicate uncertainty clearly.

Key pointReported gains support a limited conclusion: AI has advanced on traditional forecasts, while its reliability for evaluating policy effects remains unproven.
The takeaway
For businesses and readers, treat human-level claims as limited to the question types actually tested—and wait for transparent evidence on conditional and causal forecasts.

Testing Forecasts for Policy Decisions

The distinction matters because many decisions turn on what a choice causes, not simply what events are likely to occur together. A forecast about the economy after an election, for instance, may reflect conditions that helped determine the election result rather than the effect of the winning party’s policies. Treating that association as a policy effect could mislead users.

If AI systems can handle conditional questions reliably, they could make forecasts available more quickly and at lower marginal cost than commissioning a team of human forecasters, according to the source’s comparison. But ease of generating an answer is not evidence that the answer is accurate. For policy use, performance must be tested against appropriate benchmarks, and uncertainty about assumptions and relevant variables must be made clear.

The claims also point to a shift in the field’s main question: from whether AI can forecast standard, clearly resolvable events to whether it can support decisions where outcomes depend on interacting factors. Until those capabilities are measured, claims of human-level performance should remain limited to the types of questions actually evaluated.

De'Longhi Magnifica Start Automatic Espresso Machine with Milk Frother, Silver

De'Longhi Magnifica Start Automatic Espresso Machine with Milk Frother, Silver

Bose QuietComfort Ultra Earbuds (2nd Gen), Wireless Noise-Cancelling Earbuds, Black

Bose QuietComfort Ultra Earbuds (2nd Gen), Wireless Noise-Cancelling Earbuds, Black

Ninja Mega Kitchen System Blender and Food Processor, 1500W, 72 oz Pitcher, 8-Cup Bowl, Black (BL770)

Ninja Mega Kitchen System Blender and Food Processor, 1500W, 72 oz Pitcher, 8-Cup Bowl, Black (BL770)

As an affiliate, we earn on qualifying purchases.

From Event Odds to Cause and Effect

The source compares AI forecasting with earlier options: hiring superforecasters could cost thousands of dollars and take weeks, while prediction markets and platforms such as Metaculus cover a narrower set of questions. It also argues that market questions can be affected by resolution rules and incentives. These are the source’s characterizations; it does not provide cost data or a systematic comparison.

Conditional forecasts add a further challenge. Asking what the economy will look like if a party wins does not, by itself, show what that party’s victory caused. Background conditions may influence both the election and the later economy. The source says FRI’s earlier test found a gap in AI performance on conditional questions, but provides no details on the test design, sample size or models involved.

The source suggests that causal analysis may require considering combinations of relevant conditions, rather than merely asking an AI to answer a causal question directly. It says at least one company is trying an approach along these lines, but does not identify the company or describe its method. This remains a proposed direction, not evidence that AI systems can yet establish causal effects reliably.

How Far Current Results Extend

The source does not name the specific ForecastBench or Metaculus Cup results behind its claim that AI matches or beats top human performance. It gives no scores, dates, question counts, comparison methods or model versions. The claim about general-purpose models being six months behind—and reaching parity by year-end—is presented as an estimate, not a confirmed benchmark outcome.

Current performance on conditional questions is also unresolved. The account reports an earlier FRI finding of weaker AI performance, but does not describe the newer benchmark’s scope or methodology. It remains unclear how well current systems perform across different kinds of conditional questions, whether results hold outside selected benchmarks, and how reliable their answers are when important variables are omitted.

For causal forecasts, the source outlines a possible method based on combining conditional forecasts across relevant variables. It does not establish that this method identifies causal effects in practice. The company said to be testing a related approach is unnamed, and no results are provided.

The FRI Benchmark and Causal Tests

The next concrete milestone identified in the source is the FRI benchmarking experiment, expected in the coming months. Its findings could clarify whether newer AI systems have improved on the conditional questions that challenged earlier models. The source does not provide a release date or say which systems the test will evaluate.

Further evaluation will need to distinguish performance on standard event forecasts from performance on conditional questions and causal claims. Readers should look for details about question selection, scoring, comparison groups and the assumptions used to define each forecast. Until such evidence is available, the reported gains support a limited conclusion: AI forecasting has advanced on traditional questions, while its usefulness for evaluating policy effects remains under investigation.

Key Questions

What has AI forecasting achieved so far?

The source says specialized AI forecasters now match or exceed top humans on traditional forecasting questions. It does not include the benchmark scores or evaluation details needed to assess the claim independently.

What is a conditional forecast?

A conditional forecast asks what may happen if a specified event occurs, such as how an election result might relate to a later economic outcome. That relationship alone does not show that the event caused the outcome.

Can AI already make reliable causal forecasts?

The source does not establish that it can. It describes a possible approach using combinations of conditional forecasts, but provides no results demonstrating that the method reliably identifies cause and effect.

When will researchers have more evidence?

The source says the Forecasting Research Institute is working on a benchmark expected in a few months. It does not give a specific publication date or describe the planned test in detail.

Source: astralcodexten.com

You May Also Like

Spatial Focus Room: Make Distraction Impossible

A new deep-work app for Apple Vision Pro, Spatial Focus Room, removes distractions by creating immersive environments, transforming focus practices.

The labor share. Is value really moving from labor to capital? The data isn’t on anyone’s side yet.

Recent data shows mixed signals on whether AI is shifting value from labor to capital, with aggregate stability contrasted by early displacement signs.

OpenSSH SSHole: What To Know About The Overlong-Password Bug

CVE-2026-123158 can bypass SSH password checks on certain musl-based Linux systems. See the affected conditions, fix and remaining questions.

AI Showdown: Fable, Opus 5.5, Astra, Sol, And Luna – Which One Comes Out On Top?

A detailed comparison of leading AI models reveals Opus 5.5 leads in performance, Astra offers cost advantages, while Sol and Luna enable large-scale deployment.