Evaluating Generative AI for Stock Market Forecasting: Beyond the Hype

19 large language models, one narrow event, the same directional call. That's the headline from AIMultiple's recent benchmark, and it's the kind of test I actually care about — not "can GPT explain a chart" theater, but a controlled forecasting task with measurable outcomes.

The setup: 132 stocks, an unexpected event (a family manager dies), one output to predict — the 3-day cumulative abnormal return. Every model converged on the same two-force framework. Death of an entrenched underperformer unlocks the control market. Death of a value-adding owner-manager destroys hard-to-replace skill. Same thesis, 19 times.

Convergence is not edge

When 19 models produce the same directional framework on the same input, you don't have 19 independent signals. You have one signal with 19 thresholds dressed up as diversity.

The benchmark itself concedes the point. What separates the models isn't the thesis — it's where they set cutoffs, which variable breaks ties, and how readily each one defaults to neutral. The framework is shared. The cutoffs aren't.

That's a noise argument, not an alpha argument. Anyone who has backtested factor screens long enough knows the rule: when entry logic converges, the only thing left to optimize is the threshold, and threshold optimization on 132 observations is overfitting with extra steps.

The DeepSeek divergence is the lone exception worth flagging. It read family ownership in the opposite direction from the rest of the field — high concentration as a signal of smooth transition rather than succession risk. The benchmark notes the split but doesn't resolve it. I'd want to see that isolated and re-tested before I traded on it.

The one model worth studying

One model in the set is described as the most accurate, and it's the only one that abandoned absolute thresholds entirely. It builds two continuous z-score indices and issues a directional label only when one index sits in the top 30% of the sample and clears the other by at least 0.5 z-units.

That margin requirement is the structural feature I'd actually code into production. It's a relative-rank filter with a confirmation gap — the exact mechanism every mean-reversion system eventually needs to suppress whipsaw.

The benchmark also shows the same architecture pushed to its logical extreme with a top 5% gate instead of top 30%. That's the over-correction — probably under-trades until transaction costs eat whatever edge survived the threshold filter. Same architecture, opposite failure mode.

The Robinhood question

The same week, MEXC is publishing on why Robinhood stock is up — citing agentic trading, banking growth, and an unnamed product teaser. That's the marketing surface of the exact trend AIMultiple is stress-testing underneath.

Agentic trading will get sold to retail as a product. The benchmark tells you what to expect from the underlying signal: high directional agreement, no agreement on magnitude, and a structural preference for the neutral bucket when evidence is thin.

If you're a systematic trader, the rule is simple. Don't buy the "AI picks stocks" pitch. Buy the model that publishes its thresholds, its weighting order, and its arbitration rules — and backtest those on out-of-sample data yourself. The rest is threshold fitting dressed up as machine intelligence.