Why AI can't beat the market

Can artificial intelligence beat the stock market? It's the question behind every "AI trading bot" pitch and every hedge-fund headline. With enough data, fast enough models, and a clever enough multi-agent debate, surely a machine can find the edge a human can't.

We didn't want to argue about it. We wanted to test it — properly, in the open, and publish the answer whether it flattered us or not. So we built the system, pointed it at a decade of survivorship-free market data, and let the evidence decide.

The answer, after dozens of experiments, is unambiguous: no. Not with public data, not at retail cost, not against the index a normal investor already owns. This is the story of how we know.

The benchmark is the one that actually matters to a Japanese retail investor: a plain, cap-weighted index held tax-free inside NISA. Beating it has to clear a high bar — out-of-sample (no curve-fitting to the past), after realistic trading cost and tax, and across both the calm years and the rally years, not just the window that happens to look good.

Every strategy ran through the same harness, frozen, with the same rules in backtest as in paper trading. We even bought the full paid market-data archive — survivorship-free, ten years, including delisted names and the order flows professionals watch — so "we just didn't have the right data" could never be the excuse.

And we published everything. Each test appears on the live dashboard with its verdict — pass, fail, or inconclusive — and the methodology is open source. Nothing that failed is hidden.

Value. Quality. Momentum. Post-earnings drift. Supply-and-demand from margin balances. Share buybacks. Sector tilts. Liquidity events. Multi-factor blends. Leverage. Trend overlays. One by one, each was tested against the index — and one by one, each lost after cost, in both halves of the data.

A recurring pattern explained the failures. A screen would look brilliant over the full period, then split into a flat-or-negative result before 2023 and a big positive only after — meaning the "edge" was just the recent rally, not something repeatable. Another pattern: by the time a real change (a margin step-up, a buyback, an earnings inflection) was visible in the filings, the price had already moved. Chasing it bought the top.

Only two results survived, and both are risk management, not extra return: a trend rule that cuts crash drawdowns at roughly zero long-run cost, and an equity-plus-gold trend blend that diversifies. Neither beats the index on return, and neither even survives the NISA wrapper's constraints. The honest count: 15+ tested, 0 that beat the index on return.

Surely the famous flows hold a signal? We tested the two most-watched in Japan. Foreign-investor net buying correlates +0.40 with the same week's move — but about 0.00 with next week's. It's coincident, not predictive, and it's published with a six-day lag. Timing on it loses to simply holding.

The sector short-selling ratio, across 148,000 observations, showed no contrarian bounce — high-short sectors mildly underperformed. Margin-restriction flags were the one genuinely stable signal: names that spike ~35% into a restriction give back ~4% versus the market over the next quarter. But they're exactly the names you can't short, and the signal is negative — so it's an avoidance rule ("don't chase"), not an edge you can trade.

Buying the full professional dataset changed nothing. The conclusion held.

This isn't an indictment of AI. It's a demonstration of how markets work. The price is set by the most informed, fastest participant in the room. If a fact is public — in a filing, a dataset anyone can buy, a strategy in a book — it is already in the price by the time you act on it.

An edge, by definition, is something scarce, private, and decaying. The moment it's widely known, it's arbitraged away. We were searching public data with retail tools, which guaranteed from the start that nothing durable would be there to find. AI doesn't change this — it makes processing public information cheaper for everyone, which competes any fleeting edge away faster.

Most people who look like winners aren't beating the market — they're holding it during a good decade, plus survivorship (you hear about the winners, not the 90% who quietly lost). The index's own long-run return makes patient holders wealthy. That's winning; it just isn't "beating."

The genuine, durable edges come from places a retail investor can't reach: speed (high-frequency market-making measured in microseconds), scale and access (private deals, allocations, crisis-time terms), legal private information (alt-data bought and processed at scale), or illegal private information (insider trading — which a notable share of "genius" track records turned out to be). None of these is "better analysis," and none is available to you.

The one edge that is available to you isn't an edge over the market at all — it's an edge over your own behaviour.

Since prediction doesn't beat the index, the levers that remain are the ones you fully control — and the data backs each one. Front-load your contributions: deploying a year's quota early beat dollar-cost-averaging in 67% of rolling years, and waiting for a dip lost. Don't panic-sell: even down 20–30%, the market's next twelve months averaged about +8% and were positive two times out of three — selling converts a paper loss into a permanent one. Minimise cost, turnover and tax: it was the structural opponent that beat every active strategy we tested.

It's unglamorous. It's also the only thing here that reliably builds wealth, and most active participants underperform it. The job an AI can actually do for your money is not prediction — it's keeping you disciplined: automating the boring right thing and making the exciting wrong thing harder.

Don't take our word for it. Every test, every verdict, and the live paper-trading agent are on the dashboard, and the whole harness is open source so you can reproduce the result. That's the point: a machine that rejects bad strategies honestly is more useful than one that promises returns it can't deliver.