I ran 12+ trading bots for a year. Here's what survived, what died, and what I learned.

Most "I built a trading bot" content on the internet is success-bias narrative. Someone deploys a strategy, it works for a month, they write the post, the post goes viral, the bot quietly stops working three months later and the postmortem never gets published. The lesson — most retail trading bots fail in specific, repeatable ways — never enters the public corpus.

We've been running a fleet for the last twelve months. More than a dozen bots across two venues: OKX for crypto perpetuals (demo until any single strategy survives 60+ live trades net-of-fee), Interactive Brokers for tradfi equities (paper until 30+ trades + a 3-6 month soak). At peak, ~$300k of notional was deployed across the demo + paper books. Every trade audit-logged. Every PnL reconciled against the broker's own books — never against the bot's internal database.

Most of what we ran failed. The patterns of how they failed are what this article is about.

The fleet structure

Two venues, two execution surfaces:

• OKX side: BTC, ETH, SOL, and a curated list of perp pairs traded by btc-quantum, eth-quantum, regime-rotator, and the tuned-perps family. Demo-account-only until a strategy passes the live-audit gate. Live OKX positions are user capital, never customer capital — see /bots/public for the current live readout.
• IBKR side: 32+ tradfi names across the AI-supercycle universe (semi, memory, power, networking), originally split across single-name bots (nvda-ibkr, aapl-ibkr, amd-fast-rsi-bounce-ibkr, etc.), since consolidated into tradfi-ibkr-bot with the FIB_BASKET universe. Paper account; live ramp deferred until the consolidated book has the 30+ trades + soak required by our own paper-to-live policy.

Twelve months in, we've shipped enough postmortems to identify six distinct failure classes and exactly two survivors. The failure classes generalize. If you're running or planning to run a retail algo book, you're likely going to hit at least three of them — better to recognize the shape now than from inside.

Failure class 1: crypto-perp alpha exhaustion

The most painful lesson. We had four entire strategy classes — momentum, range-mean-reversion, breakout, funding-arb — running against the OKX perp universe over Q4 2025 / Q1 2026. Each had passed its walk-forward audit individually. Combined they generated 213 demo trades.

The gross PnL: +$122. The fees: $131. The net: −$9.

Four strategies, two hundred trades, a quarter of operator attention, and the net result was within rounding error of zero — and on the wrong side of zero. The alpha was real-ish at the gross level. The execution cost ate it entirely.

The lesson is brutal in its specificity: at retail-perp size in 2026, any strategy whose backtest doesn't explicitly model exchange fees IS lying to you. Maker rebates don't save you if your strategy is a taker. Fee tiers don't save you if you're not at $10M+ monthly volume. The "fees are negligible" assumption that worked in 2018-2022 crypto-perp markets doesn't hold any more — the spreads have tightened and the alpha that lived inside the wider spreads is gone.

What we did about it: trimmed regime-rotator to a 4-pair universe (SOL/INJ/LINK/AVAX), blacklisted SUI/YFI/IOTA and 8 other liquidity-marginal pairs, and made all analytics NET-of-fee from then on. The decision gate for re-expanding the universe is now 60 net-positive live trades — not gross.

Failure class 2: "passes walk-forward" ≠ "alpha"

We built two genuinely sophisticated single-name bots in early 2026:

• nvda-ibkr — a ~1,100-line NVDA swing bot with multi-strategy ensembling and a risk overlay. Walk-forward result: 5/5 PASS.
• aapl-ibkr — a more compact AAPL bot. Walk-forward result: 4/4 PASS, Profit Factor 2.05.

Both bots looked, by every standard metric the systematic-trading literature recommends, like winners. We deployed them on paper, ran them for two months, and then compared their cumulative return to the dumbest possible benchmark: holding the underlying ticker, same dollar size, no rebalancing, no overlay.

• nvda-ibkr captured 3% of buy-and-hold NVDA's return over the period.
• aapl-ibkr captured 2.2% of buy-and-hold AAPL's return.

The bots weren't losing money. They were risk-managed buy-and-hold with massive opportunity cost. The walk-forward "PASS" wasn't measuring alpha; it was measuring "doesn't blow up." Those are very different properties.

The deeper lesson: walk-forward validation tells you a strategy survives history. It does not tell you the strategy is worth running. A strategy whose alpha after costs is less than the underlying's natural drift is worse than no strategy. The benchmark you compare against matters enormously. We've since added "captures ≥30% of B&H notional return" as a hard gate in the audit pipeline; without it, "passing" is a vibe, not a fact.

Failure class 3: PnL fiction

In May 2026 we ran a reconciliation against OKX's own fill log on btc-quantum. The bot's internal SQLite vault reported cumulative PnL of +$1,309. OKX's broker truth: −$373.

Source of the gap: the bot's analytics.record_trade() was recording gross fill prices without subtracting the fee that OKX deducted from the same trade. Compound this over a hundred trades and you build a fictional $1,700 PnL drift. The bot's dashboard, the live status JSON, and the user-facing telemetry were all reporting the fictional number.

The architectural lesson: every bot's internal PnL view is a cache, not truth. The single source of truth is the broker's own fill log + position statement. Internal databases drift because bots are written by humans who forget to subtract fees, miss partial fills, or get confused when an order gets cancelled and re-submitted. The reconciliation discipline is non-negotiable — and most of the time, the first reconciliation reveals at least one bug.

We now run a nightly diff: bot-DB-PnL versus broker-fill-PnL. Any divergence > 0.5% pages an operator before the next morning's open.

Failure class 4: silent death

In May 2026 we had two separate "bot is dead but nobody noticed" incidents:

• eth-quantum hung for 18 hours on a single stale OKX TCP read. The socket never received a TCP RST, the application-layer read() never returned, the bot's main loop was blocked but the container was Up. Fix: socket.setdefaulttimeout(30) at process start, plus an application-layer heartbeat the watchdog reads.
• btc-quantum had a duplicate-block guard that was supposed to prevent the bot from re-submitting an order on the same bar twice. The guard was a no-op (a bug in the comparison) and the bot was fail-deadly on broker-side rejects, so duplicate submissions silently went through. PR #71 fixed the guard.

The same week, the watchdog false-positively alerted that eth-quantum had been dead for 7.7 hours when the bot was actually alive and trading — the watchdog was reading container uptime, not application heartbeat.

Net lesson: container Up is not health. Application heartbeat is health. Build an explicit heartbeat — a value the bot writes to a known file every N seconds — and make the watchdog read THAT, not Docker's container status. Build a socket timeout at the network layer of every external call. The combination catches both kinds of silent death.

Failure class 5: deconfliction

Multi-bot operations break in subtle ways when bots share resources.

• Three OKX bots fought over the same OKX account in April 2026 (btc-quantum, an experimental eth-grid, and tsla-bot). Without per-pair ownership rules, two bots would simultaneously decide "long ETH-PERP" with different sizes and one would adopt the other's position thinking it was its own.
• Two IBKR bots (amd-microswing-ibkr and amd-fast-rsi-bounce-ibkr) shared the paper account, both watched AMD, and the same orphan-adoption pattern played out — one bot saw an open AMD position from the other and "adopted" it into its own state, then started managing exit logic against a position it didn't open.

Solution: 1-bot-per-pair, full stop. The fleet registry assigns explicit ownership; any pair another bot wants is locked. When we consolidated five single-name IBKR bots into tradfi-ibkr-bot ($10k cap), all the deconfliction logic became internal to one process instead of negotiated between containers. Less elegant, more reliable.

Failure class 6: data-starved deployment

nvda-bot was a perp-strategy bot we deployed on OKX in early 2026. The strategy required at least 202 daily bars of historical context. OKX's NVDA-perp listing has 59 daily bars total.

The bot was making decisions on insufficient data — every signal it generated was essentially noise within its strategy's confidence interval. Killed it. The note in our project memory says "restart ~December 2026" — when OKX will have enough NVDA-perp history for the strategy's lookback window.

Lesson: before deploying a strategy, verify the data window the strategy needs against the data window the venue actually has. The mistake here was trivial to make — we'd designed the strategy on synthetic data and assumed the venue would catch up. It didn't.

What survived

Out of the dozen+ bots, two strategies have genuinely beaten their benchmarks across consecutive walk-forward cycles + soak periods:

1. reg_channel_mr + ema_crossover (tradfi mean-reversion on equity baskets). Both achieved 15 ROBUST verdicts each in the May 2026 sweep leaderboard, beating fib_mean_reversion on 31 of 32 names tested. These are now the routed strategies on the IBKR consolidated bot.
2. fib_mean_reversion with SMA50 filter, on the 5-name HIGH_VOL cohort only. Recalibrated May 2026 (PR #278): basket PF 1.53→1.76, Sharpe 1.08→1.42, +23.7% three-year cumulative, walk-forward 4/5 PASS. Coexists with the broader 1H per-ticker scope routing change from the May 19 sweep — orthogonal.

Both survivors are tradfi mean-reversion. Both run on daily timeframes. Both clear B&H after fees. That's it. Two survivors out of double-digit attempts.

Notably absent from the survivor list: any pure crypto-perp directional strategy. Per failure class 1, that alpha is currently exhausted at retail size. We keep crypto bots running on the demo gate not because we expect them to find new alpha tomorrow, but because the day the alpha returns (a market structure shift, a new venue, a new product), we want the infrastructure already in place to detect it.

Cross-cutting meta-patterns

Four lessons that generalize across all six failure classes:

1. Fees compound faster than alpha. Every strategy must be modeled net-of-fee from line one. The gap between gross and net is the most common single cause of a "winning" strategy being a net loser. Make your audit pipeline reject any backtest that doesn't have explicit fee modeling.

2. Walk-forward says "doesn't blow up." It does not say "beats benchmark." Add a hard benchmark-comparison gate to your validation. "Captures ≥30% of B&H notional return" is our current bar; before we had it, "passing strategies" were silently underperforming the underlying.

3. Internal database is a cache. Broker fill log is truth. Reconcile nightly. Page on divergence. Most live bots have at least one PnL-fiction bug; the first reconciliation usually finds it.

4. Container Up is not health. Application heartbeat is health. Build an explicit heartbeat from inside the bot process. Make the watchdog read THAT, not Docker. Add socket timeouts to every external API call so blocked reads can't hang the main loop.

What this all cost

Twelve months. Roughly five months of full operator attention (it's a side project on top of the QA platform itself). Dozens of postmortems. Multiple six-figure-notional paper losses, denominated in opportunity cost more than dollars (we're using paper + demo accounts for exactly this reason). Probably twenty rebuilds of strategies that looked promising in backtest and died in soak.

Most retail traders skip this cost by stopping after their first profitable backtest, deploying live, and discovering one of the six failure classes above with real money. The cost of skipping the postmortem discipline is the cost of finding out in live trade. We chose to find out in paper.

If you want to see the current live + paper results in real time, they're public: /bots/public. Equity curves update daily; the PnL columns are reconciled to broker truth.

If this resonates

We deploy our IBKR paper bots via the runbook documented in a separate article on running IBKR Trader Workstation for systematic bots. For the broker choice itself, see why IBKR for the AI supercycle trade. The crypto side runs against OKX for the reasons in that comparison piece.

To run your own paper fleet from scratch:

1. For tradfi: open an Interactive Brokers account — paper accounts are free, the API works from day one, and IBKR's program currently gives the new account up to $1,000 of IBKR stock (terms apply). Full reasoning + the friction we hit on our broker page.
2. For crypto perps: open an OKX account — the demo environment mirrors live, the API is documented, the fee tier ramps. Full reasoning on our broker page.
3. Build the audit discipline before the first strategy. Net-of-fee from day one. Benchmark comparison gate. Nightly reconciliation. Heartbeat-based watchdog. The four meta-patterns above are cheap to add upfront and expensive to retrofit after the first PnL-fiction incident.

The dozen+ bots will become two survivors. That's the rate. Plan capacity accordingly — the value isn't in deploying ten strategies in parallel, it's in killing the eight that don't work fast and feeding the two that do.