IBKR Trader Workstation for systematic bots: the operational map nobody publishes

Interactive Brokers has documentation. It covers the API call signatures, the symbol conventions, the order-type matrix, the limit/market/bracket semantics. What the docs don't tell you is what actually breaks when you run a real bot against the production paper account at 3am Paris time on a Tuesday, then re-deploy a code change at 9am the next morning while the European session is open.

We learned the gap the hard way. This article is the operational map we wish had existed before we started — written from twelve months of shipping bots against IBKR, eight of those months running them 24/5 against the paper account, and a 38-hour silent outage that taught us why the container being Up isn't the same thing as the bot working.

If you're evaluating IBKR as a broker for systematic execution, or you're already operating one and hitting friction you don't have words for, this is for you.

The setup architecture (what runs where)

A working IBKR systematic-trading stack has three moving pieces:

1. IB Gateway (or TWS) — the IBKR-side software that holds the network session with their servers. Gateway is the headless variant; TWS is the desktop UI version. For bots, Gateway wins — it doesn't open a window, doesn't drift on UI version updates, and the gnzsnz/ib-gateway Docker image bundles all the bootstrap pieces (IBC for non-interactive login, Xvfb for the framebuffer, the daily-forced-restart loop IBKR mandates).
2. Your bot process — the Python (or whatever) process that opens a TCP connection to the gateway, places orders, listens for fills.
3. State — the JSON / SQLite files that track positions, equity, risk metrics, trade history. These live OUTSIDE both processes, on the host filesystem, bind-mounted into the bot container.

The three pieces sit on the same host. Bot connects to Gateway over an internal Docker network. State lives on disk, bind-mounted. Simple in concept; the friction is in the details.

The port 4002 trusted-IP trap

The single most common first-week mistake: pointing your bot at port 4002.

IB Gateway listens on 127.0.0.1:4002 with a hardcoded trusted-IP allowlist of 127.0.0.1. If your bot connects from any other IP — including a sibling Docker container on the same host — the TCP handshake succeeds, then the API handshake silently drops. Your bot waits the full 15-second nextValidId timeout, throws, retries, and waits again.

The fix isn't in IBKR's docs. It's a socat forwarder inside the gateway container: socat TCP-LISTEN:4004,fork TCP:127.0.0.1:4002. Your bot connects to port 4004 externally; socat forwards it to 4002 with the source address rewritten to 127.0.0.1; the gateway sees a trusted-IP source and accepts the API session.

The gnzsnz image ships with this forwarder pre-wired. If you build your own gateway image, you need to add the socat layer yourself. We've seen this trip up every team that tries to roll a custom gateway image on day one.

(Also worth knowing: the TRUSTED_IPS env var that some docs reference is not wired into the jts.ini.tmpl template in this image. Don't waste an evening setting it — the socat path is the working one.)

IBC re-login = 90 seconds of API downtime, every time

IB Gateway must log back in to IBKR's servers every 24 hours — IBKR enforces this. The gnzsnz image handles it via a scheduled IBC (IB Controller) restart. From your bot's perspective, this means every container restart, every nightly forced re-login, every docker compose change touches the gateway triggers ~90 seconds of API downtime.

Your bot needs a sane retry policy. Connect timeout 15 seconds, exponential backoff (30s / 60s / 120s), give up after 5 attempts and alert. We've seen bots written without this assumption that fall over permanently on the first daily restart and never auto-recover.

Side effect to know: docker compose up -d <bot-service> recreates dependent services. Because tradfi-ibkr-bot depends_on ibkr-gateway, touching the bot also bounces the gateway. Every code change costs 90 seconds of API downtime. Plan deploys accordingly.

"Existing session detected" — pick a primary

If your paper credentials are logged in from a second machine — your local Win11 IB Gateway, an old VPS instance, anywhere — the new gateway hits a modal dialog: "Existing session detected. Choose: take over (primary) or run secondary (read-only)."

A headless gateway can't click. Without the right env var, it sits at the dialog forever.

Set EXISTING_SESSION_DETECTED_ACTION=primary in the gateway env. The new session takes over within ~30 seconds; the old one gets disconnected. This is the right default for a production bot — you want the deployed instance to win, not whatever's open on your laptop.

(Also: close your local Gateway when you're not actively using it. Even with primary set, the reconnect flapping wastes log lines.)

State management: positions are NOT in your files

The single biggest mental model adjustment: your bot's positions_state.json is a cache, not a source of truth. The source of truth is IBKR's server. Your bot rebuilds the cache from reqPositions() on every cold start.

Why this matters: if your bot crashes mid-trade, restarts, and finds a position in positions_state.json that doesn't match what reqPositions() returns, trust IBKR. We've shipped a force-reconcile pattern that wipes the local state and rebuilds from reqPositions() on detection of any mismatch. It's the only sane policy.

Five state files, all bind-mounted from host into container:

| File | Purpose | |---|---| | live_status.json | Heartbeat + equity, overwritten every cycle. Read by the dashboard pusher. | | risk_state.json | Daily drawdown tracking, max-equity watermark. Persists across restarts. | | trade_vault.db | SQLite of all trades (open + closed). Permanent record. | | analyst_metrics.json | Per-strategy performance breakdown. | | dynamic_pairs.json | Pairs added by the universe-expansion process. |

Critical bind-mount trap: if the file doesn't exist on the host when docker compose up runs, Docker creates a directory there, then refuses to mount-it-as-a-file. Always pre-touch the JSONs (echo '{}' > file.json) before first deploy. Skip this and you get cryptic mount errors that look like permissions issues.

SL/TP orders live at IBKR — not in your bot

This is what makes IBKR genuinely viable for unattended systematic execution: stop-loss and take-profit orders are placed at IBKR's server, not held in your bot's memory. If your bot crashes, the position keeps its protection. If your VPS reboots, the position keeps its protection. If you lose internet for 4 hours, the position keeps its protection.

The pattern is: open the position as a market or limit order; immediately submit the SL and TP as independent bracket children with outsideRTH=true and tif=GTC. They sit at IBKR until they fill or you cancel them. Your bot can be off-line for days and the position is safe.

This is the operational property that lets us run unattended overnight. Not all brokers offer this — many crypto venues require the bot to hold SL/TP orders in memory and place them reactively, which means a bot crash exposes you. IBKR's broker-side SL/TP is one of the genuine reasons we prefer it for tradfi.

The "unhealthy but Up" trap (38 hours we'll never get back)

In May 2026 we shipped a postmortem on a 38-hour silent outage: the gateway container showed Up in docker ps, the bot was running, but every API call returned not connected. TWS itself had died inside the container; the container's PID 1 was the supervisor, not TWS, so the supervisor stayed alive and Docker reported the container as healthy.

The lesson: a container being Up is not the same thing as the application inside being healthy. You need an explicit healthcheck that exercises the application path. For our IBKR gateway:

healthcheck:
  test: ["CMD-SHELL", "nc -z 127.0.0.1 4002 || exit 1"]
  interval: 60s
  timeout: 10s
  retries: 3

This catches the dead-TWS-under-live-supervisor case. We deployed it after the incident. Auto-remediation (restart on unhealthy) is still on the backlog — we want eyes on the first few unhealthy events before turning auto-restart on.

Order routing gotchas

A few non-obvious ones we hit running real strategies:

• EtradeOnly not supported error rejection: certain order types require a wrapper helper (_new_order() in our codebase) that strips IBKR's e-Trade-style flags. Without it, ~10% of orders reject inexplicably. The fix is one wrapper function; the bug is in not having it.
• [10349 TIF] log line is purely cosmetic. IBKR's API spec lists 10349 as an informational message, not an error. Suppress it in your error filter or you'll get paged for nothing.
• Pair-universe drift: if your bot has a "Hunter" component that auto-expands the trading universe, it will sometimes try to re-add tickers you've audit-banned. We use a apply_audit_locks(strategies, merged_pairs) post-Hunter call to scrub re-additions. Without it, your blacklist silently leaks.

Paper to live: what actually changes

The temptation after 30 paper trades is to flip the switch. Don't.

Our policy: 30+ paper trades AND 3-6 months of soak before any real money. The paper trades validate the wiring; the soak validates that nothing weird happens around earnings, dividends, weekend gaps, holidays, exchange disconnect windows, IBC re-login timings during volatile periods. Stuff that doesn't show up in 30 trades shows up in 6 months.

When you do flip:

• TRADING_MODE=paper → live
• IBKR_PORT=4002 → 4001 (live gateway listens on a different port)
• Swap IBKR_GATEWAY_USER and IBKR_GATEWAY_PASSWORD to live creds
• Buy a market data subscription — about $30/month for US equities at retail, more if you need exchange-specific feeds. Set IBKR_MARKET_DATA_TYPE=1 for live data; 3 is delayed.
• If your account is a cash sub-account (not margin): keep LONG_ONLY=true. The bot must not try to short. Cash accounts mechanically can't.

Ramp size slowly. We use a 25% → 50% → 100% ramp over two months. The bot's first month at real money WILL surface differences from paper — fill slippage, partial fills, the actual cost of market orders into the open. Better to discover them at 25% size.

Honest tradeoffs

IBKR is excellent for systematic execution. It is not turnkey:

• You're operating IBC (the controller that handles non-interactive Gateway login). It has its own release cadence, occasional bugs, and the gnzsnz image lags upstream by days-to-weeks.
• You're operating Docker on a VPS. If docker builder prune -f isn't in your toolbox, this isn't your stack yet.
• You're reading IBKR's API release notes for breaking changes. They happen. The Python ib_insync ecosystem helps but doesn't immunize you.
• Non-US-resident personal accounts can't withdraw margin cash. Worth checking your residency status against IBKR's matrix before funding meaningful capital.

The payoff is one of the few brokers where you can deploy systematic execution at retail-scale capital, against a global universe, with a real API and a paper environment that genuinely matches live. The crypto-venue tier has the API depth but lives offshore. The fintech-broker tier (Robinhood, Public, eToro) has the UX but doesn't have a real API. IBKR sits in the middle and pays back the operational learning curve once you're past it.

How to actually act on this

If you're seriously evaluating IBKR for systematic execution:

1. Open a paper account first — at no cost, with full API access from day one. Open an Interactive Brokers account — that's where we started, that's where we still run paper. Full reasoning and our honest tradeoffs are on our broker page.
2. Use the gnzsnz/ib-gateway Docker image as the starting point. Don't roll a custom gateway image until you understand exactly what IBC + Xvfb + socat are doing.
3. Read IBKR's "Trader Workstation API" docs in full, even the boring parts about market data subscriptions and pacing violations. The boring parts are where the friction lives.
4. Cross-reference with our stack page + the AI supercycle access argument for the editorial case alongside the operational one.

The operational learning curve is real. So is the payoff. After twelve months, our bot fleet sits comfortably on IBKR's rails because the property we needed — broker-side SL/TP that survives bot crashes, global venue access from one account, a real Python API, and a paper environment that matches live — exists at IBKR and essentially nowhere else at retail.