Think beyond the signal
An automated trading system is the whole path from incoming data to a verified portfolio state. It reads markets and accounts, proposes an action, applies risk policy, submits orders, processes fills and checks its records against the venue. The strategy is just the part that decides what it would like to trade.
Plenty of expensive failures happen even when the signal points in the right direction. The feed may be stale. A timeout may trigger a duplicate order. Position sizing may use the wrong balance. A restarted process may have no idea what already filled. Those are trading failures, not mere IT inconveniences.
A trading bot may be one program inside this architecture. An AI trading agent may supply decisions or coordinate tools. Neither replaces independent limits, verified account state or a recovery procedure.
From data feed to recovery
These are logical responsibilities, not a requirement for eight separate services. A small system can run them in one process; a larger one may separate them across machines and teams. What matters is that each responsibility has an explicit input, output, owner and failure behavior.
1. Establish trustworthy market and account state
A trading decision combines two kinds of truth. Market state includes quotes, trades, order-book depth, reference prices, funding and venue status. Account state includes cash, positions, collateral, open orders, fills, transfers and permissions. A strategy that sees only prices can send an order that conflicts with an existing position or an order already waiting at the venue.
Every input needs a source timestamp, receipt timestamp, schema version and freshness status. Adapters can normalize units and symbols, but should retain the original payload. If a field is missing or a sequence breaks, mark the state incomplete. Quietly filling the gap with an old value only hides the problem.
Coinbase Exchange's WebSocket documentation illustrates why this is necessary: consumers must handle sequence gaps and out-of-order messages or use a channel designed to maintain a synchronized book. That is one venue's implementation, but the lesson is broader: a healthy connection does not mean the local book is complete.
- Define the maximum acceptable age for every market, reference and account input.
- Detect missing, repeated and out-of-order events instead of relying on connection status.
- Keep symbol, decimal, contract-size and timezone conversions explicit and testable.
- Rebuild state from a snapshot plus subsequent events when continuity is uncertain.
- Block new risk when the system cannot prove which orders and positions already exist.
2. Make the strategy state its intent clearly
The decision layer can use fixed rules, optimization, statistical models or AI. Its output should still be a structured proposal: instrument, target exposure or quantity, direction, decision time, valid-until time and reason or strategy identifier. It should not receive unrestricted authority to call arbitrary venue functions.
Decision intent = desired portfolio change + time horizon + confidence/context Keeping intent separate from execution makes the same proposal reviewable before and after a trade. It also lets policy resize or reject an action without changing the model. If an AI component is unavailable or returns malformed output, the fallback should be a known state—usually no new exposure—not an improvised order.
3. Put risk controls before the point of no return
Pre-trade controls operate while an unwanted order can still be stopped. They should use current portfolio state and include every open order that could fill, not only completed positions. Common checks include instrument allowlists, maximum order size, price collars, notional and leverage limits, concentration, available collateral, rate of order entry and cumulative loss.
The SEC's Market Access Rule applies to covered broker-dealers accessing U.S. securities markets; it is not a universal crypto rule. Its engineering logic is still instructive: prevent orders that exceed preset credit or capital thresholds, reject erroneous price or size, detect duplicative orders, restrict system access and deliver immediate post-trade reports to appropriate surveillance personnel.
| Control point | Examples | Why timing matters |
|---|---|---|
| Before order creation | Asset, venue, strategy and account permissions | Stops an unauthorized intent before it reaches an adapter. |
| Immediately before submission | Price, size, notional, leverage, available balance and duplicate checks | Uses the freshest state at the irreversible boundary. |
| While orders are open | Exposure, fill, cancellation, staleness and price-deviation monitoring | An acceptable order can become dangerous as the market or account changes. |
| After execution | Fill, fee, position, cash and policy reconciliation | Finds disagreements that pre-trade controls could not observe. |
Risk policy should be independent of model confidence. A prediction scored at 99% does not justify bypassing position or loss limits. Threshold changes should be versioned, approved and visible in the audit record so an attractive result cannot be explained by an undocumented relaxation of controls.
4. Treat an order as a sequence of states
An order is not simply “sent” or “filled.” It can be constructed, rejected locally, submitted, acknowledged, open, partially filled, amended, pending cancellation, canceled, fully filled or unknown. The exact states vary by venue, but the system should map them into a consistent internal lifecycle without inventing certainty.
The hardest case is an uncertain acknowledgement. Suppose a request times out after leaving the local process. Retrying the same economic order can duplicate exposure if the venue accepted the first request. Assuming success can be equally dangerous if it did not. A safe recovery path uses a stable client order identifier where supported, queries venue state, checks fills and open orders, and blocks conflicting action until the ambiguity is resolved.
- Assign each intended order a stable internal identity before network submission.
- Record the request, acknowledgement, venue order ID, fills, fees and terminal state.
- Handle partial fills as real exposure rather than waiting for the requested quantity.
- Bound retry count, price movement, order age and total outstanding notional.
- Make cancel-and-replace behavior explicit; cancellation is not final until acknowledged.
5. Reconcile the database with the outside world
The local database is a working model of the account, not the final authority. A centralized system must compare it with venue orders, fills, balances and positions. An onchain system must also distinguish submitted, pending, confirmed, reverted and replaced transactions, with a chain-reorganization policy where relevant.
Reconciliation can run after each event and on a schedule. Differences should be classified: timing lag, known rounding, missing event, duplicate processing, unrecognized order or material balance mismatch. The safe response depends on the class, but an unexplained position mismatch should normally block new risk until resolved.
Reconciled position = prior confirmed position + externally confirmed fills ± transfers and adjustments 6. Design security around actual permissions
A system should receive only the authority it needs. Read-only data collectors do not need trade access; an execution service usually does not need withdrawal access; an analyst should not hold a production secret. Coinbase Exchange, for example, documents separate View, Trade, Transfer and Manage API-key permissions. Other venues use different scopes, so verify the exact account model rather than assuming the labels mean the same thing.
- Use separate credentials for environments and responsibilities.
- Prefer trade-only access and disable value transfer when the workflow does not require it.
- Store secrets outside source code and prevent them from appearing in logs or alerts.
- Rotate and revoke credentials through a tested procedure, including during an incident.
- Restrict who can deploy code, alter limits, approve assets and resume a paused system.
7. Monitor the machinery, not just profit and loss
Profit is a delayed, noisy signal of system health. A losing trade may be correct system behavior; a profitable trade may have violated its limit. Operational monitoring should expose the path that produced the result.
| Layer | Useful measurements | Example alert |
|---|---|---|
| Data | Age, gap count, reconnects, invalid messages | Book age exceeds the strategy's maximum. |
| Decision | Signal rate, rejected schema, model version, drift | Decision distribution moves outside its validated range. |
| Policy | Accept, reject and resize counts by reason | A strategy suddenly hits its size limit repeatedly. |
| Execution | Acknowledgement latency, reject rate, fill rate, slippage | Venue rejects or timeouts exceed the operating threshold. |
| Portfolio | Exposure, concentration, margin, drawdown, reconciliation breaks | Venue position differs from the local expected position. |
| Infrastructure | Process health, queues, clock drift, storage and dependencies | A consumer falls behind while orders remain active. |
Logs should connect a market observation, decision, policy result, order and fill through stable identifiers. Preserve versions of code, models, configuration and venue adapters so a trade can be reconstructed from the information actually available at the time.
8. Decide what “safe” means before something breaks
“Stop the bot” is not a recovery plan. Killing a process neither cancels venue orders nor closes exposure. Name the safe modes in advance and specify exactly what each mode may do.
| Mode | Permitted behavior | Typical trigger |
|---|---|---|
| Normal | New orders within validated strategy and policy. | All required data, controls and dependencies are healthy. |
| No new risk | Manage or reduce existing exposure; block increases. | Stale data, model failure or unresolved state. |
| Cancel only | Cancel open orders without creating replacements. | Execution degradation or strategy disablement. |
| Reduce only | Submit bounded orders that can only lower exposure. | Limit breach, margin stress or controlled shutdown. |
| Isolated | No venue action until an operator completes reconciliation. | Credential incident, unknown orders or corrupted state. |
A restart should begin by discovering external state, not by replaying the last intended action. The system should load balances, positions, open orders and recent fills; reconcile them; then require every gate to pass before normal trading resumes.
Break these things on purpose before launch
| Injected failure | Expected behavior | Evidence to retain |
|---|---|---|
| Market-data gap | Mark state stale, resynchronize and block dependent orders. | Gap, snapshot, recovery time and decisions suppressed. |
| Order timeout | Query by identity and reconcile before retrying. | Request, timeout, venue query and resolved state. |
| Partial fill | Update exposure immediately and apply the remaining-order policy. | Each fill, remaining size and policy decision. |
| Process restart | Discover venue state before enabling new risk. | Startup snapshot and reconciliation result. |
| Limit breach | Reject new exposure and alert through an independent path. | Limit version, rejected action and acknowledgement. |
| Credential revocation | Enter a safe mode without repeated unauthorized requests. | Authentication error, response and escalation record. |
| Venue outage | Stop retries at a bound and manage other venues without assuming a hedge exists. | Open exposure, retry count and operator decision. |
Move from simulation to live trading in stages
FINRA's algorithmic-trading guidance emphasizes development controls, testing before production, pilot deployment, monitoring after launch and mechanisms to disable a system. Its rules apply to member firms, but the testing sequence offers a useful benchmark for any automated system that can create financial exposure.
- Unit and property tests: verify calculations, state transitions, limits and invariants under constructed inputs.
- Integration tests: exercise adapters against recorded responses, venue sandboxes or test networks, including errors.
- Historical simulation: test strategy and execution assumptions on point-in-time data without calling it live evidence.
- Failure injection: delay, duplicate, drop and reorder events; restart components with open simulated exposure.
- Shadow mode: consume production data and create decisions without submitting orders.
- Limited live pilot: use bounded instruments, size and time while comparing expected and actual behavior.
- Controlled scaling: increase only after fill quality, reconciliation and incident response remain within thresholds.
The backtests versus live results guide explains why passing an earlier stage does not establish performance at a later one.
How to inspect someone else's platform
- Draw the boundary. Identify the data source, decision engine, risk controls, executor, custodian and operator.
- Trace one order. Ask how an intent becomes an order, how it can be rejected and how every fill changes portfolio state.
- Inspect permissions. Determine which credentials can trade, transfer value, change limits or deploy code.
- Review failure behavior. Ask what happens after stale data, a timeout, partial fill, restart, venue outage and lost credential.
- Demand separated evidence. Keep backtest, paper, shadow and live results clearly labeled, with costs and sample periods.
- Test the exit. Confirm the user can pause automation, cancel orders, reduce exposure, export records and revoke access.
- Check accountability. Every model, rule, configuration and limit change should have a version, owner and effective time.
Questions that come up often
Can an automated trading system run without AI?
Yes. Time-based rebalancing, fixed execution schedules, threshold rules and conventional statistical strategies can all be automated. AI changes how some decisions are produced; it does not define whether the surrounding system is automated.
Does paper trading prove the system works?
It can reveal data, state and workflow defects, but it does not prove production liquidity, queue position, market impact or the operator's response to real financial loss. Treat it as one test stage, not live proof.
Should a system automatically close everything after an error?
Not always. A blind emergency market order can worsen a problem during illiquidity or when local position state is wrong. Recovery policy should distinguish blocking new risk, canceling orders and reducing verified exposure, with price and size controls appropriate to the failure.
What is the most important system metric?
There is no single metric. Financial exposure, state reconciliation, data freshness and execution quality answer different questions. A system can be profitable while unhealthy, or operationally correct during a losing period.
Sources and scope
- SEC — Risk-management controls for brokers or dealers with market access
- FINRA Regulatory Notice 15-09 — Algorithmic-trading supervision and control practices
- Coinbase Developer Documentation — WebSocket sequence gaps and order-book synchronization
- Coinbase Developer Documentation — API-key permissions
- NIST AI RMF Core — AI testing, monitoring and disengagement
- Onchain Off Emotion — What trade proofs can and cannot establish
- Onchain Off Emotion — Risk disclosure
This page describes one sensible system shape. It does not show that a specific bot or platform uses these controls. Security, custody and market rules vary with the operator, instrument, venue and country.