SnakeBatch v9: Economics of the 100-Shard Mesh

Charles Dana · Monce SAS · May 2026

snakebatch.aws.monce.ai · /paper · /architecture · /concurrency · /math

1. Per-Lambda Cost

v9 uses ONE Lambda binary at 4096 MB, deployed to 100 functions.

v9-worker (4096 MB, ~3 vCPUs):
  invoke  : $0.0000002 per request
  compute : 4 GB × duration × $0.0000166667/GB-s
          = $0.0000667/s

  divide  (~80ms warm)  :  $0.000005 + $0.0000002  ≈ $0.0000054
  conquer (~150ms warm) :  $0.000010 + $0.0000002  ≈ $0.0000102
  leaf    (~400ms warm) :  $0.0000267 + $0.0000002 ≈ $0.0000269

2. Tree Topology & Worker Count

v9 partitions input via binary divide, then hands each ≤τ slice to one leaf. Per layer the tree shape is:

Snake at the leaf does its own internal IF/ELIF/ELSE partition with build_bucket_chain(bucket=user_b). Snake's partition is class-aware, not size-fixed: a slice of 500 mixed-class rows at bucket=32 emits ~4 buckets (one per class plus catch-all), not 500/32 = 16. So the assembled JSON is dramatically lighter than naive arithmetic suggests.

3. Total Cost Per Training Run

L layers each with ~⌈n/τ⌉ leaves, ~⌈n/τ⌉ conquers, and ~⌈n/τ⌉−1 divides. Production default τ = 1,000 (locked — not a tuning knob). The cost numbers below assume this.

The cost rows below are split into compute (the leaf bill, linear in N) and reservation (provisioned-slot bypass needed to keep peak invocations/s under the L-A1AFA3CF ceiling, super-linear past N≈30K). At τ=1000, all rows above 7.5K×25L need bypass to be feasible — see /paper §7 for the derivation.

Costs split: leaves are linear in N (the compute column); reservation is super-linear in N because the slot count needed to sustain peak invocations/s grows as O(L · N / log N) at fixed τ (see /paper §7.3). Past N≈30K at L=25, the reservation term overtakes compute and dominates total cost.

3.1 Measured: 3 Datasets × 5K Rows (May 2026)

Live benchmark: 3 datasets (binary, 3-class, regression), each trained twice — once on full 5K (perfect-fit), once on 4K with 1K held-out. All predicts forced through the cloud (cloud_threshold = 1). 12 model trains in total, ~30K predict round trips, single CloudWatch window:

Per-row cost decomposition

CloudWatch delta over the entire bench: $0.0434 for 387 Lambda invocations. Total work performed: 6 trainings × 4–5K rows = 27,000 training rows, plus ~24,000 inference round trips. So:

Trainings dominate compute time (52s of 73s wall clock, 71%); predicts dominate count. If we ascribe cost proportional to wall clock:

Cloud predict's per-row cost is not a constant — it scales with the assembled bucket-chain length (training size × layer depth). A second empirical anchor from a Nature consumer running a 22K-row × 5-layer model:

The lever for >>1M-prediction workloads is the local handoff (§6.1): download the model JSON once with m.to_algorithmeai() and run algorithmeai.Snake in-process. algorithmeai per-row inference is sub-millisecond regardless of model size, with no Lambda billing, no S3 fetch, no HTTP RTT. Cloud predict's pricing is for the case where the user can't or doesn't want to ship the model to the caller (multi-tenant, audit trail, central enforcement) — not for high-volume batch.

Numbers persist in v9/bench/results_3x5k.json; rerun via v9_smoke_economics.py.

3.2 Earlier Burn Snapshot (May 2026)

72-hour CloudWatch snapshot of the eu-west-3 account, all Lambdas:

4. v6 vs v9 — Why Spread the Mesh?

v9 is the version that completes at scale — v6's recursive self-invokes wedged under L-A1AFA3CF tightening. v9 routes around that with sharding (isolation) plus toggled provisioned slots (peak-rate bypass). The mesh is an availability mechanism; the slots are a throughput mechanism; neither alone replaces the other.

5. Three-Bucket Cost Structure

The per-million numbers above are compute-only. The real cost of running v9 has three buckets, and only one of them scales with rows processed:

5.1 Fixed Costs — the always-on floor

EC2 t4g.2xlarge (gatherer)   $96.77/month  (on-demand)
                             $65.30/month  (reserved 1y, 32% off)
S3 storage (~5 GB)            $0.12/month
DynamoDB on-demand            $0.06/month  (~50K writes)
CloudWatch log retention      $0.30/month
Route53 + ACM cert            $0.50/month
                             ────────────
                             $97.75/month total fixed

The gatherer is 8 vCPU / 32 GB at 99% idle. Massively overprovisioned for a single user; sized for multi-tenant. At 100K rows/month total throughput, the fixed bucket alone amortizes to $978/M rows — dwarfs every other cost. At 100M rows/month it's $0.98/M and disappears into the noise.

5.2 Activation Cost — the provisioned-concurrency toggle

L-A1AFA3CF (Lambda concurrency scaling rate) is a 1000/min account-wide ceiling AWS does not adjust. Cold starts on a chain-serialized invoke graph cap v9 at <5× parallelism for small jobs — the 100-shard mesh is structurally underutilized at the spawn-rate ceiling. Provisioned concurrency bypasses this for the provisioned fraction:

Provisioned reservation rate (eu-west-3 x86):
  $0.0000048673 per GB-second  (paid while READY, not while invoking)

Spin-up:                  ~84s, NOT billed
Tear-down:                ~0.8s, instant
Min billing granularity:  1 second

  4 GB ×   1 shard ×  60s = $0.00117  per minute warm
  4 GB ×  10 shards ×  60s = $0.0117   per minute warm
  4 GB × 100 shards ×  60s = $0.117    per minute warm  ($7.01/hour)

The spin-up tax means the toggle is a session primitive, not a per-job one. You toggle on once, run a stream of jobs against the warm mesh, toggle off when done. AWS cannot bill provisioning while the function itself is unreachable, so a reliable toggle-off (EventBridge one-shot + 60s sweep) is mandatory — a leaked toggle on 100 shards burns $168/day.

5.3 Usage Cost vs Activation Cost — the friction equation

Define friction as the fraction of warm-window time the mesh sat ready but doing nothing useful. 0% friction = perfectly back-to-back jobs. 50% friction = half your warm window was idle.

provisioned $/min  $0.117  (full mesh)
on-demand   $/min  $0.0083 (measured, 5K×5L bench)
                  ───────
ratio              14×     provisioning is 14× the price of plain Lambda

So provisioning only pays for itself when the warm window crams enough back-to-back compute to dilute the reservation tax. Concretely, for a 5K×5L job (one warm session, full 100-shard mesh, 60s budget):

The session profile that genuinely beats on-demand is 20+ back-to-back jobs through one warm window. One job per session is strictly worse than on-demand — you're paying 14× the compute cost for ~2× the wall-clock improvement.

5.4 The amortization scenario — when warm Lambdas pay off

Provisioning is a session primitive, not a per-job one. The 84s spin-up plus the 14× reservation rate means a single isolated job gets crushed by the toggle. But a session — a stretch of clock time during which you fire many jobs back to back — can amortize the warm cost down to fractions of a cent per job. Concrete numbers, full mesh (100 shards × 4 GB), 60s of spin-up amortized into the budget:

5.5 So what's it actually for?

Provisioning trades dollars for predictable latency. The 14× tax is the price of skipping cold starts and the L-A1AFA3CF spawn-rate ceiling on the provisioned fraction. Three regimes where that trade is worth taking:

For everything else — ad-hoc training, exploratory benches, the "morning bruv let me try this dataset" flow — on-demand is the right default. v9's small-job wall clock (~6 s for 5K×5L) is bad enough to justify the toggle when latency is the product, fine to live with when correctness is the product.

5.6 Honest per-million by usage profile

The blue dashed line is on-demand — the toggle curve approaches 14× that and stops. Provisioning is a wall-clock product, not a cost product.

6. EC2 Gatherer Cost (deep dive)

One t4g.2xlarge in eu-west-3, 8 vCPU / 32 GB:

On-demand : $0.1344/hour = $96.77/month
Reserved 1y : $0.0907/hour = $65.30/month  (32% off)

The gatherer hosts the FastAPI app (/v9/train, /v9/status, /v9/model, /grid/v9) and runs the SQS drainer thread. Idle CPU 99%, RAM 0.85 / 30 GB at current load — massively overprovisioned for a single user, perfect for multi-tenant scale.

7. Inference Cost

Two paths:

7.1 Local handoff — monceai.Snake.to_algorithmeai()

Download /v9/model/{id}, instantiate algorithmeai.Snake(path), predict locally. $0 per prediction, sub-millisecond per row on commodity hardware, zero network roundtrip.

This is the recommended path for v9 because the model JSON is byte-equivalent in semantics to a locally-trained Snake. There is no operational reason to keep inference in the cloud.

7.2 Cloud predict — /v9/predict/{id}

Not yet implemented. When wired, expected:

v9-predict (1 GB, ~5ms warm):
  invoke  : $0.0000002
  compute : 1 GB × 0.005s × $0.0000166667 = $0.000000083
  Total per Lambda ≈ $0.0000003 per call
  ≈ $0.30 per million predictions

8. Cost vs Provider Pricing

Comparing v9 to closed-source classification APIs at 10K-row training scale:

The SnakeBatch number isn't a discount; it's a different cost class. SAT-by-construction means we don't pay for backprop, we don't pay for hyperparameter search, we don't pay for retries. The work is polynomial in n and linear in spot Lambda time. There is no model to "fit" beyond constructing the formula.

© 2026 Charles Dana · Monce SAS · SnakeBatch v9 · /paper · /architecture · /math