Kimi K3 is open-weight, but it is not small. Moonshot AI’s new model contains 2.8 trillion total parameters, activates 104 billion per token, and ships roughly 1.56 terabytes of weight files. “Download and run it” means renting a small GPU cluster, not launching one ordinary accelerator instance.
The cleanest comparable deployment across the three major U.S. clouds is two eight-GPU H200 nodes. That 16-GPU topology comes directly from the current Kimi K3 serving guidance for H200 hardware. It gives 2.256 terabytes of aggregate high-bandwidth GPU memory and enough room for the MXFP4 weights, runtime overhead, model state, and a working cache.
Using public Linux on-demand prices, 12 hours per day, and a 30-day planning month, AWS is the least expensive of the three in this snapshot. Its compute bill is about 25% lower than either Azure or Google Cloud. Azure and Google Cloud land within five dollars of each other per month.
The important qualification
Sixteen H200s are a hardware floor, not a 100-user service-level guarantee. “100 concurrent users” does not say how many are generating at once, how long their prompts are, how many reasoning tokens they request, or what latency they expect. Kimi K3 always reasons, and its final-weight serving profiles were still awaiting full production measurement when this analysis was prepared. A representative load test is mandatory before purchase.
The cost comparison
Every row below buys the same accelerator count: 16 NVIDIA H200 GPUs in two interconnected cloud instances. The estimate assumes the cluster is fully stopped for the other 12 hours each day and restarted without paying for idle GPU capacity.
| Cloud | Two-node stack | Cluster / hour | 12-hour day | 30-day month | Annual |
|---|---|---|---|---|---|
| Azure | 2× Standard_ND96isr_H200_v5 | $169.60 | $2,035.20 | $61,056 | $742,848 |
| AWS | 2× p5en.48xlarge | $126.592 | $1,519.10 | $45,573 | $554,473 |
| Google Cloud | 2× a3-ultragpu-8g | $169.614 | $2,035.37 | $61,061 | $742,909 |
Spread across 100 named capacity slots, the compute floor is approximately $456 per user-month on AWS and $611 per user-month on Azure or Google Cloud. That is not a metered per-user price. It is simply the monthly cluster bill divided by 100, whether those users keep the GPUs busy or not.
How the estimate works
The inputs are deliberately conservative and comparable:
- Hardware: 16 NVIDIA H200 GPUs—two nodes with eight 141 GB H200s each.
- Serving software: a Kimi K3-compatible engine such as SGLang or vLLM, with tensor and expert parallelism across both nodes.
- Purchase model: public Linux on-demand rates, with no Spot pricing, private discounts, reserved-use commitments, credits, or taxes.
- Runtime: 12 billable hours per day for 30 days, or 360 hours per month; the annual figure uses 4,380 hours.
- Regions: Azure East US 2, AWS US East (Ohio), and Google Cloud Iowa.
- Scope: accelerator instances only. The VM price includes the CPU, memory, networking interfaces, and local SSD bundled with each selected instance.
Provider math
Azure: the public retail price is $84.80 per hour for one Standard_ND96isr_H200_v5 VM in East US 2. Two nodes cost $169.60 per cluster-hour. At 360 hours, that is $61,056 per month.
AWS: the current public price list gives $63.296 per hour for one Linux p5en.48xlarge instance in US East (Ohio). Two nodes cost $126.592 per cluster-hour. At 360 hours, that is $45,573.12 per month.
Google Cloud: the on-demand price is $84.806908493 per hour for one a3-ultragpu-8g VM in Iowa. Two nodes cost $169.613816986 per cluster-hour. At 360 hours, that is $61,060.97 per month.
What 100 concurrent users really changes
The model weights determine the minimum GPU memory. User behavior determines whether that minimum cluster is fast enough.
A hundred people signed in at once is easy if only a handful are generating. A hundred simultaneous long-context reasoning jobs is a different system. It increases the memory reserved for Kimi K3’s recurrent state and key-value cache, raises aggregate token demand, and can turn an apparently economical cluster into a queue.
Capacity planning needs four measurements:
- Active-generation concurrency: how many of the 100 users are actually decoding at the same moment.
- Average and p95 context: input, retained conversation, tool results, images, and output tokens.
- Service target: time to first token, output tokens per second, and maximum acceptable queue time.
- Reasoning effort: low, high, or max. Kimi K3 always reasons, so hidden reasoning tokens consume capacity too.
As a planning test, model 100 simultaneous requests at the p95 prompt size and desired output length, then replay that traffic against the exact two-node cluster. If either time to first token or per-user decode speed misses the target, the answer is another serving replica, a lower reasoning setting, shorter contexts, stronger admission control, or some combination of the four.
The production number is usually higher
A single 16-GPU stack has no serving redundancy. One failed VM, fabric problem, driver update, or model reload can take the whole endpoint down. An interactive business service would normally need at least a second stack for rolling maintenance and failover.
| Cloud | Monthly compute | Annual compute | Capacity cost / 100 users |
|---|---|---|---|
| Azure | $122,112 | $1,485,696 | $1,221 / user-month |
| AWS | $91,146 | $1,108,946 | $911 / user-month |
| Google Cloud | $122,122 | $1,485,817 | $1,221 / user-month |
That doubling is not automatically the correct architecture. If the service can tolerate scheduled downtime, a single stack may be enough. If it cannot, the resilient figures are the more honest starting budget.
Costs outside the headline
The GPU bill dominates, but it is not the whole platform:
- Persistent model storage: the official checkpoint contains 96 weight shards totaling about 1.56 TB. Local NVMe disappears when ephemeral instances are released, so daily shutdown may require persistent object or block storage and a model-loading strategy.
- Startup time: transferring and loading 1.56 TB can consume a meaningful part of a 12-hour operating window. A capacity reservation or retained node may erase some of the expected shutdown savings.
- Control plane: load balancing, API gateways, authentication, schedulers, container orchestration, secrets, monitoring, logs, and a small CPU service tier.
- Data movement: internet egress, cross-zone traffic, and especially cross-node GPU communication if the topology is not placed on the provider’s intended high-speed fabric.
- Operations: quota acquisition, capacity reservations, security hardening, patching, on-call coverage, benchmarking, and incident response.
For an early budget, add 5% to 10% for the surrounding platform before labor and redundancy, then replace that allowance with an architecture-specific bill of materials. The real swing factor is usually not storage or logging. It is whether one 16-H200 stack meets the load test.
Recommendation
Use AWS as the first cost benchmark, not as an automatic purchase decision. At public on-demand rates in the selected U.S. regions, AWS is clearly cheaper for the same 16-H200 topology. But all three providers require scarce, quota-controlled accelerators, and actual capacity may matter more than a list-price advantage.
Before committing, run a one- or two-day proof of capacity with representative prompts, reasoning settings, and 100-user arrival patterns. Capture tokens per second, time to first token, queue time, cache hit rate, GPU memory headroom, and cold-start duration. Then price the number of serving stacks the test—not the parameter count—says you need.
Decision in one sentence
Budget $45,600 to $61,100 per month for the minimum 12-hours-per-day cluster, or $91,100 to $122,100 per month for two complete stacks, and do not promise 100-user performance until the workload passes a production-shaped load test.