Self-Hosted LLM Cost Calculator

The cost of running an open large language model (LLM) depends on how much useful work a paid serving fleet completes each month. Buying graphics processing units (GPUs), renting them by the hour, and paying a hosted application programming interface (API) per token create different bills for the same workload. A fair comparison prices enough capacity to serve that workload before asking which bill is lower.

Quiet periods have different costs under each arrangement. A purchased machine still ties up capital, while hourly rentals continue to cost money for as long as they are kept available. Token pricing follows the amount processed under the provider's billing terms.

A token is a unit of text processed by a model. Input tokens make up the prompt and its context, while output tokens are generated in response. Reading long documents and returning short summaries produces a different workload from answering short prompts with long responses. The total token count alone therefore does not describe either the processing work or the API bill.

Prompt processing is called prefill, and response generation is called decode. Their processing rates can differ, so capacity planning needs both. A serving replica is one complete model copy with all the GPUs it requires. If that copy spans several GPUs, adding another replica repeats the complete GPU requirement. The useful throughput is the work completed by that whole replica across its requests.

Available hours and busy-time utilization determine how much of that throughput can be used during a month. Busy-time utilization is the share of available time budgeted for serving work. A lower target leaves more time unused and may require additional replicas to serve the same volume. Whole replicas also create spare capacity whenever demand falls between fleet sizes. Monthly capacity can look sufficient even when simultaneous requests would still cause unacceptable delays.

Electricity depends on average power over the hours the equipment remains running, including idle periods. The host machine and facility overhead, such as cooling, can add to GPU consumption. Staffing, monitoring, storage, and network charges may also matter. A comparison that includes only the purchase price or an advertised hourly GPU rate can miss recurring expenses that change the decision.

Hardware amortization spreads a purchase over a chosen number of months for a monthly cost comparison. Simple cash payback measures how long the initial outlay would take to recover through avoided API spending after recurring operating costs. Extending the amortization period reduces the monthly allocation, but it does not reduce the purchase outlay or make cash recovery faster.

The comparison remains useful only when the alternatives deliver comparable work. Model weights, numerical precision, response quality, context requirements, and acceptable response times all matter. A cheaper model or a slower service may answer a different requirement. Cost estimates also need separate checks that the model fits in GPU memory and that the proposed fleet can handle real traffic peaks.

How to Use This Tool:

Start with one model and a representative month of demand. The starting prices, throughput, and power figures are editable examples in USD. Replace them with your quotes and measurements; no live vendor prices are fetched.

1. Enter Monthly input tokens and Monthly output tokens in millions, together with the separate API input price and API output price per million tokens. Include billed reasoning tokens in the output total when applicable. Effective rates can account for discounts or caching that you have calculated separately.
2. Set GPUs per serving replica to the full GPU requirement for one model copy and choose Minimum active replicas. Enter measured prefill and decode throughput for that complete replica across its concurrent requests. Use separate phase rates while busy; a single-user rate or an end-to-end rate is not interchangeable with these inputs.
3. Choose Hours online per day and Target busy-time utilization. The estimate uses 30 days per month. Select a utilization target that reflects your serving requirements, then let the calculated fleet count show whether the minimum is sufficient.
4. Select Hosting mode and enter the corresponding costs. For Owned hardware, include each GPU's share of the host purchase in Purchase allocation per GPU, then enter the amortization period, average GPU power, and electricity tariff. For Rented GPUs, use an all-inclusive price per GPU-hour that includes the host and electricity.
5. Add relevant expenses in Advanced. Operations and other hosting costs are monthly totals for the fleet. Add paid spare replicas when required. In owned mode, enter additional host power only when it is missing from the GPU power allowance, and apply facility power usage effectiveness (PUE) without counting facility energy twice.
6. Review Monthly bill and Fleet decision after editing. Results update automatically. Check the full installed GPU count and acquisition outlay before using the cost comparison to plan a purchase or rental.
   If Check the cost inputs appears, correct the value identified in the message. Both throughput fields must remain positive, even when one of the token totals is zero.

Interpreting Results:

The large monthly amount is the modeled self-host cost. The heading identifies the cheaper option, and Savings vs API is positive when self-hosting costs less and negative when it costs more. The comparison includes any replicas added beyond your minimum.

• Fleet to provision includes active replicas and paid spares. Actual busy share describes work spread across the active replicas, so it can be below your target after whole-replica rounding. Benchmark that configuration against the required response times before committing to it.
• Self-host / 1M divides the total bill by input plus output tokens. Compare it with an API price blended at the same input/output mix, rather than with either quoted API rate alone.
• First crossover is the first monthly token volume at which the modeled bills are equal, with enough replicas to serve it at the current input/output mix. A subsequent replica addition can reverse the saving. Use Volume plan to check proportional volume changes and its Half utilization scenario to see the effect of a lower utilization target.
• Simple cash payback measures recovery of the purchase outlay. A finite result can coexist with a higher amortized monthly cost. If payback exceeds the amortization period, recovering the purchase requires the hardware to keep operating and the monthly cash savings to continue beyond that period.

The Cost curve shows a bounded range, capped at 60 times one replica's monthly capacity. Its axis may end before your current workload or a reported crossover. Read the numeric results as well as the visible curve.

Technical Details:

Monthly serving demand is expressed as replica-hours of work. Prefill time and decode time are added, then divided by the busy hours available from one complete replica. This gives a capacity estimate for the entered workload, with whole replicas priced whenever demand exceeds the current fleet.

Formula Core

The fixed 30-day month determines available hours. Token totals entered in millions are converted to tokens, and seconds of processing are converted to hours before fleet sizing.

These symbols keep the workload units separate from the GPU count.

The ceiling brackets round required capacity up to a whole replica, while the minimum keeps a paid fleet available at low demand. Adding GPUs to one replica changes its cost immediately. Its throughput changes only when new measured prefill and decode rates are supplied for that configuration.

Adding the two phase times is a planning assumption. It does not simulate overlapping prefill and decode work, queues, or changes in batching efficiency. Published aggregate token rates may include prompt processing or the full request duration. Check their measurement period before using them as the separate, decode-only rate required here.

The actual busy share divides serving work by the available hours of active replicas. Spare replicas do not reduce this reported percentage.

Owned and rented monthly costs

Owned electricity starts with average power for every installed replica. Let W_g be average watts per GPU, W_h additional host watts per replica, and η the facility PUE. PUE is total facility energy divided by information technology (IT) equipment energy. Multiplying IT energy by it includes the modeled facility overhead.

Facility energy E is measured in kilowatt-hours. The division by 1,000 converts watt-hours to kilowatt-hours.

Power entered in kW is converted to watts first, so 0.1 kW and 100 W represent the same draw. Average power must cover idle and busy periods across all available hours. Utilization is not multiplied into the electricity calculation. PUE multiplies energy only, and a value of 1 adds no facility overhead.

Let P be purchase allocation in USD per GPU, L the amortization period in months, e the USD/kWh tariff, and r the all-inclusive USD/GPU-hour rental quote. K is monthly operations plus other hosting costs for the entire fleet. API rates p_i and p_o are USD per million input and output tokens.

The monthly bills combine these costs according to the hosting arrangement.

Rental charges cover all installed GPUs for the full available hours, including idle periods and reserves. Separately entered purchase and electricity amounts do not contribute in rented mode because the quote is assumed to include the host and electricity. Operations and other hosting costs are added once per month in either mode and do not automatically grow with the replica count.

For a nonzero workload, the effective self-host price uses the selected monthly total and all input plus output tokens. Monthly savings use that same total.

Capacity and first crossover

A volume comparison holds the input/output mix constant. Let V be total monthly tokens in millions and f the input share. One replica's capacity Q depends on that mix, both phase rates, available hours, and utilization. The blended API rate p uses the same mix.

Let a be the monthly cost of one replica, including its GPUs and applicable amortization and electricity or rental. Fixed monthly cost b consists of the paid spare replicas plus operations and hosting. Self-host cost then rises in whole-replica steps, while API cost rises at the blended token rate.

For a positive blended API rate, an equality with k active replicas occurs at the following candidate volume.

The candidate counts only if it lies within the volume served by that fleet size. The minimum fleet covers zero through its full capacity. Each larger fleet covers volumes strictly above the previous fleet's capacity and up to its own capacity, including the upper boundary. The first feasible equality is the reported crossover.

If the API charge for a fully used replica's capacity is below that replica's monthly cost, growing volume cannot create monthly savings under these assumptions. Fixed overhead can postpone a crossover to a later fleet size when the per-replica economics otherwise favor self-hosting. Equality exactly at a capacity boundary can disappear as soon as another replica becomes necessary.

Cash payback

Owned cash operating cost is electricity plus operations and other hosting expenses. Acquisition is paid upfront, so amortization is excluded from the recurring cash cost used to recover it. Simple payback divides the full installed-fleet purchase outlay by the monthly API spending avoided after those cash expenses.

This assumes the same monthly cash saving continues throughout the recovery period. Changing the amortization period changes monthly accounting cost and the payback warning, but does not change cash payback itself.

Substitution for a two-GPU replica

Consider an illustrative owned fleet serving 60 million input and 30 million output tokens per month. Each complete two-GPU replica processes 1,000 input tokens/s during prefill and 100 output tokens/s during decode. It remains available 10 hours per day at a 50% target busy share, with a minimum of one active replica and no spares.

The workload requires 100 replica-hours, while one replica has 150 budgeted busy hours available.

Use a $1,200 purchase allocation per GPU over 12 months, 100 W average power per GPU, and electricity at $0.10/kWh. Set additional host power, operations, and other hosting costs to zero, with PUE 1. API prices of $2 per million input tokens and $8 per million output tokens complete this example. These are illustrative assumptions, not hardware measurements or market quotes.

Cash expenses are only $6 per month in this example, leaving $354 per month to recover the $2,400 purchase. The payback calculation therefore uses the cash saving rather than the $154 monthly saving after amortization.

Boundaries and numerical precision

Some missing results reflect an undefined comparison or the absence of cash savings. They should not be interpreted as zero cost or instant recovery.

Inputs used by the selected mode must be valid. Token totals, applicable quotes, and power cannot be negative. Both throughput rates must be at least 0.01 tokens/s. Available hours must be from 0.01 to 24 per day, utilization from 0.01% to 100%, and PUE from 1 to 5. GPUs per replica and minimum active replicas must be positive whole numbers; spare replicas can be zero or a positive whole number. An entered workload requiring more than 10,000 active replicas is rejected. This limit applies to the entered workload; extrapolated crossovers and volume scenarios may exceed it and do not establish that such a fleet can be provisioned.

Calculations retain precision until display. Monthly money normally shows two decimal places, effective token prices up to four, and general quantities up to three. A replica ratio is treated as the nearest integer when the distance is at most 10⁻¹² times the larger of 1 and its magnitude. Otherwise it rounds upward. A workload exactly at a fleet's capacity fits that fleet; an excess beyond this tolerance adds a replica. The summary treats an absolute monthly cost difference below 0.000000001 USD as a tie, while scenario rows compare unrounded amounts directly. Ordinary display rounding can also make unequal amounts look identical.

Limitations and Privacy:

These are educational planning estimates, not financial advice or a procurement quote. Throughput, power, and prices are supplied assumptions. The calculation does not run a benchmark, check GPU memory fit, or establish model quality and service equivalence.

• Taxes, financing, resale value, depreciation schedules, replacement cycles, and future changes in prices or demand are not modeled. All amounts use USD without currency conversion.
• Storage, network, software, and staff expenses enter only through your supplied allowances. Provider-specific billing rules, retries, caching, and discounts are not independently calculated.
• A zero purchase allocation applies to automatic fleet additions and reserves as well as existing equipment. A mixture of already-owned GPUs and newly purchased GPUs is not tracked separately.
• Calculations use the numbers entered in your browser. Inputs can be carried in the page address, so sharing that address can disclose workload assumptions and quotes.

Worked Examples:

References:

• Measuring electricity, U.S. Energy Information Administration.
• Power Usage Effectiveness (PUE), The Green Grid.
• LLM Inference Benchmarking: Fundamental Concepts, NVIDIA, April 2, 2025.
• AIPerf Metrics Reference, NVIDIA.
• How to run a llama.cpp benchmark, Simplified Guide.