Ceph Storage Capacity Calculator
Estimate Ceph usable capacity from node sizes, replica or erasure-code protection, failure-domain reserve, nearfull policy, and PG hints.{{ summaryTitle }}
| Metric | Value | Copy |
|---|---|---|
| {{ row.label }} | {{ row.value }} |
| Type | Signal | Detail | Next action | Copy |
|---|---|---|---|---|
| {{ row.type }} | {{ row.signal }} | {{ row.detail }} | {{ row.action }} |
| Guardrail | Value | Copy |
|---|---|---|
| {{ row.label }} | {{ row.value }} |
| Profile | Usable | Efficiency | Fault tolerance | Min spread | Delta | Verdict | Copy |
|---|---|---|---|---|---|---|---|
| {{ row.profile }} | {{ row.usable }} | {{ row.efficiency }} | {{ row.faultTolerance }} | {{ row.minimumDomains }} | {{ row.delta }} | {{ row.verdict }} |
Introduction
A Ceph pool is sized by the failures it must survive, not by drive labels alone. Installed terabytes are only the starting estate. Before that space becomes safe data capacity, the plan has to account for Object Storage Daemon (OSD) overhead, CRUSH placement, protection copies or chunks, operating fill thresholds, and the raw room needed for recovery after a host or OSD disappears.
Three capacity numbers are easy to confuse. Raw capacity is the sum of the devices or hosts you expect to contribute. Used raw capacity is the part of that estate you are willing to fill under a nearfull policy. Usable capacity is the protected data that remains after replication or erasure coding consumes some of the used raw space. A replicated pool is simple to reason about because every object is stored as full copies. An erasure-coded pool can be much more efficient, but it needs enough separate placement domains for all data and coding chunks.
| Planning term | What it controls | Common mistake |
|---|---|---|
| Protection profile | Replica count or erasure-code k+m chunks. | Comparing capacity without checking the needed placement spread. |
| Failure domain | The unit expected to disappear together, such as a host, OSD, or custom block. | Using average capacity when the largest host would create the real recovery gap. |
| Nearfull threshold | The operating fill point used for the usable-capacity estimate. | Treating nearfull as a target instead of an alert boundary. |
| Placement groups | The pool shards that CRUSH maps across OSDs. | Changing PG counts from arithmetic alone without checking autoscaler guidance. |
Fill thresholds are part of the safety margin. Nearfull is an early warning that the cluster is getting crowded. Backfillfull can block recovery or rebalancing onto an OSD that is already too full. Full can stop normal writes. A plan that treats nearfull as the desired steady-state fill level may pass a healthy-day capacity check and still fail during backfill, when the cluster most needs spare raw room.
Topology changes the answer as much as protection math. Six equal hosts can place a 4+2 erasure-coded profile across six hosts, but a smaller host count cannot satisfy the same spread even if the raw total is high. Mixed-capacity clusters add another wrinkle because losing the largest host matters more than losing an average host. Capacity, fault tolerance, and placement spread have to be read together.
A Ceph sizing result is still a model, not a live cluster audit. Device classes, CRUSH rules, balancer state, pool autoscaling, snapshots, compression, quotas, and write patterns can move the real usable limit. The estimate is most valuable before hardware purchase, pool creation, or protection-profile changes, where it can expose a bad reserve or spread assumption while it is still cheap to fix.
How to Use This Tool:
Use the calculator as a what-if planner: enter the raw estate, choose the pool protection profile, then read the capacity and guardrail outputs together.
- Set Total nodes. For equal hosts, enter one Capacity per node and unit. For mixed hosts, turn on Heterogeneous nodes and fill the per-node capacity rows; blank rows count as zero.
- Choose a Protection preset such as Replication 3x, EC 4+2, or EC 6+3. Select Custom when you need to edit size, min_size, k, or m directly.
- Set Failure domain and Domains to tolerate. Host reserve uses the largest entered hosts first, OSD reserve uses average OSD size, and custom reserve uses the capacity block you enter.
- Enter OSDs per node, Target PG per OSD, and Data pools to produce Suggested PG per pool and Estimated PG shards per OSD.
- Adjust OSD nearfull target. Use the Advanced section only when the scenario needs custom Backfillfull ratio, Full ratio, OSD or metadata overhead, CRUSH imbalance skew, or Accept degraded PGs.
- If the summary reports topology risk, a raw shortfall, or a failed recovery check, open Recovery Briefing and Cluster Guardrails. Fix the failed spread, fault-tolerance, or nearfull assumption before treating Usable capacity as a plan.
Interpreting Results:
The headline Usable capacity is protected data capacity at Nearfull used. It is lower than raw storage because the estimate first deducts overhead and skew, then keeps reserve for the selected failure target, then applies the protection efficiency.
A green summary means the entered numbers satisfy the calculator's reserve, fault-tolerance, threshold-order, and spread checks. It does not prove that a live CRUSH rule, device class, balancer state, or pool autoscaler will agree.
- Nearfull used below Requested nearfull means the recovery-safe clamp lowered the operating fill point.
- Extra raw for requested nearfull above zero means more raw capacity is needed to keep the requested nearfull value without degraded override.
- Minimum spread must be less than or equal to Available spread for host or OSD failure-domain models.
- Scheme Matrix compares protection profiles on the same raw estate. A higher usable number is useful only when its fault tolerance and spread verdict also fit.
Technical Details:
Ceph pools store RADOS objects through placement groups, and CRUSH maps those placement groups to OSD sets. In a replicated pool, each placement group is placed on as many OSDs as the pool size requires. In an erasure-coded pool, each object is split into k data chunks and m coding chunks, so the pool needs enough distinct placement targets for k+m chunks.
Capacity math starts with node capacities expressed in one selected unit. The unit selector labels the numbers and results; it does not convert mixed SI and IEC values. Keep all entries in the same unit before comparing TB, TiB, PB, or PiB scenarios.
The recovery reserve is a raw-domain estimate. Host reserve sums the largest entered hosts first, OSD reserve multiplies the selected failure count by average OSD size, and custom reserve multiplies the failure count by the custom block size. That reserve is compared with raw capacity after overhead and skew, which is why uneven host sizes or a large imbalance assumption can lower the recommended nearfull point quickly.
Formula Core
The main capacity path computes effective raw capacity, chooses the nearfull value used for the estimate, and applies protection efficiency.
In the formula, C is each entered node capacity, O is OSD or metadata overhead, S is CRUSH imbalance skew, R is the selected raw-domain reserve, and E is protection efficiency. When Accept degraded PGs is on, Nearfull used keeps the requested nearfull value instead of applying the recovery-safe minimum. That mode is useful for emergency what-if comparisons, but it intentionally removes the reserve clamp.
| Protection model | Efficiency | Healthy fault tolerance | Minimum spread |
|---|---|---|---|
| Replication n | 1 / n | n - 1 domains | n domains |
| Erasure coding k+m | k / (k + m) | m domains | k + m domains |
Placement group planning is a sizing hint, not a replacement for Ceph autoscaling. The calculator rounds the suggested pool value to a power of two after estimating how many PG shards each OSD would carry.
The shard factor is the replica count for replicated pools and k for erasure-coded pools. Estimated PG shards per OSD then multiplies the suggested pool count by data pools and shard factor, and divides by the estimated OSD count.
| Check | Boundary used | Meaning when it fails |
|---|---|---|
| Failure target fit | Domains to tolerate <= healthy fault tolerance | The selected profile cannot stay healthy through that many losses. |
| Spread fit | Available spread >= minimum spread | The chosen failure-domain model has too few placement targets. |
| Nearfull reserve | Requested nearfull <= recommended nearfull | The requested fill target leaves too little recovery reserve. |
| Threshold ordering | Nearfull < backfillfull < full | The calculation raises backfillfull and full as needed to keep the order valid. |
A six-node cluster with 10 TB per node, Replication 3x, one host reserve, no overhead, and no skew has 60 TB raw effective capacity. One host reserve is 10 TB, so recommended nearfull is 1 - 10 / 60 = 83.33%. Usable capacity is then 60 x 0.8333 x 1/3, or about 16.67 TB. The same reserve under EC 4+2 applies 4/6 efficiency and produces about 33.33 TB, but only when six suitable placement domains exist.
Accuracy Notes:
The estimate uses the values you enter. It does not read live OSD utilization, CRUSH weights, device classes, pool quotas, snapshots, compression ratios, balancer state, or autoscaler recommendations.
- Use OSD or metadata overhead when marketed drive size differs from usable OSD capacity after formatting and BlueStore overhead.
- Use CRUSH imbalance skew when mixed devices, uneven weights, or known hot OSDs make ideal distribution too optimistic.
- Treat Accept degraded PGs as an emergency what-if. It keeps the requested nearfull target even when the recovery-safe clamp would lower it.
- Confirm Suggested PG per pool with live Ceph autoscaler status before changing production
pg_num.
Worked Examples:
A six-host replicated pool with Total nodes set to 6, Capacity per node set to 10 TB, Replication 3x, one host in Domains to tolerate, and 85% requested nearfull reports Nearfull used near 83.33%. Usable capacity is about 16.67 TB because one 10 TB host has to remain outside the used raw portion.
The same six hosts under EC 4+2 can show roughly double the Usable value in Scheme Matrix because efficiency changes from 1/3 to 4/6. The comparison is acceptable only if Minimum spread is 6 and Available spread is also at least 6.
A mixed fleet with two 18 TB hosts and four 10 TB hosts should use Heterogeneous nodes. If the plan shows Extra raw for requested nearfull, lower OSD nearfull target, add raw capacity, or reduce the tolerated-domain target before relying on the headline usable number.
FAQ:
Why is usable capacity lower than raw storage?
The calculator reduces raw storage by overhead, skew, nearfull policy, failure reserve, and protection efficiency. Replication 3x turns only one third of the used raw capacity into protected data.
Does changing TB to TiB convert my numbers?
No. The selected unit labels the entered capacities and the results. Convert the numbers yourself before comparing decimal units such as TB with binary units such as TiB.
What should I do when spread fails?
Compare Minimum spread with Available spread. Add placement domains, choose a narrower protection profile, or verify the custom CRUSH path manually if the failure domain is custom.
Why is requested nearfull different from nearfull used?
Nearfull used is clamped down when the requested nearfull value leaves too little recovery reserve. Accept degraded PGs disables that clamp for emergency what-if modeling.
Can I use the PG result directly in production?
Use Suggested PG per pool as a planning starting point only. Check Ceph autoscaler status and the live pool's CRUSH rule before changing production placement groups.
Glossary:
- OSD
- An Object Storage Daemon that stores Ceph data and participates in placement, recovery, and rebalancing.
- Placement group
- A pool shard that maps many objects to a set of OSDs through CRUSH.
- CRUSH
- The Ceph placement system that chooses OSDs according to topology, weights, and rules.
- Nearfull
- The operating fill threshold used for the capacity estimate and early warning logic.
- min_size
- The minimum replica or chunk count needed for degraded I/O to continue.
- Failure domain
- The unit expected to fail together, such as a host, one OSD, or a custom capacity block.
References:
- Pools, Ceph Documentation.
- Erasure code, Ceph Documentation.
- Placement Groups, Ceph Documentation.
- Health Checks, Ceph Documentation.