TL;DR
AI infrastructure capacity is no longer measured credibly by GPU count alone. A reserved accelerator becomes usable production capacity only when the organization can also provide rack power, cooling, network and storage throughput, facility headroom, grid or on-site generation, permits, water strategy, operational support, and community acceptance.
The practical CIO model is to treat AI capacity as a chain of constrained gates. The lowest available gate sets the real capacity ceiling. A business may have thousands of GPUs under contract and still have less deployable capacity than expected because power delivery, liquid-cooling readiness, network buildout, or permitting is late.
CEOs and CIOs should therefore make AI workload placement, data-center sourcing, sustainability, and community engagement part of the same capacity decision. The question is not simply, “How many GPUs can we buy?” It is, “How much reliable AI work can we operate, where can we operate it, and what physical and social permissions must be in place first?”
Introduction
Enterprise AI planning has moved past the point where infrastructure can be treated as a background service. Training clusters, high-volume inference, retrieval systems, model-serving platforms, and agent workloads are beginning to create power and cooling requirements that influence site selection, contract structure, capital allocation, and delivery timelines.
The scale of the wider market explains why. The U.S. Department of Energy’s 2024 data-center energy report estimated that data centers consumed about 4.4 percent of U.S. electricity in 2023 and could account for approximately 6.7 to 12 percent by 2028. In June 2026, the Federal Energy Regulatory Commission directed all six regional grid operators under its jurisdiction to justify or reform how data centers and other large loads connect to the transmission system.
This is not an energy-sector issue that CIOs can delegate after the architecture is selected. It changes whether an AI platform can be delivered at all.
Bloom Energy’s 2026 industry research described power availability as the defining constraint on data-center growth. Its annual report also found a material gap between when developers expect power and when utilities believe they can deliver it. The mid-year update added another gate: community scrutiny over electricity prices, water consumption, and grid reliability is increasingly shaping which projects move forward.
The operational lesson is straightforward. AI capacity must be translated from business demand into physical infrastructure and then through the external approvals required to operate that infrastructure. Each translation introduces a constraint, an owner, a lead time, and evidence that must be available before the next investment decision.
Current-State Planning Versus Capacity-Aware Planning
Many AI infrastructure plans still begin with a model-size estimate, convert that estimate into GPU demand, and then move directly into procurement. That approach may work for a limited cloud experiment. It is inadequate for a production platform that requires firm capacity, predictable service levels, controlled data placement, and a multiyear operating model.
| Planning dimension | Common current state | Capacity-aware target state |
|---|---|---|
| Demand signal | Number of use cases or requested GPUs | Workload classes, service levels, concurrency, growth, and business criticality |
| Compute | Accelerator model and quantity | GPU, CPU, memory, topology, scheduler, software, resilience, and spare capacity |
| Facility | Existing rack space assumed available | Qualified rack density, power path, cooling path, floor loading, and maintenance access |
| Power | Utility service treated as a fixed input | Firm megawatts, delivery date, interconnection studies, rate structure, curtailment, and on-site options |
| Network and data | Connectivity reviewed after site selection | Fabric, storage, WAN, data gravity, replication, and latency included in placement decisions |
| Environmental impact | Annual sustainability reporting | Site-level energy, water, emissions, noise, and equipment lifecycle controls |
| Community and permitting | Facilities responsibility | Executive delivery gate with local engagement, approvals, and mitigation commitments |
| Capacity reporting | GPUs purchased or reserved | Useful production work supported within all physical and operational constraints |
The target state is not a facilities project with an AI label. It is a joint operating model across business sponsors, application owners, AI platform teams, infrastructure, network engineering, data-center operations, finance, procurement, sustainability, legal, utilities, and local stakeholders.
What a Megawatt Actually Means to an AI Program
A megawatt is a measure of instantaneous power capacity. A megawatt-hour is a measure of energy consumed over time. AI programs need both a sufficient power ceiling and a dependable energy supply, but the terms are often blended in executive discussions.
The second distinction is between IT load and total facility load. GPUs, CPUs, memory, storage, and switches consume the IT load. Cooling systems, pumps, power conversion, lighting, and other supporting systems add facility overhead. Power Usage Effectiveness helps describe that relationship, but it does not replace an engineered design for redundancy, derating, maintenance states, harmonics, startup behavior, or future expansion.
As a rough planning illustration, 50 racks designed for 100 kilowatts each create a 5-megawatt rack-level IT envelope. That does not mean a 5-megawatt utility service is sufficient. The site still needs to account for cooling and electrical overhead, network and storage systems, redundancy, maintenance states, growth headroom, and any limits placed on the interconnection.
A CIO should ask four different questions when someone says the organization has secured 20 megawatts:
- Is that 20 megawatts of utility service, facility capacity, or usable IT load?
- Is the capacity firm, interruptible, phased, or dependent on a future grid upgrade?
- Does it remain available during maintenance or the failure of a power path?
- Is the cooling plant capable of removing the corresponding heat at the planned rack densities?
Without those answers, the number is not a production-capacity commitment. It is a planning assumption.
The AI Capacity Translation Model
The diagram below shows the core model. The important point is that each gate can reduce, delay, or relocate the capacity that reaches production. No downstream team can recover capacity that was never secured upstream.
The real capacity ceiling is the minimum available capacity across these gates. If the GPU contract supports 40 megawatts of IT load but the qualified cooling plant supports 24, the usable ceiling is no more than 24 before other constraints are considered. If the site can cool 24 but the interconnection initially delivers 12, the near-term ceiling falls again.
This is why AI infrastructure plans need a capacity dependency map rather than a single procurement schedule.
Reserved GPU Capacity Is Not Usable Production Capacity
GPU reservations matter, especially when accelerator supply is constrained. They are only one part of the delivery chain.
| Capacity layer | What may be reserved or purchased | What must be proven before production |
|---|---|---|
| Accelerator | GPU systems, cloud instances, hosted clusters | Correct model fit, memory fit, topology, drivers, software support, and delivery date |
| Rack | Cabinet positions or deployed systems | Qualified rack power, busway, floor loading, cable paths, service clearance, and fire controls |
| Cooling | Nominal cooling tonnage or provider commitment | Supported inlet conditions, coolant supply, heat rejection, redundancy, leak detection, and maintenance procedures |
| Electrical | Utility allocation or facility nameplate | Firm delivery, energization date, protection studies, redundancy, power quality, and contract limits |
| Network | Circuits and switch ports | End-to-end bandwidth, latency, loss, fabric scale, WAN path, and operational monitoring |
| Data | Storage capacity | Throughput, metadata performance, ingestion, replication, governance, and recovery |
| Site | Land, building, or colocation suite | Permits, environmental conditions, water strategy, construction readiness, and community acceptance |
| Operations | Support contract and platform tooling | Trained staff, spares, runbooks, lifecycle control, observability, incident response, and recovery testing |
A useful executive metric is not “GPUs under contract.” It is “qualified AI capacity available by service class and date.” That metric should report the limiting gate, not hide it.
Power Availability and Interconnection Lead Times
For a traditional enterprise data center, power was often treated as a stable facility constraint. For large AI deployments, it is becoming a location and sequencing decision.
Bloom Energy’s 2026 annual survey reported that utility respondents expected time-to-power to take roughly 1.5 to 2 years longer on average than hyperscalers and colocation providers expected. The same report described widening expectation gaps in Northern Virginia, the Bay Area, and Atlanta. This is vendor-sponsored survey evidence rather than a universal utility benchmark, but the planning implication is still important: the party buying compute and the party delivering power may be working from materially different dates.
The June 2026 FERC action reinforces the point. The Commission required six regional grid operators to address study processes, cost transparency, co-location and behind-the-meter generation, flexible large loads, and the relationship between large loads and nearby generation. Interconnection is therefore not only a construction task. It is also a tariff, cost-allocation, reliability, and policy process that varies by region.
A credible AI power plan should distinguish at least five milestones:
- Power identified: A utility, developer, or provider believes capacity may be available.
- Power studied: The load and its grid impact have entered the appropriate study process.
- Power contracted: Commercial terms, upgrade obligations, and delivery conditions are documented.
- Power energized: The physical service is available to the site.
- Power qualified: The service has passed commissioning, resilience, and workload acceptance tests.
Only the final milestone should be counted as usable production capacity.
Rack Density and Cooling Readiness
High-density AI racks change the facility design, not just the server bill of materials. NVIDIA’s GB200 NVL72 reference design illustrates the direction of travel: NVIDIA described a rack requiring 120 kilowatts of cooling capacity and using direct liquid cooling.
That does not mean every enterprise AI rack will operate at that density. It means average rack-density assumptions from general-purpose virtualization environments can no longer be used safely for AI planning.
A liquid-cooled deployment may require:
- Facility water loops or an approved alternative heat-rejection design
- Coolant distribution units and appropriate redundancy
- Manifolds, quick disconnects, filtration, water chemistry, and leak detection
- Warm-water supply and return temperature designs that match the equipment
- Controls integration across the building management system and data-center infrastructure management platform
- Maintenance procedures for mixed air-cooled and liquid-cooled components
- Spare parts, trained technicians, and vendor escalation paths
- Commissioning under realistic heat load rather than an empty-room inspection
Retrofitting an existing room can be more complex than placing new racks. The electrical path may be constrained by busway, breakers, switchgear, UPS modules, or generator capacity. The cooling path may be constrained by pipe sizing, pumps, heat exchangers, chillers, dry coolers, or available water. Floor loading, rack dimensions, rear-door access, and cable routing may also become hard limits.
The practical decision is not “air versus liquid.” It is whether the complete thermal path can remove heat reliably during normal operation, maintenance, and failure conditions.
Colocation Versus Private Facilities
Private facilities and colocation can both support enterprise AI, but they move different risks onto the organization.
| Decision factor | Private facility | Colocation | Public cloud or managed AI capacity |
|---|---|---|---|
| Time to first capacity | Slow when major power or cooling upgrades are required | Potentially faster when qualified high-density capacity already exists | Often fastest for initial capacity, subject to region, quota, and service availability |
| Design control | Highest | Shared with provider and contract boundaries | Lowest physical control, highest service abstraction |
| Rack-density customization | Strong when the site can be redesigned | Depends on suite, provider standard, coolant model, and available power blocks | Provider-defined |
| Capital profile | High capital commitment and long asset life | Contracted recurring commitment with buildout charges | Consumption and commitment-based operating expense |
| Permitting and community exposure | Direct organizational responsibility | Mostly provider-managed, but customer demand still influences expansion | Provider-managed, though regional constraints still affect availability |
| Data gravity | Strong for local enterprise data | Strong when connected to enterprise campuses and carrier ecosystems | Strong when data already resides in the same cloud region |
| Operational responsibility | Full facilities and platform stack | Platform stack plus provider relationship and shared facility processes | Platform, data, application, cost, and service governance |
| Exit complexity | Physical assets and specialized facility investment | Contract terms, migration windows, cross-connects, and data movement | Data egress, service dependencies, reserved commitments, and refactoring |
Colocation is not automatically a shortcut. A provider may have building-level megawatts available while lacking the exact rack density, liquid-cooling configuration, network fabric, or delivery window the AI design requires. Contract language should identify the supported kW per rack, cooling method, delivery temperature, redundancy model, maintenance rights, expansion blocks, power curtailment terms, metering, service credits, and responsibilities at every handoff.
Private facilities provide more control, but they also bring the organization into utility planning, equipment lead times, environmental review, construction risk, and community relationships. The decision should be based on qualified capacity and operating capability, not a simple preference for ownership or outsourcing.
On-Site Generation and Microgrids
On-site generation is moving from emergency backup into long-term data-center capacity strategies. Bloom Energy’s 2026 mid-year survey reported that 61 percent of developers planned to bring their own power when the grid could not meet their needs. The Department of Energy also identifies on-site generation, storage, demand flexibility, and reuse of existing energy infrastructure as part of the broader response to data-center load growth.
The business case can include faster time-to-power, improved resilience, predictable capacity expansion, and reduced dependence on a constrained interconnection. The architecture may combine utility service, generators or fuel cells, renewable generation, batteries, thermal storage, and a microgrid controller.
However, a microgrid does not remove external dependencies. It creates a different set of them:
- Fuel availability, pipeline capacity, or renewable resource variability
- Air-quality, noise, water, and land-use permits
- Emissions controls and sustainability commitments
- Black-start, islanding, synchronization, and protection requirements
- Maintenance staffing and spare-parts strategy
- Battery duration and degradation
- Utility coordination and export restrictions
- Community concerns about local environmental impact
A bridge-to-grid design also needs an explicit transition plan. Temporary power can become permanent through inertia, leaving the organization with higher operating costs, duplicated equipment, or an emissions profile that no longer matches corporate commitments.
The right question is not whether on-site generation is good or bad. It is whether the complete power portfolio is reliable, permitted, financeable, maintainable, and aligned to the organization’s sustainability and community commitments.
Network Capacity and Data Gravity
A power-advantaged site can still be the wrong AI site.
AI training and distributed inference depend on high-throughput, low-loss network fabrics, storage bandwidth, metadata performance, and predictable data movement. Moving a large cluster farther from constrained metro areas may improve access to power while increasing WAN costs, replication time, recovery complexity, user latency, or exposure to a small number of long-haul paths.
Data gravity adds another constraint. Sensitive datasets may already reside in a private facility, specific cloud, regional data platform, or regulated geography. The energy-optimal location may not be the data-optimal location. Moving the data can consume more time, money, and operational risk than moving the workload.
The placement decision should therefore evaluate three network layers:
- Inside the cluster: GPU fabric, east-west bandwidth, congestion control, and topology.
- Inside the site: Storage, ingestion, management, backup, observability, and tenant segmentation.
- Between sites: WAN capacity, cloud interconnects, replication, user access, recovery paths, and data-sovereignty boundaries.
Reserved GPU capacity without qualified network and data paths is stranded capacity.
Water, Environmental, and Community Constraints
Cooling choices affect both power and water. The Lawrence Berkeley National Laboratory estimated direct data-center water consumption of approximately 66 billion liters in 2023, with hyperscale and colocation facilities representing most of that total. Its report also emphasized that indirect water consumption varies with the regional electricity mix, which means a low-water cooling design can still carry a material water footprint through power generation.
Water impact is local. A design that is reasonable in a water-abundant region may be unacceptable in a stressed watershed. Annual corporate averages can obscure the effect of a specific project on local water, electricity prices, noise, air quality, roads, land use, and emergency services.
Bloom Energy’s June 2026 mid-year report said community scrutiny had intensified, and cited 18 proposed state bills and 86 proposed local moratoriums in the United States as of May 2026. Its survey was commissioned by a power vendor and was weighted toward U.S. respondents, so the figures should be treated as an industry signal rather than a complete public-policy census. Even with that limitation, the operating lesson is clear: community acceptance has become a schedule and capacity dependency.
A credible community and environmental plan should address:
- Who pays for grid, road, water, and public-service upgrades
- How residential and small-business ratepayers are protected
- Expected water withdrawal and consumption under normal and peak conditions
- Noise, emissions, lighting, backup-generation testing, and construction traffic
- Local jobs, tax base, training, and other durable community benefits
- Transparency on project phases, power sources, expansion limits, and mitigation commitments
- How complaints, incidents, and environmental data will be reported after the site opens
Community engagement should begin before the design is presented as final. A technically complete project can still fail when local stakeholders believe the costs are being externalized while the benefits remain abstract.
Workload Placement Based on Energy Availability
Energy availability should become an input to workload placement, but it should not become the only input. The useful model classifies workloads by latency, data gravity, criticality, scheduling flexibility, portability, and environmental constraints.
| Workload class | Power and placement characteristics | Likely placement pattern |
|---|---|---|
| Real-time customer inference | Low latency, high availability, limited curtailment tolerance | Near users or application services, often across multiple regions or sites |
| Regulated private inference | Strong data gravity, controlled trust boundary, predictable demand | Qualified private facility or colocation with firm capacity and governed data locality |
| Large training and fine-tuning | High power density, high network demand, often schedulable with checkpointing | Power-advantaged site or cloud region with strong fabric and data-staging capability |
| Batch embedding and indexing | Flexible scheduling, significant data movement, restartable | Site or time window selected for capacity, price, and energy conditions |
| Development and experimentation | Bursty, uncertain demand, lower utilization | Cloud, shared enterprise platform, or smaller private pools |
| Business-critical agents | Moderate compute but strong dependency on data, tools, identity, and uptime | Close to governed systems of record with resilient inference capacity |
Energy-aware placement can include shifting non-urgent training, embedding, evaluation, or synthetic-data jobs to sites or time windows with available capacity. It can also include model optimization, batching, quantization, caching, and workload admission controls that reduce the need for new physical capacity.
The caveat is portability. A workload cannot be shifted safely merely because another site has spare megawatts. The destination must have compatible models, data, network paths, identity, secrets, software, observability, licensing, and recovery behavior.
Build a CIO-Level AI Capacity Ledger
The executive team needs one evidence model that connects business demand to physical capacity. A capacity ledger should identify both what has been purchased and what is actually usable.
At minimum, the ledger should contain:
- Business sponsor and technical owner
- Workload class, service level, and growth forecast
- Accelerator type, topology, and reservation status
- Site, room, row, rack, and failure domain
- Rack kW, IT megawatts, and total facility capacity allocation
- Cooling method, design point, redundancy, and commissioning status
- Utility interconnection milestone and on-site generation dependency
- Network, storage, and data-location dependencies
- Permit, environmental, and community-engagement status
- Contracted date, qualified date, expiration, and expansion rights
- Cost per useful unit of work
- Energy, water, emissions, and hardware-utilization metrics
- Limiting gate and accountable owner
This ledger should be reviewed with the AI portfolio, not only with facilities. It is the bridge between board-approved AI investment and the production capacity that investment can actually deliver.
A Phased Implementation Strategy
The transition from GPU-centric planning to capacity-aware planning should be staged. The goal is to improve decision quality before the organization commits to the largest facilities and energy investments.
| Phase | Primary work | Evidence required to exit |
|---|---|---|
| Discover demand | Classify workloads, service levels, data gravity, growth, and flexibility | Approved workload classes, baseline utilization, demand ranges, and critical dependencies |
| Translate capacity | Convert workload demand into compute, rack, cooling, network, storage, and power envelopes | Capacity model with assumptions, ranges, headroom, and limiting-gate analysis |
| Qualify sites and providers | Compare private sites, colocation, cloud regions, utilities, and on-site power options | Written evidence for delivery dates, rack density, cooling, network, permits, and expansion rights |
| Pilot the full path | Deploy representative workloads and commission the supporting facility path | Measured performance, thermal behavior, power quality, failover, recovery, and operational acceptance |
| Contract and govern | Align procurement, utility, colocation, cloud, sustainability, and community commitments | Contract-to-capacity mapping, accountable owners, escalation paths, and review triggers |
| Scale and operate | Add capacity in controlled blocks and update placement policies | Capacity ledger, telemetry, utilization, environmental metrics, and gate-based expansion approvals |
The pilot should test more than model throughput. It should include a realistic thermal load, network saturation, storage behavior, power-path maintenance, coolant alarms, recovery, and workload restart. A successful benchmark on a vendor floor does not prove that the production site can operate the same system.
Most enterprises already own many of the required data sources. The problem is that they are separated across IT and facilities systems.
A practical integration pattern connects:
- AI schedulers and GPU telemetry
- Kubernetes, virtualization, and bare-metal management
- Data-center infrastructure management platforms
- Electrical power monitoring systems
- Building management systems
- Utility and colocation metering
- Network and storage observability
- Configuration management databases and asset inventories
- FinOps, procurement, contract, and sustainability data
The shared model should allow an operator to trace a business workload to the GPUs it uses, the racks those GPUs occupy, the power and cooling paths that support the racks, the site and provider contracts involved, and the environmental and community commitments attached to that capacity.
Automation should focus on evidence and control, not only utilization. Useful capabilities include:
- Admission control when a rack, cooling loop, or site approaches a qualified limit
- Energy-aware scheduling for workloads approved as flexible
- Automated comparison of reserved, installed, energized, qualified, and consumed capacity
- Alerting when utility, provider, permit, or construction milestones slip
- Chargeback or showback based on useful work and supporting facility cost
- Forecasting that includes delivery lead times and workload efficiency improvements
- Policy checks that prevent placement in a site that violates data, resilience, or environmental requirements
The objective is not to create a single dashboard with every number. It is to create a decision system that identifies which constraint is preventing the next unit of useful AI work.
Risks, Caveats, and Operational Gotchas
Several mistakes repeatedly make AI capacity look larger than it is.
Counting Nameplate Capacity as Usable Capacity
Building, UPS, generator, or utility nameplate values may not represent the power available to IT during normal operations, maintenance, or a component failure. Capacity should be qualified against the intended redundancy and operating state.
Double Counting Redundant Power Paths
A 2N design provides resilience, not twice the sellable IT capacity. The same caution applies to redundant cooling, network, and storage paths.
Using Average Rack Density
A room averaging 30 kW per rack may still be unable to support a concentrated row of 100 kW racks. Distribution paths and local heat rejection matter.
Treating Liquid Cooling as a Server Accessory
Liquid cooling crosses facilities, controls, operations, maintenance, water treatment, safety, and vendor-support boundaries. The ownership model must be explicit.
Treating On-Site Power as a Permitting Shortcut
On-site generation may reduce grid dependency while increasing air-quality, fuel, noise, land-use, and community obligations.
Assuming Cloud Capacity Has No Physical Constraint
Cloud services abstract the facility, but they do not eliminate regional power, cooling, quota, network, or sustainability constraints. Those constraints appear through availability, price, region choice, and service terms.
Ignoring the Network When Chasing Power
A cheaper or faster power location can create higher data-movement cost, longer recovery, weaker resilience, or unacceptable latency.
Measuring Sustainability Only at Portfolio Level
Annual renewable-energy matching or corporate averages may not explain the local impact of a specific site. Site-level energy, water, emissions, and community metrics are needed for defensible decisions.
Executive Decision Criteria
Before approving a major AI infrastructure commitment, CEOs and CIOs should be able to answer these questions:
- What business services and service levels justify the capacity?
- Which workload assumptions drive the peak and sustained power requirement?
- What is the difference between reserved accelerators and qualified production capacity?
- Which gate currently limits delivery: GPUs, racks, cooling, power, network, data, permits, or operations?
- When will power be studied, contracted, energized, and qualified?
- Can the facility support the planned rack density during maintenance and failure conditions?
- What placement options exist if the preferred site misses its power or permitting date?
- Which workloads can shift across sites or time without violating latency, data, or recovery requirements?
- What direct and indirect water, emissions, noise, and land-use impacts apply to each site?
- How are utility costs and infrastructure upgrades allocated without shifting unintended costs to the community?
- Who owns community engagement and the commitments made during approval?
- What evidence will trigger the next capacity block, and what evidence will stop it?
A board does not need to design a cooling loop. It does need confidence that the AI investment depends on a complete, owned, and validated capacity chain.
Conclusion
AI power has become a business capacity decision because the physical system now influences strategy, location, schedule, cost, sustainability, and public approval.
The practical mistake is to treat GPUs as the unit of capacity and everything else as supporting infrastructure. In production, the reverse is often true. Accelerators are one component inside a chain that includes rack power, cooling, network, storage, utility service, on-site generation, permits, environmental constraints, operating capability, and community acceptance.
The strongest CIO approach is to manage that chain as a capacity portfolio. Classify workloads, translate demand into physical requirements, qualify sites and providers, separate reserved capacity from usable capacity, and expose the limiting gate in every executive review.
This model also creates better workload-placement decisions. Latency-sensitive inference can remain close to users and data. Flexible training and batch workloads can move toward power-advantaged locations or time windows. Regulated workloads can remain inside qualified private boundaries. Cloud and colocation can be used deliberately rather than as emergency overflow after a private site misses its delivery date.
The next AI capacity review should not begin with a GPU count. It should begin with a workload service class, a megawatt and cooling envelope, a site-qualified delivery date, and evidence that the organization has permission to operate at the intended scale.
