Enterprise AI agents are running into an infrastructure problem. More than four in five organizations say their systems need upgrades to support production-grade agentic AI, according to a Google Cloud survey of 1,402 global IT leaders.
Google Cloud published the findings July 8 as companies move agents beyond controlled pilots. The report points to rising inference costs, operational complexity, governance demands, and power constraints as agents take on multistep work across business systems.
Agentic AI strains more than compute capacity
The 83% figure covers a broader set of requirements than accelerator capacity alone. In its 2026 infrastructure report, Google said 52% of respondents use hybrid multicloud architectures, while 48% prioritize infrastructure with strict data-residency controls.
Agentic applications may make repeated model calls, query databases, use external tools, and coordinate with other agents. Those workflows require systems to retain context and manage activity across multiple steps.
Edge deployment is also moving into enterprise AI plans. Ninety percent of respondents called it important to their initiatives, including 72% who rated it extremely or very important.
Costs extend beyond computing capacity. Sixty-two percent reported what Google calls a significant “inference tax” involving data-egress fees, excess storage, and idle specialized hardware. Another 81% cited operational complexity as a hidden scaling cost.
Security and governance remain major obstacles. Seventy-nine percent identified security, governance, and machine learning operations as their leading challenge.
Power is influencing hardware decisions as well. Ninety-one percent said they consider power consumption when selecting hardware, and 61% rated it a primary or significant factor. The proposed Fife AI data center shows how grid capacity and planning approval can shape where new AI infrastructure is built.
Production readiness requires new controls
Infrastructure teams need to determine how agents will preserve state, recover from failures, and maintain context during long-running workflows. Systems designed for demonstrations may lack the monitoring and persistence required for live business applications.
Governance reviews should cover agent identities, permissions, approval requirements, and audit logs. Agents that can read email, query databases, or change records should receive only the access required for each task. The rise of agentic ransomware capable of multistep attacks adds urgency to tightly scoped credentials and continuous monitoring.
Workload placement also depends on latency, resilience, security, and data-residency requirements. Some applications may need to remain on-premises, at the edge, or within specific cloud regions.
Google said in its report overview that 78% of respondents now obtain generative AI products through their primary cloud provider, up 30 percentage points from 2025. Consolidation can simplify procurement and governance, but it can also increase dependence on one provider’s models, data services, and orchestration tools. Anthropic’s $19 billion TeraWulf deal shows how AI companies are also securing dedicated computing and power capacity outside traditional cloud arrangements.
Google recommends flexible computing resources, centralized agent governance, unified data access, and support for hybrid and edge deployments. Those recommendations align with products Google Cloud sells, so enterprises should compare competing architectures for portability, cost, and vendor lock-in.
Moving agents into production will require organizations to set governance rules, plan for failures, control where data is processed, and determine whether recurring infrastructure costs can be justified.
Read more: A heatwave-related Cambridge supercomputer outage shows why cooling and resilience belong in AI infrastructure planning.