Lower AI model prices do not always mean lower infrastructure costs. Learn how agent activity, data movement, and workload placement affect enterprise AI spending.
Anant Dadu, Co-Founder of Zingle AI, expected lower costs when his team switched to an AI model that was 80% cheaper per token. Instead, the team's Snowflake data warehouse spending reportedly jumped from around $14,000 to $26,300 the following day, according to a LinkedIn post he shared. Dadu said the users, questions, and underlying data had remained the same.
At the time of the post, the team had not determined what caused the increase. More database queries, inefficient SQL, or additional retries could have played a role, but none had been confirmed.
This anecdote highlights a potential cost of agentic AI: the work agents generate across databases and other systems can add expenses that token pricing alone doesn't capture.
Why AI agents increase data query volumes
AI agents can break a single request into several steps, generating queries and tool calls as they work.
Consider an employee asking an agent why regional sales declined. The agent might examine transaction records, compare inventory levels, review historical trends, and investigate possible explanations. If the results are inconclusive, it may revise its queries or retrieve additional information.
Each step consumes infrastructure resources, even when the employee sees only the final answer. Cloudera used the analytics team's experience in a video to illustrate how data infrastructure costs can rise despite lower model prices.
Gartner predicted in August 2026 that inference costs per agentic workflow would increase more than fivefold through 2028 as AI applications become more sophisticated. The forecast concerns inference costs, but it illustrates how complex workflows can offset savings from cheaper tokens.
The hidden data bill behind cheaper AI
Model prices are easy to compare, but agent activity generates expenses across multiple enterprise systems. The table below identifies the main cost categories and what organizations should monitor.
Cost category | How AI agents generate expenses | What to monitor |
Model and tool usage | Inference, repeated prompts, orchestration, and API calls | Token spending and tool calls |
Data processing | SQL queries, vector retrieval, transformations, and warehouse compute | Query volume, data scanned, and compute usage |
Infrastructure overhead | Storage, data transfers, synchronization, and supporting services | Transfer charges, storage costs, and retries |
A query that produces an unusable answer still consumes processing resources. Repeated searches can become particularly expensive when agents scan large datasets or execute inefficient SQL.
The FinOps Foundation's October 2026 guidance also identified observability, evaluation tools, orchestration platforms, and vector databases as potential sources of unexpected AI spending.
Because these services may be purchased and billed separately, organizations can struggle to see the full cost of operating an AI application.
Cost visibility also depends on knowing which applications generate the activity. Tracking queries and infrastructure consumption by agent or workflow can help teams identify expensive operations and assign responsibility for spending before usage scales.
EY's September 2026 research estimated that the full enterprise cost of AI could reach roughly three times the token invoice after accounting for infrastructure, software, governance, organizational changes, and other expenses. The estimate is modeled and should not be treated as a universal multiplier.
How data location affects AI infrastructure costs
When agents access data across cloud and private environments, copying or transferring that information can increase storage and processing costs.
Moving customer records into a separate AI environment, for example, may simplify access but creates ongoing storage and synchronization requirements. Transfers between cloud platforms or regions can add further charges.
Organizations should assess whether agents can query governed data where it resides. Data volumes, access frequency, and infrastructure pricing will determine whether local access or movement is more economical.
Choose workload placement based on cost
Public cloud infrastructure offers flexibility for variable demand, while private environments may suit sustained workloads or sensitive data. Costs depend on utilization, available capacity, and infrastructure pricing, along with any data residency requirements.
McKinsey's October 2026 research on scaling AI inference emphasized the importance of matching deployment decisions to workload requirements.
"Compute for inference will be distributed across cloud, enterprise, edge, and device, with location dictated by economics, latency, and data control," the research stated.
Cloudera's Anywhere Cloud supports data and AI workloads across public clouds, private data centers, and sovereign environments. Its open lakehouse architecture allows compatible processing engines to access shared enterprise data, reducing the need to maintain separate copies for different workloads. This gives organizations more options for placing AI workloads near governed data and managing the infrastructure costs those applications generate.
Query optimization and caching can reduce repeated processing triggered by AI agents. Table management and limits on repeated operations can also improve efficiency. Teams should account for these factors when comparing deployment costs.
Measure AI ROI by completed tasks
Models with similar capabilities can generate different infrastructure costs depending on how many queries and retries they need to complete a task. IBM's guidance on AI agent token spend management encourages organizations to ask, "How much did we spend to accomplish the task?"
Calculate the total cost of a workflow, including failed attempts and human review, then divide it by the number of successfully completed tasks. This provides a more useful measure of AI operating costs.
Teams can compare the cost per completed task with the value it delivers. An agent that costs more to operate may still produce a better return if it reduces manual work or improves customer service. Automating repetitive data engineering or security tasks, for example, can free employees to focus on more complex problems.
Improve efficiency across the AI system
Before moving an AI agent from pilot to production, organizations should examine three questions:
- Where does the required data live? Identify the datasets agents need and whether accessing them involves unnecessary movement or duplication.
- How much work does each task generate? Measure the queries, transfers, retries, and processing needed to produce a successful result.
- Are workloads running in the right environment? Compare infrastructure costs, performance, and governance requirements to determine the most economical placement.
Monitoring workload activity can help teams identify expensive operations before they scale. As model prices fall, enterprises can expand AI adoption while keeping infrastructure consumption under control.
For a closer look at how enterprises are adapting their data infrastructure to support AI, explore Cloudera's research, The Great AI Re-Architecture.