Summary Bullets:

• After a period of ‘token maxxing,’ organizations are looking to reign in and better control inference costs.

• Instead, enterprises are now embracing ‘value-maxxing,’ which focuses on outcomes.
Last week, Google announced several enhancements to Gemini Enterprise designed to help enterprises obtain greater and faster ROI on their AI projects. The improvements address one of the biggest frustrations expressed by organizations today, namely that the benefits promised by AI are taking too long to realize. Companies are clamoring for domain specific solutions in order to speed the deployment, reduce the integration complexity, and increase the value obtained from AI projects. Additionally, business leaders are eager for better tools to help them manage AI costs. They are looking for improved visibility on token use and costs, more proactive spending controls, and more flexible payment options.

On Tuesday, August 25th, Google announced Gemini Enterprise for Financial Services and Gemini Enterprise for Legal. The industry-specific solutions include out-of-the box AI capabilities such as specialized agents that provide shortcuts for directing workflows, data connectors, and sector-optimized models. Reusable packages of instructions teach AI agents to perform specialized tasks that are customized to meet company-specific requirements; connectors link agents to internal systems and data while maintaining access controls; pre-built agents are available to deploy out of the box; and software provider partnerships facilitate industry-specific customization and integration, while avoiding vendor lock-in. Though initially rolled out for the financial services and legal industries, Google plans to offer similar solutions for other industries, including healthcare, life sciences, and professional services.

The following day, August 26th, Google revealed expanded tools for managing AI spending. It announced that Google Antigravity, its AI agent development platform, and Android Studio, for building applications, will now be included in Gemini Enterprise subscriptions. Usage across Antigravity, the platform, and the app rolls up into a single view instead of separate license and billing siloes. Furthermore, Google is providing expanded billing flexibility and new cost management tools for agent workloads across Gemini Enterprise. Customers can purchase a mix of per-seat subscriptions along with a new pay-as-you-go option, to help avoid hitting token caps in the middle of a job. Companies that commit to a minimum monthly spend will receive discounts on token costs. To better control spending, Google has rolled out new guardrails that enable administrators to set limits on AI spend by project, help estimate agent runtime costs, and identify anomalies in spending. Project level guardrails can pause an agent when API call limits are reached; a FinOps agent provides spending summaries in natural language.

Google’s announcements directly address concerns many organizations have over the spend on AI inference. After a period of ‘token maxxing’ wherein greater token usage was associated with greater productivity, organizations are looking to rein in and better control inference costs. Despite declining token costs, overall consumption, and therefore spend, are skyrocketing. Thus, the industry is now embracing ‘value-maxxing,’ which focuses on outcomes. It seeks to identify and quantify results, whether they be improved performance, more insightful decisions, or greater efficiency.

Regardless of the jargon of the day, organizations are taking a more analytical and practical approach to cost, latency, and performance optimization. No longer is the fastest or most expensive model considered the best choice for all tasks; organizations are now recognizing that some workflows are served well enough by less intensive reasoning, and that the same level of accuracy is not required for all tasks. At the same time, many are considering open-source strategies, attracted to the potential of lower costs, ability to fine tune models, greater transparency, local deployment options, and the option of leveraging existing infrastructure investments. At the end of the day, the development of appropriate AI strategies relies heavily on a broader understanding of the business and its operating model; professionals that can combine this knowledge with technical AI expertise are invaluable.