Why Off-the-Shelf AI Fails the Enterprise: Building Custom Skills and Context on Gemini Enterprise Agent Platform
95% of GenAI pilots fail to scale. Learn how Custom Skills and MCP Context on the Gemini Enterprise Agent Platform bridge SAP demos and production agents.
Why Most Enterprise AI Pilots Never Leave the Conference Room
Eighty percent of enterprise AI projects fail to deliver their intended business value (Pertama Partners, 2025). The failure rate isn't a technology problem. Gemini, GPT-4, and Claude are all remarkably capable. It's an architecture problem. Teams see a compelling demo, replicate the demo's approach: one massive system prompt stuffed with every policy, API spec, and edge case the agent might ever need, then watch it collapse under real enterprise load, real SAP data, and real governance requirements.
What makes a production-grade enterprise agent different from a compelling demo? The answer isn't a better model or a smarter prompt. It's a fundamentally different architecture, one that separates what an agent needs to know permanently from what it needs to look up right now.
This post explains the root cause, the fix, and what it actually takes to build an agent that can survive contact with SAP: the most data-dense, most customized, most governance-constrained system in the enterprise.
Key Takeaways
- 95% of enterprise GenAI pilots fail to scale beyond initial demonstrations (Talyx AI, 2025)
- Monolithic system prompts create a "context tax" that inflates token costs, slows responses, and degrades reasoning at scale
- Skills define what an agent knows how to do; Context provides the real-time data it acts on. Both are required for production.
- 98% of Fortune 100 companies run SAP; every installation is uniquely customized, and generic agents can't navigate that landscape
- The enterprise AI integration consulting market reaches $14B in 2026; partners who bridge this gap own the most durable position in the market
What Is the "Context Tax" — and Why Is It Killing Your Enterprise AI Rollout?
Only 11% of enterprises currently run AI agents in production, despite 85% having adopted them in some form (Index.dev, 2025). That chasm between adoption and deployment has one dominant architectural cause: the "context tax," the practice of loading a single, monolithic system prompt with everything the agent might conceivably need, on every single request.
What does a context tax look like in practice? An order management agent gets a system prompt containing: order creation rules, return policies, customer segmentation logic, credit check procedures, SAP OData (Open Data Protocol) spec, custom Z-table mappings, error codes, escalation paths, data formatting requirements, and compliance guardrails. That's 35,000 tokens before the user types a single word.
Every request now carries that full payload. Token costs multiply. Latency climbs, because models take longer to process large contexts. And reasoning quality degrades, because the model must locate relevant instructions within a wall of noise it wasn't designed to parse efficiently.
The "just in case" prompt engineering mindset is the single biggest destroyer of production viability. It feels like thoroughness. In practice, it's architectural debt that compounds with every new requirement added to the prompt.
The enterprises that cracked the production problem didn't write better prompts. They changed the architecture entirely, separating what the agent needs to know all the time from what it needs to know right now.
For guidance on measuring your current context window costs against SAP workloads, see Agentic Balance Sheet: A2A, API Policy, and Your AI Budget.
What Are Skills and Context — and Why Does the Distinction Matter for Production?
Modular AI architectures deliver 43% faster feature delivery and 22% lower infrastructure costs compared to monolithic alternatives (Shaped AI, 2025). That's not a theoretical advantage; it's the measurable outcome of separating agent capability into two distinct layers that each carry only what's needed, when it's needed.
The Gemini Enterprise Agent Platform builds on this principle with Skills and Context.
Skills: The Agent's Domain Expertise
Skills are reusable, file-based instruction modules. Think of them as domain-specific playbooks stored on disk that give an agent specific procedural knowledge. The open SKILL.md format makes them versionable, testable, and composable like any other code artifact. A Skill might define:
- The exact sequence of OData calls to post a goods receipt in your specific SAP landscape
- The validation rules your procurement team enforces before releasing a purchase requisition above €50,000
- How to interpret the custom Z-error codes your S/4HANA instance throws on a failed credit check
- The escalation workflow when a delivery dispute falls above a regional manager's approval threshold
The critical property is that Skills load on demand. When a user asks the agent to process a return, the sap_return_handling Skill loads into context. When they ask about a payment dispute, the sap_dispute_resolution Skill loads. Every other Skill stays on disk. The active context stays lean.
Pattern recognition: The on-demand loading model mirrors how skilled SAP consultants actually work. They don't hold the entire SAP documentation in working memory. They retrieve the right procedure for the current task, apply it, and set it aside. The Skills model makes that cognitive efficiency an architectural property of the agent itself.
Why do off-the-shelf Skills fail? A generic "create sales order" Skill from a public template library doesn't know your company's custom approval thresholds. It doesn't know your region's tax handling logic, your Z-tables for customer credit classification, or which BAPI (Business Application Programming Interface) variant your specific S/4HANA release exposes. Generic Skills are starting points, not production assets. They need to be rewritten for every client's specific landscape before they're deployable.
Context: The Agent's Real-Time Intelligence
Context is the live data that makes a Skill actionable right now, for this customer, in this transaction. The Skill knows how to run a credit limit check. Context tells the agent the actual credit limit, specifically for customer 0001234567, in company code DE01, against the live S/4HANA production instance, at the moment the request fires.
Context flows through two mechanisms on the Gemini Enterprise Agent Platform:
Model Context Protocol (MCP): The open standard for connecting agents to live data sources and tools. Sixty-seven percent of CTOs have named MCP their default agent-integration standard (Zuplo State of MCP Report, 2026). An MCP server positioned in front of SAP enables real-time queries: inventory levels, customer master data, open AR items, shipping statuses, all without moving data out of SAP or creating a parallel silo. The agent queries; SAP answers; nothing is duplicated.
Memory Bank: Persistent state across multi-turn, multi-day workflows. A dispute resolution that spans three days and four system touchpoints isn't a single conversation. Memory Bank keeps the thread coherent without requiring expensive SAP re-queries at every step, and without losing the audit trail that compliance teams will demand.
Skills vs. Context at a Glance
| Skills | Context | |
|---|---|---|
| What it is | Stored procedure in SKILL.md | Live data via MCP or Memory Bank |
| When it changes | When your business rules change | On every request, in real time |
| Loaded into memory? | No — on demand only | Yes — injected per request |
| SAP example | BAPI sequence for goods receipt | Current credit limit for customer 1234567 |
| Without it... | Agent hallucinates the right procedure | Agent acts on stale or fictional data |
According to the Zuplo State of MCP Report (April 2026), 78% of enterprise AI teams have at least one MCP-backed agent running in production, with Fortune 500 adoption accelerating from 12% in 2024 to 28% in Q1 2025. This rapid trajectory confirms that MCP isn't an experimental protocol anymore. It's the current standard for enterprise agent data access, and any production SAP integration should be built on it.
For the technical walkthrough of wiring MCP to SAP OData, see Wiring an LLM Agent to S/4HANA OData Without Losing Your Mind.
Why Does Every SAP Integration Require Custom Skills — and Never Generic Agents?
Ninety-eight percent of Fortune 100 companies use SAP, and 78% of all large enterprises run at least one SAP module (HL SAP Industry Overview, 2024). SAP is the gravitational center of enterprise data, the system of record for finance, procurement, logistics, HR, and manufacturing across 450,000+ organizations worldwide. It's also the hardest system in the enterprise to connect to an AI agent, for three reasons that no generic off-the-shelf agent can navigate.
Reason 1: SAP is not standard across installations. Every SAP instance reflects 20+ years of accumulated customization. Custom Z-tables define non-standard data structures. Modified BAPIs (Business Application Programming Interfaces) expose different call signatures than the public documentation. Custom approval workflows route transactions through company-specific paths. Non-standard error codes carry meanings unique to that company's configuration. A generic agent built against SAP's public API documentation will fail on the first real customer call.
Reason 2: SAP access is tightly governed. SAP data includes financial positions, customer PII, pricing strategies, and procurement data that's often M&A-sensitive. Enterprises grant API access cautiously, require audit trails for every agent interaction, and expect compliance with SOD (Segregation of Duties) controls that have no equivalent in generic AI platforms. Agents that don't carry cryptographic identity (Agent Identity) and operate through governed API gateways (Agent Gateway) won't survive security review.
Reason 3: SAP is operationally unforgiving. A misconfigured API call doesn't return an error you can gracefully retry. It can post a duplicate goods receipt, trigger an unintended payment run, lock a customer account, or create a financial posting that requires manual reversal. Agent error handling for SAP must be deterministic, not probabilistic. Retrying a failed BAPI call without understanding the failure mode can cause exactly the damage you were trying to prevent.
This is the environment that Skills and Context are built to handle.
A custom sap_dispute_resolution Skill doesn't just know the generic dispute workflow. It knows your specific OData endpoint for pulling open AR items from company code DE01. It knows that disputes above €10,000 route through a regional approval workflow stored in Z-table ZCREDIT_APPROV. It knows that error code AR_DISPUTE_007 means the document is locked by a dunning run, not by another user. That granular knowledge doesn't exist in any public skill library. It's built by a partner who understands both the Gemini platform and the client's specific SAP configuration.
The Context layer grounds that Skill in live data through a secure MCP server: the customer's current outstanding balance, 90-day payment history, the SAP document number of the disputed invoice, the current dunning level, and the credit limit remaining. The agent acts on real operational data, not static test records or hallucinated values.
From the field: In production SAP agent deployments, the gap between a working demo and a production-ready deployment consistently comes down to three things: correct BAPI error handling, Z-table awareness, and real-time context injection. Generic agents miss all three. The first real BAPI error in production exposes every assumption the demo glossed over.
For a deeper look at the OData and MCP connection patterns, see Wiring an LLM Agent to S/4HANA OData Without Losing Your Mind.
The Last Mile: Why Google Cloud Partners Own the Most Valuable Position in This Market
The enterprise AI integration consulting market reaches $14 billion in 2026, up from $11 billion in 2025 (Mordor Intelligence, 2025). That growth is driven by a gap that no platform vendor can close from the product side alone: the distance between what Gemini can theoretically do and what it takes to make it work inside a specific enterprise's SAP landscape.
Forty-two percent of companies abandoned most of their AI initiatives in 2025, up sharply from 17% the year before (Folio3 AI, 2025). That abandonment wave reflects enterprises hitting the wall of implementation complexity without the right expertise to break through. For Google Cloud partners, that wall is an asset, not an obstacle.
So where does partner value actually live in this model?
Capability Scaffolding with ADK and agents-cli
The Agent Development Kit (ADK) and agents-cli give partners a rapid scaffolding layer for building, testing, and deploying agent networks at client engagements. Rather than building infrastructure from scratch for each project, partners template the scaffolding and focus customization work on the Skills layer, specifically the client-specific components. An engagement that took three months to deliver the first time takes three weeks on the fifth, because the reusable infrastructure is already battle-tested.
Building Proprietary Skill Libraries
The highest-leverage partner asset isn't billable services hours. It's a proprietary Skill library. A partner that builds a sap_retail_supply_chain Skill library covering demand forecast review, supplier communication workflows, and inventory reconciliation owns IP they can deploy for a new client in days rather than months, at margins that pure services work can't reach.
The specialization opportunities are real: Retail SAP Supply Chain Skills, Manufacturing Maintenance and MRO Skills, Finance Period-Close Skills, Public Sector Procurement Compliance Skills. Each vertical represents a distinct addressable segment within SAP's 450,000+ customer base, and a Skill library that covers one vertical well can anchor an entire practice.
Enforcing Governance That Enterprises Actually Require
Enterprise SAP integrations don't just need to work technically. They need to satisfy legal, security, and compliance teams before they go anywhere near production. Implementing Agent Identity (cryptographic authentication that gives each agent a verifiable ID) and Agent Gateway (governed, auditable routing for every SAP API call) isn't something a customer's IT team can configure quickly. Partners who make enterprise governance a defined deliverable, not an afterthought discovered during security review, close larger deals and maintain longer engagements.
Enterprise buyers already know Gemini exists. What they need to see is evidence that a partner can make it work inside their specific SAP landscape. That requires Skills, not slides.
For the broader implementation lifecycle these Skills fit into, see Agentic SAP: Google Cloud + BTP Implementation Lifecycle.
The market is waiting. Start by auditing your customers' three highest-friction SAP workflows: order-to-cash exceptions, procurement approval backlogs, or dispute resolution queues. Map the exact OData or BAPI calls each workflow requires. Build the Skill. Ground it with MCP Context. Deploy with Agent Identity. That sequence, executed by a partner who owns both the platform expertise and the SAP domain knowledge, is the last-mile integration that enterprises can't build themselves.
See 5 Google Cloud Agent Scenarios for SAP Integration for real-world workflow breakdowns with economics.
Frequently Asked Questions
What is the difference between Skills and Context in the Gemini Enterprise Agent Platform?
Skills are reusable, file-based instruction sets (in the open SKILL.md format) that define what an agent knows how to do: specific procedures, API call sequences, validation rules, and error handling logic. Context is the dynamic, real-time data provided through MCP servers and Memory Bank that anchors those procedures in the current transaction. Skills without Context produce agents that know procedures but can't act on real data. Context without Skills produces agents drowning in data but lacking procedure. Both layers are required for production-grade performance.
Why can't enterprises just use a large system prompt instead of modular Skills?
A monolithic system prompt creates a context tax that compounds with scale. Token costs accumulate on every request regardless of relevance. Latency grows as context size grows. And model reasoning quality drops as the signal-to-noise ratio in the prompt decreases. Modular Skills load only when the current task requires them, keeping active context lean and focused. Research confirms that modular architectures deliver 43% faster feature delivery and 22% lower infrastructure costs compared to monolithic alternatives (Shaped AI, 2025), and the advantage widens as enterprise workloads scale.
What is Model Context Protocol (MCP) and why is it critical for SAP integrations?
MCP is an open standard for providing AI agents with real-time, governed access to external data sources and tools, without hardcoding API calls into the agent itself. For SAP, an MCP server provides secure query access to live credit limits, inventory positions, open AR items, and shipping statuses without requiring data to leave the SAP environment or be duplicated into a separate store. As of April 2026, 78% of enterprise AI teams run at least one MCP-backed production agent, and 67% of CTOs have named MCP their default agent-integration standard (Zuplo State of MCP Report, 2026).
What is the Gemini Enterprise Agent Platform, and how does it replace Vertex AI?
The Gemini Enterprise Agent Platform is Google Cloud's consolidated environment for building, deploying, and governing enterprise AI agents, announced at Google Cloud Next 2026. It consolidates Vertex AI's ML infrastructure, Agentspace, Gemini Code Assist, and the Agent Development Kit (ADK) into a single product with per-agent pricing. Vertex AI branding is retired, but existing Vertex AI SDK integrations for SAP ABAP continue to function under the new platform. New capabilities include Agent Identity for cryptographic agent authentication and Agent Gateway for governed, auditable enterprise system access.
Why do SAP integrations always require custom Skills, not generic agents?
Every SAP installation reflects decades of accumulated customization: Z-tables, modified BAPIs, company-specific approval hierarchies, and non-standard error codes that bear no resemblance to SAP's public documentation. A generic agent trained on SAP's public API spec will fail the moment it encounters a real enterprise's landscape. A custom SAP Skill encodes the specific OData endpoints, BAPI call sequences, Z-table structures, and error handling logic for that customer's exact configuration. With 98% of Fortune 100 running SAP (HL SAP Industry Overview, 2024), the addressable market for those custom Skills is enormous.
The Blueprint Is Simple. The Execution Is Where Partner Value Lives.
Building a great enterprise agent isn't about using the biggest prompt. It's about building the smartest, most modular architecture. Skills make agents procedurally capable without inflating the active context. Dynamic Context makes those procedures actionable against real, live enterprise data. Together, they're the technical foundation for AI agents that can survive contact with real SAP configurations, real governance requirements, and real enterprise scale.
The Gemini Enterprise Agent Platform provides the infrastructure. Custom Skills and MCP-grounded Context provide the specificity that turns a capable platform into a production-ready integration. The enterprises that need both (all 450,000+ SAP customers) can't build that combination in-house at speed. They need partners who understand the platform, speak SAP, and can bridge the last mile between a compelling demo and a deployment that a CFO would sign off on.
The blueprint is: audit the friction, map the BAPIs, build the Skill, ground it with Context, deploy with governance. That sequence, executed well, is the most durable revenue position in enterprise AI integration for the next five years.
See 5 Google Cloud Agent Scenarios for SAP Integration for end-to-end workflow examples with cost and ROI breakdowns.
Gera Mats is an enterprise AI architect focused on Google Cloud and SAP integrations. He writes about agentic system design, MCP-based data access patterns, and the partner economics of AI-driven SAP transformation. Connect on LinkedIn.