SAP's Philipp Herzig on AI built for business data

Good morning, AI enthusiasts. AI coding tools can spin up an app in minutes, and plenty of people have taken that as a sign traditional software is finished. SAP’s CTO thinks they’re missing the harder part: running a business on that app.

In partnership with SAP, we sat down with CTO Philipp Herzig to talk about the “SaaS is dead” debate, how AI is changing the way people use work software, what agents need to know to get answers right, how to keep AI bills in check, and the models built for business data.

In today’s AI rundown:

• What the “SaaS is dead” narrative misses

• From clicking screens to stating intent

• Teaching AI how your business runs

• Cheaper tokens, better business outcomes

• Specialized AI for decisions that count

LATEST DEVELOPMENTS

INDUSTRY TAKE

🏢 What the “SaaS is dead” narrative misses

The Rundown: As AI coding tools let anyone build apps, many believe traditional software companies are over. Herzig thinks this narrative confuses building an app with running one, and says the same gap explains why so many AI pilots never make it to production.

Cheung: What’s one AI trend you think is overhyped?

Herzig: The idea that “SaaS is dead” because AI can now write software. It confuses the ability to generate an application with the responsibility of running it.

Generating an expense app is one thing. Running it across 40 countries, keeping up with tax rules, enforcing permissions, and reconciling reimbursements is another. As agents take on more autonomous work, they depend more on trusted data, business context, governance, and security. The interface can change radically. The need for reliable business processes doesn’t.

Cheung: Why do so many AI pilots never get deployed?

Herzig: The difference between a prototype and a production-ready system at enterprise scale, with all the technological dependencies and complexity under the hood while still looking simple for the user, is still substantial human work.

Cheung: What’s been harder at SAP, the technology or adoption?

Herzig: Adoption, without a doubt. Useful AI changes who makes which decisions and how. That affects expertise, authority, and professional identity.

The companies moving fastest gave access to people whose judgment the organization already trusted. A respected supply chain manager saying “this helped me catch a disruption” is more powerful than any training program.

Why it matters: SAP has an obvious stake in defending enterprise software, but the gap between generating an app and running a business on it is real. If you’re rolling out AI at work, his adoption advice applies regardless of vendor: pick one high-value use case and put the colleagues people already trust first in line.

AI IRL

💻 From clicking screens to stating intent

The Rundown: SAP’s newest AI push, Joule Work, flips how people use business software: instead of clicking through screens, you say what you want done. Herzig walked us through how it works, how he uses it himself, and what it means for developers.

Cheung: How does Joule Work change the day-to-day experience of using business software?

Herzig: For decades, enterprise software required us to learn its structure: which application to open, which screen to find, which buttons to click. With Joule Work, you start with what you want to accomplish. It brings together information and capabilities across applications.

But this isn’t about replacing every screen with a chat box. With Spaces, Joule can turn your intent into a personalized visual workspace: a chart, table, or dashboard you refine by asking. I don’t have to log into four systems, orient myself, and hope I’m looking at the right data. The relevant picture is already assembled. Web, desktop, mobile, and voice let you work in the way that fits the moment.

Cheung: What’s one way you use it yourself?

Herzig: As CTO, I want to stay close to what our engineers are building. But the question for me isn’t just “What changed?” It’s “Where would my attention be useful?” Instead of piecing together an update, I can ask Joule to bring the relevant changes together. That helps me arrive at a team conversation better informed: what should we discuss, what needs a closer look, and where can I help?

Joule doesn’t make those judgments for me. It reduces the work of gathering the information behind them. I spend less time collecting status and more time having conversations that move the work forward.

Cheung: If anyone can build with AI, what’s left for developers to do?

Herzig: The job shifts from implementation to architecture and judgment: deciding what’s worth building, evaluating business correctness, designing the system that governs agents safely, and handling edge cases and failures. The hardest engineering challenges in enterprise AI aren’t about building something that works. They’re about building something that works safely, at scale, with the right guardrails in place.

Joule Studio (SAP’s tool for building custom AI agents) removes that baseline burden. Developers start with the enterprise context, such as authentication, permissions, and compliance, already in place and focus on what delivers value.

Why it matters: If your company is handing employees agents and AI coding tools, Herzig’s point is that the easy part got easier and the hard part didn’t. Someone still has to own whether the output is correct, secure, and safe to act on, and that job is landing on developers.

EDUCATING AI

🧠 Teaching AI how your business runs

The Rundown: Herzig says the riskiest AI answers are “fluent-but-wrong”: confident, correctly worded, and blind to how the business runs. SAP’s fix is giving agents a map of what company data means, plus a record of how past decisions got made.

Cheung: Most people might think giving AI access to their company’s files is enough context. What does AI actually need to know about a business to get answers right?

Herzig: Enterprise data lives in tables, and knowing what a field means requires understanding the business process behind it: the transactions, the unwritten rules and exceptions, and identity, like knowing a supplier in procurement is the same entity in finance.

The main risk in enterprise AI is fluent-but-wrong: an answer that sounds authoritative but is contextually off.

Cheung: SAP’s Knowledge Graph maps what a company’s data means. How does that make AI better in a business context?

Herzig: It gives models a machine-readable map of business objects, relationships, and rules. Agents stop guessing what fields mean.

That improves accuracy, reduces incorrect API calls and retries, and can give smaller models enough context for more tasks. But understanding a rule isn’t the same as enforcing it: actions still run with the user’s permissions, defined boundaries, and human approval where required.

Cheung: SAP is also building Company Memory to capture how work actually gets done. What’s the most useful thing it stores?

Herzig: The most valuable thing it stores is why an exception was approved: the context, rationale, authority, and outcome. The next person doesn’t have to start from scratch.

An exception must not become a rule just because it worked once. A human process owner must review it to understand why the decision worked, whether the conditions still apply, and whether it is safe to generalize.

Why it matters: If your team is wiring agents into real workflows, more documents in the prompt won’t fix fluent-but-wrong answers. An easy check: feed the agent inputs your real system would reject, like an invoice with no currency, and see what it accepts.

TOKENOMICS

💸 Cheaper tokens, better business outcomes

The Rundown: Token prices keep falling, but company AI bills keep rising as usage spreads. Herzig says the question is shifting from who has the smartest model to who gets the best business result for the lowest cost.

Cheung: What did SAP learn as employee AI use grew?

Herzig: We cannot blindly throw everything at LLMs. An email summary doesn’t need the same model as complex, multi-step business reasoning. Route tasks by complexity, risk, and context, or solve them without an LLM call.

Reduce unnecessary token consumption without losing relevant context, including running models locally where it makes sense, so that only what actually requires LLM reasoning reaches the model. Keep models replaceable as quality and costs improve. And test at production scale: more than 60,000 SAP colleagues use Joule Work internally.

Cheung: Where do companies waste the most money on AI?

Herzig: Sending everything to frontier models, and “context bloat”: excessive, low-value information. A cheap model that is unreliable at scale isn’t a good trade either. The goal is the best outcome at the lowest possible cost.

Cheung: McKinsey found 80% of AI users feel more productive, but only 6% of companies see significant profit impact. What else closes that gap?

Herzig: In knowledge-intensive industries, people are the primary cost, so faster research, drafting, and coding can translate into margin. In capital-intensive businesses, assets, energy, and logistics can matter far more than payroll. Here, better demand forecasts or maintenance decisions may have a larger impact than faster writing. That’s why we invest in tabular AI. It’s ideal for predicting values in structured business data.

Why it matters: If your company’s AI costs are climbing, start with Herzig’s two culprits: top-tier models doing work a smaller one could handle, and prompts stuffed with information the model doesn’t need. And if your business runs on assets more than people, the bigger payoff may come from better forecasts, not faster writing.

TABULAR AI

📊 Specialized AI for decisions that count

The Rundown: Tabular AI brings the foundation-model approach to spreadsheets and databases, and the space is heating up. SAP built its own SAP-RPT models, acquired Prior Labs, and committed more than €1B to it over four years, while Google and NVIDIA have also released tabular models of their own.

Cheung: For a company that’s never used a tabular model, what’s a good first problem to try?

Herzig: Start with use cases like payment delay prediction or cash flow forecasting. For payment delays, you need historical invoice dates, due dates, actual payment dates, customers, amounts, and a few contextual attributes.

With a few thousand rows, a Tabular Foundation Model can begin predicting when a customer will pay and flag likely delays while you still have options. Reducing average collection time from 50 to 45 days at a large company can free up tens of millions in cash.

Cheung: How can people judge whether to trust a prediction?

Herzig: SAP-RPT-1.6 offers Explain by Feature, showing which variables influenced a prediction, and Explain by Example, showing similar historical records used as evidence. A finance controller can assess whether those patterns still apply.

The longer-term ambition is causal reasoning: not just predicting a late payment, but understanding which intervention, such as a change of payment terms, would change it. That’s a research direction beyond explaining a prediction.

Why it matters: If most of your company’s AI use is chat and writing help, this is a low-cost way to test something different. The late-payment data already sits in most finance systems, the payoff is easy to measure, and the explanations let your finance team check the model’s work.

GO DEEPER

🎟️ Join the SAP TechEd experience

Herzig is scheduled to deliver the SAP TechEd keynote on Oct. 27, kicking off three days of virtual sessions on building AI-powered systems that move from intent to execution.

Register for the virtual event.

See you soon,

Rowan, Zach, Shubham, Jennifer, and Nate — the humans behind The Rundown