AI Gross Margin Explained for Founders

In AI products, model costs scale with usage, so they sit in cost of goods sold. That makes the LLM bill a gross margin problem, not just an engineering one.

Quick answer

AI gross margin is revenue minus AI COGS (model APIs, inference, hosting), divided by revenue. Cutting cost per request is the fastest way to improve it.

What belongs in AI COGS

Model API and inference costs.
Hosting that scales with usage, such as vector databases and GPUs.
Third-party APIs used per request.
Support costs that scale with usage, if you include them.

How to calculate it

Gross margin = (revenue − COGS) ÷ revenue. Track it monthly and per customer segment.

Why it matters

Investors compare AI companies on gross margin because it shows whether growth makes money. A falling margin as usage grows is a red flag. See how investors diligence AI COGS.

Levers that don't touch pricing

Caching and batching.
Cheaper models for simple requests.
Output limits.
Per-customer caps on heavy users.

Model yours with the AI gross margin calculator.

Frequently asked questions

What is a good gross margin for an AI startup?

It varies by product and stage. What matters most is that it holds or improves as usage grows.

Do LLM API costs count as COGS?

Usually yes, when they scale with customer usage. Agree the definition with your accountant.

Want us to find your 20%?

Book a free 15-minute API cost teardown. Bring last month's provider usage and we'll show you your biggest waste and the first fix we'd ship.

Get a Free 15-Minute API Cost Teardown