Book inference API, GPU serving, and production hosting that power customer features as cost of revenue, or COGS. COGS means the direct costs of delivering your service, and they sit above the gross-margin line.
Book model training, fine-tuning, and evaluation as R&D operating expense below that line. This split keeps gross margin honest. It also helps founders price work, control cloud bills, and survive diligence.
Table of Contents
- What goes above the gross-margin line?
- Why split cloud bills by environment?
- What margin hit should you expect?
- How should pricing respond when costs move?
What goes above the gross-margin line?
Put every cost that scales when customers use the product into cost of revenue. That includes third-party inference calls, GPUs serving live traffic, and production hosting for paying users. FCF Startups states this rule in its FCF Startups accounting guide, with training and evaluation kept in R&D.
Keep experiments out of COGS. Research runs, model builds, and test evaluations do not serve a current customer. They belong in operating expense even when they use the same cloud vendor.
Why split cloud bills by environment?
Split the cloud bill by environment every month. Book production to cost of revenue, and book dev, staging, and CI compute to R&D.
Indinero warns in its Indinero SaaS chart of accounts that mixing them makes gross margin look high until diligence remaps it. Use a simple routine founders can repeat:.
- Tag accounts or projects as production, staging, dev, and CI.
- Assign shared inference endpoints by traffic source.
- Review the split before month-end close.
- Fix tags when a test workload touches production.
What margin hit should you expect?
Expect a lower gross margin than pure SaaS. Scaling-stage AI B2B products average about 52% gross margin versus about 80% for pure SaaS, with inference alone taking about 23% of revenue, according to SaaS Mag reporting ICONIQ data in its ICONIQ snapshot report.
That gap changes cash and valuation. Less gross profit remains from each revenue dollar. Plan hiring, burn, and fundraising around that thinner cushion.
How should pricing respond when costs move?
Track cost per task, not just cost per token. Unit prices can fall while your total bill rises because usage grows. Enterprise generative-AI spend rose from $11.5B in 2024 to $37B in 2025, and Menlo Ventures data summarized by CloudZero in its CloudZero AI statistics roundup recommends a predictable base fee plus usage metering.
Set a base fee that covers support, hosting minimums, and margin. Add metered charges tied to the work that drives inference, such as calls, pages, minutes, or resolved tickets. Review cost per task monthly and adjust tiers before heavy users turn profitable revenue into a loss.