Who Pays When AI Causes Harm? How Anthropic, Microsoft, Google, OpenAI Indemnify

Anthropic’s draft prospectus reportedly says it bears liability for some harmful model behavior. Separately, the major AI vendors publish indemnities for customers. Here is what Anthropic, Microsoft, Google and OpenAI each say they cover, and where the conditions sit.

Anthropic’s draft IPO prospectus says that in some cases the company bears liability for harmful behavior by its AI models, according to an October 20 Wall Street Journal story. That puts a question many founders skip at the center of a public filing: when an AI system causes damage, who pays? Part of the answer already sits in public view, in the AI indemnification language that Anthropic, Microsoft, Google and OpenAI publish for business customers.

Where it stands: Anthropic has not released the draft, and it declined the Journal’s request for comment. The vendor terms described below are public and can change; this article reflects the versions available on October 21.

What the draft reportedly says about model harm

The Journal described roughly a third of the draft as risk factors. On harm, its summary is specific. Anthropic “said it bears liability for certain harmful model behaviors, meaning it would be on the hook to pay for damages, such as stolen data or security incidents, caused by its models.”

The draft also warns, per the Journal, that “harm that could result from a catastrophic AI-related incident” might be large enough that the company could not get insurance to cover it.

The report does not say which contracts create that liability, to whom it runs, or how it is capped. It is a risk disclosure, not a published term. The customer-facing promises vendors do publish are narrower, and mostly about intellectual property.

Four published AI indemnification commitments

In the second half of 2023, the four vendors below announced protection for business customers against copyright or wider IP claims. The coverage and conditions differ.

Vendor What is covered Main condition
Anthropic Third-party IP claims over paid, authorized use and outputs Excludes customer changes, combinations and inputs
Microsoft Copyright claims over paid commercial Copilots and their output Customer used built-in guardrails and filters
Google Cloud IP claims over training data and over generated output No intentional infringement; responsible-use tools
OpenAI Legal costs of IP claims, per TechCrunch Generally available features and ChatGPT Enterprise

Anthropic: Section K of the Commercial Terms

Anthropic expanded its terms in late 2023, saying it would “defend our customers from any copyright infringement claim made against them for their authorized use of our services or their outputs.” The terms went live on January 1, 2024 for API customers and January 2, 2024 for Claude through Amazon Bedrock.

The current Commercial Terms put the promise in Section K. A covered Customer Claim is one alleging that the customer’s “paid use of the Services (which includes data Anthropic has used to train a model that is part of the Services) in accordance with these Terms or Outputs generated through such authorized use violates any third-party intellectual property right.” The definition reaches training data as well as outputs, but only for paid use within the terms.

Three features stand out for a buyer reading the section:

  • Exclusions. The cover does not apply where a claim arises from customer modifications, from combining the services with technology or content Anthropic did not provide, or from customer inputs, among other carve-outs.
  • It runs both ways. The customer must defend Anthropic against claims tied to the customer’s inputs or to use that breaks the usage rules.
  • Caps. The terms state that the liability limits in Section L.3 “do not apply to either party’s obligations under Section K (Indemnification).”

The process matters too. The party giving the indemnity keeps “the right to control the defense of any such Claim, including the selection of counsel,” and the strategy and any settlement, with limits on settlements that would force the other side to admit fault.

Microsoft: the Copilot Copyright Commitment

Microsoft announced its commitment in September 2023. If a commercial customer is sued for copyright infringement over Copilot use or output, Microsoft said it would defend the customer and pay adverse judgments or settlements, “as long as the customer used the guardrails and content filters we have built into our products.”

Its stated reasoning was commercial: “We are charging our commercial customers for our Copilots, and if their use creates legal issues, we should make this our problem rather than our customers’ problem.” The commitment applies to paid commercial Copilot services and Bing Chat Enterprise, as an extension of Microsoft’s existing IP indemnity.

Google Cloud and OpenAI

Google Cloud described a two-part approach in October 2023. “The first prong relates to Google’s use of training data, while the second specifically covers generated output of foundation models.” It attached conditions: the output cover applies only if the customer did not try to infringe deliberately and uses available tools, such as source citation, to handle output responsibly. Google said the terms sit on its public service terms page, so customers get them “without needing to amend their existing agreement.”

OpenAI’s program, Copyright Shield, was reported by TechCrunch in November 2023. According to TechCrunch, OpenAI said it would pay legal costs for customers on the “generally available” features of its developer platform and on ChatGPT Enterprise who face IP lawsuits over generated work. OpenAI’s own terms page was unavailable to us, which is why this section leans on the TechCrunch account.

What these commitments do not address

Set the published terms next to the draft prospectus and the gap is plain. The vendor promises above are about intellectual property. The liabilities the Journal described in Anthropic’s draft, for example data theft or security breaches traced to a model, are a different category, and how they are handled for any given customer depends on the rest of the contract: warranties, disclaimers, liability caps and any negotiated terms.

For founders building on these models, the documents point to a few areas that recur when teams review AI vendor terms:

  • Which plan is covered. Paid, commercial or generally available tiers are named again and again; free or preview features may fall outside.
  • Which conditions apply. Guardrails, filters and the absence of intentional misuse appear in Microsoft’s and Google’s commitments.
  • What you add. Modifications, combinations and your own inputs are excluded in Anthropic’s terms, which matters for anyone fine-tuning or wrapping a model in their own product.
  • What flows back to you. Anthropic’s reverse indemnity shows that customers can owe the vendor a defense too.
  • What you promise downstream. A startup reselling AI features may be asked for its own indemnity by enterprise buyers, which our piece on pricing enterprise contracts touches on.

Dependence on a single model provider raises its own questions, covered in our article on suppliers that also compete, and the cost side of long vendor contracts in our look at non-cancellable commitments.

The insurance warning is the last data point. In the Journal’s account, Anthropic told prospective investors that the damage from a catastrophic AI incident could be too large to insure.

Frequently Asked Questions

What is AI indemnification?

It is a promise in a vendor’s terms to defend a customer against certain third-party legal claims tied to the AI service, and to pay resulting judgments or approved settlements. The published versions from major AI vendors focus mainly on intellectual property claims.

Does Anthropic indemnify its customers?

Yes, under Section K of its Commercial Terms, for third-party claims that a customer’s paid, authorized use of the services or the outputs violates someone’s intellectual property rights. The cover does not apply where a claim arises from the customer’s own modifications, combinations with other technology, or inputs.

Are Microsoft’s and Google’s AI indemnities the same as Anthropic’s?

No. Microsoft’s Copilot Copyright Commitment covers copyright claims for paid commercial Copilots and requires the customer to have used Microsoft’s guardrails and content filters. Google Cloud describes two prongs, one for training data and one for generated output, with conditions such as not intentionally infringing.

What liability did Anthropic describe in its draft prospectus?

According to the Wall Street Journal, the draft says Anthropic in some cases bears liability for certain harmful model behaviors, for example data theft or breaches traced to its models, and warns that a catastrophic AI incident could exceed what insurance can cover.

Do AI vendor indemnities cover every kind of harm?

The published commitments reviewed here are framed around intellectual property, mostly copyright. Other harms, such as errors, data loss or security incidents, depend on the rest of the contract, including warranties, disclaimers and liability limits.

Sources

This article is general information for founders and is not accounting, legal or investment advice. How a financing instrument is accounted for depends on its exact terms; work through it with your accountant or auditor. Anthropic has not published its IPO prospectus; figures are as reported by the sources cited on the dates shown.