Startup Secrets Management After Railway: A Runbook Backed by 28.65M Leaks

Railway, Zeabur and Fireworks AI each reported exposed environment variables in October. For small teams, the useful question is whether every key could be rotated in an afternoon. Here is a secrets runbook built from provider docs, OWASP, CISA and GitGuardian data.

When Railway posted its security notice on October 21, it did not ask customers to check a few keys. It told all of them to “revoke all environment variables stored in Railway, including API keys and private registry credentials.” Developers spent Wednesday evening rotating keys. That scramble is the real test of secrets management at a startup: not which vault you bought, but whether you can find, replace and kill every credential in one sitting.

Where it stands: The revoke-everything advice went up on Railway’s status page at 21:59 UTC on October 21, which was 5:59 p.m. in New York, and Railway said “as we confirm which accounts were affected, we will contact those customers directly by email.” As of its 01:22 UTC update on October 22, builds were still paused, so many teams could revoke keys but not yet deploy their replacements. Our platform risk breakdown covers that collision.

Three platforms, one operational lesson

October brought several separate incidents in which stored environment variables were exposed. No source connects them, and they should not be read as one campaign.

  • Railway (October 21) said an outside party may have gotten at customers’ stored variables, in some cases in readable form.
  • Fireworks AI said “an unauthorized third party accessed environment variables used in evaluation jobs in a limited number of customer accounts.”
  • Zeabur, a Taiwan-based host, reported its own breach and its coming shutdown, covered in our Zeabur migration guide.

Each company asked customers to do the same thing: replace credentials at the source. A team that knows what it holds can do that in hours; one that does not has to hunt first.

The numbers behind secrets sprawl

Leaks through code are a separate path from platform breaches, but the scale shows how many keys float around. GitGuardian’s 2026 report counted “28.65 million new hardcoded secrets” added to public GitHub commits in 2025, “a 34% increase year over year.” Secrets for AI services alone reached “1,275,105 , up 81% year over year.”

The figure founders should remember is about cleanup. When GitGuardian retested a batch of leaked secrets first confirmed valid in 2022, “the validity rate was still above 64%” in January 2026. Leaked keys often stay live for years.

CISA explains why embedded credentials linger: “When credential material is embedded, it is difficult to discover and can enable long-term unauthorized access if exposed.” OWASP’s answer is to “centralize the storage, provisioning, auditing, rotation and management of secrets.”

Rule one: a separate key for every app and environment

A shared key turns one leak into a company-wide rotation. A key per service, per environment (production, staging, each developer tool) means you can revoke the exposed one without touching the rest, and usage logs show which system made which call.

Put a ceiling on every key that spends money. OpenRouter’s docs say: “We recommend setting a credit limit on every key.” The reason is blunt: “A key with no limit lets a leaked key or a runaway agent spend your entire balance, including any auto top-ups.” Payment keys deserve the same narrowing. Stripe recommends “generating restricted API keys for your server-side code to limit the damage to your business if your keys are ever exposed or compromised.”

Rule two: deleting is not revoking

Of all the steps, this one is the easiest to get wrong. Fireworks put it in one line: “Removing a stored secret alone does not invalidate the underlying credential.” It added that “third-party credentials require action with their issuing providers.”

In practice, clearing a variable on your host only removes your copy. Whoever took a copy can keep using it until the AI provider, cloud account, database or payment processor that issued it revokes it. Your runbook needs a column for who issued each key and where its revoke button lives.

Rule three: use the protections your host offers, and know their limits

Railway offers sealed variables. Its docs say a sealed value “is provided to builds and deployments but is never visible in the UI nor can it be retrieved via the API.” That limits who can read a value back out of the dashboard or API. It comes with trade-offs Railway lists itself: “Sealed variables cannot be un-sealed,” and “Sealed variables are not copied when duplicating an environment.”

Vercel draws a similar line between sensitive and non-sensitive variables. In its April incident, it said the variables compromised were “non-sensitive environment variables stored on Vercel (those that decrypt to plaintext).”

Neither company has said these features would stop every attack, and Railway has not described how its October incident happened. Treat sealing as one layer. It narrows who can read a value; rotation is still on you.

Rule four: scan before you push

GitHub now blocks some AI keys at the commit. Its October 13 changelog says repositories “with secret scanning enabled, including free public repositories, will have commits containing these secrets automatically blocked.” When a partner’s key reaches a public repo, GitHub “forwards it to the relevant partner so they can revoke or rotate the credential before it can be abused.”

Turn scanning and push protection on for every repository, private ones included where your plan allows. Know what it misses: it watches code, not the variables stored in your hosting dashboard.

A secrets management runbook for small teams

Write this down once, keep it next to your incident contacts, and rehearse it on a quiet day.

Step What to do Owner
1. Inventory List each key, where it lives, who issued it Tech lead
2. Rank Mark keys to cut off at once (payments, cloud admin) Founder
3. Issue Create replacements, scoped and capped Service owner
4. Deploy Ship new values to every consumer Service owner
5. Revoke Kill old keys at the issuer, not just the host Service owner
6. Review Check usage and bills for activity you did not make Finance + tech

Two habits keep the runbook alive. First, rotate on a calendar, not only after a scare: Anthropic suggests rotating “on a consistent schedule (for example, every 90 days) by creating new ones and deactivating old ones.” Second, automate where you can. OWASP warns that “key rotation is a challenging process when implemented manually, and can lead to mistakes,” and that it is better to automate it or make sure the process is properly supported.

Budget matters too; our guide to how much seed funding to reserve for cybersecurity can help you budget for this work.

Railway’s revoke-everything advice on October 21 covered every customer as a precaution, even though Railway said its initial analysis suggested the impact may be limited to a subset of users.

Frequently Asked Questions

What is secrets management for a startup?

It is the practice of controlling where API keys, passwords and tokens are stored, who can read them, how they are rotated and how they are revoked. For a small team it is mostly a written process plus a few platform settings, not an expensive tool.

Does deleting an environment variable revoke the key inside it?

No. Fireworks AI said in its October 2026 bulletin that removing a stored secret does not invalidate the underlying credential. The key has to be revoked or rotated with the company that issued it.

How often should a startup rotate API keys?

Anthropic suggests a consistent schedule, giving every 90 days as an example: issue the replacement first, then switch the old key off. Any key that may have been exposed should be rotated right away, whatever the schedule says.

Do sealed or sensitive environment variables stop a platform breach?

Neither Railway nor Vercel has said that. Railway says sealed values cannot be viewed in its UI or pulled through its API but are still provided to builds and deployments. Treat these settings as one layer, not a guarantee.

Will GitHub catch leaked keys before they cause damage?

Only some of them. GitHub secret scanning and push protection work on repositories where scanning is enabled and forward certain provider keys found in public repos to those providers. They do not see secrets stored in your hosting platform.

Sources

This article is general information for founders, not legal or security-consulting advice. Incident details are as reported by the sources cited on the dates shown and can change; for breach-notification duties, talk to counsel.