Independent Reporting · Not Legal or Security Advice · Verify With the Breached Company · Editorial Policy

Railway Breach: Rotate API Keys in This Order, Using Railway’s 13 Guides

Railway’s instruction on Wednesday, October 21 was one sentence long: users should “immediately roll all environment variables, including any API keys, stored in Railway, as well as any private registry credentials, and review logs for those services for unusual activity.” Doing that well takes longer than reading it. This checklist shows how to rotate API keys and the other secrets a Railway project tends to hold, in a sensible order, using the provider-specific guides Railway published the following night and the providers’ own documentation.

Where it stands: Railway’s builds were still paused when we last loaded its status page, a little before 10 p.m. ET on Wednesday (01:49 UTC Thursday). Its most recent update conceded the guides “may require redeployment of services which you will not be able to complete until Builds are available again,” but added: “However, there are steps you may begin to take.” Most of what follows can be done now, at the providers, without a Railway deploy. Background on the incident itself is in our Railway security incident explainer.

Before you start: write down what you hold

Open each Railway project and list every variable that is a credential, not a setting. For each one, note which provider issued it, which services use it, and what else outside Railway uses the same value, such as a laptop, a CI job or a second host. Shared values are the ones people forget, and a rotation that misses one leaves the old secret alive.

Then sort by damage. A key that can spend money or start machines goes to the top. A key that only reads public data can wait until the urgent ones are done.

How to rotate API keys without wiping the evidence

The instinct is to delete a leaked key. Several providers, and Railway’s guides, warn against doing that first, because deleting can throw away the record of what the key did.

Railway’s OpenRouter guide puts it simply: “Disabling a key stops it immediately and keeps its history for review. Deleting is permanent.” Its xAI guide makes the same point for Grok: “Disabling is reversible and keeps the key’s usage history attached to it, which helps the review below.” On AWS, Railway notes that “Access keys are long-lived, so an exposed key works until you deactivate it,” and AWS’s own knowledge-center article is explicit: “Deactivate the original access key.” It adds: “Important: Don’t delete the original access key during this step.”

OpenAI is the exception in Railway’s set. Railway summarizes its guidance as: “OpenAI’s guidance is to delete the key, review usage, and contact support with the details.” Follow each provider’s own rule rather than one habit for all of them.

AI model keys: the likeliest target

Railway said it believes the attackers were after AI inference providers, so model keys belong at the front of the queue.

  • OpenRouter. Railway warns: “If a key leaked, it spends your prepaid credits directly. Whoever has it can also route requests to any model OpenRouter offers, which shows up as models you never use.” Disable the old key, then “Create a replacement key with a spend limit and an expiration.”
  • Anthropic. “Anthropic’s guidance is to revoke the key immediately, then review the Usage and Logs pages in the Claude Console.” Railway flags one tell: “If a key you only use with one model suddenly shows another, that’s a strong signal.”
  • OpenAI. Delete the key, review usage by key, and contact OpenAI support if anything looks wrong, per Railway’s summary.
  • xAI (Grok). Disable any key that may have been exposed, check the Usage Explorer for requests you did not make, and put the new key in Railway for the next deploy.

Giving each app its own key, with a cap, makes the next incident smaller. Our guide to protecting API keys and tokens covers that setup.

Cloud credentials: AWS and Google Cloud

Cloud keys can start servers and GPUs, and those bills arrive fast. Railway’s Google Cloud guide says: “By default, service account keys don’t expire, so an exposed key works until you disable it.” It adds a detail that changes the order of steps: “Short-lived tokens already issued from the key stay valid for up to an hour, so disable the service account itself if you need to cut off access immediately.” Google’s own documentation explains why: such tokens “can’t be revoked.” When reviewing, Railway says “Compute, GPU, and Vertex AI usage are the common targets.”

For AWS, deactivate the old access key, create a new one, and search the logs. Railway notes that “CloudTrail records every management API call made with a key for 90 days, and IAM tracks when each key was last used.” Do not stop at your home Region: “Repeat the search in every Region. Event history is per Region, and attackers often launch resources in Regions you don’t watch.”

Stripe and payment keys

Railway’s Stripe guide reverses the usual order on purpose: “Stripe’s guidance is to rotate first and investigate second: a key that reached a public repository or a log line is treated as compromised even without evidence of use.” Once the key is rolled, check the webhook list. Railway warns: “An attacker-added endpoint receives a copy of every event.”

The checklist in one table

Secret First move Then check
OpenRouter key Disable Activity, unfamiliar models
Anthropic key Revoke Usage and Logs
OpenAI key Delete Usage by key
xAI key Disable Usage Explorer
AWS access key Deactivate CloudTrail, all Regions
Google Cloud key Disable key or account Cloud Audit Logs
Stripe key Roll Webhooks, activity
Database password Regenerate Open sessions
GitHub registry token Revoke on GitHub Security log

Databases and GitHub have their own wrinkles. A Railway database with public access turned on can be reached from anywhere with a leaked password; see our guide to rotating Railway database passwords. Railway removed the GitHub registry tokens it stored, but those tokens stay valid on GitHub until you kill them there; our GitHub token guide walks through it.

Working around the build freeze

Railway’s notice was frank about the constraint: “Until builds are restored, you will not be able to roll out new versions.” So a service running on Railway will keep using whatever value it started with.

That leaves a choice for each secret. Kill the old credential now and accept that the service may fail until it can be redeployed, or leave it live a little longer and watch it closely. For keys that can run up charges, waiting is usually the riskier option. Either way, put the new value into Railway’s variables now so it is ready when deploys return, update any copies outside Railway, and keep a short log of what you changed and when. That record also helps if you later need to dispute charges with a provider.

For AWS users moving fast, one sentence from Amazon’s guidance is worth keeping on screen while you work: “Important: Don’t delete the original access key during this step.”

Frequently Asked Questions

How do I rotate API keys after the Railway breach?

For each key stored in Railway, shut the old one off at its issuing provider, generate a new one, and store the new value in Railway. Where the provider offers a disable option, Railway’s guides prefer it to deleting, because a disabled key keeps its usage history. Then review that provider’s usage or logs for activity you did not cause.

Should I disable or delete a leaked API key?

It depends on the provider. Railway’s guides for OpenRouter and xAI recommend disabling, which stops the key and keeps its history. AWS says to deactivate the original access key and not delete it during that step. Railway summarizes OpenAI’s guidance as deleting the key, reviewing usage and contacting support.

Can I rotate secrets while Railway builds are paused?

You can revoke or disable old keys at each provider at any time. Running Railway services read their variables when they start, so they will not use a new value until they are redeployed, and Railway said redeploys were not possible until builds returned. Revoking first may break a service until then.

Which Railway secrets should I rotate first?

Put first anything able to run up a bill or unlock further systems, meaning AI provider keys, AWS and Google Cloud credentials, Stripe keys, database passwords and GitHub tokens. Two kinds Railway handled on its own: bucket credentials, and the GitHub tokens it kept for account connections.

Sources


This article is news reporting and general security information based on the public sources listed above, as they stood on the dates shown. Incident details often change as investigations continue; check the affected company’s own status page and notices for the latest. Data Breach Radar is not affiliated with any company named here.

Owed money from a settlement? Check what is open at OpenClassActions.com. Caring for someone with dementia? Find practical guides at HelpDementia.com. Working out a skin routine? Evidence-based answers at AcneAdvocate.com. Forgot the name of a movie? Identify it at FindThisMovie.com.

We use cookies to run this site, measure how it’s used, and show ads. Choose “Essentials only” to limit cookies to what the site needs to work. Privacy Policy. Cookie Policy.