Release should be separated from deployment, there should be virtually no difference between the code running production and other envs, instead of creating a release per environment you should just tag your code with an easy to read label (e.g. a version number) produce artifacts from that version of the code, then at deploy time you decide (either manually or through an automated logic) which version of the code you want to use. Ideally all "differences" at runtime between production and other envs should be defined at, well, runtime i.e. through environment variables (even the frontend stuff that you see usually made at build time, there should be a mechanism at runtime to avoid it) this way you have much more predictable behavior and it's easier to reason on, also you can physically separate building from deployment pipelines, so you can have much more agility on switching to whatever is needed, it's also much more easier on the brain (and for LLMs) to clearly see the steps and fix ci pipelines problems.
If you really need build time differences, then you should identify this difference by what really is not by the env you assume it will run on, because explicit is always better than implicit, especially for LLMs and new devs.
> It seemed reasonable to me record the deployment history in the repository
That doesn't seem reasonable to me. You'd be using Git for persistence concerns that are not directly related to source code management.
"Environment X had commit a1b2c3d deployed at 1970-01-01T00:00:00Z" is all you need to store, and I don't see why you'd want to do that in the repo itself. Commit a1b2c3d exists regardless of where or when it's deployed.
Store that data in the thing that handles the deployments.
Totally agree. It seems like a good idea at first as it's simple, but, since the deployment concerns don't match the source code management ones it breaks down quite quickly.
For example it's quite readable when deployment order matches the commit order, but it gets problematic when e.g. the deployed version is rolled back to a previous version.
It also gets messy as environments are added, or even worse you have to manage customer-specific deployments.
I'm a big fan of just adding a simple script like `git describe --always --dirty > VERSION` in the output files of the deployment process. Or otherwise including it in bundle or executable metadata. (Subtly reformatting it to pass to `dotnet build -p:Version $GITDESCRIBEISH`, for instance, or putting it into a .json file that a web service will statically serve.) Similarly outputting `git describe --always --dirty` into CI/CD logs can be very handy.
You can use git describe's output to exactly name a commit. You can also use annotated tags to tag semver releases or marver (Marketing Version) releases and if you output the git describe to users that care about semver or marver they see confidence in your deployment system when you deploy exactly a tag versus deploy something like a hotfix after it. But git describe is also useful for lower environments because it will have useful output when not directly on a tag.
(ETA: For a monorepo you can still use semver/marver annotated tags with `git describe --match subproject/` to use a tree structure for tags.)
I've never thought having the environment in the version tag (or whatever resembles that) was a good idea. It is a remarkably common thing, though.
But it means that, by definition, we are deploying a different tag to dev, staging, and prod. (Or whatever your pre-prod envs are called.) Hopefully, these all point at the same commit … hopefully¹. (Worse, some schemes out there — not the TFA's — force them to never specify the same commit(!) and we devolve to "let's pray those different commits specify the same tree.)
Just tag a version number. Use the date, if you can't think of a version number. Separately, you maintain some file containing what tag/revision you want on each of your dev/stage/prod/etc. environments. This is basically the same rough scheme as what TFA lays out … but not special. Just text files, in a repo², like everything else. You can do it in a separate repository (I'd recommend) or the same one (I wouldn't recommend, but it can work).
The exemplar question,
> When did we last deploy the www.our-app app to production?
Is just the `git log` of the file that contains the hash that that app should be deployed to.
The SRE in me notes, however, that gitops-based deployments (what all of these schemes are) represent the point in time that a deployment is requested — not when it actually rolls out. Deploys that fail or are blocked on some externality can converge much much later than the git commit instructing them to occur is minted. Also, the SRE in me wants to notes that deploys in any realistic scenario are not a point in time. (It starts when you start bringing up the first replica of your app, and it finishes when the last replica of the older deployment has spun down. Depending on how much bloat you have … it can be a good little span of time.)
(¹TFA's scheme seems to contain a textual reference to the tree sha, but is that checked?)
²If you have worked with Flux or ArgoCD, you're almost certainly doing exactly what I am describing, though those are not the only two such systems capable of it.
Am I missing something but tag in git says nothing about actual deploy. It's at most a deploy attempt. What's wrong with CD which tracks actual deploys and records which commits are deployed and all related info like who and where?
It’s a tradeoff for sure. But i think the problems just manifest a bit differently.
With multi repo it’s just a smaller constant friction on everything you do.
With mono repo everything is working amazing. Then you hit something big you need to solve to move on. So the friction is more spiky.
Edit: to add another thought. I feel like multi repo is more process oriented, while in mono repo you try to resolve that through tooling. Devs are just generally going to write about that more.
It's not on by default but there has been a packed ref format for git for some time that can be opted into for repos with lots of refs. It's one of the things being considered for the new default in the 3.0 defaults review.
Release should be separated from deployment, there should be virtually no difference between the code running production and other envs, instead of creating a release per environment you should just tag your code with an easy to read label (e.g. a version number) produce artifacts from that version of the code, then at deploy time you decide (either manually or through an automated logic) which version of the code you want to use. Ideally all "differences" at runtime between production and other envs should be defined at, well, runtime i.e. through environment variables (even the frontend stuff that you see usually made at build time, there should be a mechanism at runtime to avoid it) this way you have much more predictable behavior and it's easier to reason on, also you can physically separate building from deployment pipelines, so you can have much more agility on switching to whatever is needed, it's also much more easier on the brain (and for LLMs) to clearly see the steps and fix ci pipelines problems.
If you really need build time differences, then you should identify this difference by what really is not by the env you assume it will run on, because explicit is always better than implicit, especially for LLMs and new devs.
Agreed. I tend to think of it like this.
Runtime config is a pure function of environment.
A versioned binary/artifact should be the same across environments.
Release stage is a property of environment. (local/dev -> preview/staging -> prod)
If behaviour or value of something changes across environments it belongs as config and not code.
> It seemed reasonable to me record the deployment history in the repository
That doesn't seem reasonable to me. You'd be using Git for persistence concerns that are not directly related to source code management.
"Environment X had commit a1b2c3d deployed at 1970-01-01T00:00:00Z" is all you need to store, and I don't see why you'd want to do that in the repo itself. Commit a1b2c3d exists regardless of where or when it's deployed.
Store that data in the thing that handles the deployments.
Totally agree. It seems like a good idea at first as it's simple, but, since the deployment concerns don't match the source code management ones it breaks down quite quickly.
For example it's quite readable when deployment order matches the commit order, but it gets problematic when e.g. the deployed version is rolled back to a previous version.
It also gets messy as environments are added, or even worse you have to manage customer-specific deployments.
I'm a big fan of just adding a simple script like `git describe --always --dirty > VERSION` in the output files of the deployment process. Or otherwise including it in bundle or executable metadata. (Subtly reformatting it to pass to `dotnet build -p:Version $GITDESCRIBEISH`, for instance, or putting it into a .json file that a web service will statically serve.) Similarly outputting `git describe --always --dirty` into CI/CD logs can be very handy.
You can use git describe's output to exactly name a commit. You can also use annotated tags to tag semver releases or marver (Marketing Version) releases and if you output the git describe to users that care about semver or marver they see confidence in your deployment system when you deploy exactly a tag versus deploy something like a hotfix after it. But git describe is also useful for lower environments because it will have useful output when not directly on a tag.
(ETA: For a monorepo you can still use semver/marver annotated tags with `git describe --match subproject/` to use a tree structure for tags.)
I've never thought having the environment in the version tag (or whatever resembles that) was a good idea. It is a remarkably common thing, though.
But it means that, by definition, we are deploying a different tag to dev, staging, and prod. (Or whatever your pre-prod envs are called.) Hopefully, these all point at the same commit … hopefully¹. (Worse, some schemes out there — not the TFA's — force them to never specify the same commit(!) and we devolve to "let's pray those different commits specify the same tree.)
Just tag a version number. Use the date, if you can't think of a version number. Separately, you maintain some file containing what tag/revision you want on each of your dev/stage/prod/etc. environments. This is basically the same rough scheme as what TFA lays out … but not special. Just text files, in a repo², like everything else. You can do it in a separate repository (I'd recommend) or the same one (I wouldn't recommend, but it can work).
The exemplar question,
> When did we last deploy the www.our-app app to production?
Is just the `git log` of the file that contains the hash that that app should be deployed to.
The SRE in me notes, however, that gitops-based deployments (what all of these schemes are) represent the point in time that a deployment is requested — not when it actually rolls out. Deploys that fail or are blocked on some externality can converge much much later than the git commit instructing them to occur is minted. Also, the SRE in me wants to notes that deploys in any realistic scenario are not a point in time. (It starts when you start bringing up the first replica of your app, and it finishes when the last replica of the older deployment has spun down. Depending on how much bloat you have … it can be a good little span of time.)
(¹TFA's scheme seems to contain a textual reference to the tree sha, but is that checked?)
²If you have worked with Flux or ArgoCD, you're almost certainly doing exactly what I am describing, though those are not the only two such systems capable of it.
Am I missing something but tag in git says nothing about actual deploy. It's at most a deploy attempt. What's wrong with CD which tracks actual deploys and records which commits are deployed and all related info like who and where?
> our Git monorepo was developing an annoying problem
Sentences like this are so common from people using monorepos that I am convinced they actually introduce more problems than they solve
It’s a tradeoff for sure. But i think the problems just manifest a bit differently.
With multi repo it’s just a smaller constant friction on everything you do.
With mono repo everything is working amazing. Then you hit something big you need to solve to move on. So the friction is more spiky.
Edit: to add another thought. I feel like multi repo is more process oriented, while in mono repo you try to resolve that through tooling. Devs are just generally going to write about that more.
> This made a complete record of what was deployed, where, and when.
Personally I wouldn't store this info as refs in git.
But If i had to, I'd store them under subdirectories, try "git checkout -b "host/person/version/diet/cookies/bug-fix-for-joe".
It gets created as directory structure under .git/refs.
It seems like storing the refs in a trie would be a relatively easy change, though backwards compatibility would be annoying.
Alternatively the names of the refs could be sorted.
It's not on by default but there has been a packed ref format for git for some time that can be opted into for repos with lots of refs. It's one of the things being considered for the new default in the 3.0 defaults review.
https://git-scm.com/docs/git-pack-refs
Another reminder of how little of git I actually ever use.
[flagged]
Please don't post sneery tropes like this on HN. The guidelines make it clear we're trying for something better here. https://news.ycombinator.com/newsguidelines.html