That's how I learned a pretty important lesson about software engineering that still informs how I work to this day.
"A layer of abstraction on top of a stateful legacy system often doesn't result in a simpler system, it just introduces exciting new failure possibilities. This especially applies when the owners of the legacy system have no responsibility over the abstraction layer."
Uptime Kuma supports certificate expiry notifications and will send you messages in whatever channel (e.g. e-mail, Slack, ...) you configure ahead of time: https://uptimekuma.org/
That way, even if some of your automation is borked (or if you don't have any), you'll at least be reminded.
"I use arbitrarily complex software that has a rapid SDLC to obfuscate the issue with the fact that we have to have military grade encryption for displaying the equivalent of a poster over the internet".
The state of our industry is such that there will be a lot of people arguing for this absurdity in the replies to me. (or I'll be flagged to death).
Package integrity makes sense, and someone will make the complicated argument that "well ackshually someone can change the download links" completely ignoring the fact that a person doing that would be quickly found out, and if it's up the chain enough then they can get a valid LE cert anyway, it's trivially easy if you are motivated enough and have access to an ASN.
A lot of repositories and similar go offline randomly. It hasn't happened in a few months but usually the Microsoft package mirrors go past their Azure limits and I get reminders.
This is like the third or fourth time this has happened to them.
The Manjaro team has also caught flak for a bunch of other stuff. There's a page or two our there that detail the issues, which I'm too lazy to link here.
At this point we have to assume they're doing it for attention. I refuse to believe a team of people that can ship an OS, even if it's just a riced Arch, cannot figure out acme.sh. Come on...
not the first time, I stopped using manjaro when I noticed ping.manjaro.org was being pinged every 30 seconds on a new router I setup. nothanks on that.
but seriously, sudo crontab -e, @monthly cerbot renew
Note that the certbot instructions are to renew 2x a day with up to one hour of randomized delay; using @monthly as suggested here will result in occasional outages if the "once a month" renewal attempt fails in two consecutive months due to transient peak service blips (such as those caused by '@monthly' hardcoding for month X day 1 time 00:00 often UTC without randomization), especially as Let's Encrypt drops their lifetimes to 45 days over the next 2 years, which would result in certificates avoidably expiring in production. Please instead use certbot's recommended 2x/day renew with a random sleep of up to an hour before initiating each attempt; at least one of cronie, at, bash, python, perl random sleep methods are available on most* platforms, and are offered up by the crontab-command generator at https://certbot.eff.org/instructions .
* There is a stack overflow page from 2016 filled with solutions for Busybox, so I'd say 'all' rather than 'some' but someone out there is hosting a webserver on a potato, so better safe than sorry.
I used Manjaro for a few years.
That's how I learned a pretty important lesson about software engineering that still informs how I work to this day.
"A layer of abstraction on top of a stateful legacy system often doesn't result in a simpler system, it just introduces exciting new failure possibilities. This especially applies when the owners of the legacy system have no responsibility over the abstraction layer."
This comment made a lot more sense to me once I realized we weren't talking about an aggressively marketed weight loss drug.
If you never want this to happen again to your systems, we’re building a tool that bakes monitoring and validation into automatic cert renewals.
<https://www.certkit.io/>
Respectfully we have had Certbot for 11 years now.
Technically it wasn't offline, was it?
You could even browse it if you used a browser who still treats you like an adult and allows you to ignore certificate warnings.
Uptime Kuma supports certificate expiry notifications and will send you messages in whatever channel (e.g. e-mail, Slack, ...) you configure ahead of time: https://uptimekuma.org/
That way, even if some of your automation is borked (or if you don't have any), you'll at least be reminded.
Though with this being pushed, feels like nobody will have much choice, but automate: https://www.digicert.com/blog/tls-certificate-lifetimes-will...
Just use Caddy. It's that simple.
"I use arbitrarily complex software that has a rapid SDLC to obfuscate the issue with the fact that we have to have military grade encryption for displaying the equivalent of a poster over the internet".
The state of our industry is such that there will be a lot of people arguing for this absurdity in the replies to me. (or I'll be flagged to death).
Package integrity makes sense, and someone will make the complicated argument that "well ackshually someone can change the download links" completely ignoring the fact that a person doing that would be quickly found out, and if it's up the chain enough then they can get a valid LE cert anyway, it's trivially easy if you are motivated enough and have access to an ASN.
I love Manjaro too much, use it as daily distro but their certificate issues and its recursive behaviour threaten me a little bit.
A lot of repositories and similar go offline randomly. It hasn't happened in a few months but usually the Microsoft package mirrors go past their Azure limits and I get reminders.
This is like the third or fourth time this has happened to them.
The Manjaro team has also caught flak for a bunch of other stuff. There's a page or two our there that detail the issues, which I'm too lazy to link here.
But let's just say this isn't their first rodeo.
Agreed. This is not the first time Manjaro has made a boneheaded mistake, nor will it be the last. This is just the most recent.
At this point we have to assume they're doing it for attention. I refuse to believe a team of people that can ship an OS, even if it's just a riced Arch, cannot figure out acme.sh. Come on...
Oops, it's back now though...
not the first time, I stopped using manjaro when I noticed ping.manjaro.org was being pinged every 30 seconds on a new router I setup. nothanks on that.
but seriously, sudo crontab -e, @monthly cerbot renew
No excuses.
It's not uncommon for a Distro to point NetworkManager or whoever to check for connectivity using their own servers, Arch does it themselves[0].
[0] ping.archlinux.org
Note that the certbot instructions are to renew 2x a day with up to one hour of randomized delay; using @monthly as suggested here will result in occasional outages if the "once a month" renewal attempt fails in two consecutive months due to transient peak service blips (such as those caused by '@monthly' hardcoding for month X day 1 time 00:00 often UTC without randomization), especially as Let's Encrypt drops their lifetimes to 45 days over the next 2 years, which would result in certificates avoidably expiring in production. Please instead use certbot's recommended 2x/day renew with a random sleep of up to an hour before initiating each attempt; at least one of cronie, at, bash, python, perl random sleep methods are available on most* platforms, and are offered up by the crontab-command generator at https://certbot.eff.org/instructions .
* There is a stack overflow page from 2016 filled with solutions for Busybox, so I'd say 'all' rather than 'some' but someone out there is hosting a webserver on a potato, so better safe than sorry.
Certbot would be like the supply chain attack holy grail. Not sure I'd want software like that running unmonitored automatically with root privileges.