If you are joining 12 tables for operational data, you have a design flaw. That’s a reporting query pattern.
Often temp tables, CTEs etc are needed as an immediate fix with redesign as a long term fix. The query planner simply can’t optimize that in a reasonable time or it’s beyond there scope if what it can optimize. Also their solution to “joining in memory” is common.
My DB query rule of thumb:
If the query is hard for you to understand, it’s hard for the planner to understand.
Default to small fast simple queries and pipeline them together.
Sometimes longer and more complex queries run much, much faster. The users complain about "complexity", but when they see the performance comparison they change their minds.
I love these! “Not Quite the Same” sounds a lot like “The Wrong Abstraction” from my list. It’s a really important point and I’m glad to see there are more articles about it!
Yep, I have the wrong abstraction on my list as well. There's a couple on ratfactor that really hit the same point home.
I think the lwn article on "the midlayer mistake" is somewhat the same point but to a different extreme, and in some ways libraries not frameworks too.
"I must not complect.
Complexity is the mind-killer.
Complexity is the little-death that brings obliteration.
I will face complexity and I will permit it to pass over me and through me.
And when it has gone past, I will turn the inner eye to see its path.
Where the complexity has gone, there will be nothing.
Only I will remain."
— Litany Against Complexity
Pleasant to hear about the Object-Relational Impedance Mismatch [1], haven't heard about that concept for a long time - maybe a decade!. The reason I like Dapper [2] is that it makes you use your own sql.
> PostGIS is a great database and its foundational data type is geography.
Hate to "well ackchyually" the author here, but I'm gonna. The foundational data type in PostGIS is not geography, but geometry. geography is geodetic layer on top of geometry. I honestly don't use geography that much because it only supports a subset of the geometry functions.
I personally hate YAGNI, because it so often leads to balkanized APIs that only implement the "needed" features and omit things that a reasonable person would expect because they weren't needed initially. Far better to have a clear and explicable model that is fully and consistently implemented.
Yes I agree with this. It’s all a matter of taste and interpretation at the end of the day. YAGNI has been useful for me in stopping the development of truly complex featureful work that is not needed. But you can go overkill and strip away everything that makes your application nice to use, or that makes your domain representation syntactically complete. And that’s just stupid.
When you build a tool (physical or software) or an API (web or library) it should be easy to use. The easier it is to use the higher the probability that people will use it. So, if people build a kind of rudimentary thing and say that is all you need (which may be technically correct), then they do not optimize for likelihood of adoption/usage.
DDIA is great. I read and re-read the small section about LSM merge tree in the first and second edition of the book and got interested in this stuff. It is always a pleasure to read about internals.
the impact of recursivedoubts i.e htmx creator on software engineering practices and adhering to simplicity is monumental but the industry keeps heading towards complexity.
Decent list but the ORM thing makes me suspicious.
He links here as some kind of damning proof that ORMs are default bad.
https://openai.com/index/scaling-postgresql/
“It’s a poor craftsmen who blames their tools.”
My ORM rule of thumb: ORM for CRUD not Reports
If you are joining 12 tables for operational data, you have a design flaw. That’s a reporting query pattern.
Often temp tables, CTEs etc are needed as an immediate fix with redesign as a long term fix. The query planner simply can’t optimize that in a reasonable time or it’s beyond there scope if what it can optimize. Also their solution to “joining in memory” is common.
My DB query rule of thumb: If the query is hard for you to understand, it’s hard for the planner to understand.
Default to small fast simple queries and pipeline them together.
Your DB query rule goes against my experience.
Sometimes longer and more complex queries run much, much faster. The users complain about "complexity", but when they see the performance comparison they change their minds.
I have a similar list I like to share, had a few from Carl's list, will probably add a few from it.
http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom... https://www.parsonsmatt.org/2017/10/11/type_safety_back_and_... https://lwn.net/Articles/336262/ https://tomasp.net/blog/2015/library-frameworks/ https://ratfactor.com/cards/not-quite-the-same https://matklad.github.io/2023/11/15/push-ifs-up-and-fors-do...
I love these! “Not Quite the Same” sounds a lot like “The Wrong Abstraction” from my list. It’s a really important point and I’m glad to see there are more articles about it!
Yep, I have the wrong abstraction on my list as well. There's a couple on ratfactor that really hit the same point home.
I think the lwn article on "the midlayer mistake" is somewhat the same point but to a different extreme, and in some ways libraries not frameworks too.
Excellent list.
The last article links here, which I enjoyed:
https://venge.net/graydon/talks/VectorizedInterpretersTalk-2...
Is there a video presentation to go along with the slides that you know of?
Ftfy:
http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom...
https://www.parsonsmatt.org/2017/10/11/type_safety_back_and_...
https://lwn.net/Articles/336262/
https://tomasp.net/blog/2015/library-frameworks/
https://ratfactor.com/cards/not-quite-the-same
https://matklad.github.io/2023/11/15/push-ifs-up-and-fors-do...
Microservices
grug wonder why big brain take hardest problem, factoring system correctly, and introduce network call too
seem very confusing to grug</i>
^ This is the best and funniest paragraph of text that I’ve read this year
I'm a fan of the t-rex:
> given choice between complexity or one on one against t-rex, grug take t-rex: at least grug see t-rex
That section reminds me of The Litany Against Fear.
Hah, indeed!
I made that up in blog post that feels aligned with Carl's world-view. The Litany appears here in the post: https://www.evalapply.org/posts/writing-practices-to-10x-eng...
Pleasant to hear about the Object-Relational Impedance Mismatch [1], haven't heard about that concept for a long time - maybe a decade!. The reason I like Dapper [2] is that it makes you use your own sql.
Edit "lets" -> "makes".
[1] https://en.wikipedia.org/wiki/Object%E2%80%93relational_impe...
[2] https://github.com/DapperLib/Dapper
The Grug Brain article is fantastic, and even better if you run it through an LLM to convert from caveman. This thing is a gem.
Yeah I really struggled to read the style of writing and gave up even though I know I probably totally agree with the message
fun fact: https://news.ycombinator.com/item?id=49059774
I am a grug!
> PostGIS is a great database and its foundational data type is geography.
Hate to "well ackchyually" the author here, but I'm gonna. The foundational data type in PostGIS is not geography, but geometry. geography is geodetic layer on top of geometry. I honestly don't use geography that much because it only supports a subset of the geometry functions.
Thanks for the callout! I have much more experience with geography but you’re totally right. I’ll add a note
Thanks for a drop of humanity in this place. I feel saner
OP might also like Essentialism. Very in line with YAGNI - it guides things outside of API design too!
I personally hate YAGNI, because it so often leads to balkanized APIs that only implement the "needed" features and omit things that a reasonable person would expect because they weren't needed initially. Far better to have a clear and explicable model that is fully and consistently implemented.
Yes I agree with this. It’s all a matter of taste and interpretation at the end of the day. YAGNI has been useful for me in stopping the development of truly complex featureful work that is not needed. But you can go overkill and strip away everything that makes your application nice to use, or that makes your domain representation syntactically complete. And that’s just stupid.
That is something I observed too:
When you build a tool (physical or software) or an API (web or library) it should be easy to use. The easier it is to use the higher the probability that people will use it. So, if people build a kind of rudimentary thing and say that is all you need (which may be technically correct), then they do not optimize for likelihood of adoption/usage.
DDIA is great. I read and re-read the small section about LSM merge tree in the first and second edition of the book and got interested in this stuff. It is always a pleasure to read about internals.
the impact of recursivedoubts i.e htmx creator on software engineering practices and adhering to simplicity is monumental but the industry keeps heading towards complexity.
Because everyone wants to Cargo Cult the architecture they see in Big Tech blogs.
“If we build like Google, we become Google”
I don’t know why a small company would want Google’s engineering problems.
cool list. might check these books out later
great reading list, but also a great career arc OP! loved skimming through your site!
Thank you :)
Grug.