As a relative "youngin" (I'm 27), at what point did we make the shift from protocols to corporate-owned ecosystems? What caused it? The rise of the VC funding model? The Silicon Valley ethos of "build an MVP, grow quickly without making money," and users adopting corporate owned solutions because they're easy?
If so, how do we dismantle this? Not from a technical perspective -- atproto for example seems powerful enough -- but from a social/economic/mindshare perspective.
I'm a generation older. To me, there were three big shifts.
One was that Facebook/Twitter/etc. proved that web publishing could be made more convenient by making it more centralized, and that access to an audience was, in some way, more important than access to publishing tools. No matter how good open web publishing tools got, they couldn't compete with Facebook et. al. at providing some access to an audience, even if that audience was as small as your friends and family.
The second was a shift in who developed "internet infrastructure." In the 80s and 90s (and before), it was mainly academics working in the public interest, and hobbyist hackers. (Think Tim Berners-Lee, Vint Cerf, IETF for web/internet standards, or Dave Winer with RSS.) In the 00s onward, it was well-funded corporations and the engineers who worked for them. (Think Google.) So from the IETF, you have the email protocol standards, with the assumption everyone will run their own servers. But from Google, you get Gmail.
The third -- and perhaps most important shift -- was the move from desktop software to web + mobile software as the primary computing platform for most people. Such that even if you were a desktop user, you did most of your computing in the browser. This created a whole new mechanism for user comfort with proprietary fully-hosted software, e.g. Google Docs. This also sidelined many of the efforts to keep user-facing software open source. Such that even among the users who would be most receptive to a push for open protocols and open source software, you have strange compromises like GitHub: a platform that is built atop an open source piece of desktop software (git) and an open source storage format meant to be decentralized (git repo), but which is nonetheless 100% proprietary and centralized (e.g. GitHub.com repo hosting and GitHub Issues).
You ask how to "dismantle" this. I've long pondered the same question. I am not sure it can be dismantled. It doesn't seem like these shifts can be undone. Where I've personally ended up is that small communities of enthusiast programmers and power users can embrace open source, open protocols, and decentralization for its obvious benefits, but that it won't ever be a mass market again.
It's worth keeping in mind a few bright points: email, RSS/podcasts, the web
Email: One of the oldest parts of the Internet. Very open standard. Federated. Largely ad-free. Little lock-in (Though @gmail.com addresses are a potential serious risk). Lots of attempts (by Slack, etc.) to "kill" email because no corporation controls it.
RSS/podcasts: RSS (or Atom or whatever) should be way more popular, but it still lives on through podcasts where anyone can publish anywhere and subscribe to anything. hough Spotify and Apple are trying hard to lock things down, they haven't succeeded yet.
The web: Exists and is still largely open. Efforts to turn everything into a closed app haven't succeeded yet and attempts to lock down the web (e.g. web attestation) have failed so far.
What about realtime+mobile chat ?
Mastodon and RCS are lightyears from consolidating X/whatsapp/messenger/telegram/signal/discord/slack/teams/etc.
Email+notifications is a joke, lacking groups features, true undo, large attachments and video codecs, etc.
XMPP has existed since 1999, but has only seen mainstream adoption inside walled garden apps that never supported federation or shut it off early on. It was possible to use Facebook and Google chat from a generic XMPP client for a long time.
XMPP leaked the features that drove these other apps to win - not the same.
> What about realtime+mobile chat ?
https://matrix.org
See, the discontinuity is at the App Store and mobile shift. It was iPhone and App Store that destroyed equal human right to code and run, turning it into elite privilege to profit by code. And the escape hatch known as the Web is slowly closing.
All three of those are commendable open standards. All three are on life support.
Email is immensely concentrated amongst a literal handful of large players, and intercommunications between those are fraught. (My Protonmail messages to Mozilla Pocket's QA team are rejected by their Gmail service, and have been for years.) Spam, confidentiality, privacy, and trust of asserted identity are all overwhelming problems. I make all but no personal use of email and haven't for years, despite first using the medium in the 1980s.
RSS has been strongly deprecated by major players. Its one bright point are podcasts (based on RSS), though that is also increasingly centralised and commercialised. The number of pods which are ad-free has been ever-shrinking, and those ads are exceedingly poor. Major publishers managed to make their RSS feeds all but useless (teasers of stories, abysmal formatting, ads/nags). Small blogs are still a source, but they're hardly the vanguard they once were.
The Web's standards process has been utterly captured by Google on all three of the publisher, ad/tracking, and browser endpoints. Scrap it and start over is my general recommendation. Gemini (the protocol, not the AI engine) is an interesting but insufficient start. (I'd like to see more structured documents with notes/references at a minimum supported.)
So yes, open standards still "exist" for some value of existence. But they're far from healthy.
I would say when Facebook arrived. But it wasn't so much "shift to corporate-owned", it was more that it allowed non-techies to put stuff on the internet for the first time. Us techies, we already had our hand-coded html web pages hosted at some (probably commercial) provider.
I think the answer is "usability". Look at all the community-made, non-commercial projects. They tend to suck because they weren't built for you. They were built for people with similar high investment into the thing they do, for experts or power-users. For them it works.
So IMO the key question is how to find motivation or time or money to solve someone else's problem, without being forced to maximize the money-making part. Because by now we can see exactly what happens when money is the primary goal. Everyone starts with good intentions (solving a problem), but the incentives are so powerful. If you don't follow them you'll start to struggle, long-term, or get out-competed by someone who maximizes the money-making part of the job.
It’s hard to not see it as techies being sold out by non-techies.
Is it ironic that it was Facebook that helped “techies” get paid because they didn’t play along with the employee price fixing cartel of Apple/google/adobe/intel/disney/etc?
But even before that we had Livejournal, we had Geocities, we had forums. There were lots of places for non-techies to post.
There's a great CCC presentation by Moxie (Signal originator) on that
https://www.youtube.com/watch?v=DdM-XTRyC9c
~summarized in text form https://signal.org/blog/the-ecosystem-is-moving/
This is fascinating (if a bit discouraging) take from someone who would definitely know better than most!
I like to THINK that atproto's ability to easily move one's data between providers makes it less susceptible to the "Gmail problem," but I think I'm being naively optimistic
> ~summarized in text form https://signal.org/blog/the-ecosystem-is-moving/
The answer from Matrix is here: https://matrix.org/blog/2020/01/02/on-privacy-versus-freedom...
Related thread: https://news.ycombinator.com/item?id=35141223
Our age gap is less then 10 years but here's my two cents: laziness/convenience. Back in the 90's and 2000's, you had to be ready to spend a lot of time fiddling with setups and maintenance as well as some MAJOR early days security flaws(think the IRC days). Corporate-owned ecosystems solved that problem: you log in and forget about it. They won with what some people call user experience. The lower the entry barrier, the quicker something picks up. Back when I was in school I was the biggest Nokia fanboi and even then I acknowledged that downloading a shady jar file and installing it on my phone was iffy. At a later stage when I was a bit older and could afford it, I got my first Android phone and the existence of a marketplace was a breath of fresh air. The problem is that few people(annoyingly even now) fail to realize or admit that those types of centralizations put handcuffs on your wrists the moment you say "OK, that works for me". Whether that's social logins, cloud providers, services or anything else - it's all the same. For example, if today, OpenAI decided to close off their API's for good, I recon tens if not hundreds of thousands of "AI" startups will collapse immediately since they fully rely on OpenAI's API's. Same with AWS, GCP, Azure or any other provider. And as we see with the current fiasco with twitter, tiktok and bambu labs just to name a few from the past two days, it is abundantly clear that people are in dire need of backups. As much as I used to find google drive and docs convenient, I've personally moved away and self-host everything now. The only thing I rely on(and only as a backup plan to access my home network) is a VPN I host over at Hetzner. But again - this is my backup.
Whether the corporations saw that as an opportunity at an early stage or they were at the right place, at the right time, I can't say. I'm more leaning towards the latter since I've worked at corporations and success in those environments is most commonly a moderately-educated gamble.
Yeah, I think it's clear that laziness/convenience is the answer.
You're absolutely right about people needing backups -- but ofc selfhosting is too huge a hurdle to expect most folks to embark on.
I wonder what can be done to make the "better" options easier. Can this even be done by the private sector alone given the incentives of capitalism? I'm unsure.
Given how many things we've seen happen in the social media space back-to-back (Elon taking over Twitter, Meta pandering to the new US governing party, TikTok's ban), I can't imagine these events will slow down. That at least fills me with hope that more people will wonder "does it have to be this way?" ...obviously that won't be enough for true mass adoption, but it's a start
I think there are two aspects to this:
* The software: different open source solutions have very different requirements at a high level: language, platform or even system requirements. Say you want to take messaging off centralized platforms: you need to host something like Matrix, which is very well made and polished but takes a lot of resources to run. Alternatively, you could use Jabber, which scales like no other but is an absolute hell to setup and maintain. Same can be said about music, videos, movies and all other things
* Operations: probably simple if you ask someone on HN, but you still need to understand networking, operating systems and file systems. I started using Linux when I was 11 in the distant 2000, and even now I'm not very enthusiastic if I have to make some changes to my zfs. You also need to consider backups and security and resources. Say you wanna run openstreetmap(which we recently started doing at work). Awesome but that requires an ungodly amount of fiddling in addition to an astonishing amount of time needed to unpack, even on enterprise hardware.
If you are in the tech world, https://github.com/awesome-selfhosted/awesome-selfhosted is a great place to start. But if you want to make it simpler... Idk... A lot of people would need to put in a lot of effort, as in build a linux distro around this idea, along with "recommended hardware", one click install(a very dumbed down equivalent of portainer), and some backup and alerting mechanisms built into the system. It's a tough question and frankly I don't have the answer.
> As a relative "youngin" (I'm 27), at what point did we make the shift from protocols to corporate-owned ecosystems? What caused it?
41, in retrospect I'd say this change happened around 2000-2010, why being not-invented-here-syndrome as Web 2.0 became a thing with some corporations publishing free-to-integrate XML-based APIs (technically also JSON, but I never saw them until much later); every API was different, so the only part which could be seen as a "protocol" were the meta-level of "how to define any API" e.g. XML, JSON.
straight answer: facebook and linkedin. they were so good that they killed the independent, decentralized 1990s web. why bother setting up your own shop and communicating via protocol when you can just make a fb or lnkd page.
theres no dismantling it. every time we offer decentralized vs centralized solutions, the centralized wins because of convenience, funding, faster progress, take your pick (lmao look at bluesky/atproto, bitcoin/coinbase). It's not even primarily because of VC or Silicon Valley ethos. this is just raw human nature at work. you want this to change, propose whatever alternative you have to the normie in middle america and watch their blank stares.
You definitely have a point. I have trouble accepting just how much people will give up for convenience!
maybe it was the worse discoverability of groups. at some point google became more about commerce than actually listing information high in their result pages. if you search on facebook, communities related to specific topics pop up immediately. even whatsapp now shows "popular groups" around certain themes in the app, even though none of your phone contacts is in any such group.
and by google not showing forums or blogs (especially new ones) as top results any more (mostly because of pre-llm spam websites) they just didnt get any more users.
facebook split up the "advertising" part and the connecting people / groups part, e.g. facebook's search wouldn't show ads.
I personally that this lack of friction really pushed social media sites forward, while the rest of the internet got kneecapped by google more and more like a boiling frog.
It isnt so much that they "were so good".
It isnt like the people using the net before facebook etc just stopped what and how they were doing things and moved to facebook.
The large tech firm offering were easier, it allowed people access to the internet in an easy to use way who would not otherwise have done so.
The internet in 2000 was a much much smaller place with far different demographics.
> propose whatever alternative you have to the normie in middle america and watch their blank stares.
We also overestimate how important the web in general is to many 'normies'. It was only a little over 10 yrs ago that I had to convince my wife (20-something at the time) that she had a reason to get a smartphone. We're so far apart on the adoption curve that it's very difficult to understand each other. As generations shift, I expect attitudes about lock-in, privacy, dependency etc will as well.
It's because platforms can deal with feature complexity and UX standardisation in a way that protocols can't.
Multi-protocol clients tend to end up a mess compared to the integrated experience of a platform which can provide a single source of truth for identity, authentication, and so on.
Netscape Communicator ticked many of the boxes of Facebook years earlier, but by kludging together NNTP, HTTP, SMTP, POP3, FTP etc., and that's before you consider the difficulty of moderating an open syndication like Usenet or IRC, or the pain in the ass that email spam had become by the early 00s.
Protocol/standards people like to think they care about UX, but for platform companies, user growth and retention literally pays their bills. It's just a different set of incentives.
And to be clear, I prefer the more open internet, but UX wise, it never stood a chance against normie-optimised, integrated platforms.
Also, around ~2000 or so, most of the "big" movies ran their own websites. There's the infamous Space Jam site [1], but there was even websites made for relatively obscure movies like Pretty Persuasion (whose URL I cannot seem to find but I remember looking at it when it was relevant).
I remember when MySpace came along, I started to see movie studios started creating dedicated MySpace pages for their films instead of dedicated sites.
It makes sense; MySpace was free and had built-in marketing via their "friends" system. You're not messing with hosting, or domain names, or even programmers, and unlike other free hosting systems, it wasn't considered lame to have `Check Us Out On MySpace` (whereas it would have been considered lame to have `geocities.com/myMovie`).
Apply this to most other industries, and you have what we have now.
[1] https://www.spacejam.com/1996/
decentralized twitter is just useless. i don’t understand the appeal.
when it comes to things like TOR they make sense and are sticky, or minecraft servers (if that counts). decentralization can be desirable, even something bitcoin like (distributing a ledger) can probably have something to offer if used to solve a problem.
I get what you’re saying, though. I think decentralization will be in vogue again, when it solves real problems.
In theory you could create a decentralized uber, possibly even something cash based, if anonymity ever becomes a concern again. Some services don’t necessarily need to be built by companies, they can be unnecessary middle men. It makes sense for drivers to run nodes themselves, be their own bosses, etc.
Kind of a neat idea now I want to build it.
Something like that may not get users immediately but something will inevitably happen that will get people interested in that kind of idea.
Centralized moderation is a big thing.
Usenet was a very open system, where iirc moderation sometimes happened per discussion group but otherwise everyone individually had to ignore bad actors (add to killfile). It scaled badly with more people and spammers. Arguably it started going downhill 30 years ago. Found a decade old discussion:
https://news.ycombinator.com/item?id=9987679
Corporate ecosystems looked a lot like open protocols for so long, luring people in. Then things changed.
Part of it has to be zirp: when money isn’t free, companies suddenly look everywhere for extra cash flows.
Part of it is LLM training: it turns out that the free data can be packaged up and resold at astronomical valuations.
The difference between protocols and these social media giants is like the difference between C and Excel.
Basically when assholes like Paul Graham got involved and dumped absolute mountains of money into applications like Reddit and Dropbox that take concepts that exist in open protocols but implement them in closed moneytisable ways.
The trend was well underway in the mid-aughts, though some might argue that early forum systems, including Slashdot (Slashcode), phpBB, and even AOL forums were precursors (all were Web / app alternatives to Usenet / NNTP, effectively). If you count custom BBS forum software, the trend goes back even further to the early dial-up era of the 1980s. We're talking 300--1200 baud modems here, none of that fancy fast 48/56Kbps stuff.
One of the challenges with open-protocols-based systems is protocol stasis. That is, once a protocol is developed and in wide use, agreeing collectively on change is hard. I've seen this directly (largely on the user-side) with Diaspora* (the platform, whilst it has some good basics, is tragically stuck with design decisions from a decade and a half ago), and Mastodon (itself an attempt to break out of stasis within IRC, StatusNet, GNU Social, and WebFinger). The two sides of that debate tend to register as purist/absolutists who cotton no variance from spec, and expand-and-embrace radicals who are seeking to adapt the protocol for private gain. (The truth of course is that both positions are considerably more nuanced, of course, and good or bad motivations may well exist on either side.)
We're seeing part of this play out with HTML/HTTP (now largely captured by Google) and SMTP (largely moribund) where on the one hand a highly complex spec largely serving the interests of publishers and advertisers over readers exists (HTML/HTTP) (see especially Drew Devault's account of how insanely complex it is to write an HTML renderer from scratch), and in the case of SMTP, many failures (privacy, security, spam, workflow integration) of email to adapt to new needs and concerns.
The result is that we rely less on open standards (making lock-in more prevalent, and new entry more challenging), existing standards are either static (SMTP) or so bloated as to lock out new entrants (HTML/HTTP), and larger aspects of online exchange get locked into proprietary stacks, protocols, platforms, and actors, with what development does occur largely addressing corporate rather than community / societal needs.
For someone who was pitched on the promise and liberation of information technologies from the 1970s onward, and was present as the modern Web and online world has emerged, it's tremendously disappointing, though there've been some lessons learned, if by me rather than the world at large. It's been interesting to watch major social rights advocates, of both the digital and broader stripes, come to terms with this (EFF, ACLU, and others), and shift their tunes considerably.
For the younger set who didn't experience this, or the older set who've forgotten or weren't paying attention, it's increasingly revealing to visit works being published over the course of this development, beginning with some of the earliest RAND monographs in the 1960s, whether cautionary or enthusiastic. I find the cautionary takes have worn better.
A partial bibliography here: <https://toot.cat/@dredmorbius/105074933053020193>
I'd add to that Lessig's Code and Other Laws of Cyberspace and Andrew Shapiro's Control Revolution, both published in 1999.
<https://archive.org/details/codeotherlawsofc0000less>
<https://archive.org/details/controlrevolutio00andr>
Alvin Toffler's Future Shock addresses this specific issue only slightly, but is another historically interesting and significant take on what was, now over fifty years ago, the future of technological, informational, and cultural development:
<https://archive.org/details/isbn_0553132644>
As I've noted here recently, that book's prognostications can be divided into TK-count, ahem, three categories: technical, psychological, and social. The first is largely over-optimistic, with a general (though not total) exception in the case of information technology. The latter is strongly cautionary and relatively accurate. The third now reads as hopelessly outdated, largely as it has become the current socio-cultural environment.
See: <https://news.ycombinator.com/item?id=42688251>
Books on the impact of media and society are also worth considering. Elizabeth Eisenstein's The Printing Press as an Agent of Change <https://en.wikipedia.org/wiki/Elizabeth_Eisenstein#The_Print...>, as well as earlier works by McLuhan, Harold Innis, and Walter J. Ong. I'm increasingly convinced that changes to information technology and systems, from the advent of speech, writing, and maths to the present, have absolutely profound impacts on the societies in which they emerge (and those proximate to them). They act as power-multipliers on other technological advances, notably in agriculture, metallurgy, fuels, mechanics, electromagnetism, etc., but even on their own are highly underappreciated.
When Google killed RSS. That was a definite slide against interoperable protocols and towards closed platforms.
News to me. RSS is still around and it was almost a decade after Google killed its reader that my feeds stopped working.
RSS/Atom was near universal until Google killed their Reader product and reduced support in other products like Chrome. From there RSS market share has declined considerably and consistently:
https://openrss.org/blog/how-google-helped-destroy-adoption-...
It wasn't just RSS. Google search now deprioritizes smaller sites.
"Ecosystems" have a network effect. If everyone is on Facebook and you want to be seen, you have to be on Facebook. But the open web is an ecosystem. If people are going to Google Reader or web search engines to find content then if you want to be seen you create a blog.
But then Google murdered them, which damaged the ecosystem. In theory you could create a new search engine and a browser with solid RSS support etc. and if that's what people start using then you get the open web back. But that's a) not that easy to do and b) would have to gain market share fast enough that the things you want to index haven't already atrophied and died.
So now we have to push the rock back up the hill and build something good enough that it can start gaining rather than losing usage share as an ecosystem, but this time learn from past mistakes. In particular, don't let anybody become a single point of failure like Google was when they decided to kill everybody.
They also killed Usenet. Google has been a force for evil for a long time.
In the big picture, I think this is just the recurring story of capitalism. The big players can seize the market. Nearly every industry or medium offers economies of scale that favor large investors. And everything facing the public turns into this advertising and analytics game. So, yes, it's driven by VC money that can buy user attention and drown out the small hobbyists who cannot invest so much in marketing nor features.
I think the answer to your "dismantling" question would be similar to antitrust actions against railroads, steel industry, etc. a century ago. It takes political will and sensible regulation. Economics favor the capital, not democracy or other social values. As in with other mass consumer markets, I think the consumers also enable this in a tragedy of the commons scenario. They each can make self-serving compromises for convenience and enjoyment and ignore the externalities.
By the way, before the internet protocols dominated, there were bulletin board systems (BBSs) and unix-to-unix copy protocol (UUCP) networks. These had some grassroots kind of community federation but also got more commercial consolidation over time. Handwaving a bit, this included systems like Compuserve and AOL. In some ways, USENET was the biggest social media that made the transition from UUCP to internet. It too eventually suffered from the same erosion of its userbase and attacks by commercial consolidation and neglect, before the web.
You may as well wonder why there were railroad barons instead of railroads being public access.