This reminds me of a (completely different, but felt equally) bizzare problem with ghostty (the terminal emulator) on Linux and Steam, specifically Baldur's Gate 3.
At the start of the game after an update, BG3 compiles the (Vulkan) shaders. To avoid hammering the computer, it checks the load, including I/O pressure, and then throttles itself. It turns out that Ghostty uses io_uring, and even if it's not doing anything this somehow counts as pressure. This was made worse by the fact that I usually have a dozen or so Ghostty windows.
End result, BG3 compiles shaders using 1 core, throttled, instead of all available 20 cores. Instead of a minute or so, it'd take several hours.
(For closure: I configured ghostty to use epoll instead. Probably not optimal but works around the problem.)
I hate this. We have a whole operating system doing its absolute best to prioritize hardware resources. Don't be out here throttling yourself.
Especially for something like compiling shaders. It literally never happens unless the user is about to go play a game, which is going to consume most resources anyway. To the user, there's nothing more important running right now.
> To the user, there's nothing more important running right now.
Well, to a certain degree. I have a relatively uncommon GPU, and run games on Steam (and Linux), so literally every game does shader compilation for me. Even with a beefy CPU and beefy GPU, if the shader compilation uses more than 16 threads, my computer basically freezes until its done with it, as it's so performance intensive, kind of have to throttle things if I want to be able to use the desktop for the 10 minutes the shader compilation takes sometimes.
Edit: "Do you use Linux? That only happened to me when I was using a dram-less ssd for swap and doing IO heavy operations on the NFS mounted drive"
I do! I do have NFS drives, but not for Steam games or anything else relevant. I'm also not using swap at all. RAM seems fine during this, it's just pegging all cores to 100% usage immediately and stays there, making everything else suffer.
Seems a deep one if so. I've tried a bunch of alternative schedulers to try to avoid the issue, like Bore and BMQ, but seems to happen regardless. Been like this on previous computers too, fwiw, unless I turn down the concurrency for shader compilation stuff will lag heavily until it finishes.
1. Have you tried enabling full preemption as well? I run Debian, and before I did that I had lots of stutters from any kind of background load thanks to dynamic preemption.
2. Even then, ZFS was causing stutters until I lowered it's priority as well (spl_taskq_thread_priority=0)
3. On non-ZFS drives, try setting your I/O scheduler to Kyber for NVME or BFQ for everything else. "mq-deadline" isn't great for interactive use on a SATA SSD, even for non-boot drives, and even NVME sometimes stutters under "none" for some people.
> I'm also not using swap at all.
4. zswap (combined with zstd compression, "min_ttl_ms=1000", and "vm.page-cluster=0") and "defer+madvise" memory compaction can help. (Something about the kernel really liking having swap on-hand.)
Steam runs shader compilations at batch policy and nice 19, so without RAM or I/O bottlnecks even gaming shouldn't be too bad (though from personal experience it still induces some stuttering). It's just most distros are tuned horribly for this. (Not to sound flippant, I understand the frustration of "just try this" and it doesn't work, but it's everything I can think of RN.)
When you do, I forgot to mention I had difficulty getting a couple of settings to stick on reboot. Some had "proper" methods that were bugged, others seemingly needed a startup script anyways, so I just stuck those two groups into one:
...and some quick-n-dirty scripting[1] for ZFS drives.
Fellow HNer 'networked' pointed out[3] that the compressor type was because Debian wasn't loading the zstd module on startup for some reason, so you could fix it that way as well. Dunno your distro, hopefully you don't need these hacky workarounds that continue to annoy me.
You could try cpu pinning or ulimits to limit the program from using one core so there's always one free.
But it's also possible that the problem is in the kernel and the shaders are actually doing some system call that's locking up the kernel. Like maybe there's a bug in your cuda or nvidia driver that's causing the kernel to lock.
Do you use Linux? That only happened to me when I was using a dram-less ssd for swap and doing IO heavy operations on the NFS mounted drive, since the system will use ram as a buffer (which pushes things to swap)
My steam machine is netboot diskless nfs only.(probably a mistake, but I have a nice NAS and so leaned hard into it) However, having no swap worries me. for openbsd they document how to swap to nfs. But the linux docs say not to do that. What is the best practice here? swap to an iscsi block?
My motherboard crashes when I peg all the cores. Presumably it can't maintain the core voltage under load. Vendors need to guard against these scenarios by being extra cautious because it's a hard problem to diagnose that they'll get blamed for.
Guarding against that seems like a bad idea, but even then just use a couple less threads than physical cores. Don't get too clever.
You in particular should probably disable a couple cores until your hardware stops being defective. Have you looked into updates and voltage droop compensation settings?
It’s just reality that motherboards can malfunction or degrade in peculiar ways that a manufacturer’s quick boot up checks cant always catch. No matter the reliability claims.
If they want to cater to customers who foolishly reject reality and take every claim from manufacturer’s at face value… then foolishly misaimed blaming seems expected?
But if they just roll over and swallow the foolish blaming without even challenging customers to double check, then they are also facilitating more nonsense.
Users don't care that it's a problem with the motherboard. They have a working computer, they buy a game and it doesn't work, they blame and return the game.
Users can be unreasonable, but in this case I agree - if I reported an issue to a vendor and their response was "return your motherboard" I wouldn't work with that vendor again.
Sounds like they don't have a working computer. Computer users occasionally do things that end up pegging all cores, so I'm pretty sure that someone in that case would know their computer has some kind of issue.
We can't form the world around literal bad hardware. That's completely insane. The world would fall apart.
> I wouldn't work with that vendor again.
Possibly rightly so, if it's bad motherboard hardware. Could also be a bad power supply, bad ram, bad CPU, user misconfiguration, etc. It's a complex system, where everything has to NOT be bad for it to be deterministic.
There's a necessary level of competence required with self built systems, or any other task in human life that you're not paying someone else to do for you. That's fine. Go buy a MacBook or a pre built system if that threshold of competence can't be met. That's why they exist, for people that can't or don't want to bother.
Haha, that really depends. If you look at it from the viewpoint of a failsafe, like brakes in lifts that activate without central control, backup parachutes, etc. Then it makes some sense for resource intensive software to guard against weird edge cases like this. It's just that they measured the wrong thing.
Those have nothing to do with the required assumption of determinism in the hardware that software must enjoy.
You're examples are flawed. Those are to safely halt operation on failure. What you're suggesting with the software is to limp along pretending everything is ok.
The proper test, more in line with your others, would be to catch failure and halt operation. Since, in your case c the human isn't competent enough for that, it would maybe be a self test initiated by the operating system, then a refusal to continue when the hardware problem is identified, emulating all the periodic inspections that are part of all the proper operation of all the physical systems your mentioned.
That's a trick question as stated, and not really relevant to the discussion of if the end user software should put in safeguards for bad hardware. ;p
There's very very rarely cost pressure that allows non-determinism inside of the operating parameters, and these almost always involve radiation or gamma rays (which many of us, outside the banking industry, pretend aren't relevant) or some "impossibility", like the corruption of data that results on the same checksum.
We don't use elevator brakes or backup parachutes as an excuse to let the primary fail under normal load. Those are for emergencies.
I think the elevator analogy to this bug is that the elevator goes too fast and triggers the brakes if you input target floors more than 8 apart. You don't fix that by having the user or user software be more careful about button placement, you fix the hardware/firmware.
> in this case I agree - if I reported an issue to a vendor and their response was "return your motherboard" I wouldn't work with that vendor again.
Why would you say this? If the problem is your motherboard is in fact broken (or you have used a high-end CPU on a motherboard not rated to power it), and the vendor correctly diagnoses it despite it not being their problem, you think this is bad? You want them to change their software, making it worse for everyone who doesn't have broken hardware, just to make your broken hardware work?
Sometimes if such an issue affects a common hardware configuration it can make sense from a capitalistic sense to "make it work" for more users but if that's the case that implies you would know what the hardware configuration involves and test accordingly so the workaround would not have negative effects on users who don't need it, and either way that's not the situation we're talking about here where the user has already acknowledged that their hardware is wonky.
To be fair, such people don't seem to exist in real life. Between i586 and zen3, I have never owned a machine that boots the current kernel tip without there being one or more notes about some workaround in force, to deal with some detected broken-ness of hardware or firmware embedded therein.
Your computer is unstable and broken. I would try swapping PSU or troubleshooting that issue. This is not normal and no, this is not the job of anyone else to solve or consider.
Yeah I had this problem and it ended up being the PSU. I tried a lot of different bios and driver versions swapped cpu, memory, then eventually upgraded motherboard with new cpu and memory (all these are the suggestions from people who had the same bios code on forums) and it kept coming back… then finally replaced of the PSU which actually fixed it.
Don’t be like me! Replace the PSU! Do the math on how much your system could possibly need and then double it!
Checking the voltage and specifically which voltage the automatic overclock mode of the board applies though could also be worth the time, before replacing the psu otherwise.
And doubling the wattage isn't necessary, just be comfortably higher on the 12v line and get a good one.
But why use that metric. The OS already has support for this.
Why not just re-nice the process or whatever equivalent for the threads that do the compiling? Tell the kernel what’s going on, it will handle it for you.
> To the user, there's nothing more important running right now.
One special case: game streamers would likely rather the game stutters than the stream encoding stutters. (Especially as long as the capture of their own face+voice remains smooth.)
It looked at fossilize code (open source component used in the process), found a couple of threads online that suggested io_uring/iowait and told me first to try closing the ghostty windows and then to try to switch to epoll. This did the trick.
I have a very vague understanding of what's going on (actually better now that I've look at the links/resources from the replies in this thread!), and would probably have been able to figure it by googling in my own. But, my first hunch was that it's a problem with steam, proton, or the game, so I'd probably waste a lot of time in that direction first.
What are AIs going to do now that people are having these sort of debugging conversations in private with AI instead of in open forum threads? I worry that these forum posts will dry up and AI will start to get worse over a longer period of time.
Now that I write this thought out, I suppose that when AI gets worse in that way, people will go back to forum posts!
For these small scale (but significant!) debugging sessions, I think AI will be a net positive. Its between discovering the problem with AI vs spending a few minutes searching and giving up.
If I am reading the thread correctly, the io_uring maintainer's position is that applications trying to use the metric that "poll via io_uring" handles differently from "poll via epoll or select" are fundamentally misguided and measuring something that doesn't make sense, so they don't want to slow down io_uring to placate those apps.
> iowait is fundamentally incompatible with sane accounting in the first
place, that is the problem. It really tells you nothing, and was
deprecated since the days of uniprocessor systems. The problem here is
not how iowait is done in the kernel, it's that people are using this
metric in the first place. Which is why you just want to NOT do that,
and do sane accounting on your disk drives, if you care about that kind
of thing.
> Which is why I refuse to cater to any kind of iowait accounting metrics,
it's just adding overhead to support a metric that is largely
nonsensical in the first place. It doesn't answer any questions that an
application/tier owner would want to know. Those questions might be
things like "am I I/O bound?" or "is the device I'm using being slow?".
I wish people would realize that, rather than continue to place value in
iowait as a metric for ANYTHING, and stop trying to work around it.
It sounds more like their position is "iowait is a useless metric" and that polling with io_uring completely breaks iowait metrics but fixing it would add a minor slowdown so they refuse to do so because they don't think it's a valuable metric to begin with.
Actual utilization rates and latency for each device.
Even the grandparent bug report says that iowait is high while utilization is low. They know full well that utilization is the figure that reflects reality.
In this case "iowait is high while utilization is low" is a deliberate design choice made by the io_uring maintainers, though.
It used to under-report - but that led the the kernel wrongly downclocking the system as it believed it was idle when it wasn't.
It doesn't want to do somewhat-accurate reporting, as that would have a small per-operation performance penalty they are not willing to pay.
It now over-reports as that both avoids downclocking and is cheap to do, but the drawback is that it makes the iowait metric completely useless to any potential consumers. Which they don't care about because they believe it is inherently flawed and shouldn't be used.
Utilisation rate is also arguably a bad metric either. For an NVMe with deep queue buffers you can see 96% utilisation and think it's near its peak capacity, but in fact if you keep increasing the queue depth you can get like 5x more performance with only moderate increase in latency if at all :)
io_uring you say… one of the most infamous subsystems for security, best reserved for stringent performance requirements. Why on earth would Ghostty default to that?
Maybe because one of ghostty's selling points is performance.
Also if you're running untrusted code that attacks your not-very-common terminal emulator, you're probably in more danger from the "untrusted code" part. Threat model and all that stuff.
An app can use io_uring_set_iowait to change that behavior. Since a terminal’s waiting on I/O isn’t hugging the disk but just waiting on mouse and keyboard input, that seems like the honest thing to do.
In certain circumstances shader pre-compilation may only use one core, however this can be overridden by the user, example to use 8 cores:
~/.steam/steam/steam_dev.cfg
unShaderBackgroundProcessingThreads 8
What does this has to do with the `io_uring`? It uses io_uring to achieve this. After I deployed a test cluster, I saw how the CPU bar went beyond 100% and how the IO pressure was constant, completely fudging my metrics and my ways to detect any real pressure. The weirder part was that the cluster was running as usual, without any other issues apart from that false positive on the metrics.
HN doesn't use Markdown, and it supports only a limited set of formatting options to begin with (text between * is italicized, text that starts with two spaces on a new line is printed as code, and that's about it). It also disables emoji and a few other Unicode characters.
The twist is it would happen with any additional device in the signal path. The mere presence of a hub causes the USB control message to take slightly longer to reach the adapter—about the same amount of time it takes to receive 5 Ethernet frames at 2Gbps.
The fix is to cut the overhead by sending more than 4 frames per interrupt. Unfortunately that required buying a new adapter with a different chip.
The fix is actually to plug it directly into the host, and not use a hub. Very common thing to need to do with cheap or older USB Hardware. Paying more money for something new when the old one works fine is pointless and wasteful.
This isn't always a viable option, see for example the whole concept of docking stations.
The proper fix is to get the device manufacturer to release a firmware patch which accounts for a worst-case scenario acknowledgement round trip time by sending bigger bursts - but good luck with that...
The datasheet[0] makes no mention of field reprogrammability, but that likely wouldn’t help anyway. The maximum burst size is probably dictated by how much physical RAM the chip has to buffer packets.
The datasheet does mention support for EEPROM and SPI storage, eFuses, and a USB interface with a vendor-specific "in-house configuration" for "enhanced features and optimized performance". It even explicitly mentions some other features being configurable via "register", but never clarifies how to do that either.
What you've got to remember is that with nontrivial chips the datasheet is primarily intended for purchasing decisions. It is a high-level summary of the most important features and specifications. All the technical details are going to be in a separate many-hundred-page Application Manual.
Besides, the physical RAM likely isn't the limitation here. It has explicit support for 16k jumbo frames, so it is going to have at least that much buffer space.
That is exactly my point. If they had made it as easy to swap your storage as any PC, then you could pop in an 8Tb SSD for a fraction of the cost and not need to drag around an external disk everywhere.
This would have been a valid criticism 10 years ago. Today, a MacBook Pro comes with 3 Thunderbolt ports, each driven by an independent controller. You get a full 40Gb/s on each port. This is best-in-class connectivity.
> The fix is actually to plug it directly into the host, and not use a hub.
This was true a long time ago. For some reason the "level of abstraction" cancer has hit also PC HW. You have to account for countless levels of busses and hubs when working with serial ports for example.
There are plenty of USB hubs with built-in NICs behind the hub, that can achieve full ethernet bandwidth. I even have a hub with a built-in RTL8156 and it achieves 2.4 gbps. And a separate RTL8156 plugged into a random hub also hits that.
So it certainly seems possible to have an additional device in the signal path.
> 2026 Studio Display XDR, with everything plugged into it so switching laptops is one cable
I think I'd have accepted having two cables from the get-go. And if I observed this problem, the first diagnostic I would do is to plug the Ethernet directly into the laptop.
Fewer technical pieces in a puzzle sometimes trumps ease of use.
The first rule of troubleshooting is to try to cut down the search space as much as possible with each question. My first reaction would have also been to plug directly into the computer to see if there's a bottleneck or failure at the monitor.
With a $3300 monitor? Not a chance. When you are dropping that kind of money on your setup and it performs worse than a $35 docking station, having to mess around with that kind of ugly workaround is unacceptable.
"Ease of use" is the whole point of monitors with integrated docking stations, or even of USB-C in general. Going back to a worse UX than a 2005 Dell laptop dock? Hell no, not going to happen: something is broken here and it will end up getting replaced.
The Studio Display also takes an extra half second to activate its mic vs the onboard MacBook mic (~0.10 vs ~0.60) which is something I would never have noticed until I tried using the various whisper/parakeet STT apps.
I've got a few USB drives that are always attached that I do stuff with.
To pick a not-RAID example, I've got an old (2012-vintage) 180GB SATA SSD, from Intel, on my desk. It gets used as scratch space for big-ish file operations, where it gets hammered hard periodically, usually in bursts. It's plugged into a cheap SATA <> USB 3 adapter, and is otherwise-unadorned. It is bus-powered.
It's usually connected to the back of the PC, but it's been plugged into different USB ports (with different controller types) and also external hubs.
I don't recall what its exact status was when I starting doing that with it after pulling it out of a gently-used Thinkpad T530 about 5 years ago, but I do recall thinking that it had lots of life left.
Here's where it is today:
- 10.6 years powered on
- 81TB written (of 36.5TB promised); ~222% over spec
- Zero reallocated sectors
- One singular hardware-reported error (corrected)
- Zero sector reallocations
And the pièce de résistance:
- Zero reported errors according to ZFS checksums
That last one is important because in this application, ZFS automatically and continuously provides end-to-end verification, and it's been doing...perfectly fine.
The other USB storage devices here that are used 24/7 are similar in this way. It's not hellish at all. It's quite lovely.
I'm inclined to further extrapolate that it would not have better data integrity when connected directly to a SATA port on the motherboard, or in a disk shelf attached via iSCSI or SAS or some other more-stodgy thing. It's hard get results that exceed beyond perfection.
(But in this example, I'm not chasing down minute bus timing issues with multi-gig networks. That's a very different kind of discussion.)
In my experience, data integrity and long-term reliability of products that actually work correctly in the first place are not an issue, nor is performance if you don't have unrealistic expectations.
It's not like hard drives and SSDs underlying USB storage are engineered to lower quality standards than internal drives; they're typically similar if not identical. The only instance where I've observed special-purpose hardware is drives incorporating USB bridge logic directly on the drive's controller rather than on a separate board connected via a traditional SATA interface as a cost-cutting measure.
Problems I have seen are typically the result of hardware or firmware bugs leading to frequent disconnections or even failure to attach in the first place, power problems due to low-quality power supplies, overloading controllers or hubs with too many bus-powered devices, or knowingly or unknowingly using hardware clearly not fit for purpose (expecting good random read/write I/O performance from an array incorporating one or more of SMR drives, SATA port replicators, and non-UAS bridge chips; expecting hot-running drives to remain within their operating temperature range in environments with relatively high ambient temperatures when installed in enclosures with little or no airflow, etc.).
I was under the impression that in the Apple ecosystem, you pay for better integration. Apple’s XDR display should really be able to handle this better. Or have the drivers negotiate with the chipset better.
I mean, the guy’s using a USB Ethernet adapter with one of the cheapest NICs on the market that can do 2.5GbE, and it’s the behaviour of its firmware that caused this problem. It would have performed identically behind any hub. How is this apple’s fault?
"Better" is subjective. I think monitor is a better title, because it gives misdirection, so we the readers learn the same time OP did in the story. Unreliable narrator and all that.
I understand that not everybody likes this, because it can feel like a trick, or patronizing, or lying. But in my experience, it works very well for people.
Since lots of comments misunderstood the article, the extra latency from going through the monitor meant his laptop couldn't acknowledge the packets quickly enough. That led a buffer on the ethernet adapter to fill and that led it to drop packets.
A different adapter which required less frequent acknowledgement fixed the problem for him.
It is quite a nice post which gets to the bottom of incredibly opaque behaviour of USB hubs.
The fix is that it could send more packets per ack not that it needed to happen less. If the buffer was bigger but still capped at 4 packets per ack, the full buffer drain speed would remain at ~2.
Speaking of opaque behavior, I installed a 7-port PCIe USB card for a coworker last week that I noticed consists of a four-port controller with a 4-port hub connected to one of the ports, with no identifiable markings or documentation suggesting differences between any of the ports.
I also once had an unpowered USB 3.0 switch that somehow managed to supply power to my workstation's +5V standby rail after disconnecting it from the wall, presumably from the PC on the other side of the switch, since the only other connected devices were a mouse and a keyboard (not that powered devices' B connectors should be supplying power to unpowered hosts' A connectors in the first place).
Pretty much every >4-port USB hub or interface does that, yeah. USB ICs with more than four ports are extremely rare, and chaining two of them has no meaningful impact on performance. The biggest user-facing impact is a funky-looking USB tree view in the Device Explorer, so why bother drawing attention to it?
That’s really still the adapter though because daisy chaining usb-c exists and is kind of a given so wouldn’t that latency exist in any other configuration such as a dock? Do you know anyone who plugs Ethernet into their laptop? I don’t.
> Somehow I am one of those few very lucky people to have the opportunity of owning a 2.5 gig symmetrical line for my internet.
I'm not going to complain against more bandwidth (as a GenXer I stared my online journey at (IIRC) 9600bps (maybe lower): much celebration with V.32/V.90), but I'm curious to know how many residential use-cases can take advantage of that much bandwidth. Linux ISOs?
How many services allow that much speed on a single stream / (TCP) session? Maybe you could have five at 500 Mbps?
Certainly downloading security updates or games may be quicker, but I would think that above 500Mbps (and certainly 1 Gbps) there would be major diminishing returns. Is anyone regularly getting close to that utilization in their daily work / hobbies (and not just a momentary spike on patch Tuesday)?
I'm seriously considering using an old fiber optic Thunderbolt 2 cable to establish a point-to-point link between my desktop and file server.
The only thing holding me back is that'd I'd need to relocate the Fibre Channel card I use to connect its tape drives outside the case to do so, either in a Thunderbolt enclosure or by using a janky extender to connect the x8 FC card through a x1 PCIe slot.
Given that the system (a repurposed minitower workstation) already runs out of sight in a low-traffic area without a side panel due to SAS cables that don't fit with the side panel installed, the janky extender solution is tempting.
Torrents, and things like downloading games from Steam feel really good if you have a very strong connection. A lot of games can be downloaded in the time it takes to go grab a drink from the fridge. It adds some nice quality of life, there isn't as much need to keep a large downloaded set - just download when you need it, and it'll be there quickly.
I think going from something like 100 Mb download to 1 Gb is slightly overrated. You hit heavy diminishing returns. Symmetric Upload, for me, is what really changes your quality of life a lot. Uploading a youtube video suddenly isn't an event, it takes a few seconds. Want to upload an image in any chat? Done before you consciously see it uploading. Heavy commits are instant, even with a bunch of resources. It changed my behavior. I started uploading more stuff to remote servers because the cost in time got so much lower.
Yup as someone on a 1000/20 link (can’t get anything better in my little village in BE), decent upload is far more useful than more download after a point (far before 1gbps).
Depends on how many people you have in your household and what they do with the connection. Mine has two people who are both gamers and who use streaming media. 1000/50 makes a huge difference vs 500/50 when both are downloading.
Depends what you do really. I enjoy streaming 80gb bluray muxes without needing to store them first. I have an option to double to 2500mbit/s in my area for an additional £10 but can't yet think of cases where I need it.
Let's say you're on a Meet call with a dozen of people, one of them presenting their screen to explain an ongoing incident. You're following along while checking your dashboards and code on a remote server. You're of course keeping Slack sessions in the background.
I don't think you'd be saturating a 2.5G bandwidth, but perhaps about half of it ?
If your family is also using it in parallel that could get close to full use.
That wouldn’t come close to saturating 100 mbps. Screen sharing even at 4k is ridiculously efficient, maybe 4-5 mbps for what you’re describing where most of the image isn’t changing most of the time.
> Let's say you're on a Meet call with a dozen of people, one of them presenting their screen to explain an ongoing incident. You're following along while checking your dashboards and code on a remote server. You're of course keeping Slack sessions in the background.
This is closer to 1-10Mbps than 2.5Gbps. Slack is negligible bandwidth. Video conferencing is high compression, low bandwidth video.
The 1G and 2.5G lines benefit bulk file downloads.
A high quality 60fps 1080p gameplay stream is 4Mbps. That mostly still frame powerpoint presentation is completely negligible, and the 6 thumbnail sized videos you see at 30fps are too. You can run a whole office of people on that call on smaller connections.
4Mbps is pretty stingy for 1080p60, you are likely going to run into terrible artifacting on anything that isn't flat colors and static images. Twitch runs their 1080p60 at 6Mbps and a starry background will absolutely shred the image to shit.
"4Mbps" also isn't super meaningful without knowing what codec is being used. I'm currently watching a 10Mbps sports stream of some unknown codec and it's pretty good, but I can pretty readily pick out artifacts if I look for them.
> Let's say you're on a Meet call with a dozen of people […]
I have been on Zoom and Google Meets calls with that many people on my laptop which has a Wifi connection of ≤400 Mbps (sometimes <200Mbps). You don't need >500 Mbps for online meetings.
Are you in meetings where everyone is sending 4Kp60?
If it's a call with a dozen people, media will flow through Meet's server. Some group call servers do p2p for calls with two or three people, but approximately nobody does for a dozen.
Where I work, group call participants send about ~1Mbps of video, and maybe more for screen share, could be ~2Mbps.
So you're looking at 15 Mbps inbound and 1 Mbps outbound for the call. Plus some overhead for congestion control. That's not even 1% of your 2.5gbps connection. IIRC Meet uses higher limits, so maybe double it to 30 Mbps.
You don't need gigabit for video conferencing, unless you're running the conference out of your house. Or you have 20 people all on conferences.
You may want some QoS if your usage is near your bandwidth limits. Conferencing is sensitive to jitter and loss.
At my current ISP, speed is either 100M or 1G, so if you had 3+ people who conference simultaneously, 100M might well be limiting, so you would probably want 1G ... otoh, conferencing is usually bandwidth adaptive too, so it would still work, as long as everything plays nice with congestion.
An interesting side effect is that bufferbloat becomes less of a problem - you’re more likely able to pull 1gbps without spiking your ping.
But to be honest unless you have a household with multiple people, it’s something you mainly notice occasionally with updates, backups, large assets, etc.
I have 10 gig into my house, and most services can’t saturate it. Microsoft is crap, big game downloads don’t even top out the gigabit link in the xbox. Apple gets 2-3 gig downloading xcode updates and the like.
A lot of times downloads for games are going to be compressed/encrypted and decrypted/decompressed on the fly. It's why CPU power is actually usually the limiting factor for Steam downloads, then disk I/O capability.
It’s great when experimenting with LLMs because I can quickly try new models and download from the internet almost as fast as if coming from my NAS.
Game updates are also a great use case. I don’t game often so when my friends want to play I often have to update the game. Some patches are 60GB or more. When they come down at full line speed that’s valuable time I’m getting back to spend with my friends in game.
I doubled my measly 20 MPBps upload speed to 40 simply because of how long it was taking me to get rendered assets to someone. Having higher bandwidth than that (eventually, I hope) would be a real help in video editing.
My cable is I think up to 2Gbps now and I regularly see game updates in steam hit like 1.4gbps. Probably it is closely located in a CDN I would assume.
2.5Gbit (or 5Gbit in my case) is INFINITELY helpful if you run k8s clusters. Container images download as quickly as moving a big file on a spinning rust hard drive even if someone else is watching TV or doing equally intensive stuff.
I don't believe they asserted that watching TV was intensive, hence the followup "OR doing something equally intensive (as downloading a lot of container images)".
My mother, who lives alone, got hoodwinked into upgrading her 300 Mbit cable connection after letting it slip that she sometimes plays games on her phone while watching TV.
Do not underestimate the possibilities. I have 1.0/0.1Gb cable, and doing remote access is a complete joke. Even streaming a 1080p movie up from my house is extremely challenged.
This also locks me out of hosting my own “cloud” storage at home where it belongs - I don’t want to be out and waiting forever to get a file, so I stick to traditional cloud services for now. Slower than local storage on the LAN of course though.
A fiber isp is digging trenches soon. Cannot wait!
I mean, yeah, you have a 100 Mbit/s (or 12.5 MB/s) upload rate under ideal conditions. If my math is right - haven’t had my coffee yet - that’s just 4% of the OPs bandwidth.
Yeah upload is usually the game changer for home internet. I have symmetric gigabit and I would happily halve my download speed if it meant I could double my upload. Sending large backup .tars to cloud storage, sending data to VPCs, object storage, allowing more concurrent Plex users remotely, etc etc, 2Gbps would make so many things easier.
I have about 45 Mbit upstream on my gigabit connection, and I've never had a problem with remote access so long as it's not competing with other traffic on the connection, in which case USB tethering on my iPhone works well enough.
I suppose this also depends the protocol and the implementation. VNC often feels bad even over a point-to-point Ethernet link, while RDP feels fine for basic desktop applications on any sufficiently low-latency connection (within reason; it's not exactly snappy over dialup or basic rate ISDN).
Yeah basically nothing. I have 900 Mbps and pretty much the only things where the other end gets about 250 are Steam, Docker images, and Bittorrent. You really don't need more than 250 or at most 500 Mbps for the vast vast majority of cases.
I also have this on paper. It’s surprising how much home hardware caps out at 1 Gbps or below. I had to replace several routers and run cables everywhere to get the full speed. Once you add on tailscale, a VPN, etc it’s a proper challenge to get it consistently working above 1Gbps.
Some games are hundreds of GB of disk space and also have a terrible patching algorithm where they need to make an entire duplicate of all files to patch them, despite downloading less than a GB of patch. So you have to uninstall stuff. With fast Internet you can actually use most of your drive because you can uninstall then download a new install which only needs X space not 2X.
Literally everything is better with multi-gig symmetrical. I work from home and I have to pull multi-gig archives of documents for processing and that's better. Then I need to push them to web servers, and while I'm on 2Gbps I haven't managed to get an upload to anything useful higher than 600Mbps, but that was 30Mbps last month on cable, so tasks I'm waiting on are getting done 20x faster.
Yeah definitely Linux ISOs. It's nice to settledown in an evening, pick a new Linux ISO I haven't used yet, and veg on the sofa watching it install.
Of course unlike the netinstall images all the normies use, you have to download it up front before you start wa, uhm, installing it. But I prefer it that way so I can watch the install without being interrupted... Then again I guess at 2.5Gbit the only thing that's going to interrupt you is a crappy integrated network interface, aka Apple monitor. I don't have enough money to have that problem though.
Backups are a lot faster. For some that matters. Running 2.5g locally has a lot of benefits. With a 1g network, you need just 3 people on the network for UX to degrade.
I think it’s less about being able to max it out and more about having background events (like backups) or other users on the network not impact your own usage.
But I run servers at home and regularly download enormous video files so I can think of plenty of ways to max out a 2.5 Gbps connection (alas my own is nowhere near as fast).
Indeed. We have 4Gbps because it's not much more expensive than 1Gbps fiber (plus it required our ISP to upgrade our fiber connection from AON to XGS-PON, which is more future-proof).
The benefits are two-fold:
1. Background tasks don't impact your own issue (as you say).
2. Many CDNs can saturate 4 Gbps, so if you download something, the waiting time is much shorter than say 100Mbps.
Whether it's worth it all depends on the price difference.
4Gbit is kind of the sweet spot currently. 2.5Gbit equipment became quite cheap (e.g. Ubiquiti Flex Mini 2.5Gbit managed switches are often around 50 Euro), existing Cat 5e cabling still works reliably, and if you have a router with 10Gbit upstream and multiple 2.5Gbit downstream ports, none of the subnets can completely saturate the upstream bandwidth.
Also, these bandwidths rock for doing an initial backup, downloading a full backup, pulling many Docker images, large and frequent NixOS rebuilds (unstable-small, yay). Downloading GTA 6 (not me though, not a gamer).
> plus it required our ISP to upgrade our fiber connection from AON to XGS-PON, which is more future-proof
The opposite, actually! See for example Init7 in Switzerland offering 25Gbps fiber to home users, and having done so since 2021 - for exactly the same ~$80/mo as 1Gbps: that would not be possible on new XGS-PON deployments as it requires having direct access to the fiber going to this one customer. Init7 could trivially offer 100Gbps over the exact same fiber using existing off-the-shelf plug-and-play equipment, and many times that for customers willing to spend money on WD equipment.
PON inherently ties your fate to that of your neighbors: whatever they are getting, you are getting as well. You can just about have multiple generations of PON living on the same fiber, but that's it. The sensible thing to do would be to give each building dedicated fiber to a central PoP and put all the PON splitters in there, but as that would allow for legislation mandating access to the PoP to competitors wanting to offer superior services via direct fiber access, that is often made deliberately impossible in new PON installations by putting the splitters into tiny curbside cabinets where there is physically no space for anyone but the incumbent - which of course means the only way "open access" is possible is via a virtual provider scenario where any kind of meaningful differentiation is impossible.
The opposite, actually! See for example Init7 in Switzerland offering 25Gbps fiber to home users, and having done so since 2021 - for exactly the same ~$80/mo as 1Gbps: that would not be possible on new XGS-PON deployments as it requires having direct access to the fiber going to this one customer.
Sorry, I should have been more clear. The ISPs in my country are not going to offer higher speeds than 1Gbit/s over AON. They started using XGS-PON for all new connections and will only offer higher bandwidths over XGS-PON (currently 4Gbit or 8Gbit depending on the provider).
PON inherently ties your fate to that of your neighbors: whatever they are getting, you are getting as well.
You are leaving out a lot of details, such that different generations of PON can be sent over the same fiber. ISPs can migrate to newer PON versions by swapping equipment centrally at the OLT and the customer's ONT, while upgrading AON requires replacing active switches in street cabinets, which is much more expensive. Besides that, XGS-PON power consumption is much lower. So, it is more future-proof because ISPs will actually offer higher speeds due to lower cost. Before XGS-PON we were stuck with 1Gbit for over 10-15 years because replacing all existing equipment to offer higher speeds was too expensive. PON has upgrade paths to 25G-PON and 50G-PON, which should be enough for a while.
I know I'm far from saturating my 1G symmetrical on my normal PC. Cable conditions downgraded it to 100Mbps and I didn't notice for the longest time. (Nor did I fix it since then :P)
> but I'm curious to know how many residential use-cases can take advantage of that much bandwidth.
I don't know what you have in mind regarding what qualifies as residential usecases. I can tell you that I use my residence for work, and as part of my work I have to hold video calls and download all kinds of packages from the internet, including pulling container images, including media asset bundles that take several GB.
This often takes place when other members of my household are doing their work or watching streaming services.
If you have a 2.5Gbps connection, you often are not looking into sustainable transfer rates but bursts. A difference between 500Mbps and 2.5Gbps is whether it takes you half a minute to pull a package or 5-10 minutes.
Games mostly. Getting a 40gb LMU or iracing update when logging in for a session gets old quick. Maxing my 5 gig ftth connection feels good man.
I only upgraded because my isp charged $90 for 1gig and offered 5 gig at $65 for two years so for the price of a pci nic I thought why not. It’s also nice that every device in the house can max out their capability simultaneously but that’s a rare event.
Not sure this counts as a "residential" use-case, but I have 2 gig symmetrical for my office, and it lets me rent out two GPU servers to third parties offering 750M symmetrical on each, while keeping 500M reserved for my own use even if both of those are saturated. It's awesome.
I have a 5Gbps symmetric connection. It is a bit more expensive but far from prohibitve.
The main benefit is that I don't really have to care if a device is saturating it's 1Gbps link for updates or backups. It is also quite nice to have 5Gbps to my desktop for the platforms that support it like Steam.
I don't really get much benefit for work since our VPN bandwidth isn't amazing.
When FTTH showed up in my neighborhood a few weeks ago, I decided that I'd have been very happy with symmetric 500mbps. 500/500 is a lot more than the 500/20mbps I had before and I was really mostly interested in upload speed and latency improvements anyway.
The applications I had in mind for faster uploads were improving backups and sharing Linux ISOs, and the occasional one-off file transfer. All of that worked before, but it works better now.
I'm now also exploring things like routing my cell phone and laptop through my own VPN (I can take my network with me now and it can be fast!), and things like remote gaming (due to an drastic reduction in latency compared to DOCSIS).
...and I wound up getting symmetric gigabit anyway. The only reason I initially aimed for 500 to begin with was cost, but they ameliorated the cost difference in negotiations after the first install date got botched. :)
(I could order faster-than-gig service from this provider but my home network is only 1GBe to begin with. I'm not really interested in the barely-upgrades of 2.5 or 5GBe at all since that seems like extra steps, and I'm not ready to move anything to 10GBe yet because reasonably-efficient 10GBe gear is still kind of expensive. Some day.)
>>but I'm curious to know how many residential use-cases can take advantage of that much bandwidth. Linux ISOs?
I work in video games and I work remotely. On your average work day I can easily download 1TB worth of builds and other data. There's a LOT of sitting around and waiting for things to download and I have 1.2Gbps internet connection now. If my ISP offered 2.5Gb I'd pay almost any amount of money to have it.
This made me have AI look at a USB audio problem I have. My external soundcard is connected to my monitor (as KVM hub), then to a hub.
I learned today that that meant there was 4 hubs in the chain. What?
Turns out, the USB port I used was hooked up t a hub chip, then to the chipset. The LLM told me which USB port to use instead that avoids that setup. Furthermore, the last hub in the chain is a 7-port hub, which is implemented by a daisy-chain of two 4-port chips. Depending on which port you use, you can skip one step in the chain. There's no indication whatsoever on the outside of that hub.
I have nothing useful to add other than I feel so validated reading this. Ive been struggling with this issue on and off for over a year. With the release of Astra, it was able to finally unstick me to the point of enabling flow control which has been good enough.
I experienced this problem with a macbook and a thunderbolt dock. I had tried multiple different brands of docks and ethernet adapters, and various macbooks. One interesting thing I discovered is if I plugged a non-thunderbolt cable between the dock and the laptop, I would achieve full speed with no packet loss, so interestingly the issue doesn't occur over pure USB but only over thunderbolt.
This has got to be a bug on Apple's side right? Would be nice if they fixed it.
Oof. That is the kind of thing where Apple will never do anything. They implement standards, but they guarantee absolutely nothing if even so much as one cable involved isn’t Apple-branded. Using docks and Ethernet adapters? Good luck! They’ll just ask if you’ve considered buying a $5000 monitor with a $1000 stand to use as a dock instead. (And ironically, that’s what OP did and still had the problem :D)
What kind of Thunderbolt cable, and what kind of USB cable? The downside of using the USB-C connector for everything is that it obscures the difference between active and passive Thunderbolt cables.
I recently had this same problem with a multigig copper SFP+ module. These SFP+ modules (usually) advertise 10Gbps to the NIC they're plugged in, but will negotiate lower speeds on the copper side. So when NIC tries to send to SFP+ at higher rate than whatever speed SFP+ has negotiated internally, the SFP+ module will just drop a bunch of packets and incur a significant hit in throughput.
These tend to not support sending pause frames to its SFP+ side either, so the best option I found was to set bandwidth limit on the NIC…
Good tip! I was looking into those recently, as I have a switch whose SFP+ slots only support 1 or 10G, but I want the option to connect 1/2.5/5/10 devices.
Sounds like that may be more trouble than it's worth. Hmm. I may still poke at it, but this definitely opened my eyes to a potential trouble spot that hadn't been on my radar.
I had the same problem recently when trying to figure out high bandwidth game streaming from my desktop to my Steam deck in the living room. The beta streaming tech is “pyrowave” and it is super low latency and looks INCREDIBLE compared to traditional codecs, but as a trade off it is extremely sensitive to packet loss, and uses an appropriately large amount of bandwidth (7-800mbit at 4k120). Wired network isn’t _strictly_ a requirement but it might as well be, and the steam deck doesn’t have a physical port - but the steam deck dock does.
Anyway, the exact same situation arises, where some network condition or physical configuration (there’s a lot of hardware in between these two devices, including MoCA adapters) meant that the network adapter buffer fills up and starts dropping packets. The buffer size is configurable through some terminal command and maxing that out makes the issue disappear immediately.
Unrelated to this particular issue, but I've had subtle monitor behaviors catch me off guard as well. Usually I think of monitors as being unrelated to the underlying program running, but this is not always the case. For example I released a JavaScript programming game on steam that uses chrome (Electron) for sandboxing. This game has a tiny community but already there are a couple players doing speed running [1], and one discovered that you can complete the game many seconds faster if using a 120Hz monitor instead of 60Hz. The reason is that the JS API requestAnimationFrame [2] is tied to monitor's refresh rate (which I didn't realize) and I originally didn't do any sort of logic limiting it to 60fps (and actually I was using PixiJS tickers[3] which I didn't realize were using requestAnimationFrame under the hood).
Nice post. Unsatisfying conclusion - did you ever figure out why the original adapter was throttled when plugged into the monitor, and not when plugged into the computer?
I’m guessing it’s a USB bandwidth issue. I learned about these kind of limitations when I tried to have 3 kinects running at the same time on the same USB bus.
It's explicitly not a bandwidth issue (that would also affect the SSD). It's a latency issue - the chip doesn't have a big enough buffer to keep sufficient packets in flight at once.
That's explicitly the opposite of what the author says.
"the speed of packets coming into the adapter is 2.5 Gbps. That’s what the router pushes in. The speed going out is around 2 Gbps."
Of course latency is always on the denominator side in the "bandwidth = data / time" equation, but it seems wrong to claim it's latency when it could also be improved by fixing the numerator; it's the entire fraction that's insufficient, thus bandwidth. As written:
"The reason is that if you add up the average time it takes to get the go-ahead through my monitor’s hub, which is around 7.6 µs, to the average time it takes to send 4 KB, around 8.2 µs, you can find what the maximum speed is"
And later, increasing the numerator is exactly what fixes it with the other hardware: "I can see that instead of sending 4 1 KB packets per burst, it would send 8!"
> the chip doesn't have a big enough buffer to keep sufficient packets in flight at once
How does this make sense?
Even if his buffer was 100x larger, if the fill bandwidth exceeds the drain bandwidth, it'll eventually fill up.
The article says that explicitly:
"Well, if the speed of the drain from my adapter to my Mac is half a gig slower than the speed of the input, then you can imagine the buffer getting filled more and more until it overflows."
"buffer" was perhaps a poor choice of words on my part, because "buffer" is ambiguous. In this case I am referring to the notional "transmit buffer", which is 4 in the non-working case and raised to 8 in the working adapter.
> "I can see that instead of sending 4 1 KB packets per burst, it would send 8!"
Increasing the number of packets in flight does not change the bandwidth, it only improves latency sensitivity (i.e. you can fire more data before requiring an ACK).
> it seems wrong to claim it's latency when it could also be improved by fixing the numerator.
Increasing the bandwidth without increasing the transmit buffer would not fix this issue. The adapter he ends up buying does both (allow 2x the packets in flight, and 2x the bandwidth).
The bottom line here is that the original adapter handles full duplex 2.5g when plugged into the laptop directly, and the SSD gets full duplex bandwidth in excess of 2.5g through the same intermediate link (i.e. the display), so both the adapter and the display have sufficient bandwidth. The added hop of latency however makes the short transmit buffer insufficient to cover the latency.
> In this case I am referring to the notional "transmit buffer", which is 4 in the non-working case
Then I agree with what you were saying. Those 4 * 1 KiB are a protocol-level size (similar to the TCP window size), not an actual "buffer".
> Increasing the number of packets in flight does not change the bandwidth, it only improves latency sensitivity (i.e. you can fire more data before requiring an ACK).
That I don't fully follow. To me, the measured bandwidth includes all the effects of ACKs etc. If the the protocol demands that you can have at max 4 * 1 KiB unACKed in flight so you're waiting around for ACKs and not snding data, then increasing the number of packets in flight to 8 would 2x the bandwidth.
> The added hop of latency however makes [it] insufficient to cover the latency.
I this context, I also agree with your original statement; I misunderstood what you referring to with "buffer" (meaning the window, not actual storage), and "bandwidth" (where I think you're referring to the 5 Gbit/s nominal bandwidth of the bus, irrespective of the fact that the effective bandwidth of data sent is much lower due to requiring ACKs for small data).
You’re using an ancient definition of bandwidth. Bandwidth in packet switching networks is not the layer 1 signaling rate, it is packets per
unit time times the payload size of the packets.
The additional latency from going through the monitor made the bandwidth-delay product exceed the amount of data the adapter was able to send before receiving an ack. I thought that was fairly clear from the explanation, but now I notice that he didn't mention the latency without the monitor.
Negligible compared to the RTT, sure. But they’re talking about the latency between when the network adaptor signals that a packet has arrived and when the CPU actually services the interrupt to tell the card to send the packet(s) to main memory. The NIC’s buffer could only hold 4 packets, so by the time the NIC got a DMA buffer to copy the packets into a fifth or sixth packet had arrived and been dropped.
I see now, indeed. What cibyr posted is entirely correct.
I thought their "latency without the monitor" had referred to the overall end-to-end connection to the other IP endpoint, but they were talking about `NIC <-USB-> laptop`.
> The reason is that if you add up the average time it takes to get the go-ahead through my monitor’s hub, which is around 7.6 µs, to the average time it takes to send 4 KB, around 8.2 µs, you can find what the maximum speed is.
This part is saying the NIC with this monitor can transfer 4 KB to the host through the USB hub every 15.8 us. You can do the math and it's about 2gbps.
Remove the hub, and reduce the delay solves the problem.
Increase the burst size would work: 8 KB per 24.0 us would be sufficient for 2.5 gbps.
Increase the burst size and the transfer rate as done with the 5 gbps adapter, and that should work out, although that one is probably going to be insufficient for full rate 5 gbps as well.
"if the speed of the drain from my adapter to my Mac is half a gig slower than the speed of the input, then you can imagine the buffer getting filled more and more until it overflows. Those are dropped packets, which result in a hard backoff when the router resends them, so you drop loads in speed. And that’s the reason for all of this."
The monitor decides to send the adapter's packets on at only 2 Gbit/s, which leads to buffer overflow, thus packet loss, thus e.g. TCP congestion-control backoff.
The only thing we don't know is _why_ the monitor chooses to limit the bandwidth to those 2.07 Gbit/s (computed as the 41 KiB blocks per (7.6+8.2) µs). If it sent e.g. 81 KiB blocks, doubling the bandwidth, the problem wouldn't happen.
No, you’ve misunderstood. Any time the NIC signals that one or more packets have arrived the CPU must reply back to the NIC to give it a DMA buffer to copy the packet(s) into. By the time that reply arrived at the NIC it was already too late. The NIC could only buffer four packets and the fifth or even sixth packet had already arrived and been dropped. The NIC then sent the four packets it had to the DMA buffer and started over.
The monitor was not really at fault. All USB hubs must read USB packets on all of their ports and copy them over to the upstream port. Those packets must all go into a buffer first so they the hub can send them individually even when they arrive simultaneously. That introduced just enough delay to allow the NIC’s internal buffer to fill up. Any hub would have caused the same problem. The solution was to buy a NIC designed for a higher speed because it has a bigger internal buffer. That new NIC will probably have the same problem with USB hubs, but at 5Gbps instead of 2.5Gbps.
Not all hubs and computers are equal here. Having experienced this with RTL8153, the combination of the Studio Display and ARM MacBooks is especially bad. Using an Intel MacBook with the display, or a different USB hub instead of the display, gets the full NIC bandwidth.
Fair enough, but I still wouldn’t say that he monitor is at fault. It’s probably not operating outside the USB specification in any way, even if it takes longer to forward messages than other hubs. It may even be multiple hubs, internally.
It’s really the NIC’s fault because its buffer is too small to handle a little extra latency.
Where does the "4" come from? The article's diagram suggests that the NIC's buffer is "~35 KB", though I don't know where that number comes from either (I supposed measuring how long it takes until frames are dropped). "4" seems to be the number of 1 KB packets (the 3+1 from "packet size 1024 bMaxBurst 3"), so a transmission size (protocol-level "window size"), not a buffer size. I guess if you are referring to that as a different "buffer", that makes sense, but it doesn't seem to be the buffer the article is talking about.
But you are right with your main point regarding latency. My statement
> The monitor decides to send the adapter's packets on at only 2 Gbit/s
isn't really right: It's not that the monitor "decided" to only send 41 KiB instead of larger sizes; the 41 KiB is the maximum that the NIC can send because its `bMaxBurst = 3`, and the latency added by the monitor being a USB hub then limits the bandwidth to the mentioned ~2 Gbit/s.
I guess the key point not said anywhere is that `bMaxBurst = 3` and `wMaxPacketSize = 1024 B` are fixed protocol-level parameters that hubs like the monitor have no control over. So the 4 KiB windows are fixed from the NIC through the monitor to the Mac; the monitor cannot e.g. decide to buffer some of those to compensate for the latency it adds.
> the monitor cannot e.g. decide to buffer some of those to compensate for the latency it adds.
Correct. The NIC buffers the Ethernet packets, the USB hub in the monitor can only forward what the NIC sends. If the NIC drops an Ethernet packet, no amount of buffering in the USB hub can help; it’s already been dropped.
> Where does the "4" come from?
It was set by whoever designed the RTL8156B chip. They looked at the delay product of the USB link and calculated that a buffer holding four packets would be enough to maintain 2.5Gbps. And they’re right, assuming the latency of the USB link stays low.
Notice that when it came time to make the RTL8157 chip that supports 5Gbps they doubled the buffer size to 8. That also works only if you assume that the latency of the USB link is low, or if you have Ethernet packets arriving at 2.5Gbps instead. They probably double it again in the chip that supports 10Gbps.
Realtek are just being cheap. It would only have cost pennies to double the buffer sizes in both products, and the result would be a more reliable product. This is an example of why engineers specify Intel NICs and only consumers end up with Realtek NICs.
I find it kinda nuts they didn't run a speedtest on the first day of getting the Internet set up, considering they're paying a surely-non-base price for it. That, and you'd want to make sure everything is working one day 1 so you have a reference point in your mind. It sounded like that was the first time they actually ran a speedtest.
>Is the sweet spot for the current internet 500 Mbps?
Lower than 500...I'd guess 250.
Standard 4K youtube is like 20ish and a low latency link can still feel good even at low throughput because so much of browsing experience is multiple round trips sensitive not throughput.
I've had a NIC crap out and switch to 100M (1G internet)...that was barely and I mean just barely perceptible. 100M @ single digit latency still feels snappy for browsing
This was super surprising, have always thought ISPs are a super hard businesses. But good for you mate, good ISPs are lifesavers.
I have had the opposite experience with my ISP guys. One of the VMs I got as part of work only had IPv6 and to my shock my internet didn't have one. When I contacted my ISP they legit told "You can either have IPv4 or IPv6, but not both". Unfortunately I was on a 12 month plan and couldn't easily cancel so ended up working around using tailscale.
Be aware that turning on Ethernet Flow Control is not always better. In my experience it's only a source of issues, but my experience doesn't include USB adapters. Just know that it depends; measure and don't expect those results to translate to all servers and switches in general.
I personally always like every controller directly plugged into the pcie bus. USB-C and Thunderbolt does give you a PCIe connection but how internally it’s wired and implemented is a game of chance. The really high end ones usually break out the thunderbolt 3 into a pcie controller for Ethernet etc but that too depends on how many lanes there are and how many controllers are competing.
Eventually the people designing it will find a ‘one size fits’ all solution and then you end up with issues like these.
From my experience reading such issues, almost all such problems are trivial with a pcie card to the cpu.
Funnily enough I recently noticed my network was slow, told Claude to investigate, it came up with "replace the DisplayPort cable on the right monitor".
Immediately 10X'd my bandwidth, turned out I was using a cheap cable that wasn't shielded like it was supposed to be, and it was running right past the WiFi antenna and causing loads of interference.
how much back and forth was there between you and claude to troubleshoot this? you make it sound like you gave it one prompt and it found that solution, which to me seems incredibly impressive.
The original studio display had a similar issue. A 1gbps adapter worked fine on the Mac, but a good 2.5gbps adapter was needed when going through the studio display. There's a big thread on the Apple forums if people need to know which adapters chipsets work.
> Basically, when a device receives Ethernet frames it can’t keep up with, it just drops them. With flow control (it’s something your router or switch has to support), it can send the sender a “pause” frame, and the sender then buffers the data on its side, instead of the receiver (the adapter in this case) on its side.
One thing that can happen with pause frames (and why I hate them) is pause propagation. If you have a switch between your server and client and the client sends the switch a pause frame, the switch may send a pause frame to the server. If it does (and the server accepts pause frames), the server will pause sending to all destinations.
So your one slow client can disrupt all the other flows from the server. Which is totally unhinged.
Maybe I need to look into frames and dropped packets. My LAN can't quite reach 1Gbps, it's more like 7-900Mbps. Last time I investigated it, I came to the conclusion it's sub-par ethernet cables, but perhaps there's more to it.
Seems likely. BBR (v1) does not do the hard 30% "multiplicative decrease" on packet loss (caused by the frame drop in the article's diagram), so I'd expect it to be able to correctly estimate the bandwidth to the ~2 Gbit/s bottlenecked by the monitor-Mac connection.
Yeah, I'd expect it too, but doing some research it seems Apple still doesn't support BBR. But I wonder if a speedtest with quic would show better results for the author.
It doesn't really matter what the client OS supports because the problem is inbound. You'd want fast.com to support BBR (although... I think they do, fast.com is Netflix and afaik they are a significant BBR user)
Sure we are talking about a different level in the networking stack and very different protocols. But with every component you add things get less synchronous. Well, no buffering needs to involved by the middleman, the mechanism for the throttling could also be different. What do I know...
The 35 KiB buffer from the article is not very large, and the problem would exist unmodified if it was smaller.
As the article explains, "if the speed of the drain from my adapter to my Mac is half a gig slower than the speed of the input, then you can imagine the buffer getting filled more and more until it overflows", and this would hold for any buffer, smaller or larger.
Bufferbloat causes primarily latency problems, not bandwidth problems.
Bandwidth problem would occur at most as secondary effects, e.g. if a packet is delayed by a buffer for so long that the sender-expected timeout for an ACK expires, it considers the packet lost, and throttles down sending. But that's not happening here.
I've encountered a similar problem, the wifi on a MacBook Air was dying once connected to a Dell monitor over USB-C. The wifi signal was going from full to nearly no signal. Connecting to the same monitor with a dongle and HDMI didn't have the same outcome. Never had a time to find the culprit. Probably EMI or bad grounding.
Actually, the speed of gigabit Ethernet adapters are also known to cut 60% behind a studio display. TBH I assumed it was because USB ECM sucks and NCM (which RTL8156 implements) was supposed to fix it.
Used to get this with dial-up modems when they were either placed on top of the CRT (just to get very toasty?), or the incoming phone line ran too close to some CRTs.
This would reduce the established connection speed due to electrical noise from the monitor.
Almost all the screen ethernet ports I have tried, Philips, Lenovo get to about 300mbps and can not saturate 1gbps, which is totally bad for downloading Linux isos.
What a click-bait headline. The conclusion was that it was due to having a more bottlenecked USB-to-Ethernet adapter path. Would the same result have held with a discrete Thunderbolt 5 hub, either with a built-in 2.5 Gbps or faster Ethernet port or with the original USB-to-Ethernet adapter? Would it have held with another USB hub? Who knows -- just blame it on the expensive monitor!
The explanation is not that clear, but you missed the point. It is a networking issue known as 'bandwidth-delay product': the combination of latency and bandwidth is what limits the speed, and in this case the extra latency from the monitor hop was the culprit. Not the adapter. The faster adapter is a workaround.
You sound like Steve Jobs: "the adapter is fine, you just plugged it in wrong." No: if the buffers are too small to tolerate an intervening USB hub, the peripheral was designed poorly.
Edited to add: Unlike networks in general, the USB standard imposes maximum diameters and latencies, so designers know what to design for. If the higher-spec adapter also had awful goodput, then it might be reasonable to point fingers at the monitor.
I found it so funny that the author is using an M3, Studio Display, and a 2.5 GB symmetric internet connection to play Sauerbraten. A (very fun) 22 year old game made by 3 people.
> Somehow I am one of those few very lucky people to have the opportunity of owning a 2.5 gig symmetrical line for my internet. But one day, when I wanted to brag to my equally nerdy friends [...]
I'm not sure why you'd brag about having slow internet? ;) We had 10 gig here in Singapore for a while now.
My whole wifi slowed to 5kb/s because of a bluetooth smartwatch interfering with the 2.5ghz wifi. I was always told bluetooth and wifi sharing a spectrum could not possibly ever be problematic. Needless to say I hate Bluetooth even more since then.
I suspect the problem here is the UGreen ethernet adapter. I have soured on UGreen after a couple of their recent hardware bits proved flaky and one damaged two laptop motherboards (it was a usb-c hub).
Computer case windows are a big part of this too. There's a reason that high power high speed switching electronics generally need to be enclosed in a (near) full conductive surface to pass FCC emitted emissions regulations. Putting a giant hole in the meta box mitigates a lot of the point of it.
TLDR (if I've got this right): The problem was caused by the combination of his monitor's built-in USB hub (which added some microseconds of latency to the acknowledgement cycle) and his USB Ethernet adapter (which sent data in bursts of only 4 x 1kB USB packets before it had to wait for acknowledgement). Fixed by updating to a Realtek RTL8157 based Ethernet adapter (which sends 8 x 1kB packets at a time, and also supported a faster USB link). (Personally I wouldn't pipe my gigabits-speed internet connection through such an arrangement, but I guess this is how people do things nowadays).
> Personally I wouldn't pipe my gigabits-speed internet connection through such an arrangement, but I guess this is how people do things nowadays
The XDR Studio Display is a $3000 monitor that supports Thunderbolt 5 which advertises 80 Gbps speeds. So I think one could certainly be forgiven for assuming it could handle a measly 2.5 Gbps ethernet adapter (and in the end it could albeit with a different adapter).
I had an issue with a raspberry pi and a second usb WiFi adapter. It caused my terminal to be unusable, despite my connections carrying over the primary adapter.
Apparently there are usb harmonics or other power issues that were interfering with the primary wireless adapter .
Remember the machine is billions of electrical signals and EMF. It’s actually amazing that any of it works discretely and (somewhat) deterministically
The meta-issue is that the more "advanced" (and interconnected) all of our devices become, the greater the potential for harmful side effects, or as we saw in France, hacks.
A buggy monitor or adapter is one thing. But a buggy car? Yet these will soon be our only option.
I'm always jealous of people who have the willpower and time to investigate this kind of stuff, I would have plugged it into the laptop and moved on! It's perhaps the most nerdy "grass must be greener" luxury I can imagine (in a good way).
Hah, well if my monitor at home ever does this to me I can take my revenge by slowly removing my jacket, or sitting down on my chair. (Samsung Odyssey G9, it's ultra-sensitive to ESD anywhere near it so if I'm wearing the right jumper I can move my chair or wave my arms around and the capacitative heresy will make it wig out for a good 2-3 seconds.)
At work we run HP Z40c monitors and while they've been bulletproof in terms of image quality, on my old laptop the mouse cursor was definitely more laggy when running in a one-cable setup. We're using Logitech MX Master combos so I just moved the dongle from the monitor to my laptop and that fixed the lag issue.
Wow this sounds like a shill post. If anyone from HP or Logitech wants to talk message me. Otherwise Logitech Opti+ or whatever its called can fk off.
1. These 1 cable solutions are nice but (IME) incredibly finnicky. I've gone through several Caldigit docks that have died;
2. When you think about it, these docks (and monitors) are really 100+ Gb/s switches. It's kind of no wonder that there are odd behaviors;
3. IME that cable that provides power to the laptop and handles power/video (you know, so it really is 1 cable) is the absolute worst. Whenever you buy them, buy 2-3 because I can almost guarantee 1 will be faulty and it'll be weird, annoying problems eg the screen gling black for a split second randomly;
4. I did a lot of testing with 1GbE USB-Ethernet adapters and it's true that they are not all created equal. In a perfect world, they should do ~930Mb/s. Depending on which one you get it'll be as low as 880 (or lower); and
5. I would honestly expect better from Apple. With the price premium you pay for these Studio monitors, I would expect the hub part to be pretty much seamless.
I had a sort of similar thing recently , Claude found it but my wifi was super slow on my Mac until I turned off airdrop from everyone , and AirPlay handoff, it actually improved my wifi a lot.
When he plugged the original ethernet adapter straight into the laptop it worked as expected. It was the monitor not supporting that specific setup and chip.
It has nothing to do with the monitor's support. The GP is correct; the fault is in the Ethernet adapter. Specifically, it doesn't have a buffer large enough to sustain 2.5 Gbps when going via the monitor because the latency is too high. You'd have the same problem on many USB hubs and it would still be the Ethernet adapter's fault. It could have used a larger buffer or it could send more data at a time to the host so that the buffer doesn't fill up, but it doesn't.
I would say if a USB adapter works fine in a laptop USB port, then there is nothing wrong with it. If you are trying to use it in a monitor and it introduces so much latency that doesnt work, then thats a different issue. Its not a faulty adapter.
The adapter manufacturer sourced a part that could not sustain the product’s advertised bandwidth when connected through a USB hub. USB hubs are not esoteric. This is the fault of the adapter manufacturer cheaping out on a part.
I think we are all arguing the same thing right? The adapter worked fine when used directly in a device but not when latency was high due to other devices in the chain?
I have had hundreds of USB devices in the past which dont work on a USB hub, its very common. The first troubleshooting step in basically any USB devices manual is if you are having trouble and you are using a hub, remove the hub and plug directly into the device.
The user went way over the top in troubleshooting this issue. Thats the first thing I would have tried. Its not worth the post and the title should really be something like "USB device didnt work with a hub, but did directly in the host" and it would never have made it to the front page of HN other than maybe to be mocked.
No, you are blaming the monitor manufacturer for the customer not getting the advertised speed out of their Ethernet adapter when they connected it through the monitor’s integrated USB hub.
I am saying your blame is misplaced, and that the adapter manufacturer deserves the blame because they designed their product around a controller whose USB implementation has so much inherent latency that it mathematically cannot sustain the product’s advertised throughput when connected through a hub. The manufacturer should have tested this extremely common scenario and picked a part that could sustain it.
No, Im blaming the user for overengineering and sensationalising the fact that a cheap USB device didnt work on a hub. That is a problem as old as USB itself and to go this far down the rabbit hole on it is laughable. There are plenty of other people in the thread that agree.
> The manufacturer should have tested this extremely common scenario and picked a part that could sustain it.
Hard disagree. Maybe if you bought a better one you would expect that, but I would expect a $15 2.5Gb ethernet adapter to not work on a hub right off the bat tbh.
UGREEN is one of the most common peripheral brands on Amazon. It’s up there with Anker. This is the market your average consumer is going to encounter. They’re not gonna know that $15 is too cheap to afford a BOM that can deliver the promises on the box in common scenarios like “plugged into a hub.”
> Hard disagree. Maybe if you bought a better one you would expect that, but I would expect a $15 2.5Gb ethernet adapter to not work on a hub right off the bat tbh.
I would expect any adapter built around the RTL8156B to behave in a way identical to that described in the article, regardless of whether it cost $5, $50, or somewhere in-between.
(This message was delivered in part by an RTL8156B adapter from a manufacturer with a Chinese-soup name that currently sells for $13.29.)
Yes, but in retrospect I seem to have slightly overstated the compatibility issue. I have since learned that a single standard USB 3 hub probably wouldn’t introduce enough latency to cause the adapter’s buffer to fill up. The extra work of unwrapping the Thunderbolt tunnel around the USB protocol introduces additional latency. But so would adding a second USB 3 hub in the signal path, or a single USB 4 hub, since USB 4 is always tunneled within Thunderbolt when both ends of the link can speak Thunderbolt/USB 4.
And while the DRAM or SRAM or whatever-kind-of-RAM is included on these NICs is expensive, and I'm very appreciative of that expense: We're talking about just a few more kilobytes being enough to make a positive difference.
The only reason it was ever an issue is because someone was being cheap. (Cheap of labor? Cheap of space in the tape-out for the IC? Cheap of testing? Cheap by adhering resolutely to the exact confines of the spec on their desk without providing feedback? Cheap of all of these things? Whatever; cheap.)
OK So this device would have worked fine directly plugged into the host, and into a USB3 hub, but not Apples fancy TB tunnel to its single cable monitor hub?
Are you still arguing that the adapter is at fault here? Because In my opinion the dongle works fine and its the users extremely rare use case which is causing this.
Additionally, when the USB 4 spec was created did they not foresee this issue of adding extra latency? USB was supposed to be completely backwards compatible, but this seems like a glaring breaking change by sacrificing compatibility for speed.
Thunderbolt displays are effectively Thunderbolt docks with integrated LCD panels. They exist for people who use powerful laptops as desktop replacements. Apple isn’t the only company that makes them: Dell, LG, Samsung, BenQ all make them too. It’s not an “extremely rare use case,” especially not for the kind of user that also needs a 2.5Gbps Ethernet dongle to function at its stated speed.
I guess when they designed the USB 4 spec, they decided not to cater to device manufacturers who don’t design their devices with enough RAM to do large enough bursts to sustain their products’ stated capabilities in valid use cases.
I feel like I am going a bit crazy reading some of these comments. I don’t know why people don’t read the whole blog post to understand that the monitor works great and it gets maximum speeds when doing things like transferring files from my ssd.
The adapter works great and I am getting maximum speeds when connected to my laptop.
The laptop works great and I get maximum speeds when doing any of those things.
The hub inside the monitor works great and I get maximum speeds through it even with multiple things plugged in at the same time.
None of the parts are broken. Together they don’t work as they are supposed to because of some super intricate process of how usb works when it comes to acknowledging packages. That’s why I wrote the post. The title is true and yeah, it’s sensational but that’s what titles are there for. It’s not a false clickbait.
I read the entire post and enjoyed it very much. Thank you for writing it.
That said, I think you’re being too charitable to UGREEN. They shouldn’t advertise a product as 2.5Gb/s if it can’t actually transfer 2.5Gb/s to the host when the signal path includes a hub.
You are probably right. I think the reason why I cut them the slack is because of how cheap the adapter is and that it does give me the speed when connected directly or if I go with flow control. Its a weird one
Yeah, this probably speaks to our relative tolerances for fiddlyness. Despite being in love with computers since elementary school, I have no desire to muck with things anymore. And I feel for the normal consumer who just wants things to work the way they say on the box. But consumers also don’t want to pay more than they need to, and these reference-designs-in-a-box are a trap for those users.
That's been basically my consistent experience with ugreen - their products are cheap and work on the happy path, but edge cases are not accounted for.
> None of the parts are broken. Together they don’t work as they are supposed to because of some super intricate process of how usb works when it comes to acknowledging packages.
Typical Apple shit. Never had an issue with my Lenovo dock and 2.5G. Additionally, not having an eth adapter built in, in 2026, is enshittified hardware. Typical Apple again
It is to do with Apple. It is because they are wrapping their USB protocol in a Thunderbolt tunnel to the monitor which is adding the latency. This adapter plugged directly into a device, or into a normal USB 3 hub works completely fine. Only when going through a Thunderbolt tunnel does it add enough latency to buffer the connection.
You know Thunderbolt is an Intel technology, right? There are plenty of Windows laptops that have Thunderbolt ports, and people use them with Thunderbolt docks. The article author even confirmed that this issue reproduces with a Lenovo Thunderbolt dock.
I think you could improve the post's clarity of outcome a bit.
While it's completely right, even when having read it fully, it isn't so easy to understand unless the reader knows some things (see https://news.ycombinator.com/item?id=50588459 and its children), namely:
* That the "packet size 1024 bMaxBurst 3" (making the 4 KiB window size) is fixed for the whole connection, determined by the RTL8156B, and cannot be negotiated higher, no matter if the monitor is in between or not.
* That this window size is large enough for the NIC---Mac connection to run at full speed, but not enough for the NIC---Monitor-USB-Hub---Mac connection to run at full speed, because the added latency makes the BDP too large.
* If possible to measure, it might be illustrative to measure the NIC---Mac latency without hub directly, to compare it with the `7.6 µs` number going through the monitor (like cibyr suggested). That'd make it more obvious that the monitor increases the latency.
And I agree with dcrazy's sibling comment that the RTL8156B in the UGREEN _is_ to blame. It uses a send window size that cannot possibly deliver the advertised bandwidth behind a hub.
Anyway, great post that shows that just because you buy a device that says N Gbit/s and delivers N Gbit/s when directly attached, you cannot be sure that it'll be equally fast when attached to a form of "longer cable".
I've only read the headline: USB degrading speeds?
(five minutes passed, I skimmed through TFA)
Ah yup, of course it's somehow related to USB.
It's like the infamous line: "It's always DNS" (and when it's not it's the firewall).
When download speeds are degrading like that, it's always some weird USB thing.
In the earlier days people didn't know that, for example, plugging a monitor into an USB hub could, as a result, make the rest degrade to USB 2.0 speeds. Unplug the monitor, HUB is speedy again now that it doesn't need to deal with that 4K / 60 Hz or whatever monitor data.
It's not a criticism or anything and I'm not saying that's what's going on here: I'm just saying USB has many gotchas and there can be, as in TFA and as in many other (document or not) cases, weird things happening.
At times (and doubly so when troubleshooting), you're better to simply try to plug your USB cable directly in the computer instead of in a USB hub / monitor USB hub. (EDIT: ah yup, ofc eventually he try that)
There are thing too complex to ever be secure and then there are things too complex to not have gotchas.
USB is... Wild?
P.S: and then of course in addition to this USB wildness there's the mad insanity of USB... Cables.
This reminds me of a (completely different, but felt equally) bizzare problem with ghostty (the terminal emulator) on Linux and Steam, specifically Baldur's Gate 3.
At the start of the game after an update, BG3 compiles the (Vulkan) shaders. To avoid hammering the computer, it checks the load, including I/O pressure, and then throttles itself. It turns out that Ghostty uses io_uring, and even if it's not doing anything this somehow counts as pressure. This was made worse by the fact that I usually have a dozen or so Ghostty windows.
End result, BG3 compiles shaders using 1 core, throttled, instead of all available 20 cores. Instead of a minute or so, it'd take several hours.
(For closure: I configured ghostty to use epoll instead. Probably not optimal but works around the problem.)
I hate this. We have a whole operating system doing its absolute best to prioritize hardware resources. Don't be out here throttling yourself.
Especially for something like compiling shaders. It literally never happens unless the user is about to go play a game, which is going to consume most resources anyway. To the user, there's nothing more important running right now.
> To the user, there's nothing more important running right now.
Well, to a certain degree. I have a relatively uncommon GPU, and run games on Steam (and Linux), so literally every game does shader compilation for me. Even with a beefy CPU and beefy GPU, if the shader compilation uses more than 16 threads, my computer basically freezes until its done with it, as it's so performance intensive, kind of have to throttle things if I want to be able to use the desktop for the 10 minutes the shader compilation takes sometimes.
Edit: "Do you use Linux? That only happened to me when I was using a dram-less ssd for swap and doing IO heavy operations on the NFS mounted drive"
I do! I do have NFS drives, but not for Steam games or anything else relevant. I'm also not using swap at all. RAM seems fine during this, it's just pegging all cores to 100% usage immediately and stays there, making everything else suffer.
> Even with a beefy CPU and beefy GPU, if the shader compilation uses more than 16 threads, my computer basically freezes
Sounds like a scheduling problem that should be adressed at the OS level.
Seems a deep one if so. I've tried a bunch of alternative schedulers to try to avoid the issue, like Bore and BMQ, but seems to happen regardless. Been like this on previous computers too, fwiw, unless I turn down the concurrency for shader compilation stuff will lag heavily until it finishes.
> tried a bunch of alternative schedulers
1. Have you tried enabling full preemption as well? I run Debian, and before I did that I had lots of stutters from any kind of background load thanks to dynamic preemption.
2. Even then, ZFS was causing stutters until I lowered it's priority as well (spl_taskq_thread_priority=0)
3. On non-ZFS drives, try setting your I/O scheduler to Kyber for NVME or BFQ for everything else. "mq-deadline" isn't great for interactive use on a SATA SSD, even for non-boot drives, and even NVME sometimes stutters under "none" for some people.
> I'm also not using swap at all.
4. zswap (combined with zstd compression, "min_ttl_ms=1000", and "vm.page-cluster=0") and "defer+madvise" memory compaction can help. (Something about the kernel really liking having swap on-hand.)
Steam runs shader compilations at batch policy and nice 19, so without RAM or I/O bottlnecks even gaming shouldn't be too bad (though from personal experience it still induces some stuttering). It's just most distros are tuned horribly for this. (Not to sound flippant, I understand the frustration of "just try this" and it doesn't work, but it's everything I can think of RN.)
Many thanks for all the things to try, definitely will take a closer look and explore what you've mentioned next time it happens! :)
When you do, I forgot to mention I had difficulty getting a couple of settings to stick on reboot. Some had "proper" methods that were bugged, others seemingly needed a startup script anyways, so I just stuck those two groups into one:
...and some quick-n-dirty scripting[1] for ZFS drives.
Fellow HNer 'networked' pointed out[3] that the compressor type was because Debian wasn't loading the zstd module on startup for some reason, so you could fix it that way as well. Dunno your distro, hopefully you don't need these hacky workarounds that continue to annoy me.
[1]https://www.reddit.com/r/zfs/comments/1qerlvn/how_to_only_se...
[context]https://www.reddit.com/r/linuxquestions/comments/1qenr06/id_... and https://wiki.archlinux.org/title/Improving_performance#Kerne...
[3]https://news.ycombinator.com/item?id=47505661
[4]https://openzfs.github.io/openzfs-docs/Performance%20and%20T...
[5]https://wiki.archlinux.org/title/Zswap
You could try cpu pinning or ulimits to limit the program from using one core so there's always one free.
But it's also possible that the problem is in the kernel and the shaders are actually doing some system call that's locking up the kernel. Like maybe there's a bug in your cuda or nvidia driver that's causing the kernel to lock.
Do you use Linux? That only happened to me when I was using a dram-less ssd for swap and doing IO heavy operations on the NFS mounted drive, since the system will use ram as a buffer (which pushes things to swap)
My steam machine is netboot diskless nfs only.(probably a mistake, but I have a nice NAS and so leaned hard into it) However, having no swap worries me. for openbsd they document how to swap to nfs. But the linux docs say not to do that. What is the best practice here? swap to an iscsi block?
zram+earlyoom sounds like the right answer there. Compressed ram works as the low memory signal
> Even with a beefy CPU and beefy GPU, if the shader compilation uses more than 16 threads, my computer basically freezes until its done with it
Strange. This never happens to me on Windows. I only notice that the fans spin up a little because all cores are pegged.
i've only seen the shader compilation on linux.
fedora45/proton/steam user.
What you are describing is not normal. Which GPU? I have not seen this on Nvidia 4060/4070, or AMD 580, 6600, or 9060/9070.
Well, as mentioned, the hardware is relatively uncommon; 9970X + Pro 6000 GPU.
Especially on Linux, the default throttling just doesn't work very well.
My motherboard crashes when I peg all the cores. Presumably it can't maintain the core voltage under load. Vendors need to guard against these scenarios by being extra cautious because it's a hard problem to diagnose that they'll get blamed for.
Guarding against that seems like a bad idea, but even then just use a couple less threads than physical cores. Don't get too clever.
You in particular should probably disable a couple cores until your hardware stops being defective. Have you looked into updates and voltage droop compensation settings?
It’s just reality that motherboards can malfunction or degrade in peculiar ways that a manufacturer’s quick boot up checks cant always catch. No matter the reliability claims.
If they want to cater to customers who foolishly reject reality and take every claim from manufacturer’s at face value… then foolishly misaimed blaming seems expected?
But if they just roll over and swallow the foolish blaming without even challenging customers to double check, then they are also facilitating more nonsense.
That sounds like a faulty motherboard. Return it.
Users don't care that it's a problem with the motherboard. They have a working computer, they buy a game and it doesn't work, they blame and return the game.
Users can be unreasonable, but in this case I agree - if I reported an issue to a vendor and their response was "return your motherboard" I wouldn't work with that vendor again.
Sounds like they don't have a working computer. Computer users occasionally do things that end up pegging all cores, so I'm pretty sure that someone in that case would know their computer has some kind of issue.
We can't form the world around literal bad hardware. That's completely insane. The world would fall apart.
> I wouldn't work with that vendor again.
Possibly rightly so, if it's bad motherboard hardware. Could also be a bad power supply, bad ram, bad CPU, user misconfiguration, etc. It's a complex system, where everything has to NOT be bad for it to be deterministic.
There's a necessary level of competence required with self built systems, or any other task in human life that you're not paying someone else to do for you. That's fine. Go buy a MacBook or a pre built system if that threshold of competence can't be met. That's why they exist, for people that can't or don't want to bother.
Haha, that really depends. If you look at it from the viewpoint of a failsafe, like brakes in lifts that activate without central control, backup parachutes, etc. Then it makes some sense for resource intensive software to guard against weird edge cases like this. It's just that they measured the wrong thing.
Those have nothing to do with the required assumption of determinism in the hardware that software must enjoy.
You're examples are flawed. Those are to safely halt operation on failure. What you're suggesting with the software is to limp along pretending everything is ok.
The proper test, more in line with your others, would be to catch failure and halt operation. Since, in your case c the human isn't competent enough for that, it would maybe be a self test initiated by the operating system, then a refusal to continue when the hardware problem is identified, emulating all the periodic inspections that are part of all the proper operation of all the physical systems your mentioned.
Ffs Apple's touchscreen keyboard is just abysmal these days. My apologies, it's too late to edit that mess.
... but is hardware deterministic?
That's a trick question as stated, and not really relevant to the discussion of if the end user software should put in safeguards for bad hardware. ;p
There's very very rarely cost pressure that allows non-determinism inside of the operating parameters, and these almost always involve radiation or gamma rays (which many of us, outside the banking industry, pretend aren't relevant) or some "impossibility", like the corruption of data that results on the same checksum.
We don't use elevator brakes or backup parachutes as an excuse to let the primary fail under normal load. Those are for emergencies.
I think the elevator analogy to this bug is that the elevator goes too fast and triggers the brakes if you input target floors more than 8 apart. You don't fix that by having the user or user software be more careful about button placement, you fix the hardware/firmware.
It doesn't matter what's "fair". It matters what people do.
https://devblogs.microsoft.com/oldnewthing/20080206-00/?p=23...
Why aren't you storing all your variables with reed solomon encoding in case you have a user with bad ram?
> in this case I agree - if I reported an issue to a vendor and their response was "return your motherboard" I wouldn't work with that vendor again.
Why would you say this? If the problem is your motherboard is in fact broken (or you have used a high-end CPU on a motherboard not rated to power it), and the vendor correctly diagnoses it despite it not being their problem, you think this is bad? You want them to change their software, making it worse for everyone who doesn't have broken hardware, just to make your broken hardware work?
Sometimes if such an issue affects a common hardware configuration it can make sense from a capitalistic sense to "make it work" for more users but if that's the case that implies you would know what the hardware configuration involves and test accordingly so the workaround would not have negative effects on users who don't need it, and either way that's not the situation we're talking about here where the user has already acknowledged that their hardware is wonky.
>everyone who doesn't have broken hardware
To be fair, such people don't seem to exist in real life. Between i586 and zen3, I have never owned a machine that boots the current kernel tip without there being one or more notes about some workaround in force, to deal with some detected broken-ness of hardware or firmware embedded therein.
They'd probably be quite happy not to work with somebody that makes their broken hardware everybody else's problem, though.
Expecting software to deal with a user that can't be bothered to fix a pretty fundamental flaw with their machine is unreasonable.
That seems like a you problem - undervolt your cpu.
Disable turbo boost in the bios
"I have a faulty hardware so everyone in the world should take care not to trip my hardware. Also I don't care about anyone else."
Your computer is unstable and broken. I would try swapping PSU or troubleshooting that issue. This is not normal and no, this is not the job of anyone else to solve or consider.
Yeah I had this problem and it ended up being the PSU. I tried a lot of different bios and driver versions swapped cpu, memory, then eventually upgraded motherboard with new cpu and memory (all these are the suggestions from people who had the same bios code on forums) and it kept coming back… then finally replaced of the PSU which actually fixed it.
Don’t be like me! Replace the PSU! Do the math on how much your system could possibly need and then double it!
Checking the voltage and specifically which voltage the automatic overclock mode of the board applies though could also be worth the time, before replacing the psu otherwise.
And doubling the wattage isn't necessary, just be comfortably higher on the 12v line and get a good one.
But why use that metric. The OS already has support for this.
Why not just re-nice the process or whatever equivalent for the threads that do the compiling? Tell the kernel what’s going on, it will handle it for you.
Don’t hack your own solution.
I'm actually ok with a blocking progress bar. The whole async approach falls apart once it starts degrading performance.
> To the user, there's nothing more important running right now.
One special case: game streamers would likely rather the game stutters than the stream encoding stutters. (Especially as long as the capture of their own face+voice remains smooth.)
Amazing. How did you figure this out?!
I asked Claude to help me debug and research it.
It looked at fossilize code (open source component used in the process), found a couple of threads online that suggested io_uring/iowait and told me first to try closing the ghostty windows and then to try to switch to epoll. This did the trick.
I have a very vague understanding of what's going on (actually better now that I've look at the links/resources from the replies in this thread!), and would probably have been able to figure it by googling in my own. But, my first hunch was that it's a problem with steam, proton, or the game, so I'd probably waste a lot of time in that direction first.
Did you report it upstream? Proton, dxvk, Larian or Ghostly? It would be neat if we fixed it for everyone
What are AIs going to do now that people are having these sort of debugging conversations in private with AI instead of in open forum threads? I worry that these forum posts will dry up and AI will start to get worse over a longer period of time.
Now that I write this thought out, I suppose that when AI gets worse in that way, people will go back to forum posts!
I hope it is trained on people’s chats
That was my thought as well. This is precisely why the labs are so intent on using everyone's prompts. It's kind of the obvious thing really
I hope the AI agents go report these things autonomously at the right places.
For these small scale (but significant!) debugging sessions, I think AI will be a net positive. Its between discovering the problem with AI vs spending a few minutes searching and giving up.
Interesting to see the "io_uring vs io pressure measurement" issue come up here; it's turned up on qemu-devel recently too: https://lore.kernel.org/qemu-devel/4EF13FE7-E6F3-42E2-83F9-6...
If I am reading the thread correctly, the io_uring maintainer's position is that applications trying to use the metric that "poll via io_uring" handles differently from "poll via epoll or select" are fundamentally misguided and measuring something that doesn't make sense, so they don't want to slow down io_uring to placate those apps.
From https://lore.kernel.org/qemu-devel/962d44d3-ca7b-4cc8-b0f7-7...
> iowait is fundamentally incompatible with sane accounting in the first place, that is the problem. It really tells you nothing, and was deprecated since the days of uniprocessor systems. The problem here is not how iowait is done in the kernel, it's that people are using this metric in the first place. Which is why you just want to NOT do that, and do sane accounting on your disk drives, if you care about that kind of thing.
> Which is why I refuse to cater to any kind of iowait accounting metrics, it's just adding overhead to support a metric that is largely nonsensical in the first place. It doesn't answer any questions that an application/tier owner would want to know. Those questions might be things like "am I I/O bound?" or "is the device I'm using being slow?". I wish people would realize that, rather than continue to place value in iowait as a metric for ANYTHING, and stop trying to work around it.
It sounds more like their position is "iowait is a useless metric" and that polling with io_uring completely breaks iowait metrics but fixing it would add a minor slowdown so they refuse to do so because they don't think it's a valuable metric to begin with.
Is there a good replacement for iowait?
Actual utilization rates and latency for each device.
Even the grandparent bug report says that iowait is high while utilization is low. They know full well that utilization is the figure that reflects reality.
In this case "iowait is high while utilization is low" is a deliberate design choice made by the io_uring maintainers, though.
It used to under-report - but that led the the kernel wrongly downclocking the system as it believed it was idle when it wasn't.
It doesn't want to do somewhat-accurate reporting, as that would have a small per-operation performance penalty they are not willing to pay.
It now over-reports as that both avoids downclocking and is cheap to do, but the drawback is that it makes the iowait metric completely useless to any potential consumers. Which they don't care about because they believe it is inherently flawed and shouldn't be used.
Its really wonderful having such devoted and caring maintainers. Open source users are truly blessed.
Utilisation rate is also arguably a bad metric either. For an NVMe with deep queue buffers you can see 96% utilisation and think it's near its peak capacity, but in fact if you keep increasing the queue depth you can get like 5x more performance with only moderate increase in latency if at all :)
More context: https://lwn.net/Articles/989272/
iowait may be poor, but not fixing the regression added to Linux by io_uring clearly breaks userspace, as the BG3 story proves.
io_uring you say… one of the most infamous subsystems for security, best reserved for stringent performance requirements. Why on earth would Ghostty default to that?
Maybe because one of ghostty's selling points is performance.
Also if you're running untrusted code that attacks your not-very-common terminal emulator, you're probably in more danger from the "untrusted code" part. Threat model and all that stuff.
An app can use io_uring_set_iowait to change that behavior. Since a terminal’s waiting on I/O isn’t hugging the disk but just waiting on mouse and keyboard input, that seems like the honest thing to do.
From https://wiki.archlinux.org/title/Steam
Mentioning io_uring: I encountered this exact problem when I deployed Infisical on a Kubernetes cluster. Infisical requires Redis for queueing, so I used Dragonfly since I had the operator installed. The problem is that Infisical uses Bullmq and [they require you to enable the use of undeclared keys](https://www.dragonflydb.io/docs/integrations/bullmq#tldr), and more than that: [hashtag locking](https://www.dragonflydb.io/docs/integrations/bullmq#1-hashta...).
What does this has to do with the `io_uring`? It uses io_uring to achieve this. After I deployed a test cluster, I saw how the CPU bar went beyond 100% and how the IO pressure was constant, completely fudging my metrics and my ways to detect any real pressure. The weirder part was that the cluster was running as usual, without any other issues apart from that false positive on the metrics.
My fix was to [disable iowait](https://www.dragonflydb.io/docs/managing-dragonfly/flags#--u...). I don't know how it works, but I read some articles randomly at that time (1y ago), and I figured it out. But I never got the real reason why.
And yes, the performance was awful, and your comment made some more light in this encounter I had, LOL.
Edit: why is markdown not working O-O
> Edit: why is markdown not working O-O
HN doesn't use Markdown, and it supports only a limited set of formatting options to begin with (text between * is italicized, text that starts with two spaces on a new line is printed as code, and that's about it). It also disables emoji and a few other Unicode characters.
>bizzare problem with ghostty (the terminal emulator)
I think people who use hipster toys instead of tried and tested ones, basically deserve all the repercussions.
> Ghostty uses io_uring
And draws a 3D surface for a 2D terminal. We call this "software engineering".
It's there a GPU around that has 2d acceleration? My impression was that they don't exist anymore.
Better description: "My USB network switch was cutting my download speed".
The fact that it's a monitor is a little bit irrelevant.
Same old adapter worked perfectly if the monitor was taken out of the path. Kind of an interaction thing with both, really.
The twist is it would happen with any additional device in the signal path. The mere presence of a hub causes the USB control message to take slightly longer to reach the adapter—about the same amount of time it takes to receive 5 Ethernet frames at 2Gbps.
The fix is to cut the overhead by sending more than 4 frames per interrupt. Unfortunately that required buying a new adapter with a different chip.
The fix is actually to plug it directly into the host, and not use a hub. Very common thing to need to do with cheap or older USB Hardware. Paying more money for something new when the old one works fine is pointless and wasteful.
This isn't always a viable option, see for example the whole concept of docking stations.
The proper fix is to get the device manufacturer to release a firmware patch which accounts for a worst-case scenario acknowledgement round trip time by sending bigger bursts - but good luck with that...
The datasheet[0] makes no mention of field reprogrammability, but that likely wouldn’t help anyway. The maximum burst size is probably dictated by how much physical RAM the chip has to buffer packets.
[0]: https://www.lcsc.com/datasheet/C41376388.pdf
The datasheet does mention support for EEPROM and SPI storage, eFuses, and a USB interface with a vendor-specific "in-house configuration" for "enhanced features and optimized performance". It even explicitly mentions some other features being configurable via "register", but never clarifies how to do that either.
What you've got to remember is that with nontrivial chips the datasheet is primarily intended for purchasing decisions. It is a high-level summary of the most important features and specifications. All the technical details are going to be in a separate many-hundred-page Application Manual.
Besides, the physical RAM likely isn't the limitation here. It has explicit support for 16k jumbo frames, so it is going to have at least that much buffer space.
The only reason it wouldnt be a viable option is because Apple keeps removing usb ports from its laptops and forcing everyone to use hubs and dongles.
On my Macbook Pro I have three ports.
One external monitor (second via HDMI), the Ethernet dongle, and the Thunderbolt dock.
So for the external SSD, I am already relying on the dock...
Again, a failing of Apple to not provide more internal storage or allow you to change it easily.
They provide 8TB of internal storage... as a $3500 upgrade.
The failing seems rather intentional.
That is exactly my point. If they had made it as easy to swap your storage as any PC, then you could pop in an 8Tb SSD for a fraction of the cost and not need to drag around an external disk everywhere.
This would have been a valid criticism 10 years ago. Today, a MacBook Pro comes with 3 Thunderbolt ports, each driven by an independent controller. You get a full 40Gb/s on each port. This is best-in-class connectivity.
> The fix is actually to plug it directly into the host, and not use a hub.
This was true a long time ago. For some reason the "level of abstraction" cancer has hit also PC HW. You have to account for countless levels of busses and hubs when working with serial ports for example.
There are plenty of USB hubs with built-in NICs behind the hub, that can achieve full ethernet bandwidth. I even have a hub with a built-in RTL8156 and it achieves 2.4 gbps. And a separate RTL8156 plugged into a random hub also hits that.
So it certainly seems possible to have an additional device in the signal path.
> 2026 Studio Display XDR, with everything plugged into it so switching laptops is one cable
I think I'd have accepted having two cables from the get-go. And if I observed this problem, the first diagnostic I would do is to plug the Ethernet directly into the laptop.
Fewer technical pieces in a puzzle sometimes trumps ease of use.
The first rule of troubleshooting is to try to cut down the search space as much as possible with each question. My first reaction would have also been to plug directly into the computer to see if there's a bottleneck or failure at the monitor.
That's what I did in the blogpost.
With a $3300 monitor? Not a chance. When you are dropping that kind of money on your setup and it performs worse than a $35 docking station, having to mess around with that kind of ugly workaround is unacceptable.
"Ease of use" is the whole point of monitors with integrated docking stations, or even of USB-C in general. Going back to a worse UX than a 2005 Dell laptop dock? Hell no, not going to happen: something is broken here and it will end up getting replaced.
The Studio Display also takes an extra half second to activate its mic vs the onboard MacBook mic (~0.10 vs ~0.60) which is something I would never have noticed until I tried using the various whisper/parakeet STT apps.
I view anything "USB" with more than a little suspicion after my hellish journey into running RAID over USB drives.
You really must hate yourself to attempt that.
I've got a few USB drives that are always attached that I do stuff with.
To pick a not-RAID example, I've got an old (2012-vintage) 180GB SATA SSD, from Intel, on my desk. It gets used as scratch space for big-ish file operations, where it gets hammered hard periodically, usually in bursts. It's plugged into a cheap SATA <> USB 3 adapter, and is otherwise-unadorned. It is bus-powered.
It's usually connected to the back of the PC, but it's been plugged into different USB ports (with different controller types) and also external hubs.
I don't recall what its exact status was when I starting doing that with it after pulling it out of a gently-used Thinkpad T530 about 5 years ago, but I do recall thinking that it had lots of life left.
Here's where it is today:
And the pièce de résistance:
That last one is important because in this application, ZFS automatically and continuously provides end-to-end verification, and it's been doing...perfectly fine.
The other USB storage devices here that are used 24/7 are similar in this way. It's not hellish at all. It's quite lovely.
I'm inclined to further extrapolate that it would not have better data integrity when connected directly to a SATA port on the motherboard, or in a disk shelf attached via iSCSI or SAS or some other more-stodgy thing. It's hard get results that exceed beyond perfection.
(But in this example, I'm not chasing down minute bus timing issues with multi-gig networks. That's a very different kind of discussion.)
In my experience, data integrity and long-term reliability of products that actually work correctly in the first place are not an issue, nor is performance if you don't have unrealistic expectations.
It's not like hard drives and SSDs underlying USB storage are engineered to lower quality standards than internal drives; they're typically similar if not identical. The only instance where I've observed special-purpose hardware is drives incorporating USB bridge logic directly on the drive's controller rather than on a separate board connected via a traditional SATA interface as a cost-cutting measure.
Problems I have seen are typically the result of hardware or firmware bugs leading to frequent disconnections or even failure to attach in the first place, power problems due to low-quality power supplies, overloading controllers or hubs with too many bus-powered devices, or knowingly or unknowingly using hardware clearly not fit for purpose (expecting good random read/write I/O performance from an array incorporating one or more of SMR drives, SATA port replicators, and non-UAS bridge chips; expecting hot-running drives to remain within their operating temperature range in environments with relatively high ambient temperatures when installed in enclosures with little or no airflow, etc.).
I was under the impression that in the Apple ecosystem, you pay for better integration. Apple’s XDR display should really be able to handle this better. Or have the drivers negotiate with the chipset better.
I mean, the guy’s using a USB Ethernet adapter with one of the cheapest NICs on the market that can do 2.5GbE, and it’s the behaviour of its firmware that caused this problem. It would have performed identically behind any hub. How is this apple’s fault?
How cheap are we talking?
I've never had a problem getting a steady 2.5 Gbits between a pair of USB NICs purchased for $20 several years ago on Amazon.
"Better" is subjective. I think monitor is a better title, because it gives misdirection, so we the readers learn the same time OP did in the story. Unreliable narrator and all that.
I understand that not everybody likes this, because it can feel like a trick, or patronizing, or lying. But in my experience, it works very well for people.
I didn’t like it because it was a very generic and common technical occurrence that I would not be interested in reading about normally.
Now I feel as though my time was wasted by a dishonest author trying to get me to believe that their story was interesting when it was not.
In the grand scheme of things, it’s not a very big deal, but I feel quite annoyed since I was baited into expecting a fun read.
Yeah I feel you. If you start wondering how a real monitor could do this, only to discover a common other component, it's a disappointment.
…"and I didn't even notice, because I pay for at least four times more bandwidth than I'll ever use, outside of a speed test."
On a single device at least. Get a partner, a set top box, some gaming hardware, multiple computers working together…
or mentioning the family emergency, map included.
Since lots of comments misunderstood the article, the extra latency from going through the monitor meant his laptop couldn't acknowledge the packets quickly enough. That led a buffer on the ethernet adapter to fill and that led it to drop packets.
A different adapter which required less frequent acknowledgement fixed the problem for him.
It is quite a nice post which gets to the bottom of incredibly opaque behaviour of USB hubs.
Probably a stupid question, but why is USB behaviour so opaque?
There's not much monitoring built into usb hubs. It's actually far less when you think about all the monitoring a network switch has.
The fix is that it could send more packets per ack not that it needed to happen less. If the buffer was bigger but still capped at 4 packets per ack, the full buffer drain speed would remain at ~2.
Speaking of opaque behavior, I installed a 7-port PCIe USB card for a coworker last week that I noticed consists of a four-port controller with a 4-port hub connected to one of the ports, with no identifiable markings or documentation suggesting differences between any of the ports.
I also once had an unpowered USB 3.0 switch that somehow managed to supply power to my workstation's +5V standby rail after disconnecting it from the wall, presumably from the PC on the other side of the switch, since the only other connected devices were a mouse and a keyboard (not that powered devices' B connectors should be supplying power to unpowered hosts' A connectors in the first place).
Pretty much every >4-port USB hub or interface does that, yeah. USB ICs with more than four ports are extremely rare, and chaining two of them has no meaningful impact on performance. The biggest user-facing impact is a funky-looking USB tree view in the Device Explorer, so why bother drawing attention to it?
> chaining two of them has no meaningful impact on performance
Until you start to copy a lot of small files on your USB attached storage.
That’s really still the adapter though because daisy chaining usb-c exists and is kind of a given so wouldn’t that latency exist in any other configuration such as a dock? Do you know anyone who plugs Ethernet into their laptop? I don’t.
Meta:
> Somehow I am one of those few very lucky people to have the opportunity of owning a 2.5 gig symmetrical line for my internet.
I'm not going to complain against more bandwidth (as a GenXer I stared my online journey at (IIRC) 9600bps (maybe lower): much celebration with V.32/V.90), but I'm curious to know how many residential use-cases can take advantage of that much bandwidth. Linux ISOs?
How many services allow that much speed on a single stream / (TCP) session? Maybe you could have five at 500 Mbps?
Certainly downloading security updates or games may be quicker, but I would think that above 500Mbps (and certainly 1 Gbps) there would be major diminishing returns. Is anyone regularly getting close to that utilization in their daily work / hobbies (and not just a momentary spike on patch Tuesday)?
You definitely notice it
99% of the time I don’t need it. However, when I do need it, it sure is nice to move quickly during updates or downloading sdk/assets.
Even better is those can saturate as much bandwidth as possible while the rest of the family are uninterrupted with Netflix or video calls.
I'm seriously considering using an old fiber optic Thunderbolt 2 cable to establish a point-to-point link between my desktop and file server.
The only thing holding me back is that'd I'd need to relocate the Fibre Channel card I use to connect its tape drives outside the case to do so, either in a Thunderbolt enclosure or by using a janky extender to connect the x8 FC card through a x1 PCIe slot.
Given that the system (a repurposed minitower workstation) already runs out of sight in a low-traffic area without a side panel due to SAS cables that don't fit with the side panel installed, the janky extender solution is tempting.
I do homebrew updates regularly, and quite often it can take more than an instant. On my 40Mb connection it’s slow.
Run it in background, don’t update so often perhaps; I guess I just update as tic.
Torrents, and things like downloading games from Steam feel really good if you have a very strong connection. A lot of games can be downloaded in the time it takes to go grab a drink from the fridge. It adds some nice quality of life, there isn't as much need to keep a large downloaded set - just download when you need it, and it'll be there quickly.
I think going from something like 100 Mb download to 1 Gb is slightly overrated. You hit heavy diminishing returns. Symmetric Upload, for me, is what really changes your quality of life a lot. Uploading a youtube video suddenly isn't an event, it takes a few seconds. Want to upload an image in any chat? Done before you consciously see it uploading. Heavy commits are instant, even with a bunch of resources. It changed my behavior. I started uploading more stuff to remote servers because the cost in time got so much lower.
Yup as someone on a 1000/20 link (can’t get anything better in my little village in BE), decent upload is far more useful than more download after a point (far before 1gbps).
Depends on how many people you have in your household and what they do with the connection. Mine has two people who are both gamers and who use streaming media. 1000/50 makes a huge difference vs 500/50 when both are downloading.
Depends what you do really. I enjoy streaming 80gb bluray muxes without needing to store them first. I have an option to double to 2500mbit/s in my area for an additional £10 but can't yet think of cases where I need it.
> Maybe you could have five at 500 Mbps?
Let's say you're on a Meet call with a dozen of people, one of them presenting their screen to explain an ongoing incident. You're following along while checking your dashboards and code on a remote server. You're of course keeping Slack sessions in the background.
I don't think you'd be saturating a 2.5G bandwidth, but perhaps about half of it ?
If your family is also using it in parallel that could get close to full use.
That wouldn’t come close to saturating 100 mbps. Screen sharing even at 4k is ridiculously efficient, maybe 4-5 mbps for what you’re describing where most of the image isn’t changing most of the time.
> Let's say you're on a Meet call with a dozen of people, one of them presenting their screen to explain an ongoing incident. You're following along while checking your dashboards and code on a remote server. You're of course keeping Slack sessions in the background.
This is closer to 1-10Mbps than 2.5Gbps. Slack is negligible bandwidth. Video conferencing is high compression, low bandwidth video.
The 1G and 2.5G lines benefit bulk file downloads.
Maybe so but even at 10Mbps that's a lot of the 20Mbps upload you get from a residential cable line.
A high quality 60fps 1080p gameplay stream is 4Mbps. That mostly still frame powerpoint presentation is completely negligible, and the 6 thumbnail sized videos you see at 30fps are too. You can run a whole office of people on that call on smaller connections.
4Mbps is pretty stingy for 1080p60, you are likely going to run into terrible artifacting on anything that isn't flat colors and static images. Twitch runs their 1080p60 at 6Mbps and a starry background will absolutely shred the image to shit.
"4Mbps" also isn't super meaningful without knowing what codec is being used. I'm currently watching a 10Mbps sports stream of some unknown codec and it's pretty good, but I can pretty readily pick out artifacts if I look for them.
I recently moved, and was around 10~15Mbs for a while waiting for my ISP to pull the cable.
That didn't cut it at all, even just for Meet the quality was miserable.
> Let's say you're on a Meet call with a dozen of people […]
I have been on Zoom and Google Meets calls with that many people on my laptop which has a Wifi connection of ≤400 Mbps (sometimes <200Mbps). You don't need >500 Mbps for online meetings.
Are you in meetings where everyone is sending 4Kp60?
I (for one) would definitely NOT want to see the faces of my colleagues in 4K.
lowres, please.
only 4K if the feed is from a full room with +10 people in it.
If it's a call with a dozen people, media will flow through Meet's server. Some group call servers do p2p for calls with two or three people, but approximately nobody does for a dozen.
Where I work, group call participants send about ~1Mbps of video, and maybe more for screen share, could be ~2Mbps.
So you're looking at 15 Mbps inbound and 1 Mbps outbound for the call. Plus some overhead for congestion control. That's not even 1% of your 2.5gbps connection. IIRC Meet uses higher limits, so maybe double it to 30 Mbps.
You don't need gigabit for video conferencing, unless you're running the conference out of your house. Or you have 20 people all on conferences. You may want some QoS if your usage is near your bandwidth limits. Conferencing is sensitive to jitter and loss.
At my current ISP, speed is either 100M or 1G, so if you had 3+ people who conference simultaneously, 100M might well be limiting, so you would probably want 1G ... otoh, conferencing is usually bandwidth adaptive too, so it would still work, as long as everything plays nice with congestion.
I think steam lets you get to at least 2gbps.
An interesting side effect is that bufferbloat becomes less of a problem - you’re more likely able to pull 1gbps without spiking your ping.
But to be honest unless you have a household with multiple people, it’s something you mainly notice occasionally with updates, backups, large assets, etc.
I have 10 gig into my house, and most services can’t saturate it. Microsoft is crap, big game downloads don’t even top out the gigabit link in the xbox. Apple gets 2-3 gig downloading xcode updates and the like.
I wish I could get 10Gbit; friggin' AT&T lol.
Fetching from Google or anything on S3 can definitely fill your pipe
A lot of times downloads for games are going to be compressed/encrypted and decrypted/decompressed on the fly. It's why CPU power is actually usually the limiting factor for Steam downloads, then disk I/O capability.
The major CDNs routinely deliver full throughput.
It’s great when experimenting with LLMs because I can quickly try new models and download from the internet almost as fast as if coming from my NAS.
Game updates are also a great use case. I don’t game often so when my friends want to play I often have to update the game. Some patches are 60GB or more. When they come down at full line speed that’s valuable time I’m getting back to spend with my friends in game.
Also hyperscalers. On several occasions I've used the Netflix content freely available on S3[1] as an ad-hoc download speed test.
[1] https://opencontent.netflix.com/
there is also fast.com which serves from the netflix cdn
I doubled my measly 20 MPBps upload speed to 40 simply because of how long it was taking me to get rendered assets to someone. Having higher bandwidth than that (eventually, I hope) would be a real help in video editing.
My cable is I think up to 2Gbps now and I regularly see game updates in steam hit like 1.4gbps. Probably it is closely located in a CDN I would assume.
2.5Gbit (or 5Gbit in my case) is INFINITELY helpful if you run k8s clusters. Container images download as quickly as moving a big file on a spinning rust hard drive even if someone else is watching TV or doing equally intensive stuff.
Watching TV is nothing, even 4K netflix stream is around 15-20 mbit and shouldn't spawn many concurrent threads unlike say bittorrent.
I don't believe they asserted that watching TV was intensive, hence the followup "OR doing something equally intensive (as downloading a lot of container images)".
My mother, who lives alone, got hoodwinked into upgrading her 300 Mbit cable connection after letting it slip that she sometimes plays games on her phone while watching TV.
4K HDR BRrip in 1 minute
Do not underestimate the possibilities. I have 1.0/0.1Gb cable, and doing remote access is a complete joke. Even streaming a 1080p movie up from my house is extremely challenged.
This also locks me out of hosting my own “cloud” storage at home where it belongs - I don’t want to be out and waiting forever to get a file, so I stick to traditional cloud services for now. Slower than local storage on the LAN of course though.
A fiber isp is digging trenches soon. Cannot wait!
I mean, yeah, you have a 100 Mbit/s (or 12.5 MB/s) upload rate under ideal conditions. If my math is right - haven’t had my coffee yet - that’s just 4% of the OPs bandwidth.
Yeah upload is usually the game changer for home internet. I have symmetric gigabit and I would happily halve my download speed if it meant I could double my upload. Sending large backup .tars to cloud storage, sending data to VPCs, object storage, allowing more concurrent Plex users remotely, etc etc, 2Gbps would make so many things easier.
I have about 45 Mbit upstream on my gigabit connection, and I've never had a problem with remote access so long as it's not competing with other traffic on the connection, in which case USB tethering on my iPhone works well enough.
I suppose this also depends the protocol and the implementation. VNC often feels bad even over a point-to-point Ethernet link, while RDP feels fine for basic desktop applications on any sufficiently low-latency connection (within reason; it's not exactly snappy over dialup or basic rate ISDN).
Yeah basically nothing. I have 900 Mbps and pretty much the only things where the other end gets about 250 are Steam, Docker images, and Bittorrent. You really don't need more than 250 or at most 500 Mbps for the vast vast majority of cases.
You forgot sshfs. More generally self hosting basic services.
I also have this on paper. It’s surprising how much home hardware caps out at 1 Gbps or below. I had to replace several routers and run cables everywhere to get the full speed. Once you add on tailscale, a VPN, etc it’s a proper challenge to get it consistently working above 1Gbps.
Some games are hundreds of GB of disk space and also have a terrible patching algorithm where they need to make an entire duplicate of all files to patch them, despite downloading less than a GB of patch. So you have to uninstall stuff. With fast Internet you can actually use most of your drive because you can uninstall then download a new install which only needs X space not 2X.
Downloading big models is much nicer on a 1Gbit fiber line. Uploading test docker images too if you're forced into modifying them locally.
Literally everything is better with multi-gig symmetrical. I work from home and I have to pull multi-gig archives of documents for processing and that's better. Then I need to push them to web servers, and while I'm on 2Gbps I haven't managed to get an upload to anything useful higher than 600Mbps, but that was 30Mbps last month on cable, so tasks I'm waiting on are getting done 20x faster.
> Linux ISOs
Yeah definitely Linux ISOs. It's nice to settledown in an evening, pick a new Linux ISO I haven't used yet, and veg on the sofa watching it install.
Of course unlike the netinstall images all the normies use, you have to download it up front before you start wa, uhm, installing it. But I prefer it that way so I can watch the install without being interrupted... Then again I guess at 2.5Gbit the only thing that's going to interrupt you is a crappy integrated network interface, aka Apple monitor. I don't have enough money to have that problem though.
Backups are a lot faster. For some that matters. Running 2.5g locally has a lot of benefits. With a 1g network, you need just 3 people on the network for UX to degrade.
I think it’s less about being able to max it out and more about having background events (like backups) or other users on the network not impact your own usage.
But I run servers at home and regularly download enormous video files so I can think of plenty of ways to max out a 2.5 Gbps connection (alas my own is nowhere near as fast).
Indeed. We have 4Gbps because it's not much more expensive than 1Gbps fiber (plus it required our ISP to upgrade our fiber connection from AON to XGS-PON, which is more future-proof).
The benefits are two-fold:
1. Background tasks don't impact your own issue (as you say).
2. Many CDNs can saturate 4 Gbps, so if you download something, the waiting time is much shorter than say 100Mbps.
Whether it's worth it all depends on the price difference.
4Gbit is kind of the sweet spot currently. 2.5Gbit equipment became quite cheap (e.g. Ubiquiti Flex Mini 2.5Gbit managed switches are often around 50 Euro), existing Cat 5e cabling still works reliably, and if you have a router with 10Gbit upstream and multiple 2.5Gbit downstream ports, none of the subnets can completely saturate the upstream bandwidth.
Also, these bandwidths rock for doing an initial backup, downloading a full backup, pulling many Docker images, large and frequent NixOS rebuilds (unstable-small, yay). Downloading GTA 6 (not me though, not a gamer).
> plus it required our ISP to upgrade our fiber connection from AON to XGS-PON, which is more future-proof
The opposite, actually! See for example Init7 in Switzerland offering 25Gbps fiber to home users, and having done so since 2021 - for exactly the same ~$80/mo as 1Gbps: that would not be possible on new XGS-PON deployments as it requires having direct access to the fiber going to this one customer. Init7 could trivially offer 100Gbps over the exact same fiber using existing off-the-shelf plug-and-play equipment, and many times that for customers willing to spend money on WD equipment.
PON inherently ties your fate to that of your neighbors: whatever they are getting, you are getting as well. You can just about have multiple generations of PON living on the same fiber, but that's it. The sensible thing to do would be to give each building dedicated fiber to a central PoP and put all the PON splitters in there, but as that would allow for legislation mandating access to the PoP to competitors wanting to offer superior services via direct fiber access, that is often made deliberately impossible in new PON installations by putting the splitters into tiny curbside cabinets where there is physically no space for anyone but the incumbent - which of course means the only way "open access" is possible is via a virtual provider scenario where any kind of meaningful differentiation is impossible.
The opposite, actually! See for example Init7 in Switzerland offering 25Gbps fiber to home users, and having done so since 2021 - for exactly the same ~$80/mo as 1Gbps: that would not be possible on new XGS-PON deployments as it requires having direct access to the fiber going to this one customer.
Sorry, I should have been more clear. The ISPs in my country are not going to offer higher speeds than 1Gbit/s over AON. They started using XGS-PON for all new connections and will only offer higher bandwidths over XGS-PON (currently 4Gbit or 8Gbit depending on the provider).
PON inherently ties your fate to that of your neighbors: whatever they are getting, you are getting as well.
You are leaving out a lot of details, such that different generations of PON can be sent over the same fiber. ISPs can migrate to newer PON versions by swapping equipment centrally at the OLT and the customer's ONT, while upgrading AON requires replacing active switches in street cabinets, which is much more expensive. Besides that, XGS-PON power consumption is much lower. So, it is more future-proof because ISPs will actually offer higher speeds due to lower cost. Before XGS-PON we were stuck with 1Gbit for over 10-15 years because replacing all existing equipment to offer higher speeds was too expensive. PON has upgrade paths to 25G-PON and 50G-PON, which should be enough for a while.
4-8Gbps is absurd! Which country offers that if you don't mind me asking? Never seen anything as high as that where I live.
> (plus it required our ISP to upgrade our fiber connection from AON to XGS-PON, which is more future-proof).
Isn't "AON" basically switched Ethernet?
* https://www.fs.com/blog/aon-vs-pon-networks-whats-the-differ...
What is meant by XGS-PON being "more future-proof"?
I know I'm far from saturating my 1G symmetrical on my normal PC. Cable conditions downgraded it to 100Mbps and I didn't notice for the longest time. (Nor did I fix it since then :P)
> but I'm curious to know how many residential use-cases can take advantage of that much bandwidth.
I don't know what you have in mind regarding what qualifies as residential usecases. I can tell you that I use my residence for work, and as part of my work I have to hold video calls and download all kinds of packages from the internet, including pulling container images, including media asset bundles that take several GB.
This often takes place when other members of my household are doing their work or watching streaming services.
If you have a 2.5Gbps connection, you often are not looking into sustainable transfer rates but bursts. A difference between 500Mbps and 2.5Gbps is whether it takes you half a minute to pull a package or 5-10 minutes.
Games mostly. Getting a 40gb LMU or iracing update when logging in for a session gets old quick. Maxing my 5 gig ftth connection feels good man.
I only upgraded because my isp charged $90 for 1gig and offered 5 gig at $65 for two years so for the price of a pci nic I thought why not. It’s also nice that every device in the house can max out their capability simultaneously but that’s a rare event.
Not sure this counts as a "residential" use-case, but I have 2 gig symmetrical for my office, and it lets me rent out two GPU servers to third parties offering 750M symmetrical on each, while keeping 500M reserved for my own use even if both of those are saturated. It's awesome.
I have a 5Gbps symmetric connection. It is a bit more expensive but far from prohibitve.
The main benefit is that I don't really have to care if a device is saturating it's 1Gbps link for updates or backups. It is also quite nice to have 5Gbps to my desktop for the platforms that support it like Steam.
I don't really get much benefit for work since our VPN bandwidth isn't amazing.
When FTTH showed up in my neighborhood a few weeks ago, I decided that I'd have been very happy with symmetric 500mbps. 500/500 is a lot more than the 500/20mbps I had before and I was really mostly interested in upload speed and latency improvements anyway.
The applications I had in mind for faster uploads were improving backups and sharing Linux ISOs, and the occasional one-off file transfer. All of that worked before, but it works better now.
I'm now also exploring things like routing my cell phone and laptop through my own VPN (I can take my network with me now and it can be fast!), and things like remote gaming (due to an drastic reduction in latency compared to DOCSIS).
...and I wound up getting symmetric gigabit anyway. The only reason I initially aimed for 500 to begin with was cost, but they ameliorated the cost difference in negotiations after the first install date got botched. :)
(I could order faster-than-gig service from this provider but my home network is only 1GBe to begin with. I'm not really interested in the barely-upgrades of 2.5 or 5GBe at all since that seems like extra steps, and I'm not ready to move anything to 10GBe yet because reasonably-efficient 10GBe gear is still kind of expensive. Some day.)
>>but I'm curious to know how many residential use-cases can take advantage of that much bandwidth. Linux ISOs?
I work in video games and I work remotely. On your average work day I can easily download 1TB worth of builds and other data. There's a LOT of sitting around and waiting for things to download and I have 1.2Gbps internet connection now. If my ISP offered 2.5Gb I'd pay almost any amount of money to have it.
TBF, I might be a minority here but I quite frequently have to deal with DBs upwards of 100gb, so having symmetrical 1gig is a godsend.
I am exploring self hosting my own photo and MacBook Pro backups.
My first attempt took about 18 hours to upload 1Tb. I made adjustments to my network and I’m looking forward to a faster upload.
My wifi6 5g can’t keep up with my ISP rate unfortunately.
I’m only on 1000/1000 though.
This made me have AI look at a USB audio problem I have. My external soundcard is connected to my monitor (as KVM hub), then to a hub. I learned today that that meant there was 4 hubs in the chain. What? Turns out, the USB port I used was hooked up t a hub chip, then to the chipset. The LLM told me which USB port to use instead that avoids that setup. Furthermore, the last hub in the chain is a 7-port hub, which is implemented by a daisy-chain of two 4-port chips. Depending on which port you use, you can skip one step in the chain. There's no indication whatsoever on the outside of that hub.
I have nothing useful to add other than I feel so validated reading this. Ive been struggling with this issue on and off for over a year. With the release of Astra, it was able to finally unstick me to the point of enabling flow control which has been good enough.
I experienced this problem with a macbook and a thunderbolt dock. I had tried multiple different brands of docks and ethernet adapters, and various macbooks. One interesting thing I discovered is if I plugged a non-thunderbolt cable between the dock and the laptop, I would achieve full speed with no packet loss, so interestingly the issue doesn't occur over pure USB but only over thunderbolt.
This has got to be a bug on Apple's side right? Would be nice if they fixed it.
Oof. That is the kind of thing where Apple will never do anything. They implement standards, but they guarantee absolutely nothing if even so much as one cable involved isn’t Apple-branded. Using docks and Ethernet adapters? Good luck! They’ll just ask if you’ve considered buying a $5000 monitor with a $1000 stand to use as a dock instead. (And ironically, that’s what OP did and still had the problem :D)
> Using docks and Ethernet adapters? Good luck!
All the more irritating is that all this dock and dongle bullshit was popularlized by them.
What kind of Thunderbolt cable, and what kind of USB cable? The downside of using the USB-C connector for everything is that it obscures the difference between active and passive Thunderbolt cables.
I recently had this same problem with a multigig copper SFP+ module. These SFP+ modules (usually) advertise 10Gbps to the NIC they're plugged in, but will negotiate lower speeds on the copper side. So when NIC tries to send to SFP+ at higher rate than whatever speed SFP+ has negotiated internally, the SFP+ module will just drop a bunch of packets and incur a significant hit in throughput.
These tend to not support sending pause frames to its SFP+ side either, so the best option I found was to set bandwidth limit on the NIC…
Good tip! I was looking into those recently, as I have a switch whose SFP+ slots only support 1 or 10G, but I want the option to connect 1/2.5/5/10 devices.
Sounds like that may be more trouble than it's worth. Hmm. I may still poke at it, but this definitely opened my eyes to a potential trouble spot that hadn't been on my radar.
I had the same problem recently when trying to figure out high bandwidth game streaming from my desktop to my Steam deck in the living room. The beta streaming tech is “pyrowave” and it is super low latency and looks INCREDIBLE compared to traditional codecs, but as a trade off it is extremely sensitive to packet loss, and uses an appropriately large amount of bandwidth (7-800mbit at 4k120). Wired network isn’t _strictly_ a requirement but it might as well be, and the steam deck doesn’t have a physical port - but the steam deck dock does.
Anyway, the exact same situation arises, where some network condition or physical configuration (there’s a lot of hardware in between these two devices, including MoCA adapters) meant that the network adapter buffer fills up and starts dropping packets. The buffer size is configurable through some terminal command and maxing that out makes the issue disappear immediately.
Unrelated to this particular issue, but I've had subtle monitor behaviors catch me off guard as well. Usually I think of monitors as being unrelated to the underlying program running, but this is not always the case. For example I released a JavaScript programming game on steam that uses chrome (Electron) for sandboxing. This game has a tiny community but already there are a couple players doing speed running [1], and one discovered that you can complete the game many seconds faster if using a 120Hz monitor instead of 60Hz. The reason is that the JS API requestAnimationFrame [2] is tied to monitor's refresh rate (which I didn't realize) and I originally didn't do any sort of logic limiting it to 60fps (and actually I was using PixiJS tickers[3] which I didn't realize were using requestAnimationFrame under the hood).
[1] https://www.speedrun.com/Pragma_Twice
[2] https://developer.mozilla.org/en-US/docs/Web/API/Window/requ...
[3] https://pixijs.com/8.x/guides/components/ticker
Can’t send email more than 500 miles
A grand-grandfather to this article
Reading this new article I can tell that future is in the good hands
https://web.mit.edu/jemorris/humor/500-miles
Nice post. Unsatisfying conclusion - did you ever figure out why the original adapter was throttled when plugged into the monitor, and not when plugged into the computer?
I’m guessing it’s a USB bandwidth issue. I learned about these kind of limitations when I tried to have 3 kinects running at the same time on the same USB bus.
He states that high speed SSDs through the monitor are fine though.
> I’m guessing it’s a USB bandwidth issue
It's explicitly not a bandwidth issue (that would also affect the SSD). It's a latency issue - the chip doesn't have a big enough buffer to keep sufficient packets in flight at once.
> It's explicitly not a bandwidth issue
That's explicitly the opposite of what the author says.
"the speed of packets coming into the adapter is 2.5 Gbps. That’s what the router pushes in. The speed going out is around 2 Gbps."
Of course latency is always on the denominator side in the "bandwidth = data / time" equation, but it seems wrong to claim it's latency when it could also be improved by fixing the numerator; it's the entire fraction that's insufficient, thus bandwidth. As written:
"The reason is that if you add up the average time it takes to get the go-ahead through my monitor’s hub, which is around 7.6 µs, to the average time it takes to send 4 KB, around 8.2 µs, you can find what the maximum speed is"
And later, increasing the numerator is exactly what fixes it with the other hardware: "I can see that instead of sending 4 1 KB packets per burst, it would send 8!"
> the chip doesn't have a big enough buffer to keep sufficient packets in flight at once
How does this make sense? Even if his buffer was 100x larger, if the fill bandwidth exceeds the drain bandwidth, it'll eventually fill up. The article says that explicitly:
"Well, if the speed of the drain from my adapter to my Mac is half a gig slower than the speed of the input, then you can imagine the buffer getting filled more and more until it overflows."
This primer on latency and bandwidth is 13 years old but still worth sharing (for the sake of all readers)
https://hpbn.co/primer-on-latency-and-bandwidth/
> Even if his buffer was 100x larger
"buffer" was perhaps a poor choice of words on my part, because "buffer" is ambiguous. In this case I am referring to the notional "transmit buffer", which is 4 in the non-working case and raised to 8 in the working adapter.
> "I can see that instead of sending 4 1 KB packets per burst, it would send 8!"
Increasing the number of packets in flight does not change the bandwidth, it only improves latency sensitivity (i.e. you can fire more data before requiring an ACK).
> it seems wrong to claim it's latency when it could also be improved by fixing the numerator.
Increasing the bandwidth without increasing the transmit buffer would not fix this issue. The adapter he ends up buying does both (allow 2x the packets in flight, and 2x the bandwidth).
The bottom line here is that the original adapter handles full duplex 2.5g when plugged into the laptop directly, and the SSD gets full duplex bandwidth in excess of 2.5g through the same intermediate link (i.e. the display), so both the adapter and the display have sufficient bandwidth. The added hop of latency however makes the short transmit buffer insufficient to cover the latency.
> In this case I am referring to the notional "transmit buffer", which is 4 in the non-working case
Then I agree with what you were saying. Those 4 * 1 KiB are a protocol-level size (similar to the TCP window size), not an actual "buffer".
> Increasing the number of packets in flight does not change the bandwidth, it only improves latency sensitivity (i.e. you can fire more data before requiring an ACK).
That I don't fully follow. To me, the measured bandwidth includes all the effects of ACKs etc. If the the protocol demands that you can have at max 4 * 1 KiB unACKed in flight so you're waiting around for ACKs and not snding data, then increasing the number of packets in flight to 8 would 2x the bandwidth.
> The added hop of latency however makes [it] insufficient to cover the latency.
I agree with that, see also my newer reply https://news.ycombinator.com/item?id=50592931
I this context, I also agree with your original statement; I misunderstood what you referring to with "buffer" (meaning the window, not actual storage), and "bandwidth" (where I think you're referring to the 5 Gbit/s nominal bandwidth of the bus, irrespective of the fact that the effective bandwidth of data sent is much lower due to requiring ACKs for small data).
You’re using an ancient definition of bandwidth. Bandwidth in packet switching networks is not the layer 1 signaling rate, it is packets per unit time times the payload size of the packets.
Uploads were fine.
The additional latency from going through the monitor made the bandwidth-delay product exceed the amount of data the adapter was able to send before receiving an ack. I thought that was fairly clear from the explanation, but now I notice that he didn't mention the latency without the monitor.
That doesn't seem to be described by the article at all.
The latency added in the monitor is described as around 8 us, completely negligible compared to any Internet connection's latency.
So the latency without the monitor should be completely irrelevant.
Negligible compared to the RTT, sure. But they’re talking about the latency between when the network adaptor signals that a packet has arrived and when the CPU actually services the interrupt to tell the card to send the packet(s) to main memory. The NIC’s buffer could only hold 4 packets, so by the time the NIC got a DMA buffer to copy the packets into a fifth or sixth packet had arrived and been dropped.
I see now, indeed. What cibyr posted is entirely correct.
I thought their "latency without the monitor" had referred to the overall end-to-end connection to the other IP endpoint, but they were talking about `NIC <-USB-> laptop`.
> The reason is that if you add up the average time it takes to get the go-ahead through my monitor’s hub, which is around 7.6 µs, to the average time it takes to send 4 KB, around 8.2 µs, you can find what the maximum speed is.
This part is saying the NIC with this monitor can transfer 4 KB to the host through the USB hub every 15.8 us. You can do the math and it's about 2gbps.
Remove the hub, and reduce the delay solves the problem.
Increase the burst size would work: 8 KB per 24.0 us would be sufficient for 2.5 gbps.
Increase the burst size and the transfer rate as done with the 5 gbps adapter, and that should work out, although that one is probably going to be insufficient for full rate 5 gbps as well.
Yes, that is correct. I had misunderstood which connection cibyr was referring to, see sibling post.
That may be a thing where those of us who understand how TCP/IP or Ethernet works understood this paragraph but others didn’t follow.
The article answers this
"if the speed of the drain from my adapter to my Mac is half a gig slower than the speed of the input, then you can imagine the buffer getting filled more and more until it overflows. Those are dropped packets, which result in a hard backoff when the router resends them, so you drop loads in speed. And that’s the reason for all of this."
With diagram https://blog.tymscar.com/studio-display-ethernet/bottleneck....
The monitor decides to send the adapter's packets on at only 2 Gbit/s, which leads to buffer overflow, thus packet loss, thus e.g. TCP congestion-control backoff.
The only thing we don't know is _why_ the monitor chooses to limit the bandwidth to those 2.07 Gbit/s (computed as the 41 KiB blocks per (7.6+8.2) µs). If it sent e.g. 81 KiB blocks, doubling the bandwidth, the problem wouldn't happen.
That'd be between Apple and the monitor.
No, you’ve misunderstood. Any time the NIC signals that one or more packets have arrived the CPU must reply back to the NIC to give it a DMA buffer to copy the packet(s) into. By the time that reply arrived at the NIC it was already too late. The NIC could only buffer four packets and the fifth or even sixth packet had already arrived and been dropped. The NIC then sent the four packets it had to the DMA buffer and started over.
The monitor was not really at fault. All USB hubs must read USB packets on all of their ports and copy them over to the upstream port. Those packets must all go into a buffer first so they the hub can send them individually even when they arrive simultaneously. That introduced just enough delay to allow the NIC’s internal buffer to fill up. Any hub would have caused the same problem. The solution was to buy a NIC designed for a higher speed because it has a bigger internal buffer. That new NIC will probably have the same problem with USB hubs, but at 5Gbps instead of 2.5Gbps.
Not all hubs and computers are equal here. Having experienced this with RTL8153, the combination of the Studio Display and ARM MacBooks is especially bad. Using an Intel MacBook with the display, or a different USB hub instead of the display, gets the full NIC bandwidth.
Fair enough, but I still wouldn’t say that he monitor is at fault. It’s probably not operating outside the USB specification in any way, even if it takes longer to forward messages than other hubs. It may even be multiple hubs, internally.
It’s really the NIC’s fault because its buffer is too small to handle a little extra latency.
> The NIC could only buffer four packets
Where does the "4" come from? The article's diagram suggests that the NIC's buffer is "~35 KB", though I don't know where that number comes from either (I supposed measuring how long it takes until frames are dropped). "4" seems to be the number of 1 KB packets (the 3+1 from "packet size 1024 bMaxBurst 3"), so a transmission size (protocol-level "window size"), not a buffer size. I guess if you are referring to that as a different "buffer", that makes sense, but it doesn't seem to be the buffer the article is talking about.
But you are right with your main point regarding latency. My statement
> The monitor decides to send the adapter's packets on at only 2 Gbit/s
isn't really right: It's not that the monitor "decided" to only send 41 KiB instead of larger sizes; the 41 KiB is the maximum that the NIC can send because its `bMaxBurst = 3`, and the latency added by the monitor being a USB hub then limits the bandwidth to the mentioned ~2 Gbit/s.
I guess the key point not said anywhere is that `bMaxBurst = 3` and `wMaxPacketSize = 1024 B` are fixed protocol-level parameters that hubs like the monitor have no control over. So the 4 KiB windows are fixed from the NIC through the monitor to the Mac; the monitor cannot e.g. decide to buffer some of those to compensate for the latency it adds.
> the monitor cannot e.g. decide to buffer some of those to compensate for the latency it adds.
Correct. The NIC buffers the Ethernet packets, the USB hub in the monitor can only forward what the NIC sends. If the NIC drops an Ethernet packet, no amount of buffering in the USB hub can help; it’s already been dropped.
> Where does the "4" come from?
It was set by whoever designed the RTL8156B chip. They looked at the delay product of the USB link and calculated that a buffer holding four packets would be enough to maintain 2.5Gbps. And they’re right, assuming the latency of the USB link stays low.
Notice that when it came time to make the RTL8157 chip that supports 5Gbps they doubled the buffer size to 8. That also works only if you assume that the latency of the USB link is low, or if you have Ethernet packets arriving at 2.5Gbps instead. They probably double it again in the chip that supports 10Gbps.
Realtek are just being cheap. It would only have cost pennies to double the buffer sizes in both products, and the result would be a more reliable product. This is an example of why engineers specify Intel NICs and only consumers end up with Realtek NICs.
Hm, HN garbled my syntax involving asterisks. I meant to write
not
The interesting point is the OP didn't notice until they ran a benchmark :)
I happen to have two fiber connections to my home: a business line at 500 mbps and a residential line at 1 Gbps. Different providers.
In daily use I can't tell which one I'm on, just like the OP couldn't tell they were only getting 600 Mbps out of their 2.5 Gbps.
Is the sweet spot for the current internet 500 Mbps?
I find it kinda nuts they didn't run a speedtest on the first day of getting the Internet set up, considering they're paying a surely-non-base price for it. That, and you'd want to make sure everything is working one day 1 so you have a reference point in your mind. It sounded like that was the first time they actually ran a speedtest.
Im on my 2.5 gig for over a year now. I ran all sorts of speedtests. I got this monitor way after the fast internet
>Is the sweet spot for the current internet 500 Mbps?
Lower than 500...I'd guess 250.
Standard 4K youtube is like 20ish and a low latency link can still feel good even at low throughput because so much of browsing experience is multiple round trips sensitive not throughput.
I've had a NIC crap out and switch to 100M (1G internet)...that was barely and I mean just barely perceptible. 100M @ single digit latency still feels snappy for browsing
>It’s run by this one guy
This was super surprising, have always thought ISPs are a super hard businesses. But good for you mate, good ISPs are lifesavers.
I have had the opposite experience with my ISP guys. One of the VMs I got as part of work only had IPv6 and to my shock my internet didn't have one. When I contacted my ISP they legit told "You can either have IPv4 or IPv6, but not both". Unfortunately I was on a 12 month plan and couldn't easily cancel so ended up working around using tailscale.
There are communities of small towns and villages running their own ISP and providing help materials for others who want to try.
Do you have any pointers ? Idt this is a thing where I live tbh
https://news.ycombinator.com/item?id=44878916 (WHY2025: How to become your own ISP [video])
https://news.ycombinator.com/item?id=40161697 (Start Your Own ISP)
https://www.npr.org/2022/08/22/1118734792/michigan-man-isp-f...
Thanks mate
Be aware that turning on Ethernet Flow Control is not always better. In my experience it's only a source of issues, but my experience doesn't include USB adapters. Just know that it depends; measure and don't expect those results to translate to all servers and switches in general.
> have so many cool intricacies
I personally always like every controller directly plugged into the pcie bus. USB-C and Thunderbolt does give you a PCIe connection but how internally it’s wired and implemented is a game of chance. The really high end ones usually break out the thunderbolt 3 into a pcie controller for Ethernet etc but that too depends on how many lanes there are and how many controllers are competing.
Eventually the people designing it will find a ‘one size fits’ all solution and then you end up with issues like these.
From my experience reading such issues, almost all such problems are trivial with a pcie card to the cpu.
Good old desktop days might be behind us ‘^_^
Funnily enough I recently noticed my network was slow, told Claude to investigate, it came up with "replace the DisplayPort cable on the right monitor".
Immediately 10X'd my bandwidth, turned out I was using a cheap cable that wasn't shielded like it was supposed to be, and it was running right past the WiFi antenna and causing loads of interference.
how much back and forth was there between you and claude to troubleshoot this? you make it sound like you gave it one prompt and it found that solution, which to me seems incredibly impressive.
I think I meatproxied a handful of experiments (i.e. Claude said "can you unplug X for me" and then reran iperf or whatever, to rule things out)
The original studio display had a similar issue. A 1gbps adapter worked fine on the Mac, but a good 2.5gbps adapter was needed when going through the studio display. There's a big thread on the Apple forums if people need to know which adapters chipsets work.
Usually people discover the bandwidth delay product on long fat pipes. Neat to read someone discover it on a very fast very short link.
https://en.wikipedia.org/wiki/Bandwidth-delay_product
What's the difference in BOM cost for Realtek RTL8153 (1GigE) versus RTL8156 (2.5G) versus RTL8157 (5G) versus RTL8159 (10G)?
chips are probably like $2 to $50 across that range, hard to tell since realtek sucks
> Basically, when a device receives Ethernet frames it can’t keep up with, it just drops them. With flow control (it’s something your router or switch has to support), it can send the sender a “pause” frame, and the sender then buffers the data on its side, instead of the receiver (the adapter in this case) on its side.
One thing that can happen with pause frames (and why I hate them) is pause propagation. If you have a switch between your server and client and the client sends the switch a pause frame, the switch may send a pause frame to the server. If it does (and the server accepts pause frames), the server will pause sending to all destinations.
So your one slow client can disrupt all the other flows from the server. Which is totally unhinged.
Maybe I need to look into frames and dropped packets. My LAN can't quite reach 1Gbps, it's more like 7-900Mbps. Last time I investigated it, I came to the conclusion it's sub-par ethernet cables, but perhaps there's more to it.
Yeah bit wary of these modern "everything over one cable" contraptions.
I get the convenience but they're SO much harder to troubleshoot.
I recently tracked down some poor network performance to some dodgy Ethernet cables in my patch panel.
> Somehow I am one of those few very lucky people to have the opportunity of owning a 2.5 gig symmetrical line for my internet.
Uhm... is it really _that_ unusual? Spanish digi offers 10Gbps for 25€ and its availability is quite wide…
Having the correct driver is crucial for any 2.5Gb+ adapter. I know because I've run into this problem with 2.5 and 5Gb dongles too.
Would bbr congestion control help here to.get speeds to 2 Gbps?
Seems likely. BBR (v1) does not do the hard 30% "multiplicative decrease" on packet loss (caused by the frame drop in the article's diagram), so I'd expect it to be able to correctly estimate the bandwidth to the ~2 Gbit/s bottlenecked by the monitor-Mac connection.
Yeah, I'd expect it too, but doing some research it seems Apple still doesn't support BBR. But I wonder if a speedtest with quic would show better results for the author.
It doesn't really matter what the client OS supports because the problem is inbound. You'd want fast.com to support BBR (although... I think they do, fast.com is Netflix and afaik they are a significant BBR user)
Could this be some kind of https://en.wikipedia.org/wiki/Bufferbloat problem?
Sure we are talking about a different level in the networking stack and very different protocols. But with every component you add things get less synchronous. Well, no buffering needs to involved by the middleman, the mechanism for the throttling could also be different. What do I know...
No, the problem does not match the topics discussed in https://en.wikipedia.org/wiki/Bufferbloat
The 35 KiB buffer from the article is not very large, and the problem would exist unmodified if it was smaller.
As the article explains, "if the speed of the drain from my adapter to my Mac is half a gig slower than the speed of the input, then you can imagine the buffer getting filled more and more until it overflows", and this would hold for any buffer, smaller or larger.
Bufferbloat causes primarily latency problems, not bandwidth problems.
Bandwidth problem would occur at most as secondary effects, e.g. if a packet is delayed by a buffer for so long that the sender-expected timeout for an ACK expires, it considers the packet lost, and throttles down sending. But that's not happening here.
I've encountered a similar problem, the wifi on a MacBook Air was dying once connected to a Dell monitor over USB-C. The wifi signal was going from full to nearly no signal. Connecting to the same monitor with a dongle and HDMI didn't have the same outcome. Never had a time to find the culprit. Probably EMI or bad grounding.
USB3 has huge EMI issues that haven't been very discussed. It's messing with your signal.
> because the issues don’t start at 1 gig
Actually, the speed of gigabit Ethernet adapters are also known to cut 60% behind a studio display. TBH I assumed it was because USB ECM sucks and NCM (which RTL8156 implements) was supposed to fix it.
Used to get this with dial-up modems when they were either placed on top of the CRT (just to get very toasty?), or the incoming phone line ran too close to some CRTs.
This would reduce the established connection speed due to electrical noise from the monitor.
Can someone just tell us what brand and model the monitor is? So we can instantly know if we care or not?
Almost all the screen ethernet ports I have tried, Philips, Lenovo get to about 300mbps and can not saturate 1gbps, which is totally bad for downloading Linux isos.
What a click-bait headline. The conclusion was that it was due to having a more bottlenecked USB-to-Ethernet adapter path. Would the same result have held with a discrete Thunderbolt 5 hub, either with a built-in 2.5 Gbps or faster Ethernet port or with the original USB-to-Ethernet adapter? Would it have held with another USB hub? Who knows -- just blame it on the expensive monitor!
In a sense it felt like the obvious outcome of a "one cable" setup.
It's unreasonable assumption to expect such a setup to manage multiple full speed connects in my opinion.
If I was running support it and later heard on I just unplugged the ethernet from my monitor direct to my laptop and the speed improved I'd dispair.
But now it is doing exactly that.
> It's unreasonable assumption to expect such a setup to manage multiple full speed connects
The bandwidth in question is nowhere near saturating even a single USB3 connection.
And yet...
It's a lot easier to hit real world limits than theoretical spec limits.
The explanation is not that clear, but you missed the point. It is a networking issue known as 'bandwidth-delay product': the combination of latency and bandwidth is what limits the speed, and in this case the extra latency from the monitor hop was the culprit. Not the adapter. The faster adapter is a workaround.
You sound like Steve Jobs: "the adapter is fine, you just plugged it in wrong." No: if the buffers are too small to tolerate an intervening USB hub, the peripheral was designed poorly.
Edited to add: Unlike networks in general, the USB standard imposes maximum diameters and latencies, so designers know what to design for. If the higher-spec adapter also had awful goodput, then it might be reasonable to point fingers at the monitor.
>I have a cupboard full of USB-C Ethernet adapters. Don’t ask.
Weird. It's the first time I hear someone has a cupboard of those. Everyone I know keeps them in a drawer.
I found it so funny that the author is using an M3, Studio Display, and a 2.5 GB symmetric internet connection to play Sauerbraten. A (very fun) 22 year old game made by 3 people.
I used to play that a long time ago, possibly even when it was called Cube, and hung around on the IRC channel.
I should give it another go.
I got back into it for while last year. Honestly it's still great.
> Somehow I am one of those few very lucky people to have the opportunity of owning a 2.5 gig symmetrical line for my internet. But one day, when I wanted to brag to my equally nerdy friends [...]
I'm not sure why you'd brag about having slow internet? ;) We had 10 gig here in Singapore for a while now.
Haha, fair, but you dont have 10 gig symmetrical so what's the point!
My whole wifi slowed to 5kb/s because of a bluetooth smartwatch interfering with the 2.5ghz wifi. I was always told bluetooth and wifi sharing a spectrum could not possibly ever be problematic. Needless to say I hate Bluetooth even more since then.
I suspect the problem here is the UGreen ethernet adapter. I have soured on UGreen after a couple of their recent hardware bits proved flaky and one damaged two laptop motherboards (it was a usb-c hub).
Computer case windows are a big part of this too. There's a reason that high power high speed switching electronics generally need to be enclosed in a (near) full conductive surface to pass FCC emitted emissions regulations. Putting a giant hole in the meta box mitigates a lot of the point of it.
> ...considering I am 30 in a couple of months. My memory is not what it used to be.
Oh just you wait
Too bad, I have the 4mb plan and I get 6mb speed instead, I think im blessed
What's your handle on Sauerbraten?
Tymscar, as everywhere else! I don’t play often enough
TLDR (if I've got this right): The problem was caused by the combination of his monitor's built-in USB hub (which added some microseconds of latency to the acknowledgement cycle) and his USB Ethernet adapter (which sent data in bursts of only 4 x 1kB USB packets before it had to wait for acknowledgement). Fixed by updating to a Realtek RTL8157 based Ethernet adapter (which sends 8 x 1kB packets at a time, and also supported a faster USB link). (Personally I wouldn't pipe my gigabits-speed internet connection through such an arrangement, but I guess this is how people do things nowadays).
> Personally I wouldn't pipe my gigabits-speed internet connection through such an arrangement, but I guess this is how people do things nowadays
The XDR Studio Display is a $3000 monitor that supports Thunderbolt 5 which advertises 80 Gbps speeds. So I think one could certainly be forgiven for assuming it could handle a measly 2.5 Gbps ethernet adapter (and in the end it could albeit with a different adapter).
Why not, given that with the right USB to Ethernet adapter it apparently works just fine?
How did this story not end at: ".. but my modem said it was getting full speed so I immediately knew what was wrong"?
File this under “everything is analog”.
I had an issue with a raspberry pi and a second usb WiFi adapter. It caused my terminal to be unusable, despite my connections carrying over the primary adapter.
Apparently there are usb harmonics or other power issues that were interfering with the primary wireless adapter .
Remember the machine is billions of electrical signals and EMF. It’s actually amazing that any of it works discretely and (somewhat) deterministically
The meta-issue is that the more "advanced" (and interconnected) all of our devices become, the greater the potential for harmful side effects, or as we saw in France, hacks.
A buggy monitor or adapter is one thing. But a buggy car? Yet these will soon be our only option.
Sauerbraten mentioned!
What is this clickbait headline? It's the monitor hub and/or the USB adapter they were using.
I'm always jealous of people who have the willpower and time to investigate this kind of stuff, I would have plugged it into the laptop and moved on! It's perhaps the most nerdy "grass must be greener" luxury I can imagine (in a good way).
> So I thought I would do what any ~~human~~ apple user would do, and I spent more money on a new adapter.
Most people probably would buy another adapter. Most people would probably not debug the USB connection for its latency and throughput over USB.
Most Apple users would do this. Most other people would have just used wifi on the laptop and an HDMI cable for plugging their monitor in.
or have RJ45 on hardware they want to connect to the internet via cable
I do wish my think pad's Ethernet port was 2.5G...
Hah, well if my monitor at home ever does this to me I can take my revenge by slowly removing my jacket, or sitting down on my chair. (Samsung Odyssey G9, it's ultra-sensitive to ESD anywhere near it so if I'm wearing the right jumper I can move my chair or wave my arms around and the capacitative heresy will make it wig out for a good 2-3 seconds.)
At work we run HP Z40c monitors and while they've been bulletproof in terms of image quality, on my old laptop the mouse cursor was definitely more laggy when running in a one-cable setup. We're using Logitech MX Master combos so I just moved the dongle from the monitor to my laptop and that fixed the lag issue.
Wow this sounds like a shill post. If anyone from HP or Logitech wants to talk message me. Otherwise Logitech Opti+ or whatever its called can fk off.
A couple of thoughts:
1. These 1 cable solutions are nice but (IME) incredibly finnicky. I've gone through several Caldigit docks that have died;
2. When you think about it, these docks (and monitors) are really 100+ Gb/s switches. It's kind of no wonder that there are odd behaviors;
3. IME that cable that provides power to the laptop and handles power/video (you know, so it really is 1 cable) is the absolute worst. Whenever you buy them, buy 2-3 because I can almost guarantee 1 will be faulty and it'll be weird, annoying problems eg the screen gling black for a split second randomly;
4. I did a lot of testing with 1GbE USB-Ethernet adapters and it's true that they are not all created equal. In a perfect world, they should do ~930Mb/s. Depending on which one you get it'll be as low as 880 (or lower); and
5. I would honestly expect better from Apple. With the price premium you pay for these Studio monitors, I would expect the hub part to be pretty much seamless.
I had a sort of similar thing recently , Claude found it but my wifi was super slow on my Mac until I turned off airdrop from everyone , and AirPlay handoff, it actually improved my wifi a lot.
Unnecessarily long...TL;DR: The monitor introduced a tiny delay in USB communication, which caused the Ethernet adapter to lose data packets...
I knew it was Apple even before clicking
No, it was his Ethernet adapter.
When he plugged the original ethernet adapter straight into the laptop it worked as expected. It was the monitor not supporting that specific setup and chip.
It has nothing to do with the monitor's support. The GP is correct; the fault is in the Ethernet adapter. Specifically, it doesn't have a buffer large enough to sustain 2.5 Gbps when going via the monitor because the latency is too high. You'd have the same problem on many USB hubs and it would still be the Ethernet adapter's fault. It could have used a larger buffer or it could send more data at a time to the host so that the buffer doesn't fill up, but it doesn't.
I would say if a USB adapter works fine in a laptop USB port, then there is nothing wrong with it. If you are trying to use it in a monitor and it introduces so much latency that doesnt work, then thats a different issue. Its not a faulty adapter.
The adapter manufacturer sourced a part that could not sustain the product’s advertised bandwidth when connected through a USB hub. USB hubs are not esoteric. This is the fault of the adapter manufacturer cheaping out on a part.
I think we are all arguing the same thing right? The adapter worked fine when used directly in a device but not when latency was high due to other devices in the chain?
I have had hundreds of USB devices in the past which dont work on a USB hub, its very common. The first troubleshooting step in basically any USB devices manual is if you are having trouble and you are using a hub, remove the hub and plug directly into the device.
The user went way over the top in troubleshooting this issue. Thats the first thing I would have tried. Its not worth the post and the title should really be something like "USB device didnt work with a hub, but did directly in the host" and it would never have made it to the front page of HN other than maybe to be mocked.
No, you are blaming the monitor manufacturer for the customer not getting the advertised speed out of their Ethernet adapter when they connected it through the monitor’s integrated USB hub.
I am saying your blame is misplaced, and that the adapter manufacturer deserves the blame because they designed their product around a controller whose USB implementation has so much inherent latency that it mathematically cannot sustain the product’s advertised throughput when connected through a hub. The manufacturer should have tested this extremely common scenario and picked a part that could sustain it.
No, Im blaming the user for overengineering and sensationalising the fact that a cheap USB device didnt work on a hub. That is a problem as old as USB itself and to go this far down the rabbit hole on it is laughable. There are plenty of other people in the thread that agree.
> The manufacturer should have tested this extremely common scenario and picked a part that could sustain it.
Hard disagree. Maybe if you bought a better one you would expect that, but I would expect a $15 2.5Gb ethernet adapter to not work on a hub right off the bat tbh.
UGREEN is one of the most common peripheral brands on Amazon. It’s up there with Anker. This is the market your average consumer is going to encounter. They’re not gonna know that $15 is too cheap to afford a BOM that can deliver the promises on the box in common scenarios like “plugged into a hub.”
> Hard disagree. Maybe if you bought a better one you would expect that, but I would expect a $15 2.5Gb ethernet adapter to not work on a hub right off the bat tbh.
I would expect any adapter built around the RTL8156B to behave in a way identical to that described in the article, regardless of whether it cost $5, $50, or somewhere in-between.
(This message was delivered in part by an RTL8156B adapter from a manufacturer with a Chinese-soup name that currently sells for $13.29.)
Yes, but in retrospect I seem to have slightly overstated the compatibility issue. I have since learned that a single standard USB 3 hub probably wouldn’t introduce enough latency to cause the adapter’s buffer to fill up. The extra work of unwrapping the Thunderbolt tunnel around the USB protocol introduces additional latency. But so would adding a second USB 3 hub in the signal path, or a single USB 4 hub, since USB 4 is always tunneled within Thunderbolt when both ends of the link can speak Thunderbolt/USB 4.
Yeah, TB does complicate things.
And while the DRAM or SRAM or whatever-kind-of-RAM is included on these NICs is expensive, and I'm very appreciative of that expense: We're talking about just a few more kilobytes being enough to make a positive difference.
The only reason it was ever an issue is because someone was being cheap. (Cheap of labor? Cheap of space in the tape-out for the IC? Cheap of testing? Cheap by adhering resolutely to the exact confines of the spec on their desk without providing feedback? Cheap of all of these things? Whatever; cheap.)
OK So this device would have worked fine directly plugged into the host, and into a USB3 hub, but not Apples fancy TB tunnel to its single cable monitor hub?
Are you still arguing that the adapter is at fault here? Because In my opinion the dongle works fine and its the users extremely rare use case which is causing this.
Additionally, when the USB 4 spec was created did they not foresee this issue of adding extra latency? USB was supposed to be completely backwards compatible, but this seems like a glaring breaking change by sacrificing compatibility for speed.
Thunderbolt displays are effectively Thunderbolt docks with integrated LCD panels. They exist for people who use powerful laptops as desktop replacements. Apple isn’t the only company that makes them: Dell, LG, Samsung, BenQ all make them too. It’s not an “extremely rare use case,” especially not for the kind of user that also needs a 2.5Gbps Ethernet dongle to function at its stated speed.
I guess when they designed the USB 4 spec, they decided not to cater to device manufacturers who don’t design their devices with enough RAM to do large enough bursts to sustain their products’ stated capabilities in valid use cases.
I feel like I am going a bit crazy reading some of these comments. I don’t know why people don’t read the whole blog post to understand that the monitor works great and it gets maximum speeds when doing things like transferring files from my ssd. The adapter works great and I am getting maximum speeds when connected to my laptop. The laptop works great and I get maximum speeds when doing any of those things. The hub inside the monitor works great and I get maximum speeds through it even with multiple things plugged in at the same time.
None of the parts are broken. Together they don’t work as they are supposed to because of some super intricate process of how usb works when it comes to acknowledging packages. That’s why I wrote the post. The title is true and yeah, it’s sensational but that’s what titles are there for. It’s not a false clickbait.
I read the entire post and enjoyed it very much. Thank you for writing it.
That said, I think you’re being too charitable to UGREEN. They shouldn’t advertise a product as 2.5Gb/s if it can’t actually transfer 2.5Gb/s to the host when the signal path includes a hub.
You are probably right. I think the reason why I cut them the slack is because of how cheap the adapter is and that it does give me the speed when connected directly or if I go with flow control. Its a weird one
Yeah, this probably speaks to our relative tolerances for fiddlyness. Despite being in love with computers since elementary school, I have no desire to muck with things anymore. And I feel for the normal consumer who just wants things to work the way they say on the box. But consumers also don’t want to pay more than they need to, and these reference-designs-in-a-box are a trap for those users.
That's been basically my consistent experience with ugreen - their products are cheap and work on the happy path, but edge cases are not accounted for.
> None of the parts are broken. Together they don’t work as they are supposed to because of some super intricate process of how usb works when it comes to acknowledging packages.
Typical Apple shit. Never had an issue with my Lenovo dock and 2.5G. Additionally, not having an eth adapter built in, in 2026, is enshittified hardware. Typical Apple again
Nope. This doesnt work on Lenovo either. Nothing to do with Apple
It is to do with Apple. It is because they are wrapping their USB protocol in a Thunderbolt tunnel to the monitor which is adding the latency. This adapter plugged directly into a device, or into a normal USB 3 hub works completely fine. Only when going through a Thunderbolt tunnel does it add enough latency to buffer the connection.
You know Thunderbolt is an Intel technology, right? There are plenty of Windows laptops that have Thunderbolt ports, and people use them with Thunderbolt docks. The article author even confirmed that this issue reproduces with a Lenovo Thunderbolt dock.
I think you could improve the post's clarity of outcome a bit.
While it's completely right, even when having read it fully, it isn't so easy to understand unless the reader knows some things (see https://news.ycombinator.com/item?id=50588459 and its children), namely:
* That the "packet size 1024 bMaxBurst 3" (making the 4 KiB window size) is fixed for the whole connection, determined by the RTL8156B, and cannot be negotiated higher, no matter if the monitor is in between or not.
* That this window size is large enough for the NIC---Mac connection to run at full speed, but not enough for the NIC---Monitor-USB-Hub---Mac connection to run at full speed, because the added latency makes the BDP too large.
* If possible to measure, it might be illustrative to measure the NIC---Mac latency without hub directly, to compare it with the `7.6 µs` number going through the monitor (like cibyr suggested). That'd make it more obvious that the monitor increases the latency.
And I agree with dcrazy's sibling comment that the RTL8156B in the UGREEN _is_ to blame. It uses a send window size that cannot possibly deliver the advertised bandwidth behind a hub.
Anyway, great post that shows that just because you buy a device that says N Gbit/s and delivers N Gbit/s when directly attached, you cannot be sure that it'll be equally fast when attached to a form of "longer cable".
It was the crappy Apple display
> I wanted to have a one-cable setup ...
The root of all evil right there. ;)
I've only read the headline: USB degrading speeds?
(five minutes passed, I skimmed through TFA)
Ah yup, of course it's somehow related to USB.
It's like the infamous line: "It's always DNS" (and when it's not it's the firewall).
When download speeds are degrading like that, it's always some weird USB thing.
In the earlier days people didn't know that, for example, plugging a monitor into an USB hub could, as a result, make the rest degrade to USB 2.0 speeds. Unplug the monitor, HUB is speedy again now that it doesn't need to deal with that 4K / 60 Hz or whatever monitor data.
It's not a criticism or anything and I'm not saying that's what's going on here: I'm just saying USB has many gotchas and there can be, as in TFA and as in many other (document or not) cases, weird things happening.
At times (and doubly so when troubleshooting), you're better to simply try to plug your USB cable directly in the computer instead of in a USB hub / monitor USB hub. (EDIT: ah yup, ofc eventually he try that)
There are thing too complex to ever be secure and then there are things too complex to not have gotchas.
USB is... Wild?
P.S: and then of course in addition to this USB wildness there's the mad insanity of USB... Cables.