Getting 50 GB/S Back from the Apple Neural Engine
eiln.github.ioRTL performance erratum in the Apple M3 Neural Engine throttles DRAM weight streaming throughput down to 17–19 GB/s from the nominal 45–60 GB/s. Avoiding the problematic path in the kernel DMA engine's speculative prefetch ring increased Llama 3.2 1B token throughput from 10.0 to 24.3 tokens/s.
Nice investigation.
It is always surprising to me when a nice round number like 1MiB results in the “bad performance” configuration (although it happens).
Are you sure erratum is the right word in this context? I usually see it used to describe the notice that a document has an error in it.
https://en.wikipedia.org/wiki/Erratum#Errata_associated_with...:
“Design errors and mistakes in a microprocessor's hardwired logic may also be documented and described as errata. One well-publicized example is Intel's "FDIV" erratum in early Pentium processors, known as the Pentium FDIV bug. This gave incorrect answers to a floating-point division instruction (FDIV) for a small set of numbers, due to an incorrect lookup table inside the Pentium chip.
Similarly, design errors in peripheral devices, such as disk controllers and video display units, can result in abnormal operation under certain conditions.”</i>
The rest of the Wikipedia page discusses errata as the documents issued to describe the errors, not the errors themselves.
The section you’ve linked is… a bit weirdly written. I don’t think they meant to suggest an alternative definition there. For example the first sentence could reasonably be read as saying the bugs are being “described as errata” but I think the more likely reading (and the one that is compatible with the rest of the article) is that they were trying to say that the process of documenting and describing the bug is what’s being done “as errata.”
That's a more common use, but the article's use is supported by dictionaries as acceptable. I'd personally prefer "bug" as a matter of style.
Huh, interesting. After looking it up I guess I was wrong.
"Apparently the memory controller's throughput has a dominant harmonic with wavelength 2048 in tensor-dimension space."
That got a laugh out of me.
This is kinda nuts.
Can this mostly be fixed in software, or are M3 users basically stuck unless apps work around it themselves?
Going from 10 to 24 tok/s just from a workaround is a pretty crazy jump.
Not all systems affected M1 and M5MAX are OK
https://x.com/anemll/status/2098454204478366132?s=20
Just checking here- this systemverilog is a hypothetical telling of what you think is going on? Or do you have the actual source of the RTL?
This goes way over my head and I don't understand most of it lol. I noticed you're still in the middle of getting your B.S. degree and you're already writing things like this...amazing.
surely someone in apple would have run some common neural net architectures through the rtl simulator before tapeout and checked that the performance meets expectations??
I guess any processes affected by this will solve this in software as time goes on?
Nice read detective
Ummm, wow. That's really bad.
The fit is (almost) trivial.
This website hijacked my back button during a simple page load. You should fix that, it’s not an acceptable way to behave.
I’m shocked that’s still possible, shouldn’t we be able to fix that at the browser level?
Just like how browsers disable or mute auto playing videos, they should be able to prevent back button hijacking until user has clicking a link or button.
Some websites like PC Gamer have some shitty middleware (that seems to be very careful to hide its identity, only appear once per session, and not immediately register on page load, probably to try to delay browsers banning the ever loving fuck out of their practices) that uses pushState to hijack the back button to try to SPAM more articles at you, probably in hopes of grabbing your attention. I would be in favor of permanently banning domains that do this even once (intentionally and not just getting hacked) from ever executing JavaScript ever again and forcing them to switch domains and lose their SEO.
Yeah you'd think this is something Google's stuff would detect and strongly punish, but somehow we ended up in this dystopian crapified parallel universe instead.
Somehow some sites are able to get around the video autoplay, I especially notice it on news sites.
They autoplay the video on mute which is not banned. I don't know why news websites do it, seems like wasting traffic and should cost more than the hypothetical income from video ads?
We should, in the alternative timeline where apps never invaded web.
My habit of nearly always opening links to other sites in a new tab helps me avoid that annoyance for the most part.
For example, if I browse HN, I open the linked article in a new tab instead of in the same one where the comments are. And I close the tab as soon as I’m done looking at it. Whether that be a couple of seconds because of annoying dark patterns, or a long time thanks to being actually readable and interesting. From that tab I might navigate to another page on the same site if the site seems good. If they then pull horse crap with the browser history in that tab I will most likely close the tab and not read anything more on that site. If I repeatedly get annoyed by a domain from having gone to it and found dark patterns I will eventually remember that the whole site is bad and stop clicking any links to it either. But mostly I don’t have to remember so many sites to actively avoid. Closing a tab is cheap when all of the history is from the same site that tab was opened for in the first place.
It’s unfortunate and annoying that they get to play tricks with dark patterns still, and I wish nobody did that / could do that in the first place. In general I mean. I dont know if this particular page does it intentionally or accidentally.
I usually try to not comment on it even if I do come across that. As I know many consider it off topic, and even the HN guidelines say to not comment on it [1]. But I still appreciate when others, like your comment here, point those things out and I then don’t click on to the linked article in the first place when I see comments about that unless other comments are indicating that the content itself is particularly interesting.
[1]: https://news.ycombinator.com/newsguidelines.html
> Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting.
I think this rule is quite unfortunate. Wouldn't we all be better off if sites that hijacked back buttons were just banned or at least had their posts hidden until they're fixed?
Sure, they're somewhat common issues, but so are ads for malware served by Google. Surely we should call those out?
Honestly, I feel like browsers should handle this.
Sites are always going to do malicious shit. The browser should know that when I hit the back button I want to go to the place I clicked that brought me here. SPAs have severely perverted this simple idea, but I’m sure browser devs can figure it out.
It was really just a bad design decision that was exploited. It should be deprecated and removed. Allow it on intranets or some legacy website specific setting, but it should be blocked on the public internet.
It would be crossing a line for us to ban or penalize sites in response to complaints about the back button.
In this particular instance, it works fine for me on Chrome. When I test it in Safari, I can see the same behaviour that people are complaining about, but I also find that if you double-click the back button, it takes you back to HN.
I point that out because it seems clear to me that the site author has not set out to implement a “dark pattern” to trick visitors into staying on their site against their will; they've just tried to implement some in-page navigation, and perhaps only tested it in Chrome.
That's outside the boundary of reasonable grounds for us to penalize a page or site. It puts us in the position of having to investigate a page's behaviour in different browsers and to adjudicate on the severity of the dysfunctional browser behaviour and the author's intent (i.e., their inner thoughts). We clearly can't be getting into that; it takes up too much time and we will never be able to consistently reach fair decisions.
I can understand the frustration, but as well as asking HN users not to fixate on offtopic distractions like this, we also ask people to "assume good faith", and I see no reason not to follow that guideline in this case.
A quick email to the author would have more likely led to a favourable outcome than this meta-subthread.
I generally open links in a new tab because browsers never seem to be able to consistently cache the previous page, so when I hit back it has to reload. A separate tab always avoids this. I even have a button on my mouse hardwired to Ctrl-W to quickly close these extra tabs.
This only appeared to happen with Safari on my end. Edge and Firefox didn't appear to have the back button hijacked here. Maybe some browsers do block the behavior now? (or maybe ublock origin blocked it, I'm not sure)
Don't tell us, tell them.
What browser are you using an what does the back button do for you instead of going back?
Safari just reloads the page instead of going back.
going forward also doesn't work (displays an about; page) so I am wondering if the website is using React or some other framework and the routing library is setup incorrectly, causing this.
Damn bots copy and pasting a single sentence from the article as a comment.
I think that’s the poster of the link and also the person whose github it is