I strongly dislike CUDA. Once you have allowed that proprietary cr*p into your C++ codebase, it is very hard to get rid, and you end up with code that is either tied to a single vendor or an #ifdef hell, probably both.
The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, etc.
These days we even have DSLs like Triton that make kernel writing much more ergonomic than anything you would hope to achieve in Rust.
You need Objective-C, Swift, and Metal is a C++14 dialect with extensions.
You may refer to the C++ bindings, which still not obviate the need for the C++14 dialect in the shaders, and it only works, because there is a shim to call the Objective-C runtime from C++.
Likewise there are DirectX COM interfaces that are really only usable from Visual C++ COM extensions, and the HLSL semantics depend very much on which compiler is being used, hence why there is finally a language reboot going on.
> The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually
No. CUDA allows you to write all the code in a single file, and uses a preprocessor to split it back out and pass it through separate compilers, one for host and one for device.
This true, but you can write the two separately if you want.
The disadvantages of writing them together are listed in the various parent posts. But some code authors really like the convenience of having the two in the same file.
I highly recommend Julia for (scientific) GPU programming but it would be nice if there was a larger community and/or funding behind the GPU side of things. It has very few core devs for what it is.
Julia has had a great CUDA story for a few years now, and this about 9 days old. Rust rejects buffer aliasing at compile time using Rust's borrow checker, but shared memory in cuda-oxide currently requires unsafe, but then there's HuggingFace's Grout and mistral.rs, so yeah, Rust is picking up ground here on Julia. How is OpenCL's performance these days?
I fell in love with MATLAB (or GNU Octave for free since you really pay for toolboxes/packages) back around 2004, despite it warts. So I second Julia, which is similar, but is a more modern functional language instead of imperative.
I asked Google's Gemini if Julia can run on GPU unmodified without annotations, pragmas, intrinsics or similar manually-managed friction, and it said yes, but that data types must be swapped out for GPU-backed types:
If your code is written using vector/matrix operations, broadcasting, or standard linear algebra functions, it can run on the GPU entirely unmodified. You only need to change the input data type to a GPU-backed array (e.g., swapping a CPU Array for a CuArray from CUDA.jl).
# A standard Julia function — completely agnostic to hardware
function custom_math!(C, A, B)
@. C = sin(A) + 2 * B # Normal broadcasted operation
end
# Running on the CPU:
A_cpu = rand(1000)
B_cpu = rand(1000)
C_cpu = similar(A_cpu)
custom_math!(C_cpu, A_cpu, B_cpu)
# Running on the GPU (Unmodified function!):
using CUDA
A_gpu = CuArray(A_cpu)
B_gpu = CuArray(B_cpu)
C_gpu = similar(A_gpu)
custom_math!(C_gpu, A_gpu, B_gpu) # Automatically compiles to native PTX!
This is the direction we should be going. So while Nvidia's Rust port is an important first step, it's an evolutionary rather than revolutionary achievement. But that's all Nvidia can really do now, since it's locked into its own paradigm like Intel/Microsoft and has gotten too big to think outside the box.
Edit: PTX in its example stands for Parallel Thread Execution, the Virtual Machine (VM) Instruction Set Architecture (ISA) created by NVIDIA for its GPUs, which works similarly to Java byte code.
Edit 2: Broadcasting is a feature in Julia that allows you to apply a function or mathematical operation element-by-element across arrays of different shapes and sizes, without writing manual loops. In Julia, broadcasting is syntactically indicated by a dot (.) placed before an operator or function name (e.g., sin.(x) or .+). <- I was today years old when I learned the term for this
> Once you have allowed that proprietary cr*p into your C++ codebase
People have been doing that all the time for every kind of codebase. It's just part of the business. I don't see how it's worth having any emotions or opinions about it. Seems like you are wasting your energy.
Are win32 APIs proprietary? So you decide to use them, use a wrapper/UI framework, or don't develop for Windows. Easy choice.
Developing for embedded devices? So you read the manufacturers manual and implement based on the spec, use some sort of HAL if they are available, or you don't have a job. Even simpler.
> The CUDA runtime is a special case of one of the libraries provided by the CUDA Toolkit. The CUDA runtime provides both an API and some language extensions to handle common tasks such as allocating memory, copying data between GPUs and other GPUs or CPUs, and launching kernels. The API components of the CUDA runtime are referred to as the CUDA runtime API.
That's a nice way of saying that it's a dependency clusterfuck.
I've never understood why we can't just expose the GPU ISA directly the way the CPU does. It's all getting compiled down at the end of the day so someone has to write a compiler for it either way. We'd be substantially better off IMO if it was all built directly into LLVM and then let middleware sort out the details.
That would require vendors to either stick with a single backwards compatible ISA like intel did for x86 or document how their graphics cards work.
CPUs manage this by changing the internal micro-architecture, but historically GPUs only needed to support a graphics API and used that abstraction layer to freely change the hardware.
If you're going to make apps in windows, you need to call their proprietary API somehow. Maybe you do it via a wrapper library, or via electron or something. But that's the same thing, just with more indirection.
> If you're going to make apps in windows, you need to call their proprietary API somehow. Maybe you do it via a wrapper library, or via electron or something. But that's the same thing, just with more indirection.
Not even close to being true. You can invoke syscalls directly, just needs a bit of reverse engineering. I wrote a bare metal libc library, with (not a whole lot of) effort I'm fully able to interface with the kernel/open windows etc. Fully statically linked, no libc, no win32, compiled on Linux executed on Windows.
The problem is this isn't really well documented _at all_, and I even ended up attempting to get in touch with the Windows kernel dev team to give me the actual internal syscalls/endpoints, but they refuse to cooperate. Which is why writing anything for Windows is entirely pointless.
The problem is much deeper than that. Most OSes' syscall ABIs are not stable and could change without warning. What is stable is the dynamically-loaded libraries, shipped as part of the system. Linux is the notable exception here; the Linux kernel project doesn't ship a libc, and Linus is very famously opposed to "breaking userspace."
There's nothing that can stop you from using syscalls in theory, but if you want your app to be portable across different OS versions, past and future, you'd better not.
Incidentally, syscalls would also break Wine. The way Wine works is basically by shipping their own versions of Windows DLLs, which express their operations in terms of Linux APIs. Because Windows programs don't rely on syscalls, and call all system functions via the system-provided libraries, the Wine loader can just link Wine's version and let the program work normally.
If I recall correctly, the Golang team got bitten by this on MacOS.
They initially implemented the Golang runtime directly on top of MacOS syscalls (not the C runtime library), just like they did on Linux - and then those syscalls changed, breaking Golang.
They had to switch to the official stable API which on MacOS is the C runtime library, not syscalls.
> This isn’t even close to being true. Here’s a thing I did that made things way more complicated than is worth it for 99% of developers when there is a proprietary solution made so I do not need to worry about these things. Because it is so hard to work around it, it is entirely pointless to develop for one of the most used operating systems in the world.
Just being totally honest this is how I read this comment when I insert context that seems important to me. I respect having principles but at some point there needs to be more value in practicality over your codebase not being locked into a proprietary framework at all.
> Not even close to being true. You can invoke syscalls directly,
The windows syscall API is yet another proprietary windows API. Sure - you can call it without loading any DLLs. But you're still calling into a proprietary windows API.
If you really hate calling proprietary windows APIs that much, maybe stop developing for windows? Develop software for linux. Or make your own kernel, or whatever. But if you keep developing software for windows, stop fighting it. Unless you have a very good reason, your software should try to fit in on its host platform. It should behave well, and work like other windows software.
It's like travel. If you fly to France, try to fit in. Maybe learn a bit of French before you go. If you hate France, don't go.
That’s insane. Windows does not have a stable syscall ABI. The way you’re supposed to interact with the kernel is through the userspace library. Of course the kernel team refuses to cooperate.
Do you want to keep reverse engineering the syscall ABI for every Windows edition and update ever? Do you want to ask your users to disable Windows Update?
Regardless, I don’t even understand how that’s relevant, since you’re still introducing a dependency on a proprietary ABI.
The CPU on most machines is quite proprietary. I don’t understand this faux purity dogma.
Practical computing is not and never has been an abstract pure concept. It’s about making machines built by corporations to do usefull things at scale.
There is no ”non proprietary” computing unless you make your own stack.
Yes but there are business costs to using high-level proprietary tools and libraries. If you write your app using win32, you won’t be able to port is very easily. You’re also stuck with whatever bad or bizarre decisions Microsoft made.
It’s even worse for CUDA. GPUs are expensive, and now you’re vendor locked. You’re between a rock and a hard place. Either spend millions in engineering time, or millions on price-gauged hardware.
I wonder which APIs you would use to port easily, because POSIX and Khronos aren't it either, as they are industry standards driven by companies where one has to pay for a seat at Open Group and Khronos offices.
Porting has never been hard. Just follow the platform guidelines. Make sane architecture. Done.
I mean _it's just work_. You don't need to invent anything. Just do the work.
What _is_ hard is when people run after silver bullets to avoid all this work.
Because people who don't understand software decide it would be cheaper to implement something only once. Or someone who does not really understand what they are doing insists that same C++ code runs automatically on all platforms.
AI has given the software engineers permit from the beancounters to do the sane thing.
Good software development orgs _have always_ done proper per platform ports.
Also - there is nothing wrong in supporting only one platform as such!
> Good software development orgs _have always_ done proper per platform ports.
I really wonder why this was never fundamentally fixed. How performant a certain instruction on a specific platform is, how well it is supported and potential equivalents or sets of other instructions to emulate an equivalent are usually all very well understood.
So there should be some graph of operations which can transform any software from and to the specifics of each platform. Especially because firmware + compliers + platform abstracting libraries are basically already just that graph, although (usually?) to lossy to be applied in reverse. Add the recent developments in very large scale statistics to it and it'd probably be quite possible to transform from and to generic intent in the implementation to the uniqueness of each platform. E.g. the theming differences between a MacOS UI and a terminal application served over serial or the processing capabilities of a VLIW CPU compared to a FPGA or a GPU server.
Considering the enormous amount of work that went into compilers, better debugging and intermediate representations it seems like a huge missed opportunity nobody seriously asked the question whether information could be emitted that would allow for decompiling all the way back to the generic intent.
” If you write your app using win32, you won’t be able to port is very easily.”
This is wrong way around.
If you don’t support the platform your app runs on using the native api:s to the hilt your port is just bad.
If you actually want to support multiple platforms _you actually need to support_ them from the ground up.
This is speaking industrially and businesswise. A professional software business always has per-platform implementation resources. Or they have just one platform. Or they pretend they are multiplatform and then _everybody_ _daily_ fights with the problems this causes.
Obviously those elements that can be portable should be. It’s like Einsteins simplicity maxim - your codebase should be as portable as can be but not more.
” It’s even worse for CUDA…”
No these are just the business and market constraints. If this does not make sense for your offering then don’t use it. This feels like false FOMO - CUDA is not a silver bullet but it might be a specific solution to a specific problem.
It really depends on the application. The reason web is so successful as a platform is because it’s rich enough for most applications, and inherently cross-platform at an OS level.
And Re: CUDA: yes if it doesn’t make sense then dont use it. That’s sort of my whole argument. It might make some level of sense from a technical perspective, but that needs to be balanced with business risk. I’m saying a lot of people aren’t doing the balancing right, which is why these new tools have value.
Dead wrong. Win32 (externally) only seems stable, but internally it changes between Windows releases. Win7 syscalls are completely different from Win11 syscalls, meaning if I want to release a binary _without relying_ on Win32 I need to provide full syscall mappings _for each and every Windows version_. This doesn't happen on Linux.
That's literally the definition of it being stable. Programs written against an interface keep working despite the implementation changing. The Linux kernel also constantly changes internally but programs written against syscalls keep working, so it is stable; that fact doesn't stop being a fact just because I dislike perf_event_open(2) or whatever. This is all very basic and easy to understand.
Correct, most of Linux user land targets API stability, not ABI stability. Windows targets ABI stability because applications are typically distributed as binary blobs. Most applications on Linux are open-source and built per each distro, so it’s a non-issue, just recompile.
This doesn’t work for proprietary software that’s distributed as blobs and rarely updated, like say, video games. But that’s a minority of stuff on Linux. But not on windows.
Realistically, on Linux applications target specific API versions of frameworks. Like Qt 6, or GTK 3, or whatever. Then everything is compiled or dynamically linked at a per-distro level. The ABI compat can bite specifically when distros enforce strict dynamic linking. But then containerization technologies come in.
And there is a difference between API and ABI stability. For example, adding SSO to std::string in C++ broke ABI, not API. If you recompile it’s fine, everything works. If you don’t then it doesn’t.
I can program all my non-CUDA GPUs use completely open non-proprietary toolchains. And if that ceases to be the case on one platform I can switch platforms without having to rewrite all my code.
If that business makes it money selling a cross-platform AI inference engine, as was the case for my previous startup, it is bad business to tie yourself to single platform. I managed to build a single code base that would support CPUs, OpenCL, Metal, CUDA, D3D12, and WebGPU from a single set of kernel sources. As a single developer, there was no way I would have been able to, at the time, maintain separate code paths and GPU kernels for each of those platform, in addition to training the models etc.
Oh, does that mean I get to say you're ironic because, literally, they didn't tell anyone to do anything. They said they didn't understand the worth of the opinion. You're interpretation is selectively literal in order to be rhetorical.
Does that mean someone else gets say I'm being ironic because I'm selectively literal in order to be rhetorical? Well, okay, I guess it's harder now.
You're interpretation is selectively literal in order to be rhetorical.
Where do you think you are ?
Most of us are in tech/IT/research the population in the spectrum here is orders of magnitude bigger than the avg on real life. SO yeah people will be literal in order to be rhetorical. Not even selectively, this is the one site where you NEED to use /s unironically.
That opinion is work-ethic related, not CUDA-related. The stance is reasonable too; why complain about characteristics of CUDA that can't be changed?
Your job as a CUDA engineer isn't to decide whether or not a proprietary API/compiler is the right call. Your boss made that choice for you when they hired you, and you accept the tradeoff if you want to keep working there. It's like someone protesting Dotnet because they wish they spent the rest of their life working with Perl instead. You can do that, but it's a completely different job with different pay grades and demands.
> I don't see how it's worth having any emotions or opinions about it. Seems like you are wasting your energy.
Some people only care about the easiest path to their pay check. Some people actually care about software engineering. I tend to prefer the latter but hamstrung by the former.
> People have been doing that all the time for every kind of codebase. It's just part of the business.
What is your ecosystem where this is true? Embedded or industrial, maybe?
I'm guessing you assume other people also use the same windows or embedded systems you're referring to. That's an insane thought: nobody would use this if they had any chance, and you intentionally chose this misery.
Obviously, you don't need to live this way. You can be free. Breathe.
> I strongly dislike CUDA. Once you have allowed that proprietary cr*p
Genuine question...why not just type "crap"? It's not even that much of a curse, but I've never really understood the point of self-censorship. If you don't want to curse then you could just use a non-curse word.
It may be to bypass censorship, rather than self-censorship. Some platforms block or shadowban comments with curse words. Not sure about this platform.
I have written many words far worse than "crap" on this site. I haven't gotten in trouble over it yet.
I do find it a little amusing, because commenters stopped criticizing my cursing the moment I started getting a good chunk of karma here. I remember in 2016 someone criticized me for using the term "shitposting"...I don't think I've gotten that kind of criticism since 2016 though.
It doesn't read as emphasis to me. It reads like the person is trying hard not to curse, and they think "crap" is a curse word. It's a little bit adorable, like I'm reading a comment from an obedient child.
I guess you are not from the generation of texters. This how languages work we used to use * as a way to avoid getting censored it over time became a way to curse or give emphasis.
People are getting used to censor themselves in order not to be reported, banned, or «hurt » other sensibilities. The words « rape » couldn’t be written in instagram for example, what a great way to deal with such a serious issue. Mainly an American thing spreading away from young people if you ask me. Sorry America, just being honest here.
As much as I personally dislike TikTok, I don't think it is fair to it: cultural willingness for more sensor sheep on Internet started years before TikTok's popularity in the west.
It's not the sole cause, but I believe it's the main driver behind a bunch of specific substitutions that are mainstream now or nearly so. For example, dih, ahh, and unalive. They may not have been invented on tiktok, but that's where they incubated.
There's no such censorship on TikTok, it's entirely groupthink based on people saying "when I use that word my video is seen less so therefore it's being censored".
Youtube is a lot more guilty of it though, as well as demonetizing.
It sounds like there is a documented policy or proven that TikTok does it. Just people thinking it does leading them to self-censor. Then people see others doing it and copy it. So, I guess it is censorship but not by TikTok.
No, but I think ICQ and some IRC clients had profanity filters turned on by default. I remember visiting a friend once and realising he hadn't turned the profanity filter off. I teased him about it for weeks.
After reading through the threat here it seems more like a cultural thing.
The US has quite a lot of filters for profanity.
I remember from my youth that in 2009 Eminem was a guest in a Germany TV show and very happy to swear as much as possible without being censored.
https://www.youtube.com/shorts/2OC-yKZ5Yag
But typing "b*tch" with T9 is just as difficult (if not more) as typing "bitch". Anyway, I guess I never had a need to swear much over SMS. On IRC, on the other hand...
Maybe more like p**p, as in "that cunt p**ped in my yard"?
I'll admit, it never once occurred to me that people might be using censored characters to provide more emphasis that a word is a swear, but I guess it does indeed do that, at least to the writer. Whether that comes across to the reader, and whether the writer cares that their intention was understood... I'm not so sure.
I was so disappointed when I tried reading tintin in other languages and found the dear captain was straight up using slurs in those. I wonder whether the english language ones have been edited over the years to remove that sort of thing
In any context I've seen, asterisks are for wrapping formatting and said formatting it to add emphasis. So being in the habit of typing `emphasised phrase`, for italics - regardless of whether the platform parses markdown/similar formatting, e.g. SMS.
To have an unclosed asterisk replacing characters in a word? I've only ever seen that as a way to bypass censorship. This spans communications from people currently in their 40s down to 20.
I think a string of non-alphanumeric characters would work much better here, like "Once you have allowed that proprietary @#$&% into your C++ codebase”
But this isn't perceived as emphasis at all. If I wanted to emphasize something, I'd be more likely to use something like *bitch* or something along those lines. Replacing a letter with an asterisk comes across as self-censorship, which is pretty silly - just use a different word if you're that uncomfortable with swearing.
I certainly dislike how everyone on YouTube is saying “SA” and “unalive” and “corn”.
It’s one thing if it’s some funny commentary channel avoiding those words, but what bothers me is the true crime YouTubers. In the subject of true crime, rape and murder are just things that are probably going to come up, and when they refuse to use the appropriate language, it comes off as infantilizing, which is weird considering that my actual YouTube account is over 18, let alone the viewer using it.
It's become so bad that even quality history youtube channels are frequently using euphemisms like "moustache-man" instead of just saying "Hitler", to avoid their videos being buried by The Algorithm, and therefore cut severely into their viewership.
> even quality history youtube channels are frequently using euphemisms like "moustache-man" instead of just saying "Hitler"
That can be quite confusing. You had German mustache-man, Russian mustache-man, French mustache-man (Petain), French small-mustache-man (de Gaulle), Spanish small-moustache-man (Franco)
I don't think those filters are even real, I think it's just mass-hysteria. I call these kinds of behaviours "traditions", but I'm not sure if there's a better term for it.
Basically someone comes up with something which is nonsensical, but plausible. Like believing that their videos are unpopular because they said the word "rape" and the algorithm magically got them, rather than because their videos suck. Then someone else sees that and starts thinking it is true. It silently spreads across the population.
I've seen this in organisations, where new recruits haven't been properly trained. Someone has come up with a method which is wildly incorrect and illegal, but plausible. The other new people around them have copied them. They've become slightly more experienced people, they've taught the next round of new people.
Before you know it, half of the organisation is doing something hilariously wrong, and they all sincerely believe it is the right way of doing it, because everyone does it. It's just self-reinforcing at that point.
I am sure they are bullshit. Like when they mute cursing and "risky" speech, but when you enable autogenerated subtitles they show up there. Youtube knows what thay said regardless if it's censored or not. It's so fucking stupid
> I call these kinds of behaviours "traditions", but I'm not sure if there's a better term for it.
In psychology that kind of thing is referred to as "superstition".
More specifically, "superstition" in this sense refers to the phenomenon of copying someone else's successful approach to a problem you have. (In your example, getting views on youtube.) Since you don't know what parts of their approach matter, you copy the effective parts and the ineffective parts equally.
No it isn't mass hysteria. YouTube has a set advertiser friendly guideline. It will scan uploads and streams automatically.
YouTube used to demonetise profanity unless it was mild. YouTube would demonetise profanity in the first X number of seconds of the video. These rules change and have been relaxed of April last year, but generally these rules still exist.
There isn't a hard filter if you say "suicide" you automatically get it. However it increases the likely hood of demonetisation. So people avoid it to be safe. So you end up with people using stupid euphemisms all the time.
The Suicide stuff is about protecting them from people who were promoting self harm. The unalive stuff is because of threats of violence. You can reference murder if it is say part of a news story or referencing something historical.
Creators are being careful don't want their video demonetised because once it gets the yellow flag, even if it is removed later they've lost the majority of monetisation.
The problem with providing direct proof of this is that YouTube moves the goalposts quite often and their auto moderation system is very inconsistent.
A (baseless) hypothesis: perhaps there are plenty of YouTube creators who use the proper, mature terminology but you never see their videos because the algorithm really is penalizing them for it...
I guess I never understood censorship when it’s plainly obvious what you’re censoring. Anyone who can read will clearly know that it said “crap”, so I don’t see how it’s fundamentally different than just saying the word. You still put the word into my brain.
"Daddy, what does cr*p mean?" Kids aren't stupid and this self-censorship isn't protecting anyone from anything.
(if a platform is serious about Bad Words for whatever reason (moral?) they would also forbid character replacements; ultimately it's the intent, not the word itself, that they try to steer with rules like that)
Your comment only makes sense in context if you believe in a deity who is too dumb to understand the difference between cr*p and crap. I for one do not worship a Bayesian spam filter.
IMO cr*p and crap are both valid but separate swear words. People have a wide option to choose from when they want to swear, and people like variety (much much more than LLMs do). People also tend to influence each other with their usages: cr*p is popular because it is popular.
Otherwise cr*p is just as good as crap, shit, horseshit, poopoo or such.
edit: * replaced with \* as HN interprets asterisks as formatting for emphasis. Thx latexr for informing me
Because I know it is not technically crap, a lot of competent people worked on it, most with good intentions. I suppose it is better described as a cleverly designed Trojan horse than can infect your software and make that software become crap, in the sense that it becomes harder to maintain, increases code duplication, messes with your build system, ties your build system to platforms that have their toolchain binaries available, etc., etc., without bringing any long-term benefits over learning things the hard way.
For learning that may be a fine approach, but CUDA (in C++) really tries to hide what is going on behind the scenes, which is roughly:
1) code gets split between a host part that goes through your normal compiler, and a device part that goes through the GPU compiler. You may as well write the kernels separate and compile them via a separate compilation step, and keep your trusted host compiler for the host-side code.
2) data needs to move between the host and devices via explicit buffer transfers and synchronization steps, CUDA tries to hide this with annotated pointers, but it is really easier to think about those as just buffers that you allocate and transfer IMO, instead of trying to transparently share pointers between host and device like CUDA does.
3) kernel launches can we wrapped in a function similar to:
Instead of the funky <<< >>> syntax that CUDA for C/C++ imposes. The problem is that once you start putting that in your code, it stops being C++ and stops being portable to non-CUDA GPUs. The launching and grid settings can be a bit hard to grasp at first, but sugarcoating that in bastardized C++ syntax does not absolve from having to understand it eventually.
So a good place to start might be an OpenCL or Metal primer, depending on the hardware you have available. D3D12 (and probably Vulcan too) makes this much harder than it should be, with too much boilerplate but is overall a mature and well-designed API should you wish to develop for Windows. Starting with WebGPU might also be good these days. It has a very different shader language than the others, but the rest of the concepts are similar, and it has a strong emphasis on making things async, which is what you want for performance anyways.
Claude/Codex should be able to get you moving very quickly.
Thank you very much for the effort you put into your advice!! I think I will start with WebGPU (wgpu), even though I have an Apple Silicon Macbook. I would really prefer to work with Rust instead of C++ because I am not good with C++. (I believe) I am good with C, so my C++ code looks like C code, and I am kinda learning the differences as I learn CUDA, which is a terrible way to learn C++, I guess.
The long term benefit is that there are more developers with CUDA experience available to hire than there are with any of the "hard ways" you mention.
Not saying you're wrong, but my career got a lot less frustrating when I started focusing more on the product and less on the ergonomics of the implementation. If you need to build a house and the customer isn't willing to pay for brick, you use vinyl siding.
> I've never really understood the point of self-censorship.
Some platforms disallow certain words. In order to bypass that, some people use the asterisks. That's just as one possible answer to your question; there can be many different reasons for self-censorship, but to me the most logical one is when one tries to work around crappy restrictions, such as on terrible reddit (they killed old.reddit recently; I retired before that due to moderators being insane, but I also said that if old.reddit is gone, I am gone anyway - the requirement to now log in, totally defeats old.reddit com's usecase. Then again reddit went downhill many years before that already, so not a real loss.)
Launching kernels manually is an error prone PITA which I believe is the principle reason for CUDA's popularity. Having the compiler give an error when you mess up is a huge benefit. But having the compiler allow you to express "I want to launch this kernel over a grid with these dimensions, with these arguments" as a single expression is where the vast majority of the value comes from.
The having it all in a single file is mostly an artefact of the fact that it is C++, because C++ is single file at a time compilation. In D (which is multiple files in a single compiler invocation) with DCompute (which targets CUDA and OpenCL with upcoming support for Vulkan and Metal), you are required to write the kernels in a separate module, but you get all the benefits of the compiler complaining when you mess up _and_ the expressivity of "launch me this kernel".
Write python code, ask any llm to translate it to C, then compile the C code - if it produces errors or fails to run, ask LLM to fix it. Then take it a step further and ask it produce machine code, and repeat the procedure.
Then RL the llm on the above, and you basically have a Python -> Machine code compiler. If you cover every single possible python syntax, every single possible C syntax, every possible standard library call, and all the compiler optimization examples (all of which is a final set), you should get something that is extremely accurate.
Having also played with Metal and WebGPU (at least years ago), I would say that CUDA is, amazingly, the best GPGPU API we have. Do I wish we had an open source parallel programming language as good or better than it? Yes. But asymmetrically hating on CUDA like this is how we continue to lag behind it in UX.
> The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually
Not to mention that this is a completely sane way to use CUDA as well.
I know it's not the same thing because proprietary vs open software it's way less important but, generally if you are not ideologically against something you can easily follow the stream and do lot of nefarious actions, especially if the action has enough degrees of separations from the actual nefast outcome.
I don't mind CUDA, I do mind that all of the SDKs don't dynamically load the various CUDA shared libraries at runtime.. intertwining itself into your application linking process makes for extreme binary portability inconvenience.
There a flavor of CUDA runtime libraries that binds at runtime, so you can have a single binary that runs with CUDA and without it. I did this at work.
Obviously you need to check if CUDA is available before trying to execute kernels, or it will error out.
i'm surprised modular's doesn't get much traction. The promise seems super interesting, and chris latner has the record to back up his claims. If someone has an explanation..
from what i can tell, you're going to be stuck with that no matter what you do
i'm currently using vulkan, and HLSL via dxc.
which should be portable but it's not.
apple refuses to support vulkan, and relies on moltenvk
and there's a bunch of OS/hardware/driver differences no matter what you do, that you'll probably have to feature test for, and compile a few different versions of your code no matter what you do
i think if you're doing something that you don't have to distribute to customers, just picking one stack and getting locked in has some appeal.
it leaves you vulnerable to lockin. but, especially in the age of ai, "claude, port this to vulkan" seems like a good enough defense against that
> The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually,
Yes, I also prefer doing it that way, but in Cuda with the driver API. Allows you to handle kernels like shaders, including editing and hot-reloading at runtime.
The reason I'm sticking with CUDA is because it's by far the most convenient API to use, without nonsense like 50-liners to alloc memory or the need to manage descriptors, bindings, queue families, etc.
> The reason I'm sticking with CUDA is because it's by far the most convenient API to use, without nonsense like 50-liners to alloc memory or the need to manage descriptors, bindings, queue families, etc.
I was there when the OpenCL committee was deciding on that sort of stuff.
As I recall, and it's been two decades and a lot of sleepless nights since then, there was real pushback at the time against OpenGL-style default bindings. So folks didn't want to establish an implicit command queue or any other default objects attached to other objects. Part of it is because OpenGL was perceived as clumsy and passé, some of it was because it is not friendly to multi-threaded applications.
Those first meetings were a shitshow full of tension, implicit threats from Apple, and backroom deals. Kudos to Neil Trevett for chairing the group; I I bet it wasn't fun for him either.
That's unfortunate. Cuda has shown that, when done right, defaults and a convenience layer can make for a well received API without sacrificing performance.
Ive essentially followed that paradigm with Python and C. I start out writing Python code. If I need something to run fast, I build a standalone C application that either reads from a file or listens on a socket, and just invoke it from Python. No need to write the entire thing in Rust and deal with all its semantics when it will be at best like 2% faster.
You don't need to use the CUDA (SIMT) programming model if you don't like it. The project includes cutile, which lets you program the GPU using tensors. It feels a lot like programming the GPU using numpy and triton.
As it happens, I just got my employer's permission to release as open source a Triton back-end for Metal and D3D12 GPUs here: https://github.com/dropbox/neso .
Finally, I am also going to be switching the inferencing part of Witchcraft from current Candle on MacOS and OpenVINO on Windows to just Candle with Neso; https://github.com/dropbox/witchcraft
It is currently faster than the stock Candle / MPS shaders it replaces on MacOS/ARM64, and IIRC a bit slower than OpenVINO/CPU on my old Windows laptop, where I never got OpenVINO/GPU to compute correctly. Candle didn't have support for GPUs on MacOS/Intel, and OpenVINO ceased to be supported there.
Compared to OpenVINO (I tried ONNX runtime too, but never got it produce correct outputs with my quantized models) it is very nice to be able to build the exact kernels I need, at the quantization settings and precision that works for the models I have and with the custom operators required (speech models do a lot of non-standard stuff), run from a single set of sources, and not have to ship a hefty third-party DLL, and having to deal with their memory leaks and other stability issues.
Okay, so, GPUs are taking one more step towards being general purpose massively parallel machines. That's cool.
What would be even cooler though would be for GPU vendors to start giving us the user manual. An I mean the real user manual, that explains how to use their piece of metal when all you have is that piece of metal. That means a precise description of the wire protocols, the data format of the buffers we send to & get from the GPU, the ISA of the cores we have access to, the relevant performance characteristics…
In other words, enough information to write a state-of-the-art driver for any OS. That would be cool.
They don't need to do that to sell their hardware so why would they do that? On the other hand, they have strong incentives to not give you that level of access and information. The only way this will change is by having some disruption by way of a competitor who sells hardware with that feature as being a major reason why it takes off.
Or we could regulate. One hammer I'm tempted to use is to simply forbid sales and importation of hardware made by companies that also distribute software. That way the only way to sell you hardware is to make sure its interfaces are both simple and thoroughly documented.
We could possibly make an exception for open source software, or only forbid the distribution of the relevant drivers, or limit the applicability of that law to specific hardware: hard drives, mice, printers, web cams... or GPUs.
They don't release it because exposing a stable instruction set would kill their ability to quickly iterate, to release silicon with bugs that can be papered over with software fixes, as fixing bugs in chips is very expensive in terms of time to market, and undoubtedly to charge more for what looks like a hardware feature but actually is a software feature.
It's been this way for 25 years and I don't see it changing.
I've been programming GPUs since the PlayStation1. The way they work under the hood has changed fundamentally maybe 4 times in that span.
I can't compare it to changes I've seen in CPU architecture since then. Maybe like: Compare the NES with its 6502 and per-cartridge mappers vs. a IBM 386 PC. Now repeat that shift 2 or 3 more times.
> The way they work under the hood has changed fundamentally maybe 4 times in that span.
How many times since we got shaders? And when exactly? I bet each change takes longer to come than the last. I mean, GPUs are increasingly general purpose nowadays. Sure they're optimised to specific kinds of embarrassingly parallel problems, but as more and more of their capabilities move out of the fixed pipeline to shaders, the need to change lessens.
> exposing a stable instruction set would kill their ability to quickly iterate
I believe the need to quickly iterate dropped significantly since we got shaders. I would bet in fact there was few such iterations in the last 10 years. Crazy increases in computing power of course, but big breaking changes in the actual ISA? I'd be surprised.
> to release silicon with bugs that can be papered over with software fixes
Yeah that's actually one of my goals. Only ship stuff that works on pain of embarrassment and prohibitive recall costs. It's crazy hard. It's how CPUs are shipped.
> and undoubtedly to charge more for what looks like a hardware feature but actually is a software feature.
Again, that's good. Such pricing tactics are scummy, I want to end them.
---
Now of course, those reasons you cited are reasons for the vendor not to do what I'm pretty sure is very good for the consumer. I propose we force them. We could start small. Mice and keyboards first. Then printers. Then webcams, wifi modules... until we get to GPUs themselves.
i really don't think you know what you're talking about. this isn't about better or worse. there are many many many workloads a GPU cannot at all implement (hint: anything with branches).
Since NVIDIA owns huggingface now and huggingface has the excellent Candle [1] crate for inference on Rust, this seems like a good step towards nice native Rust kernels.
Nobody cares if kernels are written in Rust. Kernels were meant to be written in C, but if you want to go more high-level try Triton or a similar DSL that nicely abstract tile sizes etc.
kernels aren't meant to be written by any defined language. C is just a traditionally good default language that took over from assembly. No particular reason we have to stick with C.
What reasons would you have to prefer Rust over C for compute kernels?
I am a great fan of Rust, but I don't see any benefit for kernels, due to their relatively simple nature.
SYCL is the natively polyglot counterpart, with practical implementations of it compiling down to the same sort of SPIR-V kernels as OpenCL. (OTOH, much of the current adoption on the open standards side seems to target the more widely supported SPIR-V compute shaders, via Vulkan compute.)
Not really, first of all it is for C++, not the range of languages supported by CUDA.
Before SPIR was a thing in OpenCL, Khronos could not understand why anyone would care about anything else other than C99, or why supporting Fortran on GPUs was at all relevant.
Secondly, from the competition only Intel cares about SYCL with their own sugar on top, OpenAPI.
AMD hasn't cared one second about it.
You may mention Codeplay, which is anyway an Intel owned company since 2022.
As for Vulkan, it doesn't have neither the features, nor the tooling that CUDA enjoys, it is the usual putting up with using LEGOs from different brands, with various pin sizes, that is so common with Khronos.
I will give you an outsider's perspective on an analogy in this case. It is easy to see Candle as a ML crate to use for neural networks in rust. I have used it, and it works well.
The analogy is Tensorflow 5-10 years ago. It is popular, and there are lots of material on it. You quickly learn from talking to people that due to whims, a collection of reasons, people's love of consensus that no one is recommending it; new people are not learning it. In this case, the Torch analogy is the Burn lib.
And to tie this back into GPU programming, Burn's backends use CubeCL, which lets you write compute kernels in a Rust DSL using #[cube], with a JIT compiler and autotuning machinery. It targets CUDA, AMD, Metal, Vulkan and WebGPU.
Really exciting but it reads like Claude instead of what Nvidia posts have generally been like in the past. I don't need nor want my tech blogs to sound like a young adult novel.
I’ve had this happen to me several time over the past weeks and it’s gone from quaint to humorous to farcical to outright “is-the-world-gaslighting-me” insane.
Just today I was reading Stanley Druckenmiller’s op ed in WSJ. This dude is like 80 and has made billions of dollars, and he got Claude to write his op ed???
Unbelievable. And the tells are so obvious, yet people still love the Claude-like quips and odd grammatical choices that read like halfway asshole halfway mid-sentence confusion.
That op ed was absurd. I respect Druckenmiller a lot and am always impressed with his lucidity in interviews. The Claude “ick” was all over his writing.
Yeah definitely Claude. Lazy authors, if you're going to get AI to write for you please use Astra instead - it makes way less annoying prose than Claude.
The momentum behind rust seems absolutely unstoppable at the moment, in the light of this, the adoption of Rust into the Linux kernel, and the adoption of for formally verified software by Amazon and Microsoft.
not too long ago, commenters on HN hated Rust and would never use anything written in Rust. So, now that Rust is in Linux, they shouldn't be using Linux either.
I think it would be much nicer, although unrealistic at the moment given the number of combinations of GPU vendors and variety of hardware, to directly target the underneath GPU ISA machine code.
Since I can write a simple compiler to target x64 machine code, it should be possible to write one to target my GPU.
Though, I'm certain that vendor lock is probably more profitable for them.
My thoughts on this, as a person who has recently coming around to working in this space is that up to now the convenience and "simplicity" of working in CUDA as it is has been a giant moat for NVIDIA. Having a whole toolchain with a C++ dialect and a giant extant pile of code out there that looked familiar to people meant they've "won" the AI wars.
And in that context NVIDIA had every motivation to keep their SDK somewhat abstracted higher up the chain and fully under their control and then be free to innovate in the lower bits. And this served them well as well as their customers.
My sense is that now with agent driven development this is basically evaporating. Agents are capable of at least prototyping/writing kernels for any hardware and ISA. e.g. OpenAI built their own custom hardware and ISA for it and then set agents loose on it writing kernels and claims great success. At least they're claiming this. And from my own experiences as a n00b entering this space, I can believe it.
TLDR I don't think vendor lock on the software side is going to work out for them as a strategy.
But luckily for them they continue to have really good hardware and good access to semiconductor fabrication. But just look at HotChips 2026 a couple weeks ago and look at the huge variety of new inference hardware coming down the pipe which looks completely unlike NVIDIA/CUDA.
VectorWare founder here. We are working with them and stoked they are investing more in Rust. I just gave a talk at RustConf about our different takes (https://rustconf2026.sched.com/event/2KNQj/making-gpus-feel-...). The video isn't up yet but you should check it out when it is. The efforts are complementary.
Towards the end of the post, we (NVIDIA) mention that this work was done in collaboration with Vectorware and others in the Rust community. And we can't wait to build further with the community.
Anyone know when Rust's std::autodiff will become stable? Assuming this Rust support expands to other GPU vendors, autograd will probably be the only reason to use Slang instead of Rust anymore.
I was told in the 2025 LLVM dev meeting that it will always stay in nightly because it's not practical for them to provide long-term stability guarantees that is expected of stable Rust.
To everyone skeptical of people saying that it's better to separate CPU and GPU code into separate files, riddle me this:
With this system, how do we specify whether dependencies are needed to be compiled for the CPU, GPU, or both?
----
Technically I don't know really care whether it's multiple files or one, I just want to make sure we are not reinventing a shittier version of CFG. What they are providing looks to me like:
1. explicitly annotate some things as `cfg(GPU)` or ungated (both)
2. unlabeled means annotated `cfg(not(GPU))` by definitely
Put this way, this has nothing to do with GPUs, and just has to do with creating some crate-local CFG shorthanded. Great! Let's do that first, get a solid foundation, and then come back to whatever is remaining for CUDA Rust.
Not a cuda programmer, but since they’re making a new API, why would they already make it inconsistent at start? :-/ I’m referring to the examples a,b,c vs z,x,y (different ordering of output elements)
You write kernels in a Rust DSL using #[cube], it supports WebGPU through WGSL and Vulkan through SPIR-V, along with CUDA, AMD via ROCm, and Metal. (disclosure: I am a contributor)
I'm looking forward to trying these when they stabilize! I currently use WGPU for graphics, and cudarc for CUDA.
Note: Cuda-oxide is similar to Cudarc's host component, but uses a rust-style kernel dialect. Advantage: Share structs between host and device. Disadvantage: Trading standard Cuda kernels for a new, WIP dialect.
I haven't tried the tile API yet; looking forward to it.
The last time I checked, Cuda Oxide was Linux only, and required Async; these are why I haven't tried it yet.
cudarc been great for me, because it's easy to look up existing examples and references, and it maps 1-to-1 with what I see. I'm already having a tough time with CUDA itself, a dialect of it makes a tad harder to rely on previous work.
Seems more ergonomic in general though, both approaches they share, compared to cudarc, and less build infrastructure and fiddling with environments, which is great.
It gets fun when someone uses an uncensored model to bypass a refusal, but they accidentally pick one that was trained for erotic writing and brings its particular talent to the documentation task.
I think you can't trust Pangram in a high stakes situation, but it is absolutely better than random noise at detecting AI-generated text. Which isn't surprising. If the distribution of probabilities can yield blatant Claudisms, it's not surprising it would also have more subtle deviations.
(Addendum: As I recall, LLM-generated outputs roughly follow Zipf's law, but the distribution still tends to have some subtle distinctions vs human text; pretty interesting, but I don't know where I heard this, so nothing to cite. Sorry.)
Pangram has an extremely low false positive rate. Even on adversarial examples.
One trade-off is even some obviously LLM text won't get detected by them, but they work really hard to ensure false positives are rare since a false accusation is much worse for society than someone getting away with LLM meatpuppetry.
To be honest with you, I don't think I would be able to identify with high certainty that the bottom text is AI generated, so it definitely goes a long way to obscure the AI-generated nature of it, but I also think it still feels unnatural somehow. I realize my framing naturally calls into question whether I'm being honest, but I am being honest. Given my experience with similar "skills" (it's just chunks of prompt, nothing magical after all) I expected even less.
But still, this is all very strange because it wasn't that many generations of AI models ago that AI writing was a lot better - I'm talking GPT 4.1, Claude 4.5, that sort of era.
Anthropic newsroom posts on the other hand are carefully constructed and well-written in a way that I have not seen demonstrated by LLMs yet, past or present. I expect that they have well-paid staff who are careful with every detail of their public communications. When you put it that way, it almost feels unfathomable that they wouldn't, doesn't it?
I don't feel like either one of you really has a strong claim. "Doesn't appear to" is subjective, and of course it's impossible to prove one way or another.
You're simplifying the exchange a little too much. I said:
> Anthropic doesn't appear to use Claude for blog posts
My claim is literally the lack of evidence, which, yes, can't prove anything. This claim can be contested easily by showing evidence that they in fact, do appear to be using Claude to write prose in blog posts.
They said:
> Anthropic _absolutely_ does
Sounds pretty certain Anthropic is in fact, using Claude to write blog posts. Enough to emphasize "absolutely". That doesn't read like "I'm going off of vibes", that reads like "I can prove it". So, fine. Prove it. I don't believe it, and I want to hear the proof.
I'm skeptical, but it wouldn't be my first time being wrong. But flatly, if you make claims with this kind of certainty, yes I want to hear your proof.
My point in saying "Even Anthropic doesn't appear to be using Claude for blog posts" was not meant to be some striking revelation, I literally was considering it a prior to make another point. This on the other hand sure does sound like a striking revelation to me, that a lot of people across the Internet would be curious to hear. Like I'm sure these people would be interested:
I will admit that I am unnecessarily aggressive sometimes, but I wouldn't have changed my response much in any case. If you're going to make a strong claim like this, I want your evidence, not your vibes. Otherwise, the claim should be a lot weaker.
I also realize that this sort of brashness upsets HN a bit, but it is what it is. I pandered comments for votes in my 20s a bit, time to grow up, sometimes people won't like you. Sometimes I feel something deserves a brash response.
I don't really have any opinion on your tone; I just still don't agree with your framing. A lack of evidence would be neutral like "there's no evidence to indicate either possibility is more likely", but your phrasing conveyed that one possibility was more likely than the other. I pushed back against your follow-up because it seemed like you were arguing for a higher threshold of evidence than you provided.
Well, to be fair, you're correct. I am asking for a higher threshold of evidence. It's a stronger claim. I feel a stronger claim deserves stronger evidence.
I guess that's where we disagree. I feel like either claim is equally hard to falsify from the outside (partially because I've never had much confidence in my ability to spot whether text is from an LLM outside of the most glaringly obvious cases, and likewise don't have any clue whether people who have high confidence are accurate or deluding themselves).
According to https://news.ycombinator.com/item?id=49417480, Anthropic hires writers who do not use LLMs, and reading their updates I also feel that they don't use LLMs for communication.
I think implication being organizations with 40,000+ employees and even more consultants and contractors plus a lot of budget are also using LLMs to draft public facing content instead of paying for content writers or even just proof readers .
It points to friction rather than cost economics. Same reason we are always surprised why multi billion dollar product companies with millions of install base prefer electron instead of a native app.
This does not imply that the organization is not paying for content writers or proof readers. It does suggest that they are not getting the value of paying for content writers or proof readers.
McDonald's food is not even that unhealthy. I just tried a Burger King burger the other day and it's terrible. I think it's like 2000 calories in a single burger or something.
2k cal would be around 250ml of oil. or 350grams of peanuts. So doing with bread, meat, and other stuff alike requires over 600g of food, an excellent value to energy.
I get the impression that Nvidia employees don't care too much - I started seeing fully AI-written "documentation" on some of their smaller projects more than a year ago (i.e., before it was even slightly a good idea).
when you start to internalize that these kinds of statements are an indication of how the average developer of the last 10-15 years operated the adoption rate of AI makes a lot more sense
Yeah lol it’s basically the only reliable way to know how things work. Pre-AI, I read documentation for libraries that I used almost every day.
And now with AI I’m using it to fact check Claude. And still reading it for myself to understand why other peoples code is written a certain way. It’s basically the most important thing to reference when coding.
Sure today Claude can just read the library code and tell you what a function does or how to do something. But it still won’t tell you why something is a certain way or won’t figure out specifically-designed usage patterns as reliably as the author telling you “this is an example of doing x”
I really appreciated a friend reaching out to me with some PHP questions today. It was, to me, fairly basic but he was having a hard time grokking the documentation vs reading what his coworker wrote (some code using output buffering).
I brushed up on the docs since I haven't touched it in a couple years, explained my understanding of the ob_* functions, and gave him a very brief demo on a PHP playground.
He could have asked any LLM to tell him what that chunk of code did, and to explain the three functions, and instead he reached out to me. That felt _good_. Talking shop has always been a good way for me to form connections, because the pressure to socialize becomes task-oriented and you start to learn about how people think and feel, and that opens up easier paths for actual connection. It was nice.
Just like the Old Internet still exists - niche websites, mailing lists, probably a BBS or two (likely more right?), the pre-LLM world will trudge on, for a time. I hope LLMs actually lead to good things for people in the long run, and for now I personally will remain sparse in my usage of them.
The good news is your attention to actually reading and understanding documentation will differentiate you more and more as others (short-sighted, IMO) outsource understanding to an LLM.
I hadn't read the article and read this comment as though NVIDIA themselves were implying that this library was checked but not trusted by them since it was fully LLM generated.
Dude I am in slop fucking hell right now. There is still room for a human touch, without which the agents will lever us harder and faster into a world of incomprehensible garbage.
I totally concur. I'm almost lost for words at this stage. I need me some land to grow vegetables on and that's about it. Maybe some chickens. Every single day brings more despair (and not the prosperity we were promised).
Do you have the stomach to walk into a high school in the USA these days? Teachers use AI to generate assignments. Students feed the assignments to AI and submit the responses. Teachers feed the student submissions to an AI for grading.
FYI, the world is a lot more decentralized than we think and even during the Dark Ages, guess what, that was happening in Europe and many places in the world were booming scientifically, technologically, etc.
Meh. It's not like we forgot how to make copper wires for landlines. We'll be fine. We'll live more or less like in 1880 or 1920, it's not a horrible life. I do hope we get to keep antibiotics, though.
It's true of most work in many and soon most white collar jobs, too. Claude writes some dense useless thing, everyone else uses Claude to summarize and write a reply to the thing. The Claude-submitted PRs get automatically reviewed and commented on by a GitHub Claude review bot. The programmer asks Claude to check out Claude's review comments to Claude. Claude pushes a commit to the branch and writes a comment. The Claude review bot reviews the commit and leaves a comment. The human [...].
My hot take is that it's not really that terrible in the long run for work since I think LLMs will probably be nearly or actually AGI and better white collar workers than most humans within 5 years of today. But it is very funny and surreal in the meantime.
It is definitely bad for school, though. Kids IMO should actually be encouraged to use LLMs but not in or for class work outside of an AI best practices class. Probably stop giving them homework (90% will always try to find a way to make AI do it) and have them solve problems in class hours with no electronics so that they're forced to not defer learning. This will become even more important once we have AGI.
I have two kids in engineering programs at a state University. They are allowed to use AI for homework assignments, but the homework is no longer worth any credit. They have a lot more papers, quizzes, and tests in class that count for their entire grade.
I know many teachers who actually have respect for the profession, themselves, and the students. Thankfully that means they don't do this.
Whether this is a widespread macro trend is another issue, and would be terryfying.
If true, however, it would reflect on the values of the organization: we have spent decades underpaying teachers, and doing a poor job of pretecting schools from frivoluos lawsuits. Add into that, districts have thrown money into new buildings, have been suckered by Big Tech to adopt their policies (common core was pushed by Big Tech and has been a distaster as well as computers in classrooms). As a nation (the USA) we can't get our act together for a rigorous national exam, etc etc.
About half of the states in the USA require the ACT or SAT for high school graduation.
Alabama is one state that requires the ACT. The mean score in Alabama is below 18/36. Wisconsin is another. Its students score on average about 1 point higher than the national average of 19.4/36.
If you prefer states that require the SAT, the mean SAT score of students from Delaware is less than 980/1600, about 50 points below the national average.
I'll leave it to others to argue about whether these exams are rigorous.
As a high schooler going to a school with stricter rules on AI than most in my area, I can say that it's been going downhill ever since GPT 4. Teachers constantly use AI to create assignments(my French Teacher regularly handed us work with GPT 5.1 prose and emojis). Students are also rampantly using AI and bypassing school restrictions(We have a google account, making it easy to use Gemini if we just sign out), causing an inflation in GPAs and test scores. There's no easy solution to the problem, banning AI-tools only help somewhat as even typing into Google has AI web results, and students are quickly overcoming ways to restrict them. I have a friend that vibe coded an application that allowed his Mac Mini's desktop to be mirrored on his school chromebook, bypassing every restriction with sub 1-second latency. Of course, that opens the can of worms to whether schools should allow students to use AI...
> Of course, that opens the can of worms to whether schools should allow students to use AI...
We are starting to see results indicating cognitive decline due to AI in education, so no, we should do everything possible to ban it except for very limited fields.
LLMs aren't calculators or even computers, their generated output is too flexible, generic and basically starts replacing thinking.
Most likely they should only be allowed during late highschool years or just at university level, when people at least have a chance to learn how to research on their own.
Yeah. I think if the text is written for other machines, then by all means have an LLM generate it, but if it is intended for a human audience, have a human being write it.
We are still much better at writing in a way that doesn't waste other people's time.
It is the number 1 thing I cannot stand with Claude slop. It's a sort of anthropomorphization of language. Every "thing" does, produces, feels, wants, asks, answers, etc....
- "Launch is checked"
- "Question is asked"
- "The implementation answers"
- "The model wants"
- "The results name"
- "The connection surfaces"
- "The prompt wires"
- "The feature rides the mechanism"
Every single fucking thing is alive, wants things, and does things.
It's terrible. Infuriating. I want to rip my eyeballs out reading this filth. All. The. Time. "The anger is real".
Create any page with a file uploader. They all look the same now. It's like the Twitter Bootstrap days of responsive design. You'll get an icon which looks like ones on (on the drop space) those sites which are like "you must wait 60 seconds for this file to download".
It's so horrible. The human element has been completely removed and replaced by..... mediocre.
The human element has been completely removed and replaced by..... mediocre.
No it hasn't. The human element is still there, prompting the LLM. The change is that the human is happily accepting the first thing they get rather than critically looking at it and seeing a problem.
I don't think it's that because I see a lot of this in businesses where the human isn't paying the bill, or is even aware of what the bill is.
Humans are seeing either a shortcut to go faster (accepting low quality to move on immediately; reasonable if they're short on time) or a shortcut to lowering effort (accepting low quality because they don't care; not so reasonable but probably has a deeper root cause).
In this age of LLM written everything which has softly killed my motivation for learning Rust somewhat, this has revived my interest if not only for the fact the LLMs haven't yet been trained on this yet!
I think OP’s point is that the payoff in learning a new language has diminished in this AI era. You can call that lazy, I’d consider it smart to consider whether you could be doing other, better, things with your time.
I had LLMs write a pile of cuda-rust code and they were quite competent at it. Ported a bunch of (C++) CUDA kernels over, and ground away on them til they got equivalent performance
I was learning Rust slowly when the LLM enabled coding became good enough. I switched from learning to full on building with Rust. I still learn high level concepts as needed but I will not be able to write Rust on my own at all.
And that sounds scary but the way I got over the fear is by realizing there are many things that I do very well but I do not know their internals very well. Driving is an example. I barely understand what the steering wheel, clutch or brake pedals do. I have driven over 130,000 Kms and I will perhaps drive more than double that in the next many years.
I have been building software since PHP/Drupal days. Got into AWS S3 as a beta user. Adopted Memcached (and MQ) in 2008 out of necessity. Then Python/Django for 10 years. Then Rust. And tons of JS/TS. I owe a lot to my curiosity. I believe we can keep learning what we need and still delegate most of programming to agents.
There are two types of programmers: the pragmatists who see programming as a chore and would gladly never write a line of code again given the right tools, and the gardeners who don't want their enjoyable and rewarding garden-tending work taken away from them.
The "pragmatists" who get excited developing a prototype for a week before they realize they will never be able to ship something anyone else will use because each trivial change becomes exponentially more difficult for the LLM to implement and completely impossible for the "pragmatist" to reason about, with every new commit liable to break something else.
Still waiting for this revolution of amazing 10x software! It's been 10 months since Everything Changed in November, surely the 10x pragmatists could have leveraged their effective 8 years of development time? Or maybe we'll move the goalposts again and say that actually, Everything Changed with Astra, we'll just need to wait another three months?
I've found sorta the opposite - in any area, it can just do everything for you, or it can be an incredible teacher. I've been re-learning a lot of higher-level math and it has been an knowledgeable, infinitely patient, always-available tutor. Of course, I could just have it do just about any math I want for me, but that's not the point.
Kinda the same with language/technology stuff - it can be a great tutor and it can scaffold other parts of a project for you. It can give you feedback and let you focus on the interesting parts.
I guess the motivation itself may be hard because of the fear of it taking over much of our jobs, but having this kind of help/feedback is pretty cool for the sake of learning things just because they are interesting!
> I've found sorta the opposite - in any area, it can just do everything for you, or it can be an incredible teacher.
Please don't. I've had all of Codex, Claude and Gemini convincingly tell me absolutely wrong stuff, pointing it out with easily verifiable example they come up with more and more weird reasons.
Things don't become correct simply because most sources are again - easily and logically verifiable - wrong. This already was a plague when people "just googled" stuff and effectively returned with the most SEO optimized answer. Now we have very convincingly written instances all over the place.
If these were singular instances I wouldn't be so worried, but if you are learning it already is very easy to learn something wrong. This is why back in the days when people still used physical books to learn new things it was a good idea to check first which books are actually recommended. There have been a lot of "experts" that wrote things they clearly misunderstood but worked for all the examples in their books.
To give a common example for both the backend and frontend devs, that isn't about a specific projects. LLMs and Google searches frequently turn out wrong results regarding CORS caching and how it works in relation to domains/hostnames. The circumstances under which Content-Disposition work are another example. I think a lot of wrong statements that LLMs are "convinced" about are due to wrong statements (sometimes in otherwise correct response) of popular Stack Overflow answers.
It's saddening how much wrong "common knowledge" exists in the industry. I have been bitten by a lot of these, but it feels when people don't even actually code and think anymore this will just rise forever.
Can these be wrong? Certainly. So can humans. Many of your examples are of humans being wrong. That doesn't make LLMs - or humans - useless. The fact that they are not infallible is not a reason to avoid using them and I'm not going to throw out a tool that has been incredibly valuable to me because someone on the internet got some bad CORS advice.
Learning is about so much more than accessing information, though. It is about building mental models, resolving ambiguity, exploring things that the source doesn't explain very well, and so much more.
Questions like "What do these lines of code do?" or "How does this fit into the big picture?" or "Wait, this doesn't make sense?" are rarely answered by the source.
This is a bit fresher in my mind in the math domain, but I don't think it is any different in any number of other domains, including coding. I've been working through a math textbook, gotten confused about how the author gets from step 2 to step 3, taken a picture of the text, and had AI explain it to me - it almost always gives me a much better understanding of what is going on and helps make things so much clearer. There is a level of interactivity that can't exist in a book or other "source" of information.
I think there is a bit of tension when it comes to learning and sometimes the struggle itself is informative, but there is a reason people hire tutors and go to classes taught by teachers vs just reading a textbook, and cutting yourself off from a tool because you've seen it be wrong about something seems like a silly mistake.
what this article tells me is that no one at Nvidia actually cares about this project whatsoever. otherwise, they would have had a person actually write the announcement.
First of all, this is a pre-1.0 release that requires a nightly Rust compiler (if you choose the SIMT track with cuda-oxide) so that one is going to be unstable software.
Secondly, When an issue occurs with a kernel or you want to write your own custom kernel in Rust, now we need to diagnose if the problem came from either cuda-oxide (SIMT), Rust's side, CUDA or Tile (If you decide to choose the Tile track).
Another dependency into the list and course everything is open source except CUDA itself. So any issue that happens on the CUDA level, you are forced to wait for them to fix it.
Why? The point of a shader language is to define a function that outputs some graphics. Why should that not be portable between GPUs/CPUs of different vendors?
More generally any GPU computation, not necessarily graphics. The above argument could go that there is no point in high level languages for CPUs either and everyone should just always use assembly, which is obviously false. Same can go for GPUs.
I mean if you're worried about high performance compute on cpus, then the argument also applies. That's why there so much hand written assembler with dynamic dispatch on CPU features...
It's just that gpus have no general purpose usage .. for ai, it's basically all about perf.
Of course some cross platform stuff can be made for modest acceleration, but the state of the art is always going to be out of reach.
Because of how different the memory hierarchies and pipelines are and because if you're doing perf work, this is important enough to matter. For cpus and general purpose programs, it's not. For hpc, it is
Go's type system is not at the same level as C++, Fortran, Python, Julia, Haskell, Java, to quote the languages with CUDA support from NVIDIA and their partners.
I strongly dislike CUDA. Once you have allowed that proprietary cr*p into your C++ codebase, it is very hard to get rid, and you end up with code that is either tied to a single vendor or an #ifdef hell, probably both.
The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, etc. These days we even have DSLs like Triton that make kernel writing much more ergonomic than anything you would hope to achieve in Rust.
Is this satire? D3D12 and Metal aren't any less proprietary than CUDA.
You can call their APIs without needing to compile your code with a proprietary compiler or adopt a bastardized version of C++.
Sounds like a C problem, not a CUDA problem.
Actually no.
You need Objective-C, Swift, and Metal is a C++14 dialect with extensions.
You may refer to the C++ bindings, which still not obviate the need for the C++14 dialect in the shaders, and it only works, because there is a shim to call the Objective-C runtime from C++.
Likewise there are DirectX COM interfaces that are really only usable from Visual C++ COM extensions, and the HLSL semantics depend very much on which compiler is being used, hence why there is finally a language reboot going on.
> The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually
Isn't that how CUDA code is normally written?
No. CUDA allows you to write all the code in a single file, and uses a preprocessor to split it back out and pass it through separate compilers, one for host and one for device.
This true, but you can write the two separately if you want.
The disadvantages of writing them together are listed in the various parent posts. But some code authors really like the convenience of having the two in the same file.
You could also use Mojo, one language for all targets.
I began to lose interest after the acquisition. Have you been following along, are they still going to open source it?
I thought they already did and released the compiler source code under Apache 2.0.
The Mojo compiler has been open source for over a month now.
And the Mojo standard library has been open source for over a year.
It’s all open source. Go check it out!
Nice, thank you. There is an old python project I've been thinking about converting to Mojo.
Or julia if you want a much more mature ecosystem.
I highly recommend Julia for (scientific) GPU programming but it would be nice if there was a larger community and/or funding behind the GPU side of things. It has very few core devs for what it is.
Julia has had a great CUDA story for a few years now, and this about 9 days old. Rust rejects buffer aliasing at compile time using Rust's borrow checker, but shared memory in cuda-oxide currently requires unsafe, but then there's HuggingFace's Grout and mistral.rs, so yeah, Rust is picking up ground here on Julia. How is OpenCL's performance these days?
I fell in love with MATLAB (or GNU Octave for free since you really pay for toolboxes/packages) back around 2004, despite it warts. So I second Julia, which is similar, but is a more modern functional language instead of imperative.
I asked Google's Gemini if Julia can run on GPU unmodified without annotations, pragmas, intrinsics or similar manually-managed friction, and it said yes, but that data types must be swapped out for GPU-backed types:
If your code is written using vector/matrix operations, broadcasting, or standard linear algebra functions, it can run on the GPU entirely unmodified. You only need to change the input data type to a GPU-backed array (e.g., swapping a CPU Array for a CuArray from CUDA.jl).
https://cuda.juliagpu.org/stable/
This is the direction we should be going. So while Nvidia's Rust port is an important first step, it's an evolutionary rather than revolutionary achievement. But that's all Nvidia can really do now, since it's locked into its own paradigm like Intel/Microsoft and has gotten too big to think outside the box.
Edit: PTX in its example stands for Parallel Thread Execution, the Virtual Machine (VM) Instruction Set Architecture (ISA) created by NVIDIA for its GPUs, which works similarly to Java byte code.
Edit 2: Broadcasting is a feature in Julia that allows you to apply a function or mathematical operation element-by-element across arrays of different shapes and sizes, without writing manual loops. In Julia, broadcasting is syntactically indicated by a dot (.) placed before an operator or function name (e.g., sin.(x) or .+). <- I was today years old when I learned the term for this
> Once you have allowed that proprietary cr*p into your C++ codebase
People have been doing that all the time for every kind of codebase. It's just part of the business. I don't see how it's worth having any emotions or opinions about it. Seems like you are wasting your energy.
Are win32 APIs proprietary? So you decide to use them, use a wrapper/UI framework, or don't develop for Windows. Easy choice.
Developing for embedded devices? So you read the manufacturers manual and implement based on the spec, use some sort of HAL if they are available, or you don't have a job. Even simpler.
CUDA is not an API, CUDA is a language, so you cannot make that comparison.
> The CUDA runtime is a special case of one of the libraries provided by the CUDA Toolkit. The CUDA runtime provides both an API and some language extensions to handle common tasks such as allocating memory, copying data between GPUs and other GPUs or CPUs, and launching kernels. The API components of the CUDA runtime are referred to as the CUDA runtime API.
From: https://docs.nvidia.com/cuda/cuda-programming-guide/01-intro...
CUDA is neither an API, nor a language, it is an ecosystem.
That's a nice way of saying that it's a dependency clusterfuck.
I've never understood why we can't just expose the GPU ISA directly the way the CPU does. It's all getting compiled down at the end of the day so someone has to write a compiler for it either way. We'd be substantially better off IMO if it was all built directly into LLVM and then let middleware sort out the details.
Because even CPUs rather use JIT runtimes to deal with the various kinds of ISAs that exist.
Naturally plenty of folks rather use software that doesn't take advantage of the hardware they paid for.
That would require vendors to either stick with a single backwards compatible ISA like intel did for x86 or document how their graphics cards work.
CPUs manage this by changing the internal micro-architecture, but historically GPUs only needed to support a graphics API and used that abstraction layer to freely change the hardware.
If i'm not mistaken, this already exists, and the assembly language here is called PTX
https://llvm.org/docs/NVPTXUsage.html
PTX is a bytecode format, the CUDA driver JIT compiles it when uploading into the cards.
Can't the same also be said of much of the x86 vocabulary at this point?
I appreciate that we can upload SPIR-V directly. The API still feels overly obtuse but it's not so bad.
SYCL gets close but is language specific.
> Are win32 APIs proprietary?
Yes. And crap. Not in my code bases.
If you're going to make apps in windows, you need to call their proprietary API somehow. Maybe you do it via a wrapper library, or via electron or something. But that's the same thing, just with more indirection.
Find a way to get ring 0 without touching any system APIs and you can just make your own APIs. My programs shall never say "please."
Your programs shall never grace my systems.
What makes you think he'll let you have a say in this? Btw, you wanna buy some ~~dea~~ usb sticks?
> If you're going to make apps in windows, you need to call their proprietary API somehow. Maybe you do it via a wrapper library, or via electron or something. But that's the same thing, just with more indirection.
Not even close to being true. You can invoke syscalls directly, just needs a bit of reverse engineering. I wrote a bare metal libc library, with (not a whole lot of) effort I'm fully able to interface with the kernel/open windows etc. Fully statically linked, no libc, no win32, compiled on Linux executed on Windows.
The problem is this isn't really well documented _at all_, and I even ended up attempting to get in touch with the Windows kernel dev team to give me the actual internal syscalls/endpoints, but they refuse to cooperate. Which is why writing anything for Windows is entirely pointless.
The problem is much deeper than that. Most OSes' syscall ABIs are not stable and could change without warning. What is stable is the dynamically-loaded libraries, shipped as part of the system. Linux is the notable exception here; the Linux kernel project doesn't ship a libc, and Linus is very famously opposed to "breaking userspace."
There's nothing that can stop you from using syscalls in theory, but if you want your app to be portable across different OS versions, past and future, you'd better not.
Incidentally, syscalls would also break Wine. The way Wine works is basically by shipping their own versions of Windows DLLs, which express their operations in terms of Linux APIs. Because Windows programs don't rely on syscalls, and call all system functions via the system-provided libraries, the Wine loader can just link Wine's version and let the program work normally.
https://blog.hiler.eu/win32-the-only-stable-abi/
If I recall correctly, the Golang team got bitten by this on MacOS.
They initially implemented the Golang runtime directly on top of MacOS syscalls (not the C runtime library), just like they did on Linux - and then those syscalls changed, breaking Golang.
They had to switch to the official stable API which on MacOS is the C runtime library, not syscalls.
> This isn’t even close to being true. Here’s a thing I did that made things way more complicated than is worth it for 99% of developers when there is a proprietary solution made so I do not need to worry about these things. Because it is so hard to work around it, it is entirely pointless to develop for one of the most used operating systems in the world.
Just being totally honest this is how I read this comment when I insert context that seems important to me. I respect having principles but at some point there needs to be more value in practicality over your codebase not being locked into a proprietary framework at all.
> Not even close to being true. You can invoke syscalls directly,
The windows syscall API is yet another proprietary windows API. Sure - you can call it without loading any DLLs. But you're still calling into a proprietary windows API.
If you really hate calling proprietary windows APIs that much, maybe stop developing for windows? Develop software for linux. Or make your own kernel, or whatever. But if you keep developing software for windows, stop fighting it. Unless you have a very good reason, your software should try to fit in on its host platform. It should behave well, and work like other windows software.
It's like travel. If you fly to France, try to fit in. Maybe learn a bit of French before you go. If you hate France, don't go.
That’s insane. Windows does not have a stable syscall ABI. The way you’re supposed to interact with the kernel is through the userspace library. Of course the kernel team refuses to cooperate.
Do you want to keep reverse engineering the syscall ABI for every Windows edition and update ever? Do you want to ask your users to disable Windows Update?
Regardless, I don’t even understand how that’s relevant, since you’re still introducing a dependency on a proprietary ABI.
> If you're going to make apps in windows
...your troubles are starting
The CPU on most machines is quite proprietary. I don’t understand this faux purity dogma.
Practical computing is not and never has been an abstract pure concept. It’s about making machines built by corporations to do usefull things at scale.
There is no ”non proprietary” computing unless you make your own stack.
Yes but there are business costs to using high-level proprietary tools and libraries. If you write your app using win32, you won’t be able to port is very easily. You’re also stuck with whatever bad or bizarre decisions Microsoft made.
It’s even worse for CUDA. GPUs are expensive, and now you’re vendor locked. You’re between a rock and a hard place. Either spend millions in engineering time, or millions on price-gauged hardware.
I wonder which APIs you would use to port easily, because POSIX and Khronos aren't it either, as they are industry standards driven by companies where one has to pay for a seat at Open Group and Khronos offices.
There is no ”easy” porting.
Once this is accepted the rest becomes easier as you are not wasting time trying to find a silver bullet.
I mean it’s then ”just normal work”.
Exactly.
> If you write your app using win32, you won’t be able to port is very easily.
Is this still true? eg, Shopify saying porting is now easy so no need for abstractions.
Porting has never been hard. Just follow the platform guidelines. Make sane architecture. Done.
I mean _it's just work_. You don't need to invent anything. Just do the work.
What _is_ hard is when people run after silver bullets to avoid all this work.
Because people who don't understand software decide it would be cheaper to implement something only once. Or someone who does not really understand what they are doing insists that same C++ code runs automatically on all platforms.
AI has given the software engineers permit from the beancounters to do the sane thing.
Good software development orgs _have always_ done proper per platform ports.
Also - there is nothing wrong in supporting only one platform as such!
> Good software development orgs _have always_ done proper per platform ports.
I really wonder why this was never fundamentally fixed. How performant a certain instruction on a specific platform is, how well it is supported and potential equivalents or sets of other instructions to emulate an equivalent are usually all very well understood.
So there should be some graph of operations which can transform any software from and to the specifics of each platform. Especially because firmware + compliers + platform abstracting libraries are basically already just that graph, although (usually?) to lossy to be applied in reverse. Add the recent developments in very large scale statistics to it and it'd probably be quite possible to transform from and to generic intent in the implementation to the uniqueness of each platform. E.g. the theming differences between a MacOS UI and a terminal application served over serial or the processing capabilities of a VLIW CPU compared to a FPGA or a GPU server.
Considering the enormous amount of work that went into compilers, better debugging and intermediate representations it seems like a huge missed opportunity nobody seriously asked the question whether information could be emitted that would allow for decompiling all the way back to the generic intent.
” If you write your app using win32, you won’t be able to port is very easily.”
This is wrong way around.
If you don’t support the platform your app runs on using the native api:s to the hilt your port is just bad.
If you actually want to support multiple platforms _you actually need to support_ them from the ground up.
This is speaking industrially and businesswise. A professional software business always has per-platform implementation resources. Or they have just one platform. Or they pretend they are multiplatform and then _everybody_ _daily_ fights with the problems this causes.
Obviously those elements that can be portable should be. It’s like Einsteins simplicity maxim - your codebase should be as portable as can be but not more.
” It’s even worse for CUDA…”
No these are just the business and market constraints. If this does not make sense for your offering then don’t use it. This feels like false FOMO - CUDA is not a silver bullet but it might be a specific solution to a specific problem.
It really depends on the application. The reason web is so successful as a platform is because it’s rich enough for most applications, and inherently cross-platform at an OS level.
And Re: CUDA: yes if it doesn’t make sense then dont use it. That’s sort of my whole argument. It might make some level of sense from a technical perspective, but that needs to be balanced with business risk. I’m saying a lot of people aren’t doing the balancing right, which is why these new tools have value.
Win32 is the most stable abi on the Linux desktop.
Dead wrong. Win32 (externally) only seems stable, but internally it changes between Windows releases. Win7 syscalls are completely different from Win11 syscalls, meaning if I want to release a binary _without relying_ on Win32 I need to provide full syscall mappings _for each and every Windows version_. This doesn't happen on Linux.
>> Win32 is the most stable abi on the Linux desktop.
> Dead wrong [...] if I want to release a binary _without relying_ on Win32
Then you are not using the Win32 ABI, are you?
> only seems stable, but internally it changes
That's literally the definition of it being stable. Programs written against an interface keep working despite the implementation changing. The Linux kernel also constantly changes internally but programs written against syscalls keep working, so it is stable; that fact doesn't stop being a fact just because I dislike perf_event_open(2) or whatever. This is all very basic and easy to understand.
Those are not a part of the API contract in case with NT kernel, though, unlike Linux.
Also, there are OS-provided shims in ntdll.dll (which, by the way, isn't a part of Win32 platform API, but a part of the NT kernel interface).
Correct, most of Linux user land targets API stability, not ABI stability. Windows targets ABI stability because applications are typically distributed as binary blobs. Most applications on Linux are open-source and built per each distro, so it’s a non-issue, just recompile.
This doesn’t work for proprietary software that’s distributed as blobs and rarely updated, like say, video games. But that’s a minority of stuff on Linux. But not on windows.
Realistically, on Linux applications target specific API versions of frameworks. Like Qt 6, or GTK 3, or whatever. Then everything is compiled or dynamically linked at a per-distro level. The ABI compat can bite specifically when distros enforce strict dynamic linking. But then containerization technologies come in.
And there is a difference between API and ABI stability. For example, adding SSO to std::string in C++ broke ABI, not API. If you recompile it’s fine, everything works. If you don’t then it doesn’t.
Sadly glibc made the choice for everyone that you have to recompile constantly to keep your app working.
That’s one of the biggest issues keeping Linux small on the desktop since nearly no commercial oriented company works that way.
But it looks like we will soon be able to „virtualize“ the dynamic loader so glibc has no say in this matter anymore.
I can program all my non-CUDA GPUs use completely open non-proprietary toolchains. And if that ceases to be the case on one platform I can switch platforms without having to rewrite all my code.
Many software engineers forget they're employee of a business.
If that business makes it money selling a cross-platform AI inference engine, as was the case for my previous startup, it is bad business to tie yourself to single platform. I managed to build a single code base that would support CPUs, OpenCL, Metal, CUDA, D3D12, and WebGPU from a single set of kernel sources. As a single developer, there was no way I would have been able to, at the time, maintain separate code paths and GPU kernels for each of those platform, in addition to training the models etc.
> I don't see how it's worth having any emotions or opinions about it.
Ironic, seeing as that is an opinion about it. Also weird telling people in an online discussion forum not to have opinions.
Oh, does that mean I get to say you're ironic because, literally, they didn't tell anyone to do anything. They said they didn't understand the worth of the opinion. You're interpretation is selectively literal in order to be rhetorical.
Does that mean someone else gets say I'm being ironic because I'm selectively literal in order to be rhetorical? Well, okay, I guess it's harder now.
You're interpretation is selectively literal in order to be rhetorical.
Where do you think you are ?
Most of us are in tech/IT/research the population in the spectrum here is orders of magnitude bigger than the avg on real life. SO yeah people will be literal in order to be rhetorical. Not even selectively, this is the one site where you NEED to use /s unironically.
That opinion is work-ethic related, not CUDA-related. The stance is reasonable too; why complain about characteristics of CUDA that can't be changed?
Your job as a CUDA engineer isn't to decide whether or not a proprietary API/compiler is the right call. Your boss made that choice for you when they hired you, and you accept the tradeoff if you want to keep working there. It's like someone protesting Dotnet because they wish they spent the rest of their life working with Perl instead. You can do that, but it's a completely different job with different pay grades and demands.
A pedantic dissection of someone saying "why get worked up about it".
> I don't see how it's worth having any emotions or opinions about it. Seems like you are wasting your energy.
Some people only care about the easiest path to their pay check. Some people actually care about software engineering. I tend to prefer the latter but hamstrung by the former.
> It's just part of the business. I don't see how it's worth having any emotions or opinions about it.
It's called foresight. The ability to see that vendor lock-in is against our long-term interests.
Your windows comparison is apt - now we have tons of software tied to windows, making it hard to leave that spyware-infested OS.
> People have been doing that all the time for every kind of codebase. It's just part of the business.
What is your ecosystem where this is true? Embedded or industrial, maybe?
I'm guessing you assume other people also use the same windows or embedded systems you're referring to. That's an insane thought: nobody would use this if they had any chance, and you intentionally chose this misery.
Obviously, you don't need to live this way. You can be free. Breathe.
yeah, just write a stub/wrapper around it and abstract. it's the classic coupling problem. nothing to do with CUDA
> I strongly dislike CUDA. Once you have allowed that proprietary cr*p
Genuine question...why not just type "crap"? It's not even that much of a curse, but I've never really understood the point of self-censorship. If you don't want to curse then you could just use a non-curse word.
It may be to bypass censorship, rather than self-censorship. Some platforms block or shadowban comments with curse words. Not sure about this platform.
HN definitely doesn't give a crap about that word.
can confirm, looks like crap is not on a list
pretty crappy list
I have written many words far worse than "crap" on this site. I haven't gotten in trouble over it yet.
I do find it a little amusing, because commenters stopped criticizing my cursing the moment I started getting a good chunk of karma here. I remember in 2016 someone criticized me for using the term "shitposting"...I don't think I've gotten that kind of criticism since 2016 though.
Back then the term was still associated with 4chan.
* is used to give emphasis and show that they are using the word as curse word rather just calling it bad
It doesn't read as emphasis to me. It reads like the person is trying hard not to curse, and they think "crap" is a curse word. It's a little bit adorable, like I'm reading a comment from an obedient child.
I guess you are not from the generation of texters. This how languages work we used to use * as a way to avoid getting censored it over time became a way to curse or give emphasis.
Sounds like a generational thing.
I grew up texting. But in the 90s any profanity filters could just be turned off in settings.
People are getting used to censor themselves in order not to be reported, banned, or «hurt » other sensibilities. The words « rape » couldn’t be written in instagram for example, what a great way to deal with such a serious issue. Mainly an American thing spreading away from young people if you ask me. Sorry America, just being honest here.
America is partly guilty, but TikTok censorship is a big part of the younger generation's tendency toward self censorship.
As much as I personally dislike TikTok, I don't think it is fair to it: cultural willingness for more sensor sheep on Internet started years before TikTok's popularity in the west.
It's not the sole cause, but I believe it's the main driver behind a bunch of specific substitutions that are mainstream now or nearly so. For example, dih, ahh, and unalive. They may not have been invented on tiktok, but that's where they incubated.
There's no such censorship on TikTok, it's entirely groupthink based on people saying "when I use that word my video is seen less so therefore it's being censored".
Youtube is a lot more guilty of it though, as well as demonetizing.
well_ackshually, tik tok has a well known, long, and rich history of suppressing certain search keywords.
How is that not a form of censorship? It is direct suppression of certain forms of speech.
YouTube has its problems but I don't think it's had quite as strong of an effect on language.
It sounds like there is a documented policy or proven that TikTok does it. Just people thinking it does leading them to self-censor. Then people see others doing it and copy it. So, I guess it is censorship but not by TikTok.
You had profanity filters for SMS?
No, but I think ICQ and some IRC clients had profanity filters turned on by default. I remember visiting a friend once and realising he hadn't turned the profanity filter off. I teased him about it for weeks.
After reading through the threat here it seems more like a cultural thing. The US has quite a lot of filters for profanity. I remember from my youth that in 2009 Eminem was a guest in a Germany TV show and very happy to swear as much as possible without being censored. https://www.youtube.com/shorts/2OC-yKZ5Yag
I’ve been texting since it was first a thing (sms on Nokia phones) and no one I knows does this. We just say shit, fuck, and crap.
Is that a cultural/national thing instead of an age thing?
I've never had texts censored by texting providers, they're not supposed to read texts in the first place (at least around here).
I'm definitely from the generation of texters and there was never any censorship going on with SMSs... yours must be a cultural or regional thing.
Maybe you didn't have T9 enabled but I consider it censorship when I type bitch and it gives me chubi.
Even now I wonder if I am allowed to type bitch here...
I guess we will find out.
But typing "b*tch" with T9 is just as difficult (if not more) as typing "bitch". Anyway, I guess I never had a need to swear much over SMS. On IRC, on the other hand...
stop feeding nonsense to the masses
Am I, with around 30, in this generation? Putting * in words seems like self censorship to me. Still, might have a cultural component. German here.
I'm in my 40s and * is self-censorship to me. It must be a cultural thing.
It can be used for in-jokey comedic effect. For example, referring to M*cr*sft W*nd*ws or Br*dc*m as though they were offensive terms. Or *r*cl*.
like for example, c*nt?
Maybe more like p**p, as in "that cunt p**ped in my yard"?
I'll admit, it never once occurred to me that people might be using censored characters to provide more emphasis that a word is a swear, but I guess it does indeed do that, at least to the writer. Whether that comes across to the reader, and whether the writer cares that their intention was understood... I'm not so sure.
Hah. Yeah, I agree. It's one of those things I admit is `lost in translation` for sure.
What about ^#%& as was traditional in newspaper comics strips?
How about "Pockmark!... Freshwater swabs!... Bully!.." or "Amoeba! Bashi-bazouks! Chowderheads! Certified Diplodocuses! Nyctalop! Ectoplasm!"?
"Your mother was a hamster and your father smelt of elderberries!"
I was so disappointed when I tried reading tintin in other languages and found the dear captain was straight up using slurs in those. I wonder whether the english language ones have been edited over the years to remove that sort of thing
æselmassør, sortbørsgrosserer, søpindsvinefjæs, karnevalssørøver!
Findes der en Haddock/Egon Olsen tiradegenerator derude?
Known as grawlix or obscenicon.
https://en.wikipedia.org/wiki/Grawlix
I interpreted that as indicating something was swearing without actually swearing. So, identifying something as swearing that isn't actually swearing.
In any context I've seen, asterisks are for wrapping formatting and said formatting it to add emphasis. So being in the habit of typing `emphasised phrase`, for italics - regardless of whether the platform parses markdown/similar formatting, e.g. SMS.
To have an unclosed asterisk replacing characters in a word? I've only ever seen that as a way to bypass censorship. This spans communications from people currently in their 40s down to 20.
i do this to put emphasis, i always type "h*ck".
(although it is a half-joke since it's definitely not a curse word imo)
I think a string of non-alphanumeric characters would work much better here, like "Once you have allowed that proprietary @#$&% into your C++ codebase”
Leaves more to the imagination.
But this isn't perceived as emphasis at all. If I wanted to emphasize something, I'd be more likely to use something like *bitch* or something along those lines. Replacing a letter with an asterisk comes across as self-censorship, which is pretty silly - just use a different word if you're that uncomfortable with swearing.
My guess is that jacobgorm will not reply. I would love a reply, because I want to understand how others think.
I believe we'll be left to wonder.
Platform may retroactively make up and enforce rules that makes your content violate terms (and remove them)
See YouTube.
I certainly dislike how everyone on YouTube is saying “SA” and “unalive” and “corn”.
It’s one thing if it’s some funny commentary channel avoiding those words, but what bothers me is the true crime YouTubers. In the subject of true crime, rape and murder are just things that are probably going to come up, and when they refuse to use the appropriate language, it comes off as infantilizing, which is weird considering that my actual YouTube account is over 18, let alone the viewer using it.
Advertisers ruin everything, I guess.
It's become so bad that even quality history youtube channels are frequently using euphemisms like "moustache-man" instead of just saying "Hitler", to avoid their videos being buried by The Algorithm, and therefore cut severely into their viewership.
I think it's a win-win. Intelligent people easily knows what they're talking about, and the others don't get offended. /s
glad i found that /s at the end
> even quality history youtube channels are frequently using euphemisms like "moustache-man" instead of just saying "Hitler"
That can be quite confusing. You had German mustache-man, Russian mustache-man, French mustache-man (Petain), French small-mustache-man (de Gaulle), Spanish small-moustache-man (Franco)
If I know that your terminology includes "French small-mustache-man", I'm going to be really confused over "German mustache-man".
I don't think those filters are even real, I think it's just mass-hysteria. I call these kinds of behaviours "traditions", but I'm not sure if there's a better term for it.
Basically someone comes up with something which is nonsensical, but plausible. Like believing that their videos are unpopular because they said the word "rape" and the algorithm magically got them, rather than because their videos suck. Then someone else sees that and starts thinking it is true. It silently spreads across the population.
I've seen this in organisations, where new recruits haven't been properly trained. Someone has come up with a method which is wildly incorrect and illegal, but plausible. The other new people around them have copied them. They've become slightly more experienced people, they've taught the next round of new people.
Before you know it, half of the organisation is doing something hilariously wrong, and they all sincerely believe it is the right way of doing it, because everyone does it. It's just self-reinforcing at that point.
I am sure they are bullshit. Like when they mute cursing and "risky" speech, but when you enable autogenerated subtitles they show up there. Youtube knows what thay said regardless if it's censored or not. It's so fucking stupid
> I call these kinds of behaviours "traditions", but I'm not sure if there's a better term for it.
In psychology that kind of thing is referred to as "superstition".
More specifically, "superstition" in this sense refers to the phenomenon of copying someone else's successful approach to a problem you have. (In your example, getting views on youtube.) Since you don't know what parts of their approach matter, you copy the effective parts and the ineffective parts equally.
I always associated the term “cargo culting” with that but I think that term has largely fallen out of fashion (probably for the best).
Actually, I was a little too specific here - superstition also refers to copying your own successful approach.
No it isn't mass hysteria. YouTube has a set advertiser friendly guideline. It will scan uploads and streams automatically.
YouTube used to demonetise profanity unless it was mild. YouTube would demonetise profanity in the first X number of seconds of the video. These rules change and have been relaxed of April last year, but generally these rules still exist.
There isn't a hard filter if you say "suicide" you automatically get it. However it increases the likely hood of demonetisation. So people avoid it to be safe. So you end up with people using stupid euphemisms all the time.
But do we have any evidence that “suicide” counts as a negative signal and “unalive” doesn’t?
The Suicide stuff is about protecting them from people who were promoting self harm. The unalive stuff is because of threats of violence. You can reference murder if it is say part of a news story or referencing something historical.
https://support.google.com/youtube/answer/6162278?hl=en#Harm...
Creators are being careful don't want their video demonetised because once it gets the yellow flag, even if it is removed later they've lost the majority of monetisation.
The problem with providing direct proof of this is that YouTube moves the goalposts quite often and their auto moderation system is very inconsistent.
A (baseless) hypothesis: perhaps there are plenty of YouTube creators who use the proper, mature terminology but you never see their videos because the algorithm really is penalizing them for it...
His kids were probably watching him type over his shoulder and he didn’t want to hear, “Daddy, what does crap mean?”
I am arguing that they would ask that anyway.
I guess I never understood censorship when it’s plainly obvious what you’re censoring. Anyone who can read will clearly know that it said “crap”, so I don’t see how it’s fundamentally different than just saying the word. You still put the word into my brain.
"Daddy, what does cr*p mean?" Kids aren't stupid and this self-censorship isn't protecting anyone from anything.
(if a platform is serious about Bad Words for whatever reason (moral?) they would also forbid character replacements; ultimately it's the intent, not the word itself, that they try to steer with rules like that)
I'd consider that a joke - but also zero issue using any words talking in front of kids. You might wish to explain them anyways.
cause you might go to the eternal flames if you say a no-no word online
Your comment only makes sense in context if you believe in a deity who is too dumb to understand the difference between cr*p and crap. I for one do not worship a Bayesian spam filter.
What if a toddler is browser HN and sees the curse word?
Oh, my, indeed! That's gonna traumatize the poor dude for life.
IMO cr*p and crap are both valid but separate swear words. People have a wide option to choose from when they want to swear, and people like variety (much much more than LLMs do). People also tend to influence each other with their usages: cr*p is popular because it is popular.
Otherwise cr*p is just as good as crap, shit, horseshit, poopoo or such.
edit: * replaced with \* as HN interprets asterisks as formatting for emphasis. Thx latexr for informing me
To use a literal asterisk on HN, do ** or \*. Your single usage in two places instead turned the majority of the post italic.
Normative behavior has shifted due to pervasive censorship and surveillance.
I thought that cp*p is some kind of ugly cuda pointer declaration :D And being non-standard C++ syntax it wouldn’t compile.
Least ugly cpp syntax.
As a person who prefers Rust more than cpp, gotta say it's also "Least ugly Rust syntax"
Because I know it is not technically crap, a lot of competent people worked on it, most with good intentions. I suppose it is better described as a cleverly designed Trojan horse than can infect your software and make that software become crap, in the sense that it becomes harder to maintain, increases code duplication, messes with your build system, ties your build system to platforms that have their toolchain binaries available, etc., etc., without bringing any long-term benefits over learning things the hard way.
As someone only recently getting into HPC, what do you mean when you say learning things the hard way? What would you suggest?
I recently started learning CUDA and parallel programming paradigms.
For learning that may be a fine approach, but CUDA (in C++) really tries to hide what is going on behind the scenes, which is roughly:
1) code gets split between a host part that goes through your normal compiler, and a device part that goes through the GPU compiler. You may as well write the kernels separate and compile them via a separate compilation step, and keep your trusted host compiler for the host-side code.
2) data needs to move between the host and devices via explicit buffer transfers and synchronization steps, CUDA tries to hide this with annotated pointers, but it is really easier to think about those as just buffers that you allocate and transfer IMO, instead of trying to transparently share pointers between host and device like CUDA does.
3) kernel launches can we wrapped in a function similar to:
void RunKernel(const char *kernel_name, size_t width, size_t height, size_t depth);
Instead of the funky <<< >>> syntax that CUDA for C/C++ imposes. The problem is that once you start putting that in your code, it stops being C++ and stops being portable to non-CUDA GPUs. The launching and grid settings can be a bit hard to grasp at first, but sugarcoating that in bastardized C++ syntax does not absolve from having to understand it eventually.
So a good place to start might be an OpenCL or Metal primer, depending on the hardware you have available. D3D12 (and probably Vulcan too) makes this much harder than it should be, with too much boilerplate but is overall a mature and well-designed API should you wish to develop for Windows. Starting with WebGPU might also be good these days. It has a very different shader language than the others, but the rest of the concepts are similar, and it has a strong emphasis on making things async, which is what you want for performance anyways.
Claude/Codex should be able to get you moving very quickly.
Thank you very much for the effort you put into your advice!! I think I will start with WebGPU (wgpu), even though I have an Apple Silicon Macbook. I would really prefer to work with Rust instead of C++ because I am not good with C++. (I believe) I am good with C, so my C++ code looks like C code, and I am kinda learning the differences as I learn CUDA, which is a terrible way to learn C++, I guess.
The long term benefit is that there are more developers with CUDA experience available to hire than there are with any of the "hard ways" you mention.
Not saying you're wrong, but my career got a lot less frustrating when I started focusing more on the product and less on the ergonomics of the implementation. If you need to build a house and the customer isn't willing to pay for brick, you use vinyl siding.
Because nanny states are tracking your keyboard nowadays.
Not all crap is created equal, some needs censoring.
> I've never really understood the point of self-censorship.
Some platforms disallow certain words. In order to bypass that, some people use the asterisks. That's just as one possible answer to your question; there can be many different reasons for self-censorship, but to me the most logical one is when one tries to work around crappy restrictions, such as on terrible reddit (they killed old.reddit recently; I retired before that due to moderators being insane, but I also said that if old.reddit is gone, I am gone anyway - the requirement to now log in, totally defeats old.reddit com's usecase. Then again reddit went downhill many years before that already, so not a real loss.)
he could be a farmer and didn't want to type crop.
alternately, perhaps he meant to match all of cp, crp, crrp, crrrp, and so on. the dude might really like regexes.
/s
Launching kernels manually is an error prone PITA which I believe is the principle reason for CUDA's popularity. Having the compiler give an error when you mess up is a huge benefit. But having the compiler allow you to express "I want to launch this kernel over a grid with these dimensions, with these arguments" as a single expression is where the vast majority of the value comes from.
The having it all in a single file is mostly an artefact of the fact that it is C++, because C++ is single file at a time compilation. In D (which is multiple files in a single compiler invocation) with DCompute (which targets CUDA and OpenCL with upcoming support for Vulkan and Metal), you are required to write the kernels in a separate module, but you get all the benefits of the compiler complaining when you mess up _and_ the expressivity of "launch me this kernel".
> Having the compiler give an error when you mess up is a huge benefit.
Shouldn't this be alleviated by the current code generation machines?
Well yeah, but then you are using code generation, not writing code directly.
I meant LLMs :-)
You are trying to say that llm can replace compiler?
In general, no? But they should help with this part:
> Launching kernels manually is an error prone PITA which I believe is the principle reason for CUDA's popularity.
Why is this even a question, of course they can.
Write python code, ask any llm to translate it to C, then compile the C code - if it produces errors or fails to run, ask LLM to fix it. Then take it a step further and ask it produce machine code, and repeat the procedure.
Then RL the llm on the above, and you basically have a Python -> Machine code compiler. If you cover every single possible python syntax, every single possible C syntax, every possible standard library call, and all the compiler optimization examples (all of which is a final set), you should get something that is extremely accurate.
Having also played with Metal and WebGPU (at least years ago), I would say that CUDA is, amazingly, the best GPGPU API we have. Do I wish we had an open source parallel programming language as good or better than it? Yes. But asymmetrically hating on CUDA like this is how we continue to lag behind it in UX.
> The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually
Not to mention that this is a completely sane way to use CUDA as well.
People that attack proprietary APIs always miss the point why most devs outside FOSS circles prefer them.
Turns out when one isn't ideologically against something they aren't willing to put up with a lesser experience just for the cause.
I know it's not the same thing because proprietary vs open software it's way less important but, generally if you are not ideologically against something you can easily follow the stream and do lot of nefarious actions, especially if the action has enough degrees of separations from the actual nefast outcome.
I don't mind CUDA, I do mind that all of the SDKs don't dynamically load the various CUDA shared libraries at runtime.. intertwining itself into your application linking process makes for extreme binary portability inconvenience.
There a flavor of CUDA runtime libraries that binds at runtime, so you can have a single binary that runs with CUDA and without it. I did this at work.
Obviously you need to check if CUDA is available before trying to execute kernels, or it will error out.
Sure, I've written them. None of the NVIDIA-provided SDKs are like this, including the new Rust one.
? I find it hard to see the issue here. Just put it in a separate file and call it?
Anyone here looking at Modular's offerings?
i'm surprised modular's doesn't get much traction. The promise seems super interesting, and chris latner has the record to back up his claims. If someone has an explanation..
from what i can tell, you're going to be stuck with that no matter what you do
i'm currently using vulkan, and HLSL via dxc. which should be portable but it's not.
apple refuses to support vulkan, and relies on moltenvk and there's a bunch of OS/hardware/driver differences no matter what you do, that you'll probably have to feature test for, and compile a few different versions of your code no matter what you do
i think if you're doing something that you don't have to distribute to customers, just picking one stack and getting locked in has some appeal.
it leaves you vulnerable to lockin. but, especially in the age of ai, "claude, port this to vulkan" seems like a good enough defense against that
What is a proprietary crop?
It's actually creep.
> The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually,
Yes, I also prefer doing it that way, but in Cuda with the driver API. Allows you to handle kernels like shaders, including editing and hot-reloading at runtime.
The reason I'm sticking with CUDA is because it's by far the most convenient API to use, without nonsense like 50-liners to alloc memory or the need to manage descriptors, bindings, queue families, etc.
> The reason I'm sticking with CUDA is because it's by far the most convenient API to use, without nonsense like 50-liners to alloc memory or the need to manage descriptors, bindings, queue families, etc.
I was there when the OpenCL committee was deciding on that sort of stuff.
As I recall, and it's been two decades and a lot of sleepless nights since then, there was real pushback at the time against OpenGL-style default bindings. So folks didn't want to establish an implicit command queue or any other default objects attached to other objects. Part of it is because OpenGL was perceived as clumsy and passé, some of it was because it is not friendly to multi-threaded applications.
Those first meetings were a shitshow full of tension, implicit threats from Apple, and backroom deals. Kudos to Neil Trevett for chairing the group; I I bet it wasn't fun for him either.
That's unfortunate. Cuda has shown that, when done right, defaults and a convenience layer can make for a well received API without sacrificing performance.
Yes, I wanted defaults as well, particularly a default context and command queue.
Design by committee is a real phenomenon. And people in a committee know that, but they are also helpless.
Yep.
Ive essentially followed that paradigm with Python and C. I start out writing Python code. If I need something to run fast, I build a standalone C application that either reads from a file or listens on a socket, and just invoke it from Python. No need to write the entire thing in Rust and deal with all its semantics when it will be at best like 2% faster.
You don't need to use the CUDA (SIMT) programming model if you don't like it. The project includes cutile, which lets you program the GPU using tensors. It feels a lot like programming the GPU using numpy and triton.
As it happens, I just got my employer's permission to release as open source a Triton back-end for Metal and D3D12 GPUs here: https://github.com/dropbox/neso .
As an example of you how can use it to deploy real models there is this project doing ASR and TTS: https://github.com/dropbox/nspeech .
Finally, I am also going to be switching the inferencing part of Witchcraft from current Candle on MacOS and OpenVINO on Windows to just Candle with Neso; https://github.com/dropbox/witchcraft
That sounds like it'll be easier to maintain. Will it also be faster?
It is currently faster than the stock Candle / MPS shaders it replaces on MacOS/ARM64, and IIRC a bit slower than OpenVINO/CPU on my old Windows laptop, where I never got OpenVINO/GPU to compute correctly. Candle didn't have support for GPUs on MacOS/Intel, and OpenVINO ceased to be supported there.
Compared to OpenVINO (I tried ONNX runtime too, but never got it produce correct outputs with my quantized models) it is very nice to be able to build the exact kernels I need, at the quantization settings and precision that works for the models I have and with the custom operators required (speech models do a lot of non-standard stuff), run from a single set of sources, and not have to ship a hefty third-party DLL, and having to deal with their memory leaks and other stability issues.
Okay, so, GPUs are taking one more step towards being general purpose massively parallel machines. That's cool.
What would be even cooler though would be for GPU vendors to start giving us the user manual. An I mean the real user manual, that explains how to use their piece of metal when all you have is that piece of metal. That means a precise description of the wire protocols, the data format of the buffers we send to & get from the GPU, the ISA of the cores we have access to, the relevant performance characteristics…
In other words, enough information to write a state-of-the-art driver for any OS. That would be cool.
They don't need to do that to sell their hardware so why would they do that? On the other hand, they have strong incentives to not give you that level of access and information. The only way this will change is by having some disruption by way of a competitor who sells hardware with that feature as being a major reason why it takes off.
Or we could regulate. One hammer I'm tempted to use is to simply forbid sales and importation of hardware made by companies that also distribute software. That way the only way to sell you hardware is to make sure its interfaces are both simple and thoroughly documented.
We could possibly make an exception for open source software, or only forbid the distribution of the relevant drivers, or limit the applicability of that law to specific hardware: hard drives, mice, printers, web cams... or GPUs.
Anyway, that should be disruption enough.
GPGPU https://en.wikipedia.org/wiki/General-purpose_computing_on_g...
They don't release it because exposing a stable instruction set would kill their ability to quickly iterate, to release silicon with bugs that can be papered over with software fixes, as fixing bugs in chips is very expensive in terms of time to market, and undoubtedly to charge more for what looks like a hardware feature but actually is a software feature.
It's been this way for 25 years and I don't see it changing.
A stable instruction set would be nice.
But hi, if I spent $10,000 on a piece of hardware, let me program the metal, thanks.
I've been programming GPUs since the PlayStation1. The way they work under the hood has changed fundamentally maybe 4 times in that span.
I can't compare it to changes I've seen in CPU architecture since then. Maybe like: Compare the NES with its 6502 and per-cartridge mappers vs. a IBM 386 PC. Now repeat that shift 2 or 3 more times.
> The way they work under the hood has changed fundamentally maybe 4 times in that span.
How many times since we got shaders? And when exactly? I bet each change takes longer to come than the last. I mean, GPUs are increasingly general purpose nowadays. Sure they're optimised to specific kinds of embarrassingly parallel problems, but as more and more of their capabilities move out of the fixed pipeline to shaders, the need to change lessens.
How is this different than CPUs? I suppose in the last 5 years the architecture and tape has changed a lot as they move to make more LLM capable?
> exposing a stable instruction set would kill their ability to quickly iterate
I believe the need to quickly iterate dropped significantly since we got shaders. I would bet in fact there was few such iterations in the last 10 years. Crazy increases in computing power of course, but big breaking changes in the actual ISA? I'd be surprised.
> to release silicon with bugs that can be papered over with software fixes
Yeah that's actually one of my goals. Only ship stuff that works on pain of embarrassment and prohibitive recall costs. It's crazy hard. It's how CPUs are shipped.
> and undoubtedly to charge more for what looks like a hardware feature but actually is a software feature.
Again, that's good. Such pricing tactics are scummy, I want to end them.
---
Now of course, those reasons you cited are reasons for the vendor not to do what I'm pretty sure is very good for the consumer. I propose we force them. We could start small. Mice and keyboards first. Then printers. Then webcams, wifi modules... until we get to GPUs themselves.
> GPUs are taking one more step towards being general purpose massively parallel machines
this has nothing to do with becoming more general purpose (GPUs will never be general purpose - it's literally physically impossible).
> GPUs will never be general purpose
Well in that sense, neither will CPUs. Just like GPUs, some workloads are better left to other kinds of hardware.
i really don't think you know what you're talking about. this isn't about better or worse. there are many many many workloads a GPU cannot at all implement (hint: anything with branches).
See also https://m.youtube.com/watch?v=kZRE7HIO3vk
And
https://www.sebastianaaltonen.com/blog/no-graphics-api
Since NVIDIA owns huggingface now and huggingface has the excellent Candle [1] crate for inference on Rust, this seems like a good step towards nice native Rust kernels.
[1] https://github.com/huggingface/candle
Nobody cares if kernels are written in Rust. Kernels were meant to be written in C, but if you want to go more high-level try Triton or a similar DSL that nicely abstract tile sizes etc.
oh no no no, this is going to break the CPP hold on AI and game dev.
Nah, Rust compiler still needs C++ to be built in first place, and everyone on AI uses LLVM as infrastructure.
kernels aren't meant to be written by any defined language. C is just a traditionally good default language that took over from assembly. No particular reason we have to stick with C.
And quite a few reasons that something better than C should be used. Rust seems a good candidate.
What reasons would you have to prefer Rust over C for compute kernels? I am a great fan of Rust, but I don't see any benefit for kernels, due to their relatively simple nature.
I'm not sure why being relatively simple would mean C over Rust? Rust still has the safety advantages.
That is exactly why OpenCL failed adoption, focusing on C, instead of being polyglot like CUDA.
SYCL is the natively polyglot counterpart, with practical implementations of it compiling down to the same sort of SPIR-V kernels as OpenCL. (OTOH, much of the current adoption on the open standards side seems to target the more widely supported SPIR-V compute shaders, via Vulkan compute.)
Not really, first of all it is for C++, not the range of languages supported by CUDA.
Before SPIR was a thing in OpenCL, Khronos could not understand why anyone would care about anything else other than C99, or why supporting Fortran on GPUs was at all relevant.
Secondly, from the competition only Intel cares about SYCL with their own sugar on top, OpenAPI.
AMD hasn't cared one second about it.
You may mention Codeplay, which is anyway an Intel owned company since 2022.
As for Vulkan, it doesn't have neither the features, nor the tooling that CUDA enjoys, it is the usual putting up with using LEGOs from different brands, with various pin sizes, that is so common with Khronos.
I have never seen a comment this gray
I haven't felt this popular since then 1990s when I was opposing Visual J++ and IIS.
in all of hackernews' shitty UX decisions, gray unreadable comments is one of the worst ones
noob here - what's the benefit of this? Will using Rust lead to more optimal LLMs or code or both?
I view it more of supporting an expanding use case. If rust gets popular then you'll want to support it.
I will give you an outsider's perspective on an analogy in this case. It is easy to see Candle as a ML crate to use for neural networks in rust. I have used it, and it works well.
The analogy is Tensorflow 5-10 years ago. It is popular, and there are lots of material on it. You quickly learn from talking to people that due to whims, a collection of reasons, people's love of consensus that no one is recommending it; new people are not learning it. In this case, the Torch analogy is the Burn lib.
And to tie this back into GPU programming, Burn's backends use CubeCL, which lets you write compute kernels in a Rust DSL using #[cube], with a JIT compiler and autotuning machinery. It targets CUDA, AMD, Metal, Vulkan and WebGPU.
(disclosure: I am a contributor)
It's very cool. If Rust had comptime, apart from macros, it would be unmatched in capabilities.
Integration with candle is already available! https://github.com/huggingface/candle/pull/3934/
Really exciting but it reads like Claude instead of what Nvidia posts have generally been like in the past. I don't need nor want my tech blogs to sound like a young adult novel.
I’ve had this happen to me several time over the past weeks and it’s gone from quaint to humorous to farcical to outright “is-the-world-gaslighting-me” insane.
Just today I was reading Stanley Druckenmiller’s op ed in WSJ. This dude is like 80 and has made billions of dollars, and he got Claude to write his op ed???
Unbelievable. And the tells are so obvious, yet people still love the Claude-like quips and odd grammatical choices that read like halfway asshole halfway mid-sentence confusion.
That op ed was absurd. I respect Druckenmiller a lot and am always impressed with his lucidity in interviews. The Claude “ick” was all over his writing.
Yeah definitely Claude. Lazy authors, if you're going to get AI to write for you please use Astra instead - it makes way less annoying prose than Claude.
NVIDIA is part of that shift NVIDIA CUDA Rust closes that gap
The momentum behind rust seems absolutely unstoppable at the moment, in the light of this, the adoption of Rust into the Linux kernel, and the adoption of for formally verified software by Amazon and Microsoft.
Deservedly so.
not too long ago, commenters on HN hated Rust and would never use anything written in Rust. So, now that Rust is in Linux, they shouldn't be using Linux either.
I think it would be much nicer, although unrealistic at the moment given the number of combinations of GPU vendors and variety of hardware, to directly target the underneath GPU ISA machine code.
Since I can write a simple compiler to target x64 machine code, it should be possible to write one to target my GPU.
Though, I'm certain that vendor lock is probably more profitable for them.
I don’t know for sure but I’m pretty sure the ISA changes quite frequently for Nvidia.
My thoughts on this, as a person who has recently coming around to working in this space is that up to now the convenience and "simplicity" of working in CUDA as it is has been a giant moat for NVIDIA. Having a whole toolchain with a C++ dialect and a giant extant pile of code out there that looked familiar to people meant they've "won" the AI wars.
And in that context NVIDIA had every motivation to keep their SDK somewhat abstracted higher up the chain and fully under their control and then be free to innovate in the lower bits. And this served them well as well as their customers.
My sense is that now with agent driven development this is basically evaporating. Agents are capable of at least prototyping/writing kernels for any hardware and ISA. e.g. OpenAI built their own custom hardware and ISA for it and then set agents loose on it writing kernels and claims great success. At least they're claiming this. And from my own experiences as a n00b entering this space, I can believe it.
TLDR I don't think vendor lock on the software side is going to work out for them as a strategy.
But luckily for them they continue to have really good hardware and good access to semiconductor fabrication. But just look at HotChips 2026 a couple weeks ago and look at the huge variety of new inference hardware coming down the pipe which looks completely unlike NVIDIA/CUDA.
How does this compare to vectorware? (https://www.vectorware.com/blog/)
VectorWare founder here. We are working with them and stoked they are investing more in Rust. I just gave a talk at RustConf about our different takes (https://rustconf2026.sched.com/event/2KNQj/making-gpus-feel-...). The video isn't up yet but you should check it out when it is. The efforts are complementary.
Towards the end of the post, we (NVIDIA) mention that this work was done in collaboration with Vectorware and others in the Rust community. And we can't wait to build further with the community.
Anyone know when Rust's std::autodiff will become stable? Assuming this Rust support expands to other GPU vendors, autograd will probably be the only reason to use Slang instead of Rust anymore.
Not in 2026.
I was told in the 2025 LLVM dev meeting that it will always stay in nightly because it's not practical for them to provide long-term stability guarantees that is expected of stable Rust.
That's a shame
To everyone skeptical of people saying that it's better to separate CPU and GPU code into separate files, riddle me this:
With this system, how do we specify whether dependencies are needed to be compiled for the CPU, GPU, or both?
----
Technically I don't know really care whether it's multiple files or one, I just want to make sure we are not reinventing a shittier version of CFG. What they are providing looks to me like:
1. explicitly annotate some things as `cfg(GPU)` or ungated (both)
2. unlabeled means annotated `cfg(not(GPU))` by definitely
Put this way, this has nothing to do with GPUs, and just has to do with creating some crate-local CFG shorthanded. Great! Let's do that first, get a solid foundation, and then come back to whatever is remaining for CUDA Rust.
Interesting direction from Nvidia. Anything that makes writing reliable GPU code less painful is definitely a good thing.
Not a cuda programmer, but since they’re making a new API, why would they already make it inconsistent at start? :-/ I’m referring to the examples a,b,c vs z,x,y (different ordering of output elements)
I read this the other day - definitely think it is the right direction Nvidia is taking.
Thank you NVIDIA - for once (not twice though - you've given us nothing but despair for Linux+GPU).
Does this mean I can write shaders in Rust for use with WGPU or Vulkan?
WGPU/Vulkan don't work with PTX shaders by default, additional translation would be needed.
On a side note, Vulkan has extension to launch CUDA kernels: https://docs.vulkan.org/refpages/latest/refpages/source/VK_N...
The way I understood it, rust would become an option next to Vulkan, WGPU (and opengl etc?).
But only for compute tasks. So, practically an alternative language to write compute shaders in.
You have been able to for a while with rust-gpu https://github.com/Rust-GPU/rust-gpu.
For compute shaders, you can already do this with CubeCL: https://github.com/tracel-ai/cubecl
You write kernels in a Rust DSL using #[cube], it supports WebGPU through WGSL and Vulkan through SPIR-V, along with CUDA, AMD via ROCm, and Metal. (disclosure: I am a contributor)
Does this weld Rust to CUDA? Can we use the Rust code to run on other archs?
I've been enjoying writing Julia code and then having it run on the GPU via the packages at https://juliagpu.org
I'm looking forward to trying these when they stabilize! I currently use WGPU for graphics, and cudarc for CUDA.
Note: Cuda-oxide is similar to Cudarc's host component, but uses a rust-style kernel dialect. Advantage: Share structs between host and device. Disadvantage: Trading standard Cuda kernels for a new, WIP dialect.
I haven't tried the tile API yet; looking forward to it.
The last time I checked, Cuda Oxide was Linux only, and required Async; these are why I haven't tried it yet.
cudarc been great for me, because it's easy to look up existing examples and references, and it maps 1-to-1 with what I see. I'm already having a tough time with CUDA itself, a dialect of it makes a tad harder to rely on previous work.
Seems more ergonomic in general though, both approaches they share, compared to cudarc, and less build infrastructure and fiddling with environments, which is great.
It's still linux-only but doesn't require async. You should be able to execute and compose kernels synchronously.
My understanding is not so deep regarding GPU programming or Rust... Does this mean anything regarding Nvidia GPUs and WebAssembly / WebGPU?
No. Rust is a non-web programming language.
> The launch is checked rather than trusted.
Damn even Nvidia is putting out fully Claude-written articles.
not to worry, they have an 'AI generated summary' box too
Its a recursive summarization pyramid
This thread flags an honest assessment of AI signal detection.
It gets fun when someone uses an uncensored model to bypass a refusal, but they accidentally pick one that was trained for erotic writing and brings its particular talent to the documentation task.
Thanks, saved me a few minutes
That's actually a magnificent observation. This is not only an indication of a keen eye, but a trained brilliant mind as well.
You’re absolutely right!
One might even say it is load-bearing on the seam!
Why this is important: it's the honest take.
I hate this
Genuinely, why?
this is literally a reddit comment chain
We do this wicked sin called having fun once in a blue moon here.
Slashdot's spirit shall live somewhere, no? Rent is all-time high and it can only afford here, for now.
> even Nvidia
Why "even Nvidia"?
They are fully behind using AI for basically everything.
What's next? "Damn, even McDonald's is putting out unhealthy food"
Sure, but even Anthropic doesn't appear to use Claude for blog posts. (I don't think anyone should. The prose stinks.)
Anthropic _absolutely_ does
They are just better at hiding it or configuring Claude.
I have several skills that reformat text to remove AI-speak tells.
Prove it.
https://news.ycombinator.com/item?id=49249269
I put the "humanized" output through Pangram and it still comes out as 100% AI generated.
That's about as useful as saying you asked the magical sky fairy.
I think you can't trust Pangram in a high stakes situation, but it is absolutely better than random noise at detecting AI-generated text. Which isn't surprising. If the distribution of probabilities can yield blatant Claudisms, it's not surprising it would also have more subtle deviations.
(Addendum: As I recall, LLM-generated outputs roughly follow Zipf's law, but the distribution still tends to have some subtle distinctions vs human text; pretty interesting, but I don't know where I heard this, so nothing to cite. Sorry.)
Pangram has an extremely low false positive rate. Even on adversarial examples.
One trade-off is even some obviously LLM text won't get detected by them, but they work really hard to ensure false positives are rare since a false accusation is much worse for society than someone getting away with LLM meatpuppetry.
To be honest with you, I don't think I would be able to identify with high certainty that the bottom text is AI generated, so it definitely goes a long way to obscure the AI-generated nature of it, but I also think it still feels unnatural somehow. I realize my framing naturally calls into question whether I'm being honest, but I am being honest. Given my experience with similar "skills" (it's just chunks of prompt, nothing magical after all) I expected even less.
But still, this is all very strange because it wasn't that many generations of AI models ago that AI writing was a lot better - I'm talking GPT 4.1, Claude 4.5, that sort of era.
Anthropic newsroom posts on the other hand are carefully constructed and well-written in a way that I have not seen demonstrated by LLMs yet, past or present. I expect that they have well-paid staff who are careful with every detail of their public communications. When you put it that way, it almost feels unfathomable that they wouldn't, doesn't it?
GPT 4.5 was really good.
I don't feel like either one of you really has a strong claim. "Doesn't appear to" is subjective, and of course it's impossible to prove one way or another.
You're simplifying the exchange a little too much. I said:
> Anthropic doesn't appear to use Claude for blog posts
My claim is literally the lack of evidence, which, yes, can't prove anything. This claim can be contested easily by showing evidence that they in fact, do appear to be using Claude to write prose in blog posts.
They said:
> Anthropic _absolutely_ does
Sounds pretty certain Anthropic is in fact, using Claude to write blog posts. Enough to emphasize "absolutely". That doesn't read like "I'm going off of vibes", that reads like "I can prove it". So, fine. Prove it. I don't believe it, and I want to hear the proof.
I'm skeptical, but it wouldn't be my first time being wrong. But flatly, if you make claims with this kind of certainty, yes I want to hear your proof.
My point in saying "Even Anthropic doesn't appear to be using Claude for blog posts" was not meant to be some striking revelation, I literally was considering it a prior to make another point. This on the other hand sure does sound like a striking revelation to me, that a lot of people across the Internet would be curious to hear. Like I'm sure these people would be interested:
https://www.reddit.com/r/ClaudeAI/comments/1wdfd92/are_anthr...
I will admit that I am unnecessarily aggressive sometimes, but I wouldn't have changed my response much in any case. If you're going to make a strong claim like this, I want your evidence, not your vibes. Otherwise, the claim should be a lot weaker.
I also realize that this sort of brashness upsets HN a bit, but it is what it is. I pandered comments for votes in my 20s a bit, time to grow up, sometimes people won't like you. Sometimes I feel something deserves a brash response.
I don't really have any opinion on your tone; I just still don't agree with your framing. A lack of evidence would be neutral like "there's no evidence to indicate either possibility is more likely", but your phrasing conveyed that one possibility was more likely than the other. I pushed back against your follow-up because it seemed like you were arguing for a higher threshold of evidence than you provided.
Well, to be fair, you're correct. I am asking for a higher threshold of evidence. It's a stronger claim. I feel a stronger claim deserves stronger evidence.
I guess that's where we disagree. I feel like either claim is equally hard to falsify from the outside (partially because I've never had much confidence in my ability to spot whether text is from an LLM outside of the most glaringly obvious cases, and likewise don't have any clue whether people who have high confidence are accurate or deluding themselves).
According to https://news.ycombinator.com/item?id=49417480, Anthropic hires writers who do not use LLMs, and reading their updates I also feel that they don't use LLMs for communication.
I think implication being organizations with 40,000+ employees and even more consultants and contractors plus a lot of budget are also using LLMs to draft public facing content instead of paying for content writers or even just proof readers .
It points to friction rather than cost economics. Same reason we are always surprised why multi billion dollar product companies with millions of install base prefer electron instead of a native app.
This does not imply that the organization is not paying for content writers or proof readers. It does suggest that they are not getting the value of paying for content writers or proof readers.
It does suggest that they are not accurately measuring the value of paying for content writers or proof readers.
People and companies are hungry for knowledge about people's reactions, but the modern internet DOES NOT give an accurate image of people's views.
No. It suggests they don't mind littering slop into the information environment.
Of course, that is the whole point of using AI to replace workers.
Only devs think it isn't coming for them, it is empowering and nothing else will happen, no team reductions, nah how come.
I believe most of their marketing videos use fairly convincing text to speech too, not voice actors.
McDonald's food is not even that unhealthy. I just tried a Burger King burger the other day and it's terrible. I think it's like 2000 calories in a single burger or something.
found the McShill. The King will hear of this!
> I think it's like 2000 calories in a single burger
that would be pretty cool, you can just get your entire day's calories from one burger
I couldn't even finish it man, it was so fucking sloppy. I had to throw away like 40% of it.
2k cal would be around 250ml of oil. or 350grams of peanuts. So doing with bread, meat, and other stuff alike requires over 600g of food, an excellent value to energy.
Big Mac 580 calories, Whopper 680 calories. At least based on the first official numbers in each google search.
They both have a variety of sizes but it's a similar range and they have similar contents.
Their CEO also has the dubious distinction of claiming we've reached AGI more then once: https://www.theverge.com/ai-artificial-intelligence/985597/j...
Is Jensen Huang still all-in on OpenClaw? That moment feels more like a flash in the pan.
I get the impression that Nvidia employees don't care too much - I started seeing fully AI-written "documentation" on some of their smaller projects more than a year ago (i.e., before it was even slightly a good idea).
People never really read documentation before. Agents do read it now, and they seem to understand LLM-written text just fine.
> People never really read documentation before.
The heck you talking about? How do you think we wrote software for the last 50 years?
when you start to internalize that these kinds of statements are an indication of how the average developer of the last 10-15 years operated the adoption rate of AI makes a lot more sense
This is extremely depressing, I think I'm coming around to the realization that you may be right and I've been naive my entire career.
Yeah lol it’s basically the only reliable way to know how things work. Pre-AI, I read documentation for libraries that I used almost every day.
And now with AI I’m using it to fact check Claude. And still reading it for myself to understand why other peoples code is written a certain way. It’s basically the most important thing to reference when coding.
Sure today Claude can just read the library code and tell you what a function does or how to do something. But it still won’t tell you why something is a certain way or won’t figure out specifically-designed usage patterns as reliably as the author telling you “this is an example of doing x”
only reliable way to know how things work is to reverse engineer them
But that’s my point, it’ll tell you how things work. In libraries I’m using, you can just read the code yourself.
You need the documentation to know why certain things work a particular way, or to know why some relationships or methods are the way they are
I really appreciated a friend reaching out to me with some PHP questions today. It was, to me, fairly basic but he was having a hard time grokking the documentation vs reading what his coworker wrote (some code using output buffering).
I brushed up on the docs since I haven't touched it in a couple years, explained my understanding of the ob_* functions, and gave him a very brief demo on a PHP playground.
He could have asked any LLM to tell him what that chunk of code did, and to explain the three functions, and instead he reached out to me. That felt _good_. Talking shop has always been a good way for me to form connections, because the pressure to socialize becomes task-oriented and you start to learn about how people think and feel, and that opens up easier paths for actual connection. It was nice.
Just like the Old Internet still exists - niche websites, mailing lists, probably a BBS or two (likely more right?), the pre-LLM world will trudge on, for a time. I hope LLMs actually lead to good things for people in the long run, and for now I personally will remain sparse in my usage of them.
Where do you work? Sounds nice!
A small percentage of engineers
Copy past the example code, then tweak until it breaks? If we were meant to read documentation, not reading it would cause a compiler error!
My absolute greatest skill in my career, that has consistently set me apart from my peers, is that I read documentation thoroughly.
It is shocking how much of a differentiator this is. You will discover that the software you’re already using is much more capable than you realized.
The good news is your attention to actually reading and understanding documentation will differentiate you more and more as others (short-sighted, IMO) outsource understanding to an LLM.
What? People never read documentation?
I start with reading and exploring documentation first; with the codebase as a secondary tab.
When it’s not LLM generated, documentation is supposed to be easier to read and more insightful than code.
>People never really read documentation before.
Speak for yourself. I read it.
Is that your honest load bearing assessment you’re going to flag?
I hadn't read the article and read this comment as though NVIDIA themselves were implying that this library was checked but not trusted by them since it was fully LLM generated.
What are we for, I ask? What the hell are we now. Chatters to LLMs now? Is this our future? It really is starting to feel like it now.
Dude I am in slop fucking hell right now. There is still room for a human touch, without which the agents will lever us harder and faster into a world of incomprehensible garbage.
I totally concur. I'm almost lost for words at this stage. I need me some land to grow vegetables on and that's about it. Maybe some chickens. Every single day brings more despair (and not the prosperity we were promised).
Good luck with that. You will have to pry the water from the AI datacenters.
Do you know that's not really a thing or are you just wanting to help spread the propaganda?
... doesn't one of those imply the other?
I have land and chickens, and I'm really excited about the future & AI.
I'm also an optimist but the crash is imminent. I hope I am wrong.
Land, chickens, and private local AI running sustainably on the farm sounds like the least dystopian version of this AI future!
Do you have the stomach to walk into a high school in the USA these days? Teachers use AI to generate assignments. Students feed the assignments to AI and submit the responses. Teachers feed the student submissions to an AI for grading.
Please tell me this is not true.
This is true even at the undergraduate level unfortunately…
Then here we are. AI apocalypse. Something of note. I've started to pay more attention to canned goods. Soups with lentils and so forth.
FYI, the world is a lot more decentralized than we think and even during the Dark Ages, guess what, that was happening in Europe and many places in the world were booming scientifically, technologically, etc.
Dark Ages weren't as interconnected by communications and wrapped by the tentacles of transnational corpocracy as modern world, though...
Meh. It's not like we forgot how to make copper wires for landlines. We'll be fine. We'll live more or less like in 1880 or 1920, it's not a horrible life. I do hope we get to keep antibiotics, though.
It's true of most work in many and soon most white collar jobs, too. Claude writes some dense useless thing, everyone else uses Claude to summarize and write a reply to the thing. The Claude-submitted PRs get automatically reviewed and commented on by a GitHub Claude review bot. The programmer asks Claude to check out Claude's review comments to Claude. Claude pushes a commit to the branch and writes a comment. The Claude review bot reviews the commit and leaves a comment. The human [...].
My hot take is that it's not really that terrible in the long run for work since I think LLMs will probably be nearly or actually AGI and better white collar workers than most humans within 5 years of today. But it is very funny and surreal in the meantime.
It is definitely bad for school, though. Kids IMO should actually be encouraged to use LLMs but not in or for class work outside of an AI best practices class. Probably stop giving them homework (90% will always try to find a way to make AI do it) and have them solve problems in class hours with no electronics so that they're forced to not defer learning. This will become even more important once we have AGI.
> LLMs will probably be nearly or actually AGI
What if they don't?
> This will become even more important once we have AGI.
What if we achieve AGI in 50+ years? Should everyone live in this Kafkaesque world until then?
I have two kids in engineering programs at a state University. They are allowed to use AI for homework assignments, but the homework is no longer worth any credit. They have a lot more papers, quizzes, and tests in class that count for their entire grade.
I know many teachers who actually have respect for the profession, themselves, and the students. Thankfully that means they don't do this.
Whether this is a widespread macro trend is another issue, and would be terryfying.
If true, however, it would reflect on the values of the organization: we have spent decades underpaying teachers, and doing a poor job of pretecting schools from frivoluos lawsuits. Add into that, districts have thrown money into new buildings, have been suckered by Big Tech to adopt their policies (common core was pushed by Big Tech and has been a distaster as well as computers in classrooms). As a nation (the USA) we can't get our act together for a rigorous national exam, etc etc.
About half of the states in the USA require the ACT or SAT for high school graduation.
Alabama is one state that requires the ACT. The mean score in Alabama is below 18/36. Wisconsin is another. Its students score on average about 1 point higher than the national average of 19.4/36.
If you prefer states that require the SAT, the mean SAT score of students from Delaware is less than 980/1600, about 50 points below the national average.
I'll leave it to others to argue about whether these exams are rigorous.
As a high schooler going to a school with stricter rules on AI than most in my area, I can say that it's been going downhill ever since GPT 4. Teachers constantly use AI to create assignments(my French Teacher regularly handed us work with GPT 5.1 prose and emojis). Students are also rampantly using AI and bypassing school restrictions(We have a google account, making it easy to use Gemini if we just sign out), causing an inflation in GPAs and test scores. There's no easy solution to the problem, banning AI-tools only help somewhat as even typing into Google has AI web results, and students are quickly overcoming ways to restrict them. I have a friend that vibe coded an application that allowed his Mac Mini's desktop to be mirrored on his school chromebook, bypassing every restriction with sub 1-second latency. Of course, that opens the can of worms to whether schools should allow students to use AI...
> Of course, that opens the can of worms to whether schools should allow students to use AI...
We are starting to see results indicating cognitive decline due to AI in education, so no, we should do everything possible to ban it except for very limited fields.
LLMs aren't calculators or even computers, their generated output is too flexible, generic and basically starts replacing thinking.
Most likely they should only be allowed during late highschool years or just at university level, when people at least have a chance to learn how to research on their own.
Remembering my teachers 20 years ago talking about staying in school grading until 8-9 PM, I wonder if these things are a symptom instead of a disease
I find it interesting that this was downvoted twice without explanation.
Claude, rewrite my graphics card in Rust. Make no mistakes.
Yeah. I think if the text is written for other machines, then by all means have an LLM generate it, but if it is intended for a human audience, have a human being write it.
We are still much better at writing in a way that doesn't waste other people's time.
It is the number 1 thing I cannot stand with Claude slop. It's a sort of anthropomorphization of language. Every "thing" does, produces, feels, wants, asks, answers, etc....
- "Launch is checked"
- "Question is asked"
- "The implementation answers"
- "The model wants"
- "The results name"
- "The connection surfaces"
- "The prompt wires"
- "The feature rides the mechanism"
Every single fucking thing is alive, wants things, and does things.
It's terrible. Infuriating. I want to rip my eyeballs out reading this filth. All. The. Time. "The anger is real".
Create any page with a file uploader. They all look the same now. It's like the Twitter Bootstrap days of responsive design. You'll get an icon which looks like ones on (on the drop space) those sites which are like "you must wait 60 seconds for this file to download".
It's so horrible. The human element has been completely removed and replaced by..... mediocre.
The human element has been completely removed and replaced by..... mediocre.
No it hasn't. The human element is still there, prompting the LLM. The change is that the human is happily accepting the first thing they get rather than critically looking at it and seeing a problem.
The real problem is that the human is only seeing dollar signs.
I don't think it's that because I see a lot of this in businesses where the human isn't paying the bill, or is even aware of what the bill is.
Humans are seeing either a shortcut to go faster (accepting low quality to move on immediately; reasonable if they're short on time) or a shortcut to lowering effort (accepting low quality because they don't care; not so reasonable but probably has a deeper root cause).
All of the examples read like: "The dude abides", except in a grotesque/parody way.
Even their writing skills are getting rusty.
it seems like that's the way the industry is headed
Another of those AI is bad for articles, great for coding.
Plenty of us share the same opinion on doing reviews of AI generated code.
In this age of LLM written everything which has softly killed my motivation for learning Rust somewhat, this has revived my interest if not only for the fact the LLMs haven't yet been trained on this yet!
LLM don't need to be trained in a library to use it well. It's just Rust which they know well.
And what prevent you exactly ?
There were humans far superior than you for writting Rust before LLM, now there's a LLM. The only difference is price and time execution.
You get an awesome teacher (LLM) ready to answer all your questions about Rust.
And you still find excuses not to learn it ?
At some point, just realize you've been lazy to learn it and LLMs are just an excuse.
I think OP’s point is that the payoff in learning a new language has diminished in this AI era. You can call that lazy, I’d consider it smart to consider whether you could be doing other, better, things with your time.
If the sole motivation for learning things are payoff, then sure.
the sole motivation is feeding and sheltering my family. in the time BC (before Clankers), rust was a better way to do that.
> in the time BC (before Clankers)
Nice one!
Which year shall we count as 1 AD (Anno Delirii (or should it be Darii))?
Sad to break it to you, but...
I had LLMs write a pile of cuda-rust code and they were quite competent at it. Ported a bunch of (C++) CUDA kernels over, and ground away on them til they got equivalent performance
https://github.com/rdaum/eider/tree/main/backends/cuda-oxide
And mostly just DeepSeek 4.1 Flash, too. Not even a frontier model.
Sorry.
C'est la vie
I was learning Rust slowly when the LLM enabled coding became good enough. I switched from learning to full on building with Rust. I still learn high level concepts as needed but I will not be able to write Rust on my own at all.
And that sounds scary but the way I got over the fear is by realizing there are many things that I do very well but I do not know their internals very well. Driving is an example. I barely understand what the steering wheel, clutch or brake pedals do. I have driven over 130,000 Kms and I will perhaps drive more than double that in the next many years.
I have been building software since PHP/Drupal days. Got into AWS S3 as a beta user. Adopted Memcached (and MQ) in 2008 out of necessity. Then Python/Django for 10 years. Then Rust. And tons of JS/TS. I owe a lot to my curiosity. I believe we can keep learning what we need and still delegate most of programming to agents.
There are two types of programmers: the pragmatists who see programming as a chore and would gladly never write a line of code again given the right tools, and the gardeners who don't want their enjoyable and rewarding garden-tending work taken away from them.
The "pragmatists" who get excited developing a prototype for a week before they realize they will never be able to ship something anyone else will use because each trivial change becomes exponentially more difficult for the LLM to implement and completely impossible for the "pragmatist" to reason about, with every new commit liable to break something else.
Still waiting for this revolution of amazing 10x software! It's been 10 months since Everything Changed in November, surely the 10x pragmatists could have leveraged their effective 8 years of development time? Or maybe we'll move the goalposts again and say that actually, Everything Changed with Astra, we'll just need to wait another three months?
> pragmatists
Which is a collective term for transactionalists, short-termists and profit-seekers of all kinds in this case.
I've found sorta the opposite - in any area, it can just do everything for you, or it can be an incredible teacher. I've been re-learning a lot of higher-level math and it has been an knowledgeable, infinitely patient, always-available tutor. Of course, I could just have it do just about any math I want for me, but that's not the point.
Kinda the same with language/technology stuff - it can be a great tutor and it can scaffold other parts of a project for you. It can give you feedback and let you focus on the interesting parts.
I guess the motivation itself may be hard because of the fear of it taking over much of our jobs, but having this kind of help/feedback is pretty cool for the sake of learning things just because they are interesting!
> I've found sorta the opposite - in any area, it can just do everything for you, or it can be an incredible teacher.
Please don't. I've had all of Codex, Claude and Gemini convincingly tell me absolutely wrong stuff, pointing it out with easily verifiable example they come up with more and more weird reasons.
Things don't become correct simply because most sources are again - easily and logically verifiable - wrong. This already was a plague when people "just googled" stuff and effectively returned with the most SEO optimized answer. Now we have very convincingly written instances all over the place.
If these were singular instances I wouldn't be so worried, but if you are learning it already is very easy to learn something wrong. This is why back in the days when people still used physical books to learn new things it was a good idea to check first which books are actually recommended. There have been a lot of "experts" that wrote things they clearly misunderstood but worked for all the examples in their books.
To give a common example for both the backend and frontend devs, that isn't about a specific projects. LLMs and Google searches frequently turn out wrong results regarding CORS caching and how it works in relation to domains/hostnames. The circumstances under which Content-Disposition work are another example. I think a lot of wrong statements that LLMs are "convinced" about are due to wrong statements (sometimes in otherwise correct response) of popular Stack Overflow answers.
It's saddening how much wrong "common knowledge" exists in the industry. I have been bitten by a lot of these, but it feels when people don't even actually code and think anymore this will just rise forever.
> Please don't.
I will.
Can these be wrong? Certainly. So can humans. Many of your examples are of humans being wrong. That doesn't make LLMs - or humans - useless. The fact that they are not infallible is not a reason to avoid using them and I'm not going to throw out a tool that has been incredibly valuable to me because someone on the internet got some bad CORS advice.
There is another option. Going to the source of information (eg. the official project site or code), trying stuff yourself.
Learning is about so much more than accessing information, though. It is about building mental models, resolving ambiguity, exploring things that the source doesn't explain very well, and so much more.
Questions like "What do these lines of code do?" or "How does this fit into the big picture?" or "Wait, this doesn't make sense?" are rarely answered by the source.
This is a bit fresher in my mind in the math domain, but I don't think it is any different in any number of other domains, including coding. I've been working through a math textbook, gotten confused about how the author gets from step 2 to step 3, taken a picture of the text, and had AI explain it to me - it almost always gives me a much better understanding of what is going on and helps make things so much clearer. There is a level of interactivity that can't exist in a book or other "source" of information.
I think there is a bit of tension when it comes to learning and sometimes the struggle itself is informative, but there is a reason people hire tutors and go to classes taught by teachers vs just reading a textbook, and cutting yourself off from a tool because you've seen it be wrong about something seems like a silly mistake.
Worth mentioning that Nvidia open sourced CUDA Tile IR ~8 months ago. And yes the code is open source too. https://news.ycombinator.com/item?id=46330732
I wonder how many parallels there are between CUDA's Tile abstraction and that of Metal.
what this article tells me is that no one at Nvidia actually cares about this project whatsoever. otherwise, they would have had a person actually write the announcement.
Will it then be possible to query TJunc hotspot temps on linux?
The recent circular moves that Nvidia is making is designed to wrap things around them, anything to keep the AI model party going.
Been waiting for something like this. CUDA C++ is a pain; Rust's safety for kernel programming could be a game changer.
yikes, I prefer python taichi similar to
@fast def calc(x,y):pass
So now every already written kernel can be re-written in Rust and have competitive performance to the cpp version?
If so, that's really big.
And to add the Next natural strp - custom codegen for simulating gpu compute and memory without Nvidia gpu.
First of all, this is a pre-1.0 release that requires a nightly Rust compiler (if you choose the SIMT track with cuda-oxide) so that one is going to be unstable software.
Secondly, When an issue occurs with a kernel or you want to write your own custom kernel in Rust, now we need to diagnose if the problem came from either cuda-oxide (SIMT), Rust's side, CUDA or Tile (If you decide to choose the Tile track).
Another dependency into the list and course everything is open source except CUDA itself. So any issue that happens on the CUDA level, you are forced to wait for them to fix it.
Do it in Go and I’m interested
Go is simply not feasible for CUDA work mainly because of the go runtime that manages GC, allocation, scheduling etc
GPU kernels want explicit memory control and as little go-runtime like overhead as possible
hmmm interesting
AI slop article, how can I trust on this?
The same way as people trust AI sloppy on their code.
Thank you Nvidia !! You're in the right path.
Nvidia only? Typical.
This is more promising: https://github.com/Rust-GPU/rust-gpu/
Once again... There's literally no point to a low level shader language for heterogenous back ends.
Why? The point of a shader language is to define a function that outputs some graphics. Why should that not be portable between GPUs/CPUs of different vendors?
More generally any GPU computation, not necessarily graphics. The above argument could go that there is no point in high level languages for CPUs either and everyone should just always use assembly, which is obviously false. Same can go for GPUs.
I mean if you're worried about high performance compute on cpus, then the argument also applies. That's why there so much hand written assembler with dynamic dispatch on CPU features...
It's just that gpus have no general purpose usage .. for ai, it's basically all about perf.
Of course some cross platform stuff can be made for modest acceleration, but the state of the art is always going to be out of reach.
Because of how different the memory hierarchies and pipelines are and because if you're doing perf work, this is important enough to matter. For cpus and general purpose programs, it's not. For hpc, it is
The world is unsafe
Rust just makes you sign a waiver first.
Rust for GPU programming? My CUDA debugging sessions just got a whole lot less painful, hopefully.
There's a lot of unsafe code at that level... Rust probably makes it more painful for little gain
Makes me sad that Go doesn't get love. I feel like Go is perfect for LLMs.
Don't think we're gonna see garbage collections anywhere near gpu for many reasons.
Go's type system is not at the same level as C++, Fortran, Python, Julia, Haskell, Java, to quote the languages with CUDA support from NVIDIA and their partners.