The alternative to CUDA is called OpenCL. There has been speculations that poor performance of NVIDIA GPUs with OpenCL compared to CUDA is an intentional anticompetitive practice, but I don't feel confident to tell for sure.
OpenCL is an alternative to CUDA just like Legos are an alternative to bricks. The problem with OpenCL isn't even the performance, it's everything. If OpenCL were any good, people could use it to build similarly powerful applications on cheaper AMD GPUs.
OpenCL is just a spec. It's up to companies to implement it in a successful way or not. There is no reason in and of itself that OpenCL can't compete with CUDA on performance. The fact that Apple's Metal, which is pretty good, is actually implemented with a private OpenCL system is proof that the spec is not to blame.
Well, performance isn't the issue. Like the parent said, the problem is mostly that CUDA is such a radically mature API and everything else isn't. You might be able to reimplement all the PCIe operations using OpenCL, but how long would that take and who would sponsor the work? Nvidia simply does it, because they know the work is valuable to their customers and will add to their vertical integration.
OpenCL isn't bad, and I'd love to see it get to the point where it competes with CUDA as originally intended. The peaceable sentiment of "let's work together to kill the big bad demon" seems to be dead today, though. Everyone would rather sell their own CUDA-killer than work together to defeat it.
For similar reasons why Microsoft created DirectX. It allows them to have a system where the software and hardware are more tightly integrated than using a cross-platform spec. It also allows them to situate the API within the context of the operating system and other languages that are used on Apple platforms, making things easier on developers. And at least in that regard, they certainly succeeded. Metal is probably the easiest GPU API offered on any platform. Not necessarily the most powerful, but it’s almost trivial to hand write a compute kernel and spin it off.
AMD’s first attempt at displacing CUDA-as-a-runtime was called AMD APP (advanced parallel processing) so this layer did indeed exist. It just sucked as badly as the rest of AMD’s slop.
Bonus points: the rest of the software libraries intended to compete with the CUDA ecosystem are still online in the “HSA Toolkit” GitHub repo. Here’s their counterpart to the Thrust library (last updated 10 years ago):
That theory doesn't really make sense. Nvidia didn't need AMD's permission to write a high-quality compute interface. Why would AMD need Nvidia's? AMD has their own internal opinions for how well they want compute to work; if OpenCL isn't doing it for them they should have figured that out quickly and then built their own layer as a stopgap. But we've seen what happens when they do that; ROCm. The problem is that they don't really know how and it is a glaring capability gap (didn't know I suppose - I expect they're learning fast).
If anything, the situation with OpenCL suggests AMD and friends were the ones dragging their feet. Nvidia correctly identified that being led by the OpenCL committee would lock them out of billions (trillions, if you believe the stock market) of profit and routed around the blockage rather than compromise their engineering standards.
And much more importantly for ooencl's success, poor compatibility of OpenCL implementations on Nvidia hardware by Nvidia + Microsoft platform-level software
Eg, opencl on Nvidia GPUs for windows is/was missing (I have a many-year GitHub issue with them + Microsoft), which matters for individuals, and Nvidia does not support OpenCL for its core convenience analytics libraries like the RAPIDS Python ecosystem, which is core to its massive data center market. We initially built for compatibility/distribution, but it didn't matter: That gap closed a lot of doors for us as a small ISV choosing what & how to build, and in turn, ultimately prevents our customers from buying AMD, Intel, etc
I'm not up to snuff on whether that qualifies as using its position anti-competitively anywhere, but it's a real market issue
How do we know the strategy wasn’t to avoid releasing a CUDA alternative until NVIDIA is wounded by an anti-trust lawsuit and then when they get broken up deploy your CUDA alternative and speed past them? Could be playing the long game.
That’s not the case at all. AMD and Intel could be making money hand over fist right now if they had products that worked with the ML ecosystem. If they had any magic bullet they’d be idiots to not have used it years ago, they’re missing out on tens of billions of dollars on purpose for what? Even if NVIDIA got broken up their cards would still have massive demand and profit margins. I don’t think you understand how much of a shortage of compute there currently is, anyone who isn’t blatantly incompetent or ignoring ML can make money. AMD and Intel are just fat corporations who didn’t see the potential and thought business as usual would stay number one. Now they’re standing on the sidelines with their dicks in their hands while Jensen is leading the parade.
