從 NVIDIA 這邊的新聞稿列出來的則比較合理，是透過硬體的觀點提到這個 12B model 可以跑在一張 4090 上 (24GB VRAM)：
Designed to fit on the memory of a single NVIDIA L40S, NVIDIA GeForce RTX 4090 or NVIDIA RTX 4500 GPU, the Mistral NeMo NIM offers high efficiency, low compute cost, and enhanced security and privacy.
不過即使可以這樣跑，目前比較有效率的跑法應該是應該都會找 quantization 版本來跑，通常 model 會變小不少，而且損失應該也還在能接受的範圍。
For cutting-edge platforms such as NVIDIA Grace Hopper or NVIDIA Blackwell, you must use the open-source GPU kernel modules. The proprietary drivers are unsupported on these platforms.
For older GPUs from the Maxwell, Pascal, or Volta architectures, the open-source GPU kernel modules are not compatible with your platform. Continue to use the NVIDIA proprietary driver.
Starting in the release 560 series, it will be recommended to use the open flavor of NVIDIA Linux Kernel Modules 204 wherever possible (Turing or later GPUs, or Ada or later when using GPU virtualization).
點進去看「Open Linux Kernel Modules」這頁可以看到開源版本有一些專屬功能 (在「The following features will only work with the open kernel modules flavor of the driver」這段)，蛋也有一些功能是開源版本沒有的 (在「The following features are not yet supported by the open kernel modules」這段)。
[Edit 3/4/24 11:30am PT: Clarified article to reflect that this clause is available on the online listing of Nvidia's EULA, but has not been in the EULA text file included in the downloaded software. The warning text was added to 11.6 and newer versions of the installed CUDA documentation.]
主要是這條：
You may not reverse engineer, decompile or disassemble any portion of the output generated using SDK elements for the purpose of translating such output artifacts to target a non-NVIDIA platform.
In 2021 I was contacted by Intel about the development of ZLUDA. I was an Intel employee at the time. While we were building a case for ZLUDA internally, I was asked for a far-reaching discretion: not to advertise the fact that Intel was evaluating ZLUDA and definitely not to make any commits to the public ZLUDA repo. After some deliberation, Intel decided that there is no business case for running CUDA applications on Intel GPUs.
Shortly thereafter I got in contact with AMD and in early 2022 I have left Intel and signed a ZLUDA development contract with AMD. Once again I was asked for a far-reaching discretion: not to advertise the fact that AMD is evaluating ZLUDA and definitely not to make any commits to the public ZLUDA repo. After two years of development and some deliberation, AMD decided that there is no business case for running CUDA applications on AMD GPUs.
Too many changes to list, but broadly:
* Remove Intel GPU support from the compiler
* Add AMD GPU support to the compiler
* Remove Intel GPU host code
* Add AMD GPU host code
* More device instructions. From 40 to 68
* More host functions. From 48 to 184
* Add proof of concept implementation of OptiX framework
* Add minimal support of cuDNN, cuBLAS, cuSPARSE, cuFFT, NCCL, NVML
* Improve ZLUDA launcher for Windows
Why is this project suddenly back after 3 years? What happened to Intel GPU support?
In 2021 I was contacted by Intel about the development od ZLUDA. I was an Intel employee at the time. While we were building a case for ZLUDA internally, I was asked for a far-reaching discretion: not to advertise the fact that Intel was evaluating ZLUDA and definitely not to make any commits to the public ZLUDA repo. After some deliberation, Intel decided that there is no business case for running CUDA applications on Intel GPUs.
Shortly thereafter I got in contact with AMD and in early 2022 I have left Intel and signed a ZLUDA development contract with AMD. Once again I was asked for a far-reaching discretion: not to advertise the fact that AMD is evaluating ZLUDA and definitely not to make any commits to the public ZLUDA repo. After two years of development and some deliberation, AMD decided that there is no business case for running CUDA applications on AMD GPUs.
One of the terms of my contract with AMD was that if AMD did not find it fit for further development, I could release it. Which brings us to today.
這個其實還蠻好理解的，CUDA 畢竟是 Nvidia 家的 ecosystem，除非你反超越後自己定義一堆自家專屬的功能 (像是當年 Microsoft 在 IE 上的玩法)，不然只是幫人抬轎。
Phoronix 在 open source 前幾天先拿到軟體進行測試，而他這幾天測試的結果給了「頗不賴」的評價：
Andrzej Janik reached out and provided access to the new ZLUDA implementation for AMD ROCm to allow me to test it out and benchmark it in advance of today's planned public announcement. I've been testing it out for a few days and it's been a positive experience: CUDA-enabled software indeed running atop ROCm and without any changes. Even proprietary renderers and the like working with this "CUDA on Radeon" implementation.
另外為了避免測試時有些測試軟體會回傳到伺服器造成資訊外洩，ZLUDA 在這邊故意設定為 Graphics Device，而在這次 open source 公開後會改回正式的名稱：
In my screenshots and for the past two years of development the exposed device name for Radeon GPUs via CUDA has just been "Graphics Device" rather than the actual AMD Radeon graphics adapter with ROCm. The reason for this has been due to CUDA benchmarks auto-reporting results and other software that may have automated telemetry, to avoid leaking the fact of Radeon GPU use under CUDA, it's been set to the generic "Graphics Device" string. I'm told as part of today's open-sourcing of this ZLUDA on Radeon code that the change will be in place to expose the actual Radeon graphics card string rather than the generic "Graphics Device" concealer.
