cuda – Gea-Suan Lin's BLOG

SCALE：另外一個試著在 AMD GPU 上面跑 CUDA 程式的嘗試

昨天看到討論的蠻熱烈的東西，又是一個試著在 AMD GPU 上面跑 CUDA 程式的嘗試：「Run CUDA, unmodified, on AMD GPUs (scale-lang.com)」，連結的原文在「SCALE documentation」這邊。

跟之前提過的 ZLUDA 有些差異 (關於之前寫的紋章，可以參考「讓 ~~Intel~~AMD GPU 直接跑 CUDA 程式的 ZLUDA」這邊)，在 ZCUDA 這邊是用 LD_PRELOAD 的方式達成的，像是 README 提到的這樣：

LD_LIBRARY_PATH="<ZLUDA_DIRECTORY>:$LD_LIBRARY_PATH" <APPLICATION> <APPLICATION_ARGUMENTS>

而從 SCLAE 的作法則是提供一整包編譯工具，讓你可以不用修改 CUDA 的程式，直接重新編就可以跑在 AMD GPU 上面，所以只能用在你有 source code 的情況下，並不是拿到 CUDA 程式的 binary 就可以跑。

算是個還蠻有趣的方法...

Nvidia CUDA 的 EULA 禁止跑在模擬層上面

在 Hacker News 上看到「Nvidia bans using translation layers for CUDA software to run on other chips (tomshardware.com)」這個討論串，原報導「Nvidia bans using translation layers for CUDA software — previously the prohibition was only listed in the online EULA, now included in installed files」。

文章最上面更新的這段把重點講完了，在 11.6 之後的版本就有加入這段文字了：

[Edit 3/4/24 11:30am PT: Clarified article to reflect that this clause is available on the online listing of Nvidia's EULA, but has not been in the EULA text file included in the downloaded software. The warning text was added to 11.6 and newer versions of the installed CUDA documentation.]

主要是這條：

You may not reverse engineer, decompile or disassemble any portion of the output generated using SDK elements for the purpose of translating such output artifacts to target a non-NVIDIA platform.

從「CUDA Toolkit Archive」這邊可以看到 11.6.0 是 2022 年一月的事情：

CUDA Toolkit 11.6.0 (January 2022), Versioned Online Documentation

看起來跟 ZLUDA 有關 (參考「讓 ~~Intel~~AMD GPU 直接跑 CUDA 程式的 ZLUDA」)。

時間軸看起來有對起來，2021 年的時候被 Intel 找去聊，2022 年的時候改跟 AMD 簽約，然後 AMD 後來也放掉：

In 2021 I was contacted by Intel about the development of ZLUDA. I was an Intel employee at the time. While we were building a case for ZLUDA internally, I was asked for a far-reaching discretion: not to advertise the fact that Intel was evaluating ZLUDA and definitely not to make any commits to the public ZLUDA repo. After some deliberation, Intel decided that there is no business case for running CUDA applications on Intel GPUs.

Shortly thereafter I got in contact with AMD and in early 2022 I have left Intel and signed a ZLUDA development contract with AMD. Once again I was asked for a far-reaching discretion: not to advertise the fact that AMD is evaluating ZLUDA and definitely not to make any commits to the public ZLUDA repo. After two years of development and some deliberation, AMD decided that there is no business case for running CUDA applications on AMD GPUs.

讓 IntelAMD GPU 直接跑 CUDA 程式的 ZLUDA

先前提過「在 Intel 內顯上面直接跑 CUDA 程式的 ZLUDA」，結果後來事情大翻轉，AMD 跑去贊助專案，變成支援 AMD GPU 了：「AMD Quietly Funded A Drop-In CUDA Implementation Built On ROCm: It's Now Open-Source」，專案在 GitHub 的 vosen/ZLUDA 這邊，而這包支援 AMD GPU 的 commit log 則是在 1b9ba2b2333746c5e2b05a2bf24fa6ec3828dcdf 這包巨大的 commit：

Nobody expects the Red Team

Too many changes to list, but broadly:
* Remove Intel GPU support from the compiler
* Add AMD GPU support to the compiler
* Remove Intel GPU host code
* Add AMD GPU host code
* More device instructions. From 40 to 68
* More host functions. From 48 to 184
* Add proof of concept implementation of OptiX framework
* Add minimal support of cuDNN, cuBLAS, cuSPARSE, cuFFT, NCCL, NVML
* Improve ZLUDA launcher for Windows

其中的轉折以及後續的故事其實還蠻不知道怎麼說的... 作者一開始在 Intel 上班，弄一弄 Intel 覺得這沒前景，然後 AMD 接觸後贊助這個專案，到後面也覺得沒前景，於是依照後來跟 AMD 的合約，如果 AMD 覺得沒前景，可以 open source 出來：

Why is this project suddenly back after 3 years? What happened to Intel GPU support?

In 2021 I was contacted by Intel about the development od ZLUDA. I was an Intel employee at the time. While we were building a case for ZLUDA internally, I was asked for a far-reaching discretion: not to advertise the fact that Intel was evaluating ZLUDA and definitely not to make any commits to the public ZLUDA repo. After some deliberation, Intel decided that there is no business case for running CUDA applications on Intel GPUs.

Shortly thereafter I got in contact with AMD and in early 2022 I have left Intel and signed a ZLUDA development contract with AMD. Once again I was asked for a far-reaching discretion: not to advertise the fact that AMD is evaluating ZLUDA and definitely not to make any commits to the public ZLUDA repo. After two years of development and some deliberation, AMD decided that there is no business case for running CUDA applications on AMD GPUs.

One of the terms of my contract with AMD was that if AMD did not find it fit for further development, I could release it. Which brings us to today.

這個其實還蠻好理解的，CUDA 畢竟是 Nvidia 家的 ecosystem，除非你反超越後自己定義一堆自家專屬的功能 (像是當年 Microsoft 在 IE 上的玩法)，不然只是幫人抬轎。

Phoronix 在 open source 前幾天先拿到軟體進行測試，而他這幾天測試的結果給了「頗不賴」的評價：

Andrzej Janik reached out and provided access to the new ZLUDA implementation for AMD ROCm to allow me to test it out and benchmark it in advance of today's planned public announcement. I've been testing it out for a few days and it's been a positive experience: CUDA-enabled software indeed running atop ROCm and without any changes. Even proprietary renderers and the like working with this "CUDA on Radeon" implementation.

另外為了避免測試時有些測試軟體會回傳到伺服器造成資訊外洩，ZLUDA 在這邊故意設定為 Graphics Device，而在這次 open source 公開後會改回正式的名稱：

In my screenshots and for the past two years of development the exposed device name for Radeon GPUs via CUDA has just been "Graphics Device" rather than the actual AMD Radeon graphics adapter with ROCm. The reason for this has been due to CUDA benchmarks auto-reporting results and other software that may have automated telemetry, to avoid leaking the fact of Radeon GPU use under CUDA, it's been set to the generic "Graphics Device" string. I'm told as part of today's open-sourcing of this ZLUDA on Radeon code that the change will be in place to expose the actual Radeon graphics card string rather than the generic "Graphics Device" concealer.

作者的測試看起來在不同的測試項目下差異頗大，但如果依照作者的計算方式，整體效能跟 OpenCL 版本差不多：

Phoronix 那邊則是做了與 Nvidia 比較的測試... 這邊拿的是同樣都有支援 Nvidia 與 AMD 家的卡的 Blender 測試，然後跑出來的結果讓人傻眼，透過 ZLUDA 轉譯出來的速度比原生支援的速度還快，這 optimization 看起來又有得討論了：(這是 BMW27 的測試，在 Classroom 的測試也發現一樣的情況)