Ingo Molnár 提出讓 Linux Kernel 編譯速度提昇的 mega patch

Hacker News 首頁上看到「Massive ~2.3k Patch Series Would Improve Linux Build Times 50~80% & Fix "Dependency Hell"」這個,對應到 mailing list 上的信件是「* [PATCH 0000/2297] [ANNOUNCE, RFC] "Fast Kernel Headers" Tree -v1: Eliminate the Linux kernel's "Dependency Hell"」這個,看到「0000/2297」這個 prefix XDDD

他主要是想要改善 Linux Kernel 的 compile 時間 (從 project 的名稱「Fast Kernel Headers」可以看到),只是沒想到會縮短這麼多。另外一方面也順便處理了 dependency hell 的問題 (改善維護性)。

測試出來的結果相當驚人,從 231.34 +- 0.60 secs (15.5 builds/hour) 到 129.97 +- 0.51 secs (27.7 builds/hour),以編譯次數來看的話是 78% 的改善。如果以 CPU time 來看的話,從 11,474,982.05 msec cpu-clock 降到 7,100,730.37 msec cpu-clock,也是以編譯次數來算的話,有 61.6% 的改善...

這是花了一年多的時間嘗試才達成的目標,嘗試不同的方法,前幾次雖然都有改善,但改善幅度太小,變動卻太大,他覺得不值得丟出來,直到第三次才達成這樣的目標...

第一次:

When I started this project, late 2020, I expected there to be maybe 50-100 patches. I did a few crude measurements that suggested that about 20% build speed improvement could be gained by reducing header dependencies, without having a substantial runtime effect on the kernel. Seemed substantial enough to justify 50-100 commits.

第二次:

But as the number of patches increased, I saw only limited performance increases. By mid-2021 I got to over 500 commits in this tree and had to throw away my second attempt (!), the first two approaches simply didn't scale, weren't maintainable and barely offered a 4% build speedup, not worth the churn of 500 patches and not worth even announcing.

第三次:

With the third attempt I introduced the per_task() machinery which brought the necessary flexibility to reduce dependencies drastically, and it was a type-clean approach that improved maintainability. But even at 1,000 commits I barely got to a 10% build speed improvement. Again this was not something I felt comfortable pushing upstream, or even announcing. :-/

然後基於第三次的成果覺得有望,意外的發現後續的速度改善比想像中的多非常多:

But the numbers were pretty clear: 20% performance gains were very much possible. So I kept developing this tree, and most of the speedups started arriving after over 1,500 commits, in the fall of 2021. I was very surprised when it went beyond 20% speedup and more, then arrived at the current 78% with my reference config. There's a clear super-linear improvement property of kernel build overhead, once the number of dependencies is reduced to the bare minimum.

這次的 patch 雖然超大包,但看起來對於 compile 時間改善非常多,應該會有不少討論... 消息還蠻新的 (台灣時間今天早上五點的信),晚點可以看一下其他大老出來回什麼...

在 ZFS 上跑 PostgreSQL 的調校

在「Everything I've seen on optimizing Postgres on ZFS」這邊看到如果要在 ZFS 上面跑 PostgreSQL 時的調校方式,看起來作者有一直在更新這篇,所以需要的時候可以跑去看...

主要的族群是要搞 self-hosted PostgreSQL 的人,相較於 ext4 或是 XFS,底層如果使用 ZFS 可以做許多事情,像是 compression 與 snapshot,這對於很多 DBA 相關的操作會方便不少,但也因為 ZFS 的關係,兩邊 (& PostgreSQL) 需要一起調整以確保效能...

不過短期應該還是用 RDS 就是了...

Linux Kernel 裡的 RNG 從 SHA-1 換成 BLAKE2s

Hacker News Daily 上看到的消息,Linux Kernel 裡的 RNG,裡面用到的 SHA-1 演算法換成 BLAKE2s 了:

SHA-1 已知的問題是個隱患,不過換成 BLAKE2s 應該是 maintainer 的偏好,Jason Donenfeld 在 WireGuard 裡面也是用 BLAKE2s...

在 2021 年 (2022 年?) 跑 Windows 2000 的方法

OSnews 上看到「Windows 2000 modernization guide」這個介紹文,題到了 2021 年的現在要怎麼跑 Windows 2000 的方式,原文在「Windows 2000 Modernization Guide」這邊,除了 Windows 2000 以外,還說明了在 Windows 2000 上面要怎麼跑 Microsoft Office 2007

裡面提到了 KernelEx 這個改裝套件,不過查了一下,這本來是試著在 Windows 98 上面跑 Windows 2000 或是 Windows XP 的程式,看起來不是講這個...

作者文章裡提到的是日本的 BlackWingCat 的改裝版本 (突然有種不怎麼意外的感覺),可以在 Windows 2000 上面跑 Windows XP 的程式,但即使如此也只是 kernel 層,還是有很多 library 層的東西要 porting...

看起來是很好玩,但希望用不到 XDDD

Apple 對 PNG 解碼的 bug

前幾天的 Hacker News Daily 上看到「PNG Parser Differential」這個,對應的討論在「PNG Parser Differential (vidbuchanan.co.uk)」這邊可以看到。

作者想要利用 threading 加速處理 PNG 格式,結果寫出了 bug,但意外發現 Apple 的 PNG decoder 也犯了一樣的問題:「Ambiguous Decodes #3」。

作者做了一個特別的 PNG 放在網站上,在非 Safari 的情況下會是:

而拿 iPhone 的 Safari 讀的話,可以看到:

應該是 implementation bug,還蠻有趣的 bug...

用 Exodus 打包 Linux ELF 檔案到其他機器上

前幾天在 Hacker News Daily 上看到的工具:「Exodus」,官方的說明是這樣:

Painless relocation of Linux binaries–and all of their dependencies–without containers.

技術上是把 Linux ELF 檔案搬到其他機器上以外,也幫你把對應的 dynamic library 都一起包進去:

  • Finding and bundling all of a binary's dependencies.
  • Launching the binary in such a way that the proper dependencies are used without any potential interaction from system libraries on the destination machine.

而 Linux 的 Kernel 因為有儘量維持 ABI compatibility,應該是不會有太大的問題,除非剛好用到新的 API...

看起來是個除了用 static compile 以外的解法,好像可以來弄弄看 FFmpeg

在非 4K 的螢幕上跑 HiDPI

前幾天看到 BetterDummy 這個專案,作者在 M1 上面外接 24" 1440p 的螢幕,但沒辦法啟用 HiDPI,於是就寫了一個軟體來解:

M1 macs tend to have issues with custom resolutions. Notoriously they don't allow sub-4K resolution displays to have HiDPI ("Retina") resolutions even though (for example) a 24" QHD 1440p display would greatly benefit from having an 1920x1080 HiDPI "Retina" mode.

在這之前的解法都有些麻煩,一種是買個 dummy dongle 去騙 macOS,另外是用 mirror 的方式使用:

To fix this issue, some resort to buying a 4K HDMI dummy dongle to fool macOS into thinking that a 4K display is connected and then mirror the contents of this dummy display to their actual monitor in order to have HiDPI resolutions available. Others use the built in screens of their MacBooks to mirror to the external display. These approaches have obvious drawbacks and cannot solve all problems.

作者提供的軟體可以先建立 Dummy Monitor,然後再透過 mirror 掛到實際螢幕上:

不確定用起來如何,但如果之後有需要的話好像可以測試看看...

不使用 Google 服務的 Android 手機

一樣是在 Lobsters Daily 上翻到的,去 Google 服務的 Android 系統搞法:「Lineage with microG on a Sony XA2」。

主要是看關鍵字的部份,TWRP 換掉 recovery image,然後 LineageOS 是系統底,microG 是 open source 版本的 Google 專屬 API 的相容層,Magisk 則是負責 root 相關的事情,F-Droid 是 open source 軟體的 app store,可以用他來裝 Aurora Store,就可以裝 Play Store 裡的 app。

會這樣搞的人主要還是考慮到 privacy,可以預期有不少應用程式是不會動的...

XFCE 配上 Chromium 系列瀏覽器 (Chrome/Brave/...),視窗最大化時的問題

今天發現 Brave 在視窗最大化時會超出預期的邊界,而非放大到螢幕的邊緣,找了一下發現有人已經在 Brave 的 GitHub 上開了「Incorrect scale if browser is full screen #18964」這張票,後來看到有人說在 Chromium 的 bug system 上已經有人提出來了:「Issue 1257119: Goes under the taskbar when maximized」、「Issue 1260821: maximise gets screen dimension wron」與「Issue 1261797: [User Feedback - Stable] Reports that when Chrome is maximized after being minimized, it launches to beyond the window frame on Linux」。

這次遇到的 bug 看起來是只有用 XFCE 的使用者才會中獎,目前先摸索出一套 workaround 是透過 wmctrl 操作修改瀏覽器的位置與視窗大小。

方法是先用 wmctrl -l -G 列出所有視窗的資訊,包括 geometry 的資料,接著再用 wmctrl -i -r 0x12345678 -e 0,3760,15,1232,1935 這樣的指令去指定瀏覽器的位置與視窗大小。

不知道要撐多久就是了...

獨立遊戲創作者推出 Linux 版的好處

標題不知道怎麼下,大概就是這樣...

Hacker News 首頁上翻到的,以這個 upvote 數量來看,應該會收到今天的 Hacker News Daily 上:「Despite having just 5.8% sales, over 38% of bug reports come from Linux (reddit.com)」。

作者是一位獨立遊戲開發者,在兩年前推出了「ΔV: Rings of Saturn」這款遊戲,並且也發佈了 Linux 版。

作者先就數字來比較,他賣出了 12000 套,其中 700 套是 Linux 玩家;另外他收到了 1040 個 bug report,其中大約 400 個是從 Linux 玩家回報的:

As of today, I sold a little over 12,000 units of ΔV in total. 700 of these units were bought by Linux players. That’s 5.8%. I got 1040 bug reports in total, out of which roughly 400 are made by Linux players.

That’s one report per 11.5 users on average, and one report per 1.75 Linux players. That’s right, an average Linux player will get you 650% more bug reports.

看文章時可能會覺得「Linux 玩家真難伺候」,但實際上作者講到,這 400 個 bug 裡面只有 3 個 bug 是平台相關的 (只會發生在 Linux 上),其他的 bug 其實是所有平台都會發生:

A lot of extra work for just 5.8% of extra units, right?

Wrong. Bugs exist whenever you know about them, or not.

Do you know how many of these 400 bug reports were actually platform-specific? 3. Literally only 3 things were problems that came out just on Linux. The rest of them were affecting everyone[.]

原因是 Linux 社群在參與各種 open source project 養成的習慣,會盡可能把各種資訊講清楚,並且找出可以重製問題的方式:

The thing is, the Linux community is exceptionally well trained in reporting bugs. That is just the open-source way. This 5.8% of players found 38% of all the bugs that affected everyone. Just like having your own 700-person strong QA team. That was not 38% extra work for me, that was just free QA!

But that’s not all. The report quality is stellar.

與一般玩家的回報方式完全不同,Linux 玩家很習慣就會附上基本的環境資訊,系統紀錄,甚至有時候還會包括 core dump 與 reproducible steps:

I mean we have all seen bug reports like: “it crashes for me after a few hours”. Do you know what a developer can do with such a report? Feel sorry at best. You can’t really fix any bug unless you can replicate it, see it with your own eyes, peek inside and finally see that it’s fixed.

And with bug reports from Linux players is just something else. You get all the software/os versions, all the logs, you get core dumps and you get replication steps. Sometimes I got with the player over discord and we quickly iterated a few versions with progressive fixes to isolate the problem. You just don’t get that kind of engagement from anyone else.

不知道有沒有遇到回報 GDB 資訊的...

很特別的分享 XDDD