Brendan Gregg 的工具與 profiling 技巧

這篇放在 browser tab 上一陣子了,今天找機會玩一下... 這是 Brendan Gregg 的工具,以及他說明怎麼用他來 profiling:「Analyzing a High Rate of Paging」。

裡面提到的 iostatfree 算是基本盤了,但裡面提到了很多看起來很神奇的東西,像是 biolatency、bitesize、cachestat,可以裝 bpfcc-tools 來用,檔名會加上 -bpfcc 的 suffix (像是 biolatency-bpfcc 這樣)。

三隻程式都需要 root 才能跑,對我來說,在 database 上面分析效能應該是很好用... 好像可以考慮直接當作預設安裝裝到每台機器上?

Brendan Gregg 遇到的 An Unbelievable Demo

Hacker News Daily 上看到的熱門話題,Brendan Gregg 是效能分析領域的大老,現在在 Netflix 工作,在維基百科的條目「Brendan Gregg」上也有提到他的一些知名發明,像是 Flame Graphs:

He has also created visualization types to aid performance analysis, including latency heat maps, utilization heat maps, subsecond offset heat maps, and flame graphs.

昨天他發了一篇文章在講之間他遇到的事情,原文把過程寫的很有戲劇性,值得去看一看:「An Unbelievable Demo」,而 Hacker News 上的討論也很精彩:「An Unbelievable Demo (brendangregg.com)」,還引出了 Colin Percival 也分享他的經驗。

快速講 Brendan Gregg 遇到的事情,2005 年時 Brendan Gregg 因為業務上的需要 (他當年是效能分析的顧問),幫 Sun 推出的 DTrace 寫了一包工具,叫做 DTraceToolkit,用 GPLv2 或是 CDDL 釋出。

這包工具被 Sun 的人拿去用,並且拔掉作者與授權資訊,然後還被拿去「世界巡迴」介紹這個工具,最後在雪梨的時候居然是拿來介紹給 Brendan Gregg,然後被原作者打臉。

不過他後來還是加入了 Sun... XDDD

Colin Percival 的故事則沒有牽扯到 copyright issue,不過也很有趣,這邊提到的是 bsdiff,也是個經典的工具:

Reminds me of when Apple started providing "smaller size updates" to OS X. I was curious about the details since my doctorate had touched on the topic, so I worked my contacts (I had a few in Apple engineering from the FreeBSD / OS X relationship) and after a few months I got back as answer: "We're using a tool called bsdiff, are you familiar with it?" I was indeed, since I was the author of said tool.

(Just to be clear, there was no license violation involved in this case; just a lack of awareness of the provenance of the open source software they were using.)

另外在其他的 thread 裡面,可以看到 Brendan Gregg 也有浮上來回應 (可以直接字串搜尋 brendangregg),裡面也提到了有趣的故事,像是他另外一個發明 latency heat map 在一些會場上的交流,以及自己也有遇到其他工具的作者:

Thanks. There was a time when many observability products were adding latency heat maps, and at one conference expo floor there were three companies with latency heat maps on their screen at the same time, pitching them as a flagship feature. If I walked near them they'd start trying to explain them to me, and I never figured out an appropriate response. If I said "hey, great to see you added them, I invented these back at Sun" I'd get funny looks.

I think it's a small world, and everything is software, so the chance you'll bump into someone who wrote software you are using I think is pretty high. I was once trying to get my head around Andi Kleen's pmu-tools, and I had the github repo open in my browser on my laptop I was carrying, when the guy sitting next to me on a bus says he's Andi Kleen. (Ok, it was a bus taking Linux conference attendees to an event, not a random bus, but I still found it remarkable timing -- I was studying pmu-tools at that exact time!)

拿來配啤酒的文章 XD