用 Py-Spy 分析 Python 程式效率

這之後應該會變成 Python community 的神器之一...

剛剛看到分析 Python 程式效率的工具,只要有 pid 或是直接包著跑就可以分析:「Py-Spy: A sampling profiler for Python programs.」,執行起來長這樣:

而且還可以直接產生火焰圖讓開發者直接確認,超友善:

在 FAQ 的地方也有提到作者開發這套軟體的原因。有些在開發環境根本看不出問題的,可以很快的透過這個工具在 production 上看:

This project aims to let you profile and debug any running Python program, even if the program is serving production traffic.

另外一個重點在於其他常見的 profiling 工具通常都要改程式引用進來使用,這通常會使得程式效率慢下來,而 Pyflame 支援的平台比較少:

While there are many other python profiling projects, almost all of them require modifying the profiled program in some way. Usually, the profiling code runs inside of the target python process, which will slow down and change how the program operates. This means it's not generally safe to use these profilers for debugging issues in production services since they will usually have a noticeable impact on performance. The only other Python profiler that runs totally in a separate process is pyflame, which profiles remote python processes by using the ptrace system call. While pyflame is a great project, it doesn't support Python 3.7 yet and doesn't work on OSX or Windows.

CPU 成為現代網站的速度瓶頸

在「Tracking CPU with Long Tasks API」這邊提到的現象,雖然是在提新的 API,不過裡面提到了很重要的問題。

以前的網站因為 js 都沒有用的那麼多,所以主要的瓶頸在於網路速度。所以大家最佳化的方向都是往「如何讓傳輸量變小」的方式進行,像是各類 js 的 minify,甚至是對 Gzip 演算法的暴力改善 (維持相容的 Zopfli,以及新的 Brotli):

In the old days, delivering a fast user experience depended primarily on download speed. One reason why the network was the main bottleneck back then is that JavaScript and CSS weren’t used as much as they are now, so CPU wasn’t a critical factor.

而現代網站使用 js 的情況已經是來到了新的境界 (甚至很多網站是沒有 js 就不會動),於是對於 CPU 的能力就愈來愈要求:

According to the HTTP Archive, the top 1000 websites download five times more JavaScript today compared to seven years ago.

而手機也愈來愈普及,CPU 的能力相較起來就更嚴峻了...