curl 將支援 Brotli 壓縮

Twitter 上看到有人提到 curl 支援 Brotli 了:「HTTP: implement Brotli content encoding」。

Brotli 對文字系列的資料比較有幫助 (像是 html):

Unlike most general purpose compression algorithms, Brotli uses a pre-defined 120 kilobyte dictionary, in addition to the dynamically populated ("sliding window") dictionary. The pre-defined dictionary contains over 13000 common words, phrases and other substrings derived from a large corpus of text and HTML documents. Using a pre-defined dictionary has been shown to increase compression where a file mostly contains commonly-used words.

現在還在 master 裡面,之後的 release 版本應該就會支援了...

curl 的 refactor 減少 malloc() 的次數

curl 的作者 Daniel Stenberg 寫了一篇關於他 refactor 大幅改善了 curl 的效能:「Fewer mallocs in curl」。

藉由大幅減少 malloc() 次數,在抓大檔案時會有明顯的改善:

7.54.1-DEV repeatedly performed 30% faster! The 2200MB/sec in my build of the earlier release increased to over 2900 MB/sec with the current version.

另外如果對 localhost 抓 80GB 的話,可以看到時間大幅縮短:

Old code:

real    0m36.705s
user    0m20.176s
sys     0m16.072s

New code:

real    0m29.032s
user    0m12.196s
sys     0m12.820s

cURL 接下來的安全性更新...

cURL 的維護老大放話要大家注意接下來的安全性更新:「An alert on the upcoming 7.51.0 release」。

最少 11 個安全性更新:

This release will bundle no less than _eleven_ security advisories and their associated fixes (unless we get more reported in the time we have left).

由於這些 security issue 的特性,會採取不公開的 branch 修正再 merge 回來,再加上這麼大的數量,對於穩定性的衝擊是未知的:

Merging eleven previously non-disclosed branches into master just before a release is not ideal but done so to minimize the security impact on existing users when the problems get known.

所以目前的規劃是會在 release 的 48 個小時前公開 (希望藉由這封信讓有能力的人一起集中來看),藉此來降低衝擊:

My plan is to merge them all into master and push around 48 hours before release, watch the autobuilds closesly, have a few extra coverity scans done and then fix up what's found before the release.

這安全更新的數量好像有點多 orz

用 curl 測試 Reserve Proxy 是否正確運作

架好 reverse proxy 後要測試可以用 curl--resolve 的功能來確認。

curl -v --resolve i.kfs.io:443:68.232.45.191 https://i.kfs.io/article5/global/364,324,6v1/original.png > /dev/null

其中 --resolve 的第三個參數一定要用 IP address,你可以看到他的運作原理:

* Added i.kfs.io:443:68.232.45.191 to DNS cache