curl 將支援 DNS over HTTPS

curl 的維護者 Daniel Stenberg 在他的 blog 上宣佈 curl 將會支援 DNS over HTTPS:「DoH in curl」。

從給的範例可以看出來就是多一個 CURLOPT_DOH_URL 可以設,對於要用的人很簡單:

curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_URL,
                 "https://curl.haxx.se/");
curl_easy_setopt(curl, CURLOPT_DOH_URL,
                 "https://doh.example.com/");
res = curl_easy_perform(curl);

提供 Cheatsheet 的服務:cheat.sh

看到「chubin/cheat.sh」這個服務:

the only cheat sheet you need

他提供了 HTTP(S) 介面讓你存取 (透過 curl 或是其他的軟體),像是這樣的操作:

而且提供了 shell script 讓你可以更方便在 command line 下使用:

curl https://cht.sh/:cht.sh > ~/bin/cht.sh
chmod +x ~/bin/cht.sh

你可以把 cht.sh 換成其他你喜歡的名字,或是用 alias 方便操作...

curl 將支援 Brotli 壓縮

Twitter 上看到有人提到 curl 支援 Brotli 了:「HTTP: implement Brotli content encoding」。

Brotli 對文字系列的資料比較有幫助 (像是 html):

Unlike most general purpose compression algorithms, Brotli uses a pre-defined 120 kilobyte dictionary, in addition to the dynamically populated ("sliding window") dictionary. The pre-defined dictionary contains over 13000 common words, phrases and other substrings derived from a large corpus of text and HTML documents. Using a pre-defined dictionary has been shown to increase compression where a file mostly contains commonly-used words.

現在還在 master 裡面,之後的 release 版本應該就會支援了...

curl 的 refactor 減少 malloc() 的次數

curl 的作者 Daniel Stenberg 寫了一篇關於他 refactor 大幅改善了 curl 的效能:「Fewer mallocs in curl」。

藉由大幅減少 malloc() 次數,在抓大檔案時會有明顯的改善:

7.54.1-DEV repeatedly performed 30% faster! The 2200MB/sec in my build of the earlier release increased to over 2900 MB/sec with the current version.

另外如果對 localhost 抓 80GB 的話,可以看到時間大幅縮短:

Old code:

real    0m36.705s
user    0m20.176s
sys     0m16.072s

New code:

real    0m29.032s
user    0m12.196s
sys     0m12.820s

cURL 接下來的安全性更新...

cURL 的維護老大放話要大家注意接下來的安全性更新:「An alert on the upcoming 7.51.0 release」。

最少 11 個安全性更新:

This release will bundle no less than _eleven_ security advisories and their associated fixes (unless we get more reported in the time we have left).

由於這些 security issue 的特性,會採取不公開的 branch 修正再 merge 回來,再加上這麼大的數量,對於穩定性的衝擊是未知的:

Merging eleven previously non-disclosed branches into master just before a release is not ideal but done so to minimize the security impact on existing users when the problems get known.

所以目前的規劃是會在 release 的 48 個小時前公開 (希望藉由這封信讓有能力的人一起集中來看),藉此來降低衝擊:

My plan is to merge them all into master and push around 48 hours before release, watch the autobuilds closesly, have a few extra coverity scans done and then fix up what's found before the release.

這安全更新的數量好像有點多 orz

用 curl 測試 Reserve Proxy 是否正確運作

架好 reverse proxy 後要測試可以用 curl--resolve 的功能來確認。

curl -v --resolve i.kfs.io:443:68.232.45.191 https://i.kfs.io/article5/global/364,324,6v1/original.png > /dev/null

其中 --resolve 的第三個參數一定要用 IP address,你可以看到他的運作原理:

* Added i.kfs.io:443:68.232.45.191 to DNS cache