Dehydrated 取得憑證的預設演算法改成 secp384r1

這兩天弄 dehydrated,結果發現 v0.7.0 取得憑證的預設演算法改成 ECCsecp384r1 了:

Using EC secp384r1 as default certificate type

這會導致很多「稍微舊一點」的 client 失效 (瀏覽器與 library),不知道為什麼要預設... 目前避開的方法是強制在 /etc/dehydrated/config 內設定使用 rsa

KEY_ALGO=rsa

剛剛把公司一堆機器改上去,然後把自己的 server 也加一加...

Let's Encrypt 升級資料庫伺服器 (AMD YES?)

Let's Encrypt 升級了 MariaDB 資料庫的伺服器 (跑 InnoDB),特地寫了一篇文章出來講:「The Next Gen Database Servers Powering Let's Encrypt」。

CPU 的部份從本來的 2x Intel Xeon E5-2650 (Total 24 cores / 48 threads) 換成了 2x AMD EPYC 7542 (Total 64 cores / 128 threads),這點在本來就是 CPU 滿載的情境下改善很大:

而本來的瓶頸一解決,也使得 API 的 latency 直接降下去:

回頭看一下架構,可以看到他們提到沒有使用分散式的資料庫,而是單台 database 硬撐,驗證了即使到了 Let's Encrypt 這種規模,以暴制暴還是很有效的:

We run the CA against a single database in order to minimize complexity. Minimizing complexity is good for security, reliability, and reducing maintenance burden. We have a number of replicas of the database active at any given time, and we direct some read operations to replica database servers to reduce load on the primary.

除了 CPU 暴力外,2TB RAM 與 24 顆 NVMe SSD 的搞法也是很讚的,擺明就是用記憶體拼 cache 的量,以及用大量的 NVMe SSD 疊 IOPS。

然後硬體還在成長,看起來暴力解應該會變成以後的基本答案了...

IdenTrust 願意再幫 Let's Encrypt 交叉簽三年

先前在「Let's Encrypt 在 Android 平台上遇到的問題」這邊提到了 IdenTrustLet's Encrypt 交叉簽名的有效日會在 2021 年的八月底左右到期,而這會導致比較舊的 Android 平台因為沒有內建 ISRG Root X1 這個憑證,造成 Let's Encrypt 簽出來的憑證在這些舊的 Android 裝置上都認不出來。

文章出來過了一個多月後,剛剛看到 Let's Encrypt 發佈消息,IdenTrust 願意再交叉簽名三年:「Extending Android Device Compatibility for Let's Encrypt Certificates」,當時猜測發文是要讓 IdenTrust 表態,看起來目的達成了...

話說中間跑出來的「ZeroSSL 也提供免費的 SSL Certificate (DV) 了」不知道後續會怎麼樣,之後可以看看 Certificate Transparency 的資料來看看到底有多少人用...

Dehydrated 使用 ZeroSSL 的方式

上一篇「ZeroSSL 也提供免費的 SSL Certificate (DV) 了」提到 ZeroSSL 的服務,而各家 acme client 也都陸陸續續支援了。

Dehydrated 是在 2020/09/15 的時候實做:「EAB + ZeroSSL support」,我就抓了個新版,更新之前自己包的 PPA dehydrated-lite...

Dehydrated 的設定方式還蠻簡單的,在 /etc/dehydrated/config 裡面這樣寫:

CA=zerossl
EAB_HMAC_KEY=x
EAB_KID=y

其中兩個 EAB 的資訊可以在 ZeroSSL 網站上面取得,然後其他就照舊跑...

ZeroSSL 也提供免費的 SSL Certificate (DV) 了

Facebook 上被朋友敲可以測 ZeroSSL,另外一個透過 ACME 協定提供免費的 SSL Certificate,不過目前只有支援單一網域名稱 (DV):「Another free CA as an alternative to Let's Encrypt (scotthelme.co.uk)」。

我先前就有在測 ZeroSSL,不過驗證一直過不去,當時有在 Twitter 上找 ZeroSSL 帳號問,但 ZeroSSL 的人說 ACME 的部份不在客服範圍,就先丟著...

剛剛發現是自己耍笨了,原因是 nginx 沒設好造成驗證卡住,一改好後就正常了。

SSL LabsSSL Server Test 翻了一下,他的 Root CA 看起來歷史更久,應該是有機會解決 Let's Encrypt 明年會產生的 Root CA 憑證信任問題,也就是先前在「Let's Encrypt 在 Android 平台上遇到的問題」提到的問題,在 Hacker News 上的討論也可以看到有人提到這點:

Good to know, and I'm glad there's an alternative to Let's Encrypt, just in case. Is ZeroSSL trusted by old Android devices? If so, that might be a work-around for Let's Encrypt's cross-signing from IdenTrust expiring.

不過也有些人有疑慮,畢竟提供這個服務後面的公司幹過不少壞事:

If zerossl is reselling/a subsidiary of sectigo, that’s enough reason to never use this.
Sectigo is the new name for Comodo. The same bunch of pricks who tried to trademark “Let’s Encrypt”.

Other players in the acme cert “business” is great. Renaming a slime ball name and carrying on like nothing happened is not ok.

但看起來至少是多了一個選擇...

Let's Encrypt 在 Android 平台上遇到的問題

同樣是「Standing on Our Own Two Feet」這篇文章,Let's Encrypt 預期明年九月後會在 Android 上遇到嚴重的相容性問題。

很舊的裝置主要是透過 IdenTrust 的 Root CA (DST Root CA X3) 對 Let's Encrypt 的 Intermediate CA (目前主要是 Let's Encrypt Authority X3) 簽名,從而建立憑證的信任鍊,而新的裝置除了 IdenTrust 的 CA 外,也信任了 Let's Encrypt 自家的 Root CA (ISRG Root X1):(出自「Chain of Trust」)

在 2016 年四月正式對外啟用時主要是靠 IdenTrust 的 cross-sign,而也是在 2016 年時 Let's Encrypt 自家的 Root CA (ISRG Root X1) 陸陸續續被各家收進 CA store。

所以這個時間點之前的 Android (大約是 7.1.1) 算是個相容性的分界線,在這個版本前 (而且系統無法更新的) 都只能靠 IdenTrust 的 cross-sign,這看起來大約有 33.8%,實際的流量大約是 1%~5%:

Currently, 66.2% of Android devices are running version 7.1 or above. The remaining 33.8% of Android devices will eventually start getting certificate errors when users visit sites that have a Let’s Encrypt certificate. In our communications with large integrators, we have found that this represents around 1-5% of traffic to their sites. Hopefully these numbers will be lower by the time DST Root X3 expires next year, but the change may not be very significant.

目前還有大約十個月左右的緩衝期,但大家都知道 Android 的更新速度,就十個月來說看起來不太樂觀...

官方有給他們不願意再取得一次 cross-sign 的原因,不過我覺得這個理由就很怪了,這個描述看起來是 IdenTrust 不願意再簽發一次?直覺覺得 IdenTrust 站在商業立場應該是很願意才對?而且除了 IdenTrust,應該也有其他家會有興趣?

Can we get another cross-signature? We’ve explored this option and it seems unlikely. It’s a big risk for a CA to cross-sign another CA’s certificate, since they become responsible for everything that CA does.

也有可能是放個話讓 IdenTrust 表態?先繼續看下去...

最差的情況應該就是沒有 cross-sign,然後也沒提供其他的 workaround,這樣就是買一般的 SSL certificate 來解了...

Let's Encrypt 生了新的 Root 與 Intermediate Certificate

Let's Encrypt 弄了新的 Root Certificate 與 Intermediate Certificate:「Let's Encrypt's New Root and Intermediate Certificates」。

一方面是本來的 Intermediate Certificate 也快要要過期了,另外一方面是要利用 ECDSA 降低傳輸時的頻寬成本:

On Thursday, September 3rd, 2020, Let’s Encrypt issued six new certificates: one root, four intermediates, and one cross-sign. These new certificates are part of our larger plan to improve privacy on the web, by making ECDSA end-entity certificates widely available, and by making certificates smaller.

本來有 Let's Encrypt Authority {X1,X2,X3,X4} 四組 Intermediate Certificate,都是 RSA 2048 bits。

其中 X1 與 X2 差不多都到期了 (cross-signed 的已經過了,自家 ISRG Root X1 簽的剩不到一個月),不過這兩組已經沒在用了,這次就不管他了。

而 X3 與 X4 這兩組則是明年到期,會產生出新的 Intermediate Certificate,會叫做 R3 與 R4,跟之前一樣會被自家 ISRG Root X1 簽,以及 IdenTrust DST Root CA X3 簽:

For starters, we’ve issued two new 2048-bit RSA intermediates which we’re calling R3 and R4. These are both issued by ISRG Root X1, and have 5-year lifetimes. They will also be cross-signed by IdenTrust. They’re basically direct replacements for our current X3 and X4, which are expiring in a year. We expect to switch our primary issuance pipeline to use R3 later this year, which won’t have any real effect on issuance or renewal.

然後是本次的重頭戲,會弄出一個新的 Root Certificate,叫做 ISRG Root X2,以及兩個 Intermediate Certificate,叫做 E1 與 E2:

The other new certificates are more interesting. First up, we have the new ISRG Root X2, which has an ECDSA P-384 key instead of RSA, and is valid until 2040. Issued from that, we have two new intermediates, E1 and E2, which are both also ECDSA and are valid for 5 years.

主要的目的就是降低 TLS 連線時的 bandwidth,這次的設計預期可以降低將近 400 bytes:

While a 2048-bit RSA public key is about 256 bytes long, an ECDSA P-384 public key is only about 48 bytes. Similarly, the RSA signature will be another 256 bytes, while the ECDSA signature will only be 96 bytes. Factoring in some additional overhead, that’s a savings of nearly 400 bytes per certificate. Multiply that by how many certificates are in your chain, and how many connections you get in a day, and the bandwidth savings add up fast.

另外一個特別的修改是把名字改短 (把「Let's Encrypt Authority」拿掉),也是為了省傳輸的成本:

As an aside: since we’re concerned about certificate sizes, we’ve also taken a few other measures to save bytes in our new certificates. We’ve shortened their Subject Common Names from “Let’s Encrypt Authority X3” to just “R3”, relying on the previously-redundant Organization Name field to supply the words “Let’s Encrypt”. We’ve shortened their Authority Information Access Issuer and CRL Distribution Point URLs, and we’ve dropped their CPS and OCSP urls entirely. All of this adds up to another approximately 120 bytes of savings without making any substantive change to the useful information in the certificate.

這個部份讓我想到之前寫的「省頻寬的方法:終極版本...」這篇,裡面提到 AWS 自家的 SSL Certificate 太胖,改用 DigiCert 的反而可以省下不少錢 XDDD

另外也提到了這次 cross-sign 的部份是對 ECDSA Root Certificate 簽 (ISRG Root X2),而不是對 ECDSA Intermediate Certificate 簽 (E1 與 E2),主因是不希望多一次切換的轉移期:

In the end, we decided that providing the option of all-ECDSA chains was more important, and so opted to go with the first option, and cross-sign the ISRG Root X2 itself.

這算是蠻重要的進展,看起來各家 client 最近應該都會推出新版支援。

Let's Encrypt 在檢查 CAA 時出包

Let's Encrypt 發現在檢查 CAA 的程式碼有問題,發了說明:「2020.02.29 CAA Rechecking Bug」,以及預定的處理方式:「Revoking certain certificates on March 4」。

問題是當一個 certificate request 包含了 N 個 domain 時,本來的 CAA 檢查應該要對這 N 個檢查,但程式寫成只會抓一個,然後檢查了 N 次:

The bug: when a certificate request contained N domain names that needed CAA rechecking, Boulder would pick one domain name and check it N times. What this means in practice is that if a subscriber validated a domain name at time X, and the CAA records for that domain at time X allowed Let’s Encrypt issuance, that subscriber would be able to issue a certificate containing that domain name until X+30 days, even if someone later installed CAA records on that domain name that prohibit issuance by Let’s Encrypt.

2020/02/29 發現的,就程式碼的部屬時間,發現應該從去年 2019/07/25 開始就有這個 bug:

We confirmed the bug at 2020-02-29 03:08 UTC, and halted issuance at 03:10. We deployed a fix at 05:22 UTC and then re-enabled issuance.

Our preliminary investigation suggests the bug was introduced on 2019-07-25. We will conduct a more detailed investigation and provide a postmortem when it is complete.

然後決定要 revoke 這些可能會有問題的 SSL certificate,大約佔現有還有效的 SSL certificate 的 2.6%,大約三百萬筆:

Q: How many certificates are affected?
A: 2.6%. That is 3,048,289 currently-valid certificates are affected, out of ~116 million overall active Let’s Encrypt certificates. Of the affected certificates, about 1 million are duplicates of other affected certificates, in the sense of covering the same set of domain names.

在「Check whether a host's certificate needs replacement」這邊可以偵測線上使用的 SSL certificate 是否受到影響。

另外在「Download affected certificate serials for 2020.02.29 CAA Rechecking Incident」這邊可以抓到所有受到影響,預定要 revoke 的 SSL certificate 的序號。關於取得序號的方式,官方也有提供 CLI 的指令可以操作確認,對於有很多網域名稱需要確認的人可以用這組指令編寫程式判斷:

openssl s_client -connect example.com:443 -servername example.com -showcerts </dev/null 2>/dev/null | openssl x509 -text -noout | grep -A 1 Serial\ Number | tr -d :

照目前的描述,如果申請時只有一個 domain 應該是不會中這個問題,再來是最壞的情況大概會維持三個月 (網站主人沒管他,等到時間到了自動 renew)。

Let's Encrypt 的 ACME v1/v2 透過多個節點認證

從 2020/02/19 開始,Let's Encrypt 的 ACME v1/v2 都會透過多個節點認證:「ACME v1/v2: Validating challenges from multiple network vantage points」。

從多個節點認證可以降低路由被劫持後被拿來產生 SSL Certificate 的風險,看起來會有三個機房參與,需要兩個確認:

After Feb 19th we will make four total validation requests (1 from a primary datacentre, and 3 from remote datacentres). The primary request and at least 2 of the 3 remote requests must receive the correct challenge response value for the domain to be considered authorized.

也如同標題寫的,包括了現有的 v2 與將要淘汰的 v1。

讓 pfSense 的管理界面接上 Let's Encrypt

其實網路上有一些教學,但大多數的作法都是把 port 80 空出來跑認證用的 HTTP 伺服器,而我這邊的作法是希望維持 nginx,用 webroot local folder 的方式認證。

pfSense 上先安裝 acme 套件,然後先用 Create new account key 產生出自己的金鑰:

然後選擇申請 Let's Encrypt Production ACME V2 的帳號:

接著填一下 Name 的部份,按下 Register ACME account key 註冊後就可以按下 Save 存進系統。

然後就是拿這個帳號申請網域名稱,重點在於,如果選擇 webroot local folder 時,需要填寫對應的 RootFolder 參數 (這塊做得很怪,沒有預設值 XD):

然後按下 Save 後存起來,回到頁面上就可以按 Issue/Renew 申請了,申請成功以後就可以到最上方 tab 的 System,在裡面的 Advanced 就可以改 HTTPS 管理界面使用的憑證,改好後把瀏覽器關掉重開就可以確認是不是有設定好...