musl 的 DNS reolsver 支援 TCP fallback

Facebook 上看到這篇:

剛好想起前陣子在 Hacker News 上看到「Musl 1.2.4 adds TCP DNS fallback (openwall.com)」這個消息,裡面的連結是今年五月 musl 1.2.4 的出版公告:「musl 1.2.4 released」(話說 openwall 網站似乎有擋 HiNet 的 IP,我是走第四台網路看的,或是參考 Internet Archive 上面的連結)。

musl 1.2.4 一個很重要的改變是在 DNS resolver 上支援了 TCP fallback (也就是支援 DNS over TCP),這改善了長久以來在 container 裡面使用 Alpine Linux 偶而會因為 DNS 遇到沒有照標準做的 server 而中雷的問題:

This release adds TCP fallback to the DNS stub resolver, fixing the longstanding inability to query large DNS records and incompatibility with recursive nameservers that don't give partial results in truncated UDP responses. It also makes a number of other bug fixes and improvements in DNS and related functionality, including making both the modern and legacy API results differentiate between NODATA and NxDomain conditions so that the caller can handle them differently.

查了一下對應的標準,跑去問 ChatGPT 的 GPT-4:

但 ChatGPT 引用的東西都不能直接當作是實際的文字,只能當作一個起點去找。實際翻 RFC 1035 可以翻到:

Messages carried by UDP are restricted to 512 bytes (not counting the IP or UDP headers). Longer messages are truncated and the TC bit is set in the header.

所以的確在 UDP response 的規範是 512 bytes,要取得完整的資料只能往 TCP 查詢。而 musl 有了這個 TCP fallback 總算是補掉了 Alpine Linux 的一個大坑。

而 musl 1.2.4 則是在 Alpine Linux 3.18 才開始使用:「Alpine 3.18.0 released」。

musl libc 1.2.4 – now with TCP fallback in DNS resolver

所以回到開頭,提到 Alpine Linux 3.16 還是有問題的人,是應該會遇到問題沒錯,因為 3.16 的 musl 本來就沒 TCP fallback?遇到不標準的 DNS server 的確是會噴...

Anyway,Alpine Linux 的 DNS 問題應該會變成過去式...

省頻寬的方法:終極版本...

看到「Three ways to reduce the costs of your HTTP(S) API on AWS」這邊介紹在 AWS 上省頻寬費用的方法,看了只能一直笑 XD

第一個是降低 HTTP response 裡沒有用到的 header,因為每天有五十億個 HTTP request,所以只要省 1byte 就是省下 USD$0.25/day:

Since we would send this five billion times per day, every byte we could shave off would save five gigabytes of outgoing data, for a saving of 25 cents per day per byte removed.

然後調了一些參數後省下 USD$1,500/month:

Sending 109 bytes instead of 333 means saving $56 per day, or a bit over $1,500 per month.

第二個是想辦法在 TLS 這邊下手,一開始其中一個方向是利用 TLS session resumption 降低第二次連線的成本,但他們發現沒有什麼參數可以調整:

One thing that reduces handshake transfer size is TLS session resumption. Basically, when a client connects to the service for the second time, it can ask the server to resume the previous TLS session instead of starting a new one, meaning that it doesn’t have to send the certificate again. By looking at access logs, we found that 11% of requests were using a reused TLS session. However, we have a very diverse set of clients that we don’t have much control over, and we also couldn’t find any settings for the AWS Application Load Balancer for session cache size or similar, so there isn’t really anything we can do to affect this.

所以改成把 idle 時間拉長 (避免重新連線):

That leaves reducing the number of handshakes required by reducing the number of connections that the clients need to establish. The default setting for AWS load balancers is to close idle connections after 60 seconds, but it seems to be beneficial to raise this to 10 minutes. This reduced data transfer costs by an additional 8%.

再來是 AWS 本身發的 SSL certification 太肥,所以他們換成 DigiCert 發的,大幅降低憑證本身的大小,反而省下 USD$200/day:

So given that the clients establish approximately two billion connections per day, we’d expect to save four terabytes of outgoing data every day. The actual savings were closer to three terabytes, but this still reduced data transfer costs for a typical day by almost $200.

這些方法真的是頗有趣的 XDDD

不過這些方法也是在想辦法壓榨降低與 client 之間的傳輸量啦,比起成本來說反而是提昇網路反應速度...

RFC8482 廢掉 DNS 查詢的 ANY query 了...

看到 Cloudflare 的「RFC8482 - Saying goodbye to ANY」這篇,裡面提到 RFC8482 廢掉了 ANY 查詢:「Providing Minimal-Sized Responses to DNS Queries That Have QTYPE=ANY」。

The Domain Name System (DNS) specifies a query type (QTYPE) "ANY". The operator of an authoritative DNS server might choose not to respond to such queries for reasons of local policy, motivated by security, performance, or other reasons.

對 Cloudflare 的痛點主要在於營運上的困難,因為 ANY 回應的 UDP packet size 很大,很容易造成放大攻擊:

把拒絕 ANY 查詢變成標準後,讓 DNS provider 手上多了一把武器可以用。

Amazon API Gateway 支援壓縮了...

Amazon API Gateway 支援壓縮了:「Amazon API Gateway Supports Content Encoding for API Responses」。

You can now enable content encoding support for API Responses in Amazon API Gateway. Content encoding allows API clients to request content to be compressed before being sent back in the response to an API request. This reduces the amount of data that is sent from API Gateway to API clients and decreases the time it takes to transfer the data. You can enable content encoding in the API definition. You can also set the minimum response size that triggers compression. By default, APIs do not have content encoding support enabled.

打開後傳回的資料就會自動壓縮了,然後還可以設定觸發的 response size... 依照文件 (Content Codings Supported by API Gateway),目前支援的壓縮格式應該是最常見的 gzipdeflate

這功能好像是一開始有 API Gateway 就一直被提出來的 feature request...

Mozilla 的提案「HTTP Immutable Responses」

狀態已經是 Category: Standards Track 了,RFC 8246 的「HTTP Immutable Responses」:

The immutable HTTP response Cache-Control extension allows servers to identify resources that will not be updated during their freshness lifetime. This ensures that a client never needs to revalidate a cached fresh resource to be certain it has not been modified.

Cache-Control 介紹了 immutable,像是這樣:

Cache-Control: max-age=31536000, immutable

依照 MDN 上的資料 (Cache-Control - HTTP | MDN),目前只有 EdgeFirefox 支援,不過既然成為標準了,後續其他瀏覽器應該都會支援 (吧):

Lambda@Edge 的 GA

AWSLambda@Edge 宣佈 GA 了:「Lambda@Edge – Intelligent Processing of HTTP Requests at the Edge」。

最直接的應用就是在 CloudFront 的 edge 上執行一小段 code,修改 HTTP request 或是 HTTP response 了,不過可以看到一些限制:

不過要用來解哪些問題要再想一下...

HTTP Header 裡的 Location 使用相對路徑...

HTTP Response Header 的 Location (俗稱「轉址」) 被用在不少地方,剛好今天被 ccn 戳到相關的問題...

在維基百科的「HTTP location」條目裡面有說明 HTTP/1.1 的規範裡要求必須是 absolute URI:

Location       = "Location" ":" absoluteURI

但實務上,目前市場上常見的瀏覽器都支援相對路徑。而且在 HTTP/1.1 修正版 (目前還在 draft) 裡被修正成:

Location = URI-reference

並且說明 relative 時的判定方式:

The field value consists of a single URI-reference. When it has theform of a relative reference ([RFC3986], Section 4.2), the final value is computed by resolving it against the effective request URI ([RFC3986], Section 5).

所以就大膽用吧...

429 Too Many Requests

剛剛在「Apache HTTP ServerをRFC6585の"429 Too Many Requests"に(とりあえず)対応させるパッチ」看到的... 429 Too Many Requests 是在四月才被提出來的 (RFC 6585),雖然還不是 Standard (目前是 Proposed Standard),但看起來不錯啊...

一定要放一張經典畫面...