Cloudflare Workers 推出新的計費模式:以 CPU time 收費

CloudflareWorkers 這個產品 (serverless 產品線) 推出了以 CPU time 收費的模式:「New Workers pricing — never pay to wait on I/O again」。

在這之前大家都是以 wall time 在計費,但這對於會卡在 I/O 很久的應用來說很不利,這次 Cloudflare 提出方案改用 CPU time 來計費,的確有吸引到我的目光...

這是舊的:

這是新的:

就這兩張比較起來有個不是很確定的部分,現在不看 memory 用量收費了?Pricing 這頁裡面的資量已經把 「Standard」更新上去了,但好像還是沒提到 memory?

不過不是馬上生效,而是這個月的月底 2023/10/31 可以選擇切到新的方案:

Starting October 31, 2023, you will have the option to opt in individual Workers and Pages Functions projects on your account to new pricing, and newly created projects will default to new pricing. You’ll be able to estimate how much new pricing will cost in the Cloudflare dashboard. For the majority of current applications, new pricing is the same or less expensive than the previous Bundled and Unbound pricing plans.

另外明年 2024/03/01 會全部強制切到新的方案:

If you’re on our Workers Paid plan, you will have until March 1, 2024 to switch to the new pricing on your own, after which all of your projects will be automatically migrated to new pricing. If you’re an Enterprise customer, any contract renewals after March 1, 2024, will use the new pricing.

用 CPU time 的確是好不少,但不知道這個 billing 的方式沒有其他地雷...

Cloudflare 正式推出 ECH (Encrypted Client Hello)

Cloudflare 宣佈所有的 Plan 都支援 ECH 了:「Encrypted Client Hello - the last puzzle piece to privacy」。

要講 ECH 之前,會先提到 Domain fronting 這個避開網路審查 (censorship) 的技巧,這是利用 CDN 服務裡對 SNI 的處理與對 HTTP header 的處理分開的特性,算是一種 side effect 的搞法,不保證會動,但蠻多 CDN 的服務都會動。

實際上的作法就是在 SNI 層的 hostname 填寫一個沒有被審查的 Hostname (像是 www.akamai.com),然後連到 www.akamai.com:443 上,也都使用 www.akamai.com 的 SSL certificate 交換。

但在 HTTP header 上面則是送出不一樣的 hostname (但也必須是在這家 CDN 的機器上有提供服務的網域),通常是被管制的網域名稱,像是 Host: www.cnbc.com 這樣的 HTTP header;而因為這部份被 TLS 保護,無法從外部看到內容而可以穿過網路審查。

理論上 CDN 服務是可以 reject 這樣的連線的,但大多數的 CDN 廠商並沒有阻擋這麼嚴格,於是算是一種堪用的方式。而這次 ECH 算是把這個行為標準化了,就不需要再用 side effect 的方式了。

Cloudflare 本身就很大,可以來看看後續的效應...

GCP 提供每個區域的 Standard Tier Networking 每個月 200GB 的免費頻寬

GCP 提供每個區域 Standard Tier Networking 每個月有 200GB 的免費頻寬:「Announcing 200 GB free Standard Tier internet data transfer per month」。

這類優惠應該是要給練習或是試用的人用的,吸引他們放一些量上來...

另外雖然是每個區域都有 200GB/mo 的 free bandwidth,但通常也只會用一個或兩個區域,以台灣的 asia-east1 來說,如果有量在上面跑的話,省下來的會是 $0.07/GB 這個部分,大概就是 $14/mo 左右?

Cavium (被 Marvell 併購) 在 Snowden leak 中被列為 SIGINT "enabled" vendor

標題可能會有點難懂,比較簡單的意思就是在 Snowden 當年 (2013) 洩漏的資料裡面發現了不太妙的東西,發現 Cavium (現在的 Marvell) 的 CPU 有可能被埋入後門,而他們家的產品被一堆廠商提供的「資安產品」使用。

出自 X (Twitter) 上面提到的:

這段出可以從 2022 年的「Communication in a world of pervasive surveillance」這份文件裡面找到,就在他寫的 page 71 (PDF 的 page 90) 的 note 21:

While working on documents in the Snowden archive the thesis author learned that an American fabless semiconductor CPU vendor named Cavium is listed as a successful SIGINT "enabled" CPU vendor. By chance this was the same CPU present in the thesis author’s Internet router (UniFi USG3). The entire Snowden archive should be open for academic researchers to better understand more of the history of such behavior.

Ubiquiti 直接中槍...

而另一方面,在 Hacker News 上的討論「Snowden leak: Cavium networking hardware may contain NSA backdoor (twitter.com/matthew_d_green)」就讓人頭更痛了,像是當初 Cavium 就有發過新聞稿提到他們是 AWS CloudHSM 的供應商:「Cavium's LiquidSecurity® HSM Enables Hybrid Cloud Users to Synchronize Keys Between AWS CloudHSM and Private Clouds」。

而使用者也確認有從 log 裡面看到看到 Cavium 的記錄:

Ayup. We use AWS CloudHSM to hold our private signing keys for deploying field upgrades to our hardware. And when we break the CI scripts I see Cavium in the AWS logs.

Now I gotta take this to our security team and figure out what to do.

居然是 CloudHSM 這種在架構上幾乎是放在 root of trust 上的東西...

IPv6 Excuse Bingo (IPv6 理由伯賓果?)

在「AWS IPv4 Estate Now Worth $4.5B (toonk.io)」這邊的討論意外的看到「IPv6 Excuse Bingo」這個網站...

這個 bingo 是動態的,每次 reload 都會有不同的版本出來,理由超多...

不過實際用 IPv6 network 後會發現各種鳥問題真的多,之前 (到現在) 最經典的就是 HECogent 的 IPv6 network 因為錢的問題談不攏而不通的問題,在維基百科上面就有提到從 2009 年開始就不通了:

There is a long-running dispute between the provider Cogent Communications and Hurricane Electric. Cogent has been refusing to peer settlement-free with Hurricane Electric since 2009.

所以夠大的服務如果要弄 IPv6 都得注意到這點,像是 CDN 或是 GeoIP-based load balancer 就不能把 Cogent 的用戶導到 HE 的位置上面,反之亦然。

不過話說 AWS 手上有 128M 個 IPv4 address,整個 IPv4 address 的空間也才 4.29B,也就是說光 AWS 手上就有大約 3% 的 IPv4 address 空間,如果扣掉不可用的區段的話就更高了...

在實體機上跑 K8s 的記錄

Hacker News 上翻到「Bare-Metal Kubernetes, Part I: Talos on Hetzner (datavirke.dk)」,看起來作者試著記錄下實體機器上跑 K8s 的各種事情,系列文章的第一篇在「Bare-metal Kubernetes, Part I: Talos on Hetzner」這邊。

跟朋友聊天的時候會聊到 K8s 就是個 mini AWS,裡面包括了雲端基本該有的 building blocks,對於有自己地端機房的人可以用純軟體的方式把這些東西組出來,不用去買特殊的硬體設備來架設 (像是 F5 的 load balancer)。

另外一種用 K8s 的情境是需要真正的跨環境 (像是 container 可以自由的切來切去),像是地端與雲端的整合,或是雙雲端系統的整合。

但畢竟是自己架設一套 mini AWS,複雜度當然就不是一般架設服務可以比擬的。作者用了八篇把各個角落都帶過一次,還寫了一篇炸掉的處理:「Bare-metal Kubernetes: First Incident」。

一般的情況下應該是不需要用到這個東西,但記錄一下。

EC2-Classic 完全退役

Amazon 家的老大 (CTO & VP) Werner Vogels 貼了關於 EC2-Classic 完全退役的文章:「Farewell EC2-Classic, it’s been swell」。

2021 年的時候 AWS 的 Jeff Barr 宣布了 EC2-Classic 的退役計畫:「EC2-Classic Networking is Retiring – Here’s How to Prepare」,我當時也整理了「AWS 宣佈 EC2-Classic 退役的計畫」這篇。

當時的時間表是期望在 2022/08/15 全部退役:

On August 15, 2022 we expect all migrations to be complete, with no remaining EC2-Classic resources present in any AWS account.

但後來還是晚了整整一年,到 2023/08/15 (剛好晚了一年) 才全部退役:

On August 15, 2023, we shut down the last instance of Classic.

而公告上面的更新則是在 2023/08/23 更新:

Update (August 23, 2023) – The retirement announced in this blog post is now complete. There are no more EC2 instances running with EC2-Classic networking.

因為真的太久沒用了,看了 Werner Vogels 的描述才能回想起來當時的架構,似乎是有一大鍋這件事情,靠 security group 拆開大家:

When we launched EC2 in 2006, it was one giant network of 10.2.0.0/8. All instances ran on a single, flat network shared with other customers. It exposed a handful of features, like security groups and Public IP addresses that were assigned when an instance was spun up.

順便提一下 Werner Vogels 文章的開頭提到了 AWS 很少將服務退役,即使是 2007 年推出的 Amazon SimpleDB 也還是繼續在跑,即使現在主推的是 DynamoDB

Retiring services isn’t something we do at AWS. It’s quite rare. Companies rely on our offerings – their businesses literally live on these services – and it’s something that we take seriously. For example SimpleDB is still around, even though DynamoDB is the “NoSQL” DB of choice for our customers.

用「List of AWS Services Available by Region」這頁查了一下,SimpleDB 的區域意外的還不少,在 us-east-1us-west-1us-west-2ap-southeast-1ap-southeast-2ap-northeast-1eu-west-1 以及 sa-east-1

不過話說這開頭是不是在偷臭隔壁棚 XDDD

Amazon SES 寄到 Gmail 受到阻擋的情況

我自己沒遇過,但是 Hacker News 上看到有人有遇到,所以記錄起來:「Tell HN: Gmail rate limiting emails from AWS SES」。

Amazon SES 預設是共用 IP pool,所以遇到這種情況不算太意外,但應該是暫時性的,不過發問的作者有提到後來的解法是花 US$25/mo 使用 Dedicated IP 解決 IP reputation 的問題 (在 id=37177533 這邊):

Thanks you all for comments. I have made a decision to subscribed to dedicated IPs (credits: @slau).

The differentiating factor between our current AWS SES plan and the competitors (mentioned in the comments) is having a dedicated IP. With our current volume, none of the competitors are anyway near AWS SES costs. So, moving to a dedicated IPs thats cost 25$ extra not only solves our issue, but also no change in code/infrastructure.

記得以前另外一個教訓是,寄信還是儘量用 IPv4 address 去寄,因為 IPv6 address 的 reputation 得養頗久... 不過這個也是很久前的事情了。

AWS 弄出了 AWS Dedicated Local Zones,很像 AWS Outposts...

AWS 推出了 AWS Dedicated Local Zones:「Announcing AWS Dedicated Local Zones」。

先講 AWS Outposts,他就是提供 AWS 自己的硬體,放到用戶的機房裡面,所以依照需求有不同大小的機器,甚至是整個機櫃:

AWS Outposts is a family of fully managed solutions delivering AWS infrastructure and services to virtually any on-premises or edge location for a truly consistent hybrid experience. Outposts solutions allow you to extend and run native AWS services on premises, and is available in a variety of form factors, from 1U and 2U Outposts servers to 42U Outposts racks, and multiple rack deployments.

在「What is AWS Outposts?」這邊有詳細列出有哪些服務可以跑在上面,可以看到主要就是基礎服務,以及一些吃 local 特性的服務。

另外在「How AWS Outposts works」這邊可以看出架構上會在同一個 VPC 裡面,但是不屬於同一個 AZ 下:

而這次推出的 Dedicated Local Zones 還是有些地方沒看懂跟 AWS Outposts 差在哪裡,看起來很像是重新包裝而已...

首先是首頁提到的,這邊有提到 AWS Nitro System,所以猜測這是 AWS 的硬體,而不是自己的硬體:

Build with AWS managed secure cloud infrastructure

Benefit from the same AWS security standards that apply to AWS Regions and AWS Local Zones and are delivered with the security of the AWS Nitro System to help ensure confidentiality and integrity of customer data.

另外在公告裡面提到的服務,跟 Outposts 有些差異:

AWS services, such as Amazon EC2, Amazon Virtual Private Cloud (Amazon VPC), Amazon Elastic Block Store (Amazon EBS), Elastic Load Balancing (ELB), Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS), and AWS Direct Connect are available in Dedicated Local Zones.

另外在「AWS Dedicated Local Zones FAQs」這邊則試著說明兩者差異,但就這些句子看起來,只是不同面向的東西:

Q: How are AWS Dedicated Local Zones different from AWS Outposts?

AWS Outposts is designed for workloads that need to remain on-premises due to latency requirements, where customers want those workloads to run seamlessly with their other workloads in AWS. AWS Outposts racks are fully managed and configurable compute and storage racks built with AWS-designed hardware that allow customers to run compute and storage on-premises, while seamlessly connecting to AWS’s broad array of services in the cloud.

AWS Dedicated Local Zones are designed to eliminate the operational overhead of managing on-premises infrastructure at scale. Some customers have long-term, complex cloud migration projects and need infrastructure that seamlessly scales to support their large-scale demand. Some of these customers represent the interests of a customer community and also need multi-tenancy features to efficiently coordinate across their stakeholders. Dedicated Local Zones enable these customers to reduce the administrative burden of managing their own infrastructure on-premises with scalable, resilient, and multitenant cloud infrastructure that is fully AWS-managed and built exclusively for their use.

另外回到首頁看使用單位,目前是 GovTech Singapore,看起來就是重新包裝?

另外一個猜測是在客戶的機器上面裝 AWS Nitro System,然後裝 AWS 的軟體?這就有點怪了,而且這樣相容性之類的問題也頗麻煩,也許要指定配合的機種?

等有機會遇到的時候再跟 AWS 的人問問看好了,目前也還用不到...

Backblaze 宣佈漲價

Backblaze 宣佈漲價:「Backblaze Product and Pricing Updates」。

其中 B2 Cloud Storage 這邊最主要的改變在 Storage 的部分,這次漲了 20%,從 $5/TB 變成 $6/TB:

Storage Price: Effective October 3, 2023, we are increasing the monthly pay-as-you-go storage rate from $5/TB to $6/TB. The price of B2 Reserve will not change.

頻寬的部分增加了一些 free quota,不過在意頻寬成本的人都會用 Cloudflare 之類的方式避開了,這個其實沒有什麼差... (因為 Backblaze 流出到 Cloudflare 的流量是不計費的)

Backblaze Computer Backup 的部分沒有什麼在碰,但看起來最主要的改變是從現有的 $7/mo 漲到 $9/mo,大約 28.57%:

Computer Backup Pricing: Effective October 3, new purchases and renewals will be $9/month, $99/year, and $189 for two-year subscription plans, and Forever Version History pricing will be $0.006/GB/month.

漲幅其實頗高的,但漲完後還是市場上比較低價的產品...