Amazon S3 改變 403 的收費方式

這算是一連串的故事,首先是四月底的時候「How an empty S3 bucket can make your AWS bill explode」這篇,提到了他一個晚上收到了 US$1,300 的帳單,因為有人 (沒有權限的人) 對他的 S3 bucket 狂打了 100M requests (一億筆),雖然都是 403 的 access denied,但還是得付 request 與頻寬的費用。

對於想要搞的人來說,us-east-1 的 Amazon S3 費用是 $0.005/1K requests (PUT, COPY, POST, LIST requests),換算大一點的單位是 $5/1M requests,拿個 ab 之類的工具超級簡單就可以打出破千 reqs/sec,如果是 k6 之類的工具,其實一台電腦就蠻容易打爆?

作者聯絡 AWS 客服後,客服回答你需要付這筆費用 (「這不是 bug,是 feature」):

Yes, S3 charges for unauthorized requests (4xx) as well[1]. That’s expected behavior.

然後這件事情就在社群傳開了,傳到 Jeff Barr 後直接公開提到他認為客戶不應該付 unauthorized request 的 cost (應該是先跟內部其他高層討論過了),等於是宣佈了會改掉:

不過這件事情之前應該就有人提過了,結果 Colin Percival 直接戳,他在 2006 年 Amazon S3 剛出來的時候就提過了:

Anyway,兩個禮拜過去後,剛剛看到宣佈收費方式修改:「Amazon S3 will no longer charge for several HTTP error codes」。

針對從不屬於自己帳號所產生的 403 不收費 (包括 request 與頻寬費用):

With this change, bucket owners will never incur request or bandwidth charges for requests that return an HTTP 403 (Access Denied) error response if initiated from outside their individual AWS account or AWS Organization.

然後多了一頁「Billing for Amazon S3 error responses」專門說明這件事情,這邊列的比較完整,除了 403 以外也包含了其他的 HTTP response code 是不收費的:

The current page shows a full list of HTTP 3XX and 4XX status codes that won't be billed.

補了一個 18 年的洞...

Cloudflare Zaraz 的新價錢

Cloudflare Zaraz 有點像是 GA4Mixpanel 或是 Amplitude 這種產品:

Cloudflare Zaraz is a solution that developers and marketers use to load third-party tools like Google Analytics 4, Facebook CAPI, TikTok, and others.

剛剛看到 Cloudflare Zaraz 的宣佈的新價錢:「Zaraz launches new pricing」,翻資料的時候發現去年七月的時候也有宣佈過價錢,但當時看起來反彈頗大而暫緩:「Cloudflare Zaraz steps up: general availability and new pricing」。

先看一下去年七月當時宣佈的價錢,這邊用了 Zaraz Loads 這個特別的 term:

If you exceed the free Zaraz Loads allocations, you'll be charged $0.50 for every additional 1,000 Zaraz Loads, but the service will continue to function.

看起來 Zaraz Load 包括了 script loading 以及 pageview event 都各算一次:

A Zaraz Load is counted each time a web page loads the Zaraz script within it, and/or the Pageview trigger is being activated.

這樣的話 1M event 就要 US$500,隔壁棚 Mixpanel 可以在「Pricing - Mixpanel | Product Analytics」這邊試拉,同樣 1M event 只要 $140,難怪當時被幹剿到不行...

這次公佈的價錢則是又用了新的 term,叫做 Zaraz Events,這個看起來就是一般理解的 event:

One of the biggest changes we made was changing the metric we used for pricing Zaraz. One Zaraz Event is an event you’re sending to Zaraz, whether that’s a pageview, a zaraz.track event, or similar.

而價錢直接下殺到 1M event 收 US$5,跟之前的差異巨大:

With the new Zaraz pricing model, every Cloudflare account gets 1,000,000 Zaraz Events per month for free. If your account needs more than that, every additional 1,000,000 Zaraz Events are only $5 USD, with volume discounting available for Enterprise accounts.

只是不知道 Zaraz 可以做到什麼程度,「理論上」會比較陽春,但不知道夠不夠用?

一個害我嗆到的故事... (Netlify 帳單的故事?)

故事本身其實還蠻普通的,只是我的閱讀順序害我嗆到...

首先是在 Hacker Newsbest 頁上看到「Netlify just sent me a $104k bill for a simple static site (reddit.com)」這篇,點進去以後是 Reddit 的「Netlify just sent me a $104K bill for a simple static site」這篇,看了一下作者的敘述,是個用 Netlify 的服務,上面有個 3.44MB 的音檔被針對攻擊,造成 190TB 的流量,以及 $104K 的帳單 (十萬多美金),之後 Netlify 的客服同意這是 DDoS 攻擊,給他 95% discount,也就是還是要付 $5K 左右...

Reddit 下面最高分的回應是:

[–]thankyoufatmember 2262 points 14 hours ago
Don't pay, post the story to Hackernews!

Okay,我想說我就是從 Hacker News 上看到點過來的... 回去看一下好了,結果在 Hacker News 的留言最上方是:

bobfunk 10 hours ago | next [–]

Netlify CEO here.

Our support team has reached out to the user from the thread to let them know they're not getting charged for this.

It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact.

Apologies that this didn't come through in the initial support reply.

然後我剛好在喝茶,就嗆到了...

人家常說 Ptt 的電蝦板 (PC_Shopping) 是全台灣最大的客服中心,遇到各種不公不義的問題貼上去就會解決了... 這點倒是頗像的。

雲端的流量費用

在「Cloud Egress Costs (getdeploying.com)」這邊看到的文章,原文在「Cloud Egress Costs」這邊,主要是整理了表格出來可以快速了解不同雲端的流量費用差異,裡面不是單純 VPS 比較,而是各類的服務都拿出來比,像是 storage 類的以及 CDN 類的都有放進來...

Backblaze 的頻寬費用算法頗有趣,每個月給資料量的三倍大小當作免費頻寬,沒記錯的話因為 Cloudflare 是 Backblaze 的 partner,兩邊的傳輸費用不計費,如果資料是可以公開的,可以透過這個方式接出來;如果真的得走一般的流量輸出,收費是 US$0.01/GB (所以換算後是 US$10/TB)。

三家常被擺在一起的 VPS (LinodeDigitalOceanVultr) 的頻寬也都是 US$10/TB。

以前沒注意到的是 OVH CloudScaleway 的頻寬費用居然是免費的?另外 Hetzner 雖然要收費但也很低?有機會好像該玩看看,看一下品質如何?

IFTTT 將原本的 Legacy Pro 漲價一倍

收到 IFTTT 的通知信件,本來的 Legacy Pro (US$1.99/mo) 要變成 Pro+ 變成 US$3.99/mo:

翻了一下 Hacker News,看起來前幾天有提到:「IFTTT is killing its pay-what-you-want Legacy Pro plan (2023) (techhive.com)」,原文看起來是去年六月的文章:「IFTTT is killing its pay-what-you-want Legacy Pro plan」,當時 IFTTT 的公告則是在「Important Update to the Legacy Pro Plan」這邊。

這看起來不是什麼好消息... 不是因為對 Legacy Pro 動手這件事情,而是這次的動作代表 (i.e. 我的解讀) 沒有足夠的新進用戶去「忽略」掉 Legacy Pro 的成本。

接下來應該是要思考拆出來的事情?畢竟手上有 VPS instance 可以跑...

美國大麥克的價錢

Hacker News 上看到「A site that tracks the price of a Big Mac in every US McDonald's (pantryandlarder.com)」這個,追蹤美國麥當勞裡的大麥克價錢:「McCheapest」。

這個明顯跟「大麥克指數」有關,而且因為美國不同州會有不同的稅務結構,所以可以預期不同地區的麥當勞價位不同,不過沒有預期到同一個州裡面也是各家店自行定價的...

目前網站上列出來最便宜的是奧克拉荷馬州的 US$3.49,最貴的在麻薩諸塞州的 US$8.09,價差到一倍以上... 順便提一下,目前在台灣大麥克的價錢是相同的 NT$75,約 US$2.4。

另外一個感到驚訝的是麥當勞東半部的密度比西半部高好多?還是這張資料沒顯示出來而已?

Twitch 宣佈退出韓國市場

Twitch 宣佈 2024/02/27 (星期二) 退出韓國市場:「An Update on Twitch in Korea」。日期不知道是怎麼選的,可能跟某些合約有關?

Twitch 目前的公告會有繁體中文,也可以看這份:「Twitch 韓國現況更新」。

另外今天早上找了一下,Hacker News 也有討論了:「An update on Twitch in Korea (twitch.tv)」。

目前官方給出來的理由是虧本,而且找不到方法克服虧本的問題:

Ultimately, the cost to operate Twitch in Korea is prohibitively expensive and we have spent significant effort working to reduce these costs so that we could find a way for the Twitch business to remain in Korea.

這邊提到的包括了 p2p model 以及降到 720p,但即使如此網路費用 (應該就是頻寬費用) 是其他區域的十倍以上:

First, we experimented with a peer-to-peer model for source quality. Then, we adjusted source quality to a maximum of 720p. While we have lowered costs from these efforts, our network fees in Korea are still 10 times more expensive than in most other countries. Twitch has been operating in Korea at a significant loss, and unfortunately there is no pathway forward for our business to run more sustainably in that country.

Cloudflare 這邊,2016 年還叫做 CloudFlare 的時候也有抱怨過:「CloudFlare 對 HiNet 成本的抱怨 (還有其他 ISP...)」。

當年是這樣寫 HiNetKT,成本大約是歐美區的 15 倍:

Two Asian locations stand out as being especially expensive: Seoul and Taipei. In these markets, with powerful incumbents (Korea Telecom and HiNet), transit costs 15x as much as in Europe or North America, or 150 units.

而尤其是韓國的部分,政府介入讓降價的速度比全世界慢,所以時間拉長後成本相較於其他地區就貴很多:

South Korea is perhaps the only country in the world where bandwidth costs are going up. This may be driven by new regulations from the Ministry of Science, ICT and Future Planning, which mandate the commercial terms of domestic interconnection, based on predetermined “Tiers” of participating networks. This is contrary to the model in most parts of the world, where networks self-regulate, and often peer without settlement. The government even prescribes the rate at which prices should decrease per year (-7.5%), which is significantly slower than the annual drop in unit bandwidth costs elsewhere in the world. We are only able to peer 2% of our traffic in South Korea.

不過不確定現在的情況,2016 年的 CloudFlare 跟 2023 年的 Cloudflare 已經差了七年了...

AWS 推出 Amazon S3 Express One Zone

AWS 推出了以效能為導向的 Amazon S3 Express One Zone:「Announcing the new Amazon S3 Express One Zone high performance storage class」。

從名字裡的 One Zone 可以看到這是只有在一個 AZ,主打超低 latency:

The new Amazon S3 Express One Zone storage class is designed to deliver up to 10x better performance than the S3 Standard storage class while handling hundreds of thousands of requests per second with consistent single-digit millisecond latency, making it a great fit for your most frequently accessed data and your most demanding applications.

但費用相當貴,以 us-east-1 來看的話是 $0.16/GB/mo,如果拿其他一些 storage 方案來比,可以看到非常大的差距:

  • S3 Standard:$0.023/GB/mo
  • General Purpose SSD (gp3):$0.08/GB/mo
  • General Purpose SSD (gp2):$0.1/GB/mo

可以猜測後面應該全是 NVM 之類的 storage (不過文章裡沒有提到)。

這次的 Amazon S3 Express One Zone 也多出了很多特別的限制。

首先是新的 bucket type,在這個 bucket type 下面 ListObjectsV2 呼叫就必須以 / 結尾 (這暗示後面的資料處理有對這點 optimization),另外傳回的資料不保證順序了:

The path delimiter must be “/“, and any prefixes that you supply to ListObjectsV2 must end with a delimiter. Also, list operations return results without first sorting them, so you cannot do a “start after” retrieval.

另外看起來是在 AZ 裡面直接認證,所以有新的 authentication model:

The new CreateSession function returns a session token that grants access to a specific bucket for five minutes.

然後 bucket naming 因為有後處理,在命名上不需要在整個 AWS 是唯一的 (因為被加料了):

Directory bucket names must be unique within their AWS Region, and must specify an Availability Zone ID in a specially formed suffix. If my base bucket name is jbarr and it exists in Availability Zone use1-az5 (Availability Zone 5 in the US East (N. Virginia) Region) the name that I supply to CreateBucket would be jbarr--use1-az5--x-s3.

另外資料還是可以在同一個 region 下跨 AZ 存取,而且同一個 region 下面的 compute resources (像是 EC2) 不收傳輸費用:

Although the bucket exists within a specific Availability Zone, it is accessible from the other zones in the region, and there are no data transfer charges for requests from compute resources in one Availability Zone to directory buckets in another one in the same region.

費用的部分還有個比較特別的但書,超過 512KB 的 request 會需要額外收費:

You pay an additional per-GB fee for the portion of any request that exceeds 512 KB. For more information, see the Amazon S3 Pricing page.

主要是給自己開發的應用程式用的,現有的 framework 大多都有利用 batch & buffering 的技巧降低 latency 所帶來的效能影響。

平常應該是用不太到,但就有個印象,真的在架構設計上跑不掉的時候有個選擇...

Amazon EFS 漲價,再推出給更「冷」的資料儲存的空間:Amazon EFS Archive

Amazon EFS 這次推出的是再多推出一個 storage class:「Optimize your storage costs for rarely-accessed files with Amazon EFS Archive」。

先前應該是 2019 的時候推出了 IA:「Amazon EFS 的 IA Storage Class」,現在的 Archive 就是新的 storage class,儲存成本更便宜,但取用成本更高。

us-east-1 的價錢來看,可以到 Archive 的成本是 IA 的一半:

Standard (GB-Month)	$0.30
Infrequent Access (GB-Month)	$0.016
Archive (GB-Month)	$0.008
Backup - Warm / Cold (GB-Month)	$0.05 / $0.01

讀取成本則是 IA 的三倍:(這邊的 Tiering 指的是自動化的搬遷的服務)

All storage classes - Reads (per GB transferred)	$0.03
All storage classes - Writes (per GB transferred)	$0.06
Infrequent Access - Reads (incremental charge per GB transferred)	$0.01
Infrequent Access - Tiering (per GB transferred)*	$0.01
Archive - Reads (incremental charge per GB transferred)	$0.03
Archive - Tiering (per GB transferred)*	$0.03

基本上就是 Amazon S3 那套分級方法陸陸續續搬過來的感覺。

然後注意到這個「Regional (Multi-AZ) with Elastic Throughput」是新的計價方案,就算是 Standard storage class,I/O 是要算錢的。

在舊的方案「Regional (Multi-AZ) with legacy throughput modes」裡面,Standard 的 I/O 是不用額外付費,已經包在裡面,除非你直接購買 Provisioned (保證速度):

Standard (GB-Month)	$0.30
Infrequent Acces (GB-Month)	$0.025
Backup - Warm / Cold (GB-Month)	$0.05 / $0.01
Provisioned Throughput (MB/s-Month)	$6.00
Infrequent Access - Reads (per GB transferred)	$0.01
Infrequent Access - Tiering (per GB transferred)*	$0.01

翻了一下 Internet Archive 可以確認前幾天 2023/11/26 的 pricing 頁面還是舊的,也就是說這是這次推出來的改變:「Amazon EFS Pricing」。

看了一下目前 blog 上最近掛 Amazon EFS 類別的三篇都沒提到這件事情 (「New – Announcing Amazon EFS Elastic Throughput」、「Optimize your storage costs for rarely-accessed files with Amazon EFS Archive」以及「Replication failback and increased IOPS are new for Amazon EFS」),要用的人自己注意一下?