Backblaze B2 支援相容 Amazon S3 的 API

Backblaze 宣佈支援相容 Amazon S3 的 API:「Backblaze B2 Cloud Storage Now Has S3 Compatible APIs」。

Amazon S3 的 API 算是 object storage 這個領域的 de facto standard 了,支援 Amazon S3 相容層可以讓現有的工具直接套用上去。

很多 client 軟體都藉著設定 API endpoint 的方式來支援 (通常預設會是 Amazon S3 的,這次的 endpoint 可以從 B2 的文件「S3 Compatible API」裡看到:

The format for endpoints for the Backblaze S3 Compatible API:


The Backblaze S3 Compatible API endpoints only accept connections over HTTPS. Non-secure connections will be rejected. The AWS SDKs and most integrations only require an Endpoint URL like the above (without the bucket name included).

另外也支援使用 bucket name 的形式操作:

If making the HTTP calls directly, the Backblaze S3 Compatible API supports specifying the bucket name in the hostname of the URL or in the path section of the URL. Both URLs below are valid examples of an endpoint calling a bucket:

B2 的另外一個優勢是 2018 的時候就跟 Cloudflare 合作 (參考「Backblaze 與 Cloudflare 合作,免除傳輸費用」),從 B2 到 Cloudflare 的流量是不收費的,再加上 Cloudflare 的流量也可以是免費的,組合起來就變成一個很便宜的方案 (只有 B2 的 storage cost)。

Backblaze 的 2019 年度硬碟報告

Backblaze 丟出去年的報告了:「Backblaze Hard Drive Stats for 2019」。

WD/HGST 的還是最耐用,再來是 Toshiba 的,最後是 Seagate 的。

不過有一些硬碟沒有列到表上,像是「Seagate 16 TB Drives」這組因為 2019Q4 才剛裝上去,所以才 1440 drive days,因此還沒到門檻所以沒放進報告,但就 Backblaze 測試起來看起來是個好的開始:

In Q4 2019 we started qualifying Seagate 16 TB drives, model: ST16000NM001G. As of the end of Q4 we had 40 (forty) drives in operation, with a total of 1,440 drive days—well below our 5,000 drive day threshold for Q4, so they didn’t make the 2019 chart. There have been 0 (zero) failures through Q4, making the AFR 0%, a good start for any drive. Assuming they continue to pass our drive qualification process, they will be used in the 12 TB migration project and to add capacity as needed in 2020.

再來是把 2017/2018/2019 擺在一起看:

馬上可以看到的是 AFR 上升了不少,一個是因為 8TB 系列的硬碟進入中年期,另外是 Seagate 12TB 硬碟的問題:

The total AFR for 2019 rose significantly in 2019. About 75% of the different drive models experienced a rise in AFR from 2018 to 2019. There are two primary drivers behind this rise. First, the 8 TB drives as a group seem to be having a mid-life crisis as they get older, with each model exhibiting their highest failure rates recorded. While none of the rates is cause for worry, they contribute roughly one fourth (1/4) of the drive days to the total, so any rise in their failure rate will affect the total. The second factor is the Seagate 12 TB drives, this issue is being aggressively addressed by the 12 TB migration project reported on previously.

所以大原則還是跟以前差不多,沒有時間特別研究的話就先往 WD/HGST 這邊找...

Backblaze 採購硬碟的策略

在「How Backblaze Buys Hard Drives」這篇裡面提到了 Backblaze 採購硬碟的策略,可以看到完全都是偏成本走向,所以裡面的策略一般個人用不太到,一般企業也不應該照抄,但拿來看看還蠻有趣的...


Power draw is a very important metric for us and the high speed enterprise drives are expensive in terms of power cost. We now total around 1.5 megawatts in power consumption in our centers, and I can tell you that every watt matters for reducing costs.

另外也提到了 SMR 硬碟的特性,在單位成本雖然有比較高的容量,但導致架構面需要配合 (cache),而也會有工程端的成本提昇,所以不是很愛:

SMR would give us a 10-15% capacity-to-dollar boost, but it also requires host-level management of sequential data writing. Additionally, the new archive type of drives require a flash-based caching layer. Both of these requirements would mean significant increases in engineering resources to support and thereby even more investment. So all-in-all, SMR isn’t cost-effective in our system.

成本面上,他們觀察到的現象是每季會降 5%~10%:

Ideally, I can achieve a 5-10% cost reduction per terabyte per quarter, which is a number based on historical price trends and our performance for the past 10 years.

另外提到了用 SAS controller 可以接多個 SATA 硬碟的事情 (雖然還是成本考量),但這塊也蠻有趣的:

Longer term, one thing we’re looking toward is phasing out SATA controller/port multiplier combo. This might be more technical than some of our readers want to go, but: SAS controllers are a more commonly used method in dense storage servers. Using SATA drives with SAS controllers can provide as much as a 2x improvement in system throughput vs SATA, which is important to me, even though serial ATA (SATA) port multipliers are slightly less expensive. When we started our Storage Pod construction, using SATA controller/port multiplier combo was a great way to keep costs down. But since then, the cost for using SAS controllers and backplanes has come down significantly.

Backblaze 開了歐洲區機房

Backblaze 開了歐洲機房,所以包括了一般性的 Computer BackupB2 Cloud Storage 都可以選擇要放哪邊了...


Big news: Our first European data center, in Amsterdam, is open and accepting customer data!


Whether you choose EU Central or US West, your pricing for our products will be unchanged:


Backblaze B2 的 Copy File API 終於開放

BackblazeB2 算是我還蠻愛用來丟一些東西的地方 (配合他們與 Cloudflare 合作的免費頻寬)。

先前 B2 一直沒有複製檔案的功能,如果要有同樣檔案,變成得自己再上傳一次,這對於網路沒有很快的使用者會很痛苦,現在總算是提供 API 可以直接複製了:「B2 Copy File is Now Public」。


另外這次也推出了「b2_copy_part」,針對檔案的合併所提供的 API。

Backblaze 與 Cloudflare 合作,免除傳輸費用

先前知道不少單位會選擇用 CloudFront 的原因就是 S3 到 CloudFront 這段是不需要傳輸費用的。畢竟 CDN 的 hit rate 還是有限,用其他家 CDN 得付這塊費用。

而現在 Backblaze 宣佈跟 Cloudflare 合作,免除掉 Backblaze 到 Cloudflare 的費用:「Backblaze and Cloudflare Partner to Provide Free Data Transfer」。

Today we are announcing that beginning immediately, Backblaze B2 customers will be able to download data stored in B2 to Cloudflare for zero transfer fees.

AWS 這邊會不會有其他動作呢...

Backblaze 在 2018 Q2 的硬碟故障率報告

Backblaze 照慣例發表了 2018 Q2 的硬碟狀況:「Hard Drive Stats for Q2 2018」。可以看出來他們基本上只用 SeagateHGST 了,其他的應該是嘗試性質:


WD 差了一大截,所以現在消費級的硬碟會選 Seagate 或是 HGST?

Backblaze 的 2017 年硬碟年度報告

Backblaze 照慣例發表了 2017Q4 與 2017 全年的硬碟報告出來了:「Backblaze Hard Drive Stats for 2017」。

最重要就這三張圖表,第一張是 2017Q4 資料,第二張是從 2013/04 到 2017/12 的資料,第三張是這三年的資料 (2015/2016/2017):

我先說一下結論,因為這幾年幾乎都只採購 SeagateHGST 的硬碟,所以要用他們的資料判斷 WDToshiba 的硬碟已經沒有價值了。

唯一有價值的資料是 HGST 的硬碟比 Seagate 好不少,要做出其他結論的樣本數都不夠。

Backblaze 的資料量

Backblaze 在「Yes, Backblaze Just Ordered 100 Petabytes of Hard Drives」這篇提到他們這一次買了 100PB 的硬碟,不過這應該還好 XD


破 400PB 了... 而且看起來成長速度頗穩定的。

我也用 rclone 把自己的資料往上面備份 (先加密再傳上去),目前連 free tier 都還沒用完 (雖然信用卡卡號掛著)。

Backblaze 2017Q1 對硬碟的分析

Backblaze 放出 2017Q1 對硬碟的分析資料:「Hard Drive Stats for Q1 2017」。

相較於之前的報告 (Backblaze Hard Drive Stats for 2016),這次則是把硬碟數量考慮進去,做了一份有正負誤差的:

最近的趨勢沒什麼變,整體上來看 HGST 的品質還是最好的。