Backblaze 在 2020 年對機械硬碟的回顧

前幾天 Backblaze 放了 2020 年的回顧資料出來:「Backblaze Hard Drive Stats for 2020」。

整體的 AFR (Annualized Failure Rate) 在 0.93% 左右,而如果照品牌拆開,HGST 的數字依然是最漂亮的 (雖然他現在是 WD 的品牌),大約在 0.36% 左右 (111/(1083774+4663049+372000+820272+275779+3968475)),Toshiba 次之,大約低了平均值一些落在 0.89%,而 Seagate 光是看就就知道會超過 1%...

官方有提到,低於 250,000 drive days 以下的數據僅供參考,因為資料量太少,在統計上無法提供結論:

For drives which have less than 250,000 drive days, any conclusions about drive failure rates are not justified. There is not enough data over the year-long period to reach any conclusions. We present the models with less than 250,000 drive days for completeness only.

然後 WD 本家的硬碟回到戰線了,記得之前基本上算是被唾棄 XDDD

另外一張表則是講到這三年的情況,可以看出來 2020 年的 AFR 數字降了不少,裡面也解釋了為什麼 (看起來就是活下來的穩下來了...):

The answer: It was a group effort. To start, the older drives: 4TB, 6TB, 8TB, and 10TB drives as a group were significantly better in 2020, decreasing from a 1.35% AFR in 2019 to a 0.96% AFR in 2020. At the other end of the size spectrum, we added over 30,000 larger drives: 14TB, 16TB, and 18TB, which as a group recorded an AFR of 0.89% for 2020. Finally, the 12TB drives as a group had a 2020 AFR of 0.98%. In other words, whether a drive was old or new, or big or small, they performed well in our environment in 2020.

Backblaze 與 Fastly 合作

Twitter 上看到的消息,BackblazeFastly,免除 Origin 到 CDN 這段的流量費用:

新聞稿是「Set Your Content Free With Fastly and Backblaze B2」這篇:

Our new collaboration with Fastly, a global edge cloud platform and CDN, offers an integrated solution that will let you store and serve rich media files seamlessly, free from the lock-in fees and functionality of closed “goliath” cloud storage platforms, and all with free egress from Backblaze B2 Cloud Storage to Fastly.

不過這不是第一家提供這樣的方案,在 2018 年的時候就有跟 Cloudflare 合作,免除了 Origin 端到 CDN 端這段費用:「Backblaze 與 Cloudflare 合作,免除傳輸費用」。

Backblaze 的 2020Q2 硬碟報告

在「Backblaze Hard Drive Stats Q2 2020」這邊又有資料可以看啦,主要是這張表:

比較讓我注意到的是,其中有個一千台的 HUH728080ALE600,AFR 居然是 0.00%,仔細看算了一下發現應該是弄來一批貨,上線約三個月 (91 天),而目前還沒有壞掉而已...

不過 HUH728080ALE600 這個料號很有趣啊,在搜尋的時候發現 Ptt 上 2018 年的文章「Re: [請益] 關於HUH728080ALE604 這顆8t硬碟」這邊有提到這個號碼,裡面有猜測這個料號的來源:

查了一下發現有趣的事實
國外資料都指出,OEM的原廠HGST沒有給予保固
且驗證序號會是無效的序號
但中國地區驗證 OEM序號卻是有效
所以合理懷疑HGST將生產過剩的 OEM產品轉到中國銷售?
並將這些 OEM序號登錄至HGST資料庫內
當然這些標籤都是貼HGST的, 貼DELL或HP標的都是查無有效序號
而且這些都是保五年, 保固低於五年可能就是有問題的
HUH728080ALE600 是目前有看到在中國銷售的
當然這都是沒有外盒包裝

不知道 Backblaze 是怎麼弄到這批貨的...

不過就算不管這批貨,HGST 整體上看起來還是很不錯,不過現在掛 HGST 的應該都是庫存了。

Backblaze B2 支援相容 Amazon S3 的 API

Backblaze 宣佈支援相容 Amazon S3 的 API:「Backblaze B2 Cloud Storage Now Has S3 Compatible APIs」。

Amazon S3 的 API 算是 object storage 這個領域的 de facto standard 了,支援 Amazon S3 相容層可以讓現有的工具直接套用上去。

很多 client 軟體都藉著設定 API endpoint 的方式來支援 (通常預設會是 Amazon S3 的 s3.amazonaws.com),這次的 endpoint 可以從 B2 的文件「S3 Compatible API」裡看到:

The format for endpoints for the Backblaze S3 Compatible API:

https://s3.<region>.backblazeb2.com

The Backblaze S3 Compatible API endpoints only accept connections over HTTPS. Non-secure connections will be rejected. The AWS SDKs and most integrations only require an Endpoint URL like the above (without the bucket name included).

另外也支援使用 bucket name 的形式操作:

If making the HTTP calls directly, the Backblaze S3 Compatible API supports specifying the bucket name in the hostname of the URL or in the path section of the URL. Both URLs below are valid examples of an endpoint calling a bucket:

https://bucketname.s3.us-west-001.backblazeb2.com

https://s3.us-west-001.backblazeb2.com/bucketname

B2 的另外一個優勢是 2018 的時候就跟 Cloudflare 合作 (參考「Backblaze 與 Cloudflare 合作,免除傳輸費用」),從 B2 到 Cloudflare 的流量是不收費的,再加上 Cloudflare 的流量也可以是免費的,組合起來就變成一個很便宜的方案 (只有 B2 的 storage cost)。

Backblaze 的 2019 年度硬碟報告

Backblaze 丟出去年的報告了:「Backblaze Hard Drive Stats for 2019」。

WD/HGST 的還是最耐用,再來是 Toshiba 的,最後是 Seagate 的。

不過有一些硬碟沒有列到表上,像是「Seagate 16 TB Drives」這組因為 2019Q4 才剛裝上去,所以才 1440 drive days,因此還沒到門檻所以沒放進報告,但就 Backblaze 測試起來看起來是個好的開始:

In Q4 2019 we started qualifying Seagate 16 TB drives, model: ST16000NM001G. As of the end of Q4 we had 40 (forty) drives in operation, with a total of 1,440 drive days—well below our 5,000 drive day threshold for Q4, so they didn’t make the 2019 chart. There have been 0 (zero) failures through Q4, making the AFR 0%, a good start for any drive. Assuming they continue to pass our drive qualification process, they will be used in the 12 TB migration project and to add capacity as needed in 2020.

再來是把 2017/2018/2019 擺在一起看:

馬上可以看到的是 AFR 上升了不少,一個是因為 8TB 系列的硬碟進入中年期,另外是 Seagate 12TB 硬碟的問題:

The total AFR for 2019 rose significantly in 2019. About 75% of the different drive models experienced a rise in AFR from 2018 to 2019. There are two primary drivers behind this rise. First, the 8 TB drives as a group seem to be having a mid-life crisis as they get older, with each model exhibiting their highest failure rates recorded. While none of the rates is cause for worry, they contribute roughly one fourth (1/4) of the drive days to the total, so any rise in their failure rate will affect the total. The second factor is the Seagate 12 TB drives, this issue is being aggressively addressed by the 12 TB migration project reported on previously.

所以大原則還是跟以前差不多,沒有時間特別研究的話就先往 WD/HGST 這邊找...

Backblaze 採購硬碟的策略

在「How Backblaze Buys Hard Drives」這篇裡面提到了 Backblaze 採購硬碟的策略,可以看到完全都是偏成本走向,所以裡面的策略一般個人用不太到,一般企業也不應該照抄,但拿來看看還蠻有趣的...

像是因為硬碟太多,所以硬碟的使用電量是他們在評估成本時蠻重要的一環,這點在一般的情境下不太會考慮到:

Power draw is a very important metric for us and the high speed enterprise drives are expensive in terms of power cost. We now total around 1.5 megawatts in power consumption in our centers, and I can tell you that every watt matters for reducing costs.

另外也提到了 SMR 硬碟的特性,在單位成本雖然有比較高的容量,但導致架構面需要配合 (cache),而也會有工程端的成本提昇,所以不是很愛:

SMR would give us a 10-15% capacity-to-dollar boost, but it also requires host-level management of sequential data writing. Additionally, the new archive type of drives require a flash-based caching layer. Both of these requirements would mean significant increases in engineering resources to support and thereby even more investment. So all-in-all, SMR isn’t cost-effective in our system.

成本面上,他們觀察到的現象是每季會降 5%~10%:

Ideally, I can achieve a 5-10% cost reduction per terabyte per quarter, which is a number based on historical price trends and our performance for the past 10 years.

另外提到了用 SAS controller 可以接多個 SATA 硬碟的事情 (雖然還是成本考量),但這塊也蠻有趣的:

Longer term, one thing we’re looking toward is phasing out SATA controller/port multiplier combo. This might be more technical than some of our readers want to go, but: SAS controllers are a more commonly used method in dense storage servers. Using SATA drives with SAS controllers can provide as much as a 2x improvement in system throughput vs SATA, which is important to me, even though serial ATA (SATA) port multipliers are slightly less expensive. When we started our Storage Pod construction, using SATA controller/port multiplier combo was a great way to keep costs down. But since then, the cost for using SAS controllers and backplanes has come down significantly.

Backblaze 開了歐洲區機房

Backblaze 開了歐洲機房,所以包括了一般性的 Computer BackupB2 Cloud Storage 都可以選擇要放哪邊了...

歐洲的點是放在荷蘭:

Big news: Our first European data center, in Amsterdam, is open and accepting customer data!

價錢也都跟美國的相同:

Whether you choose EU Central or US West, your pricing for our products will be unchanged:

對於在意資料放美國機房的問題應該有緩解一些...

Backblaze B2 的 Copy File API 終於開放

BackblazeB2 算是我還蠻愛用來丟一些東西的地方 (配合他們與 Cloudflare 合作的免費頻寬)。

先前 B2 一直沒有複製檔案的功能,如果要有同樣檔案,變成得自己再上傳一次,這對於網路沒有很快的使用者會很痛苦,現在總算是提供 API 可以直接複製了:「B2 Copy File is Now Public」。

這個功能主要的文件在「b2_copy_file」。

另外這次也推出了「b2_copy_part」,針對檔案的合併所提供的 API。

Backblaze 與 Cloudflare 合作,免除傳輸費用

先前知道不少單位會選擇用 CloudFront 的原因就是 S3 到 CloudFront 這段是不需要傳輸費用的。畢竟 CDN 的 hit rate 還是有限,用其他家 CDN 得付這塊費用。

而現在 Backblaze 宣佈跟 Cloudflare 合作,免除掉 Backblaze 到 Cloudflare 的費用:「Backblaze and Cloudflare Partner to Provide Free Data Transfer」。

Today we are announcing that beginning immediately, Backblaze B2 customers will be able to download data stored in B2 to Cloudflare for zero transfer fees.

AWS 這邊會不會有其他動作呢...

Backblaze 在 2018 Q2 的硬碟故障率報告

Backblaze 照慣例發表了 2018 Q2 的硬碟狀況:「Hard Drive Stats for Q2 2018」。可以看出來他們基本上只用 SeagateHGST 了,其他的應該是嘗試性質:

第二張因為把信心區間標出來,會更清楚:

WD 差了一大截,所以現在消費級的硬碟會選 Seagate 或是 HGST?