ebs – Gea-Suan Lin's BLOG

EBS io1 推出可以同時掛到多台的選項

EBS 的 io1 推出了可以同時掛到 16 台 EC2 instance 的選項：「New – Multi-Attach for Provisioned IOPS (io1) Amazon EBS Volumes」。

先看支援的區域，傳統主力區域 (us-east-1 與 eu-west-1) 都支援了，而亞洲區這邊反倒是南韓先支援了：

Multi-Attach for Provisioned IOPS (io1) volumes on Amazon Elastic Block Store (EBS) is available today at no extra charge to customers in the US East (N. Virginia & Ohio), US West (Oregon), EU (Ireland), and Asia Pacific (Seoul) regions.

其中常用的目的是 HA：

Multi-Attach capability makes it easier to achieve higher availability for applications that provide write ordering to maintain storage consistency.

Heartbeat 類的應用應該可以用上這個東西，不過本來就可以透過 command line API 做到 detach & attach，用這個只是少了一個動作...

第二個想到的是，在實體機房的環境下，有些 filesystem (在「Shared-disk file systems」裡面可以翻到一些) 可以同時掛同一個 block storage (通常是透過 SAN)，現在在 AWS 上面也可以這樣搞了。

不過 io1 記得不便宜啊...

Amazon Elasticsearch Service 可以利用 S3 當作二級儲存空間了

Amazon Elasticsearch Service 的新功能，使用 Amazon S3 當作第二級儲存空間 (UltraWarm)：「Announcing UltraWarm (Preview) for Amazon Elasticsearch Service」。

UltraWarm 需要不同的機器 (跑不同版本？)，機器的規格 (vCPU 與記憶體的比率) 接近 Memory Optimized 的版本，但是貴了不少，所以需要夠大的資料量才會打平回來...

拿 us-east-1 來看，SSD EBS 的空間成本就是 USD$0.135/GB，而傳統磁性硬碟是 USD$0.067/GB (不知道收不收 I/O 費用？)，但 storage 的價錢是 USD$0.024/GB。這邊值得一提的是 Amazon S3 是 USD$0.023/GB，看起來是直接包括了 API 的呼叫費用？

AWS 上用空間買 IOPS 的故事...

在「A web performance issue」這邊講到 Mozilla 的系統產生效能問題，後續的 trouble shooting 以及解決問題的方案。

這個系統跑在 AWS 上，在一連串確認後發現是 RDS 所使用的 EBS 的 IOPS 滿了：

After reading a lot of documentation about Amazon’s RDS set-up I determined that slow downs in the database were related to IOPS spikes. Amazon gives you 3 IOPS per Gb and with a storage of 1 Terabyte we had 3,000 IOPS as our baseline. The graph below shows that at times we would get above that max baseline.

然後大家對於解法都差不多，因為 Provisioned IOPS 太貴，所以直接加大空間換 IOPS 出來 (因為 General SSD 裡 1 GB 給 3 IOPS)：

To increase the IOPS baseline we could either increase the storage size or switch from General SSD to Provisioned IOPS storage. The cost of the different storage type was much higher so we decided to double our storage, thus, doubling our IOPS baseline. You can see in the graph below that we’re constantly above our previous baseline. This change helped Treeherder’s performance a lot.

然後再設警告機制，下次就可以提前再拉昇：

In order to prevent getting into such a state in the future, I also created a CloudWatch alert. We would get alerted if the combined IOPS is greater than 5,700 IOPS for 6 datapoints within 10 minutes.

不過 General SSD 的 IOPS 是沒有 100% 保證的，只有這樣寫：

AWS designs gp2 volumes to deliver 90% of the provisioned performance 99% of the time.

大多數的情況應該是夠用啦...

AWS 的 EBS 預設型態改為 GP2 (SSD)

AWS 宣佈 EBS 的預設型態從 Standard 變成 GP2：「EBS default volume type updated to GP2」。

包括 web console 與 API 的預設值都改成 GP2：

The AWS console defaults to GP2 in all regions. On July 29th the default EBS volume type was updated in thirteen regions from Standard to GP2. Now AWS API calls for volume, image, and instance creation also default to GP2 in all regions.

GP2 是 SSD，所以可以提供比較低的 latency，而另外一個用 GP2 的好處是 i/o 的費用已經含在內了 (Standard 會另外收取費用)，對於成本估算會比較簡單一些，尤其是 i/o 量比較大的時候。

AWS Cloud 的用法

Hacker News Daily 上看到這則，分享了 AWS (他的前東家，超過八年) 的使用經驗：

This is how I use the good parts of @awscloud, while filtering out all the distracting hype.
👇
1/25
— Daniel Vassallo (@dvassallo) July 25, 2019

除了可以在 Twitter 上看以外，也可以用 Thread reader 直接讀整條 thread，應該也還算清楚：「This is how I use the good parts of @awscloud, while filtering out all the distracting hype.」。

這邊的經驗談主要是在 web 與 app 相關的服務這塊：

My experience is in web- sites/apps/services. From tiny personal projects to commercial apps running on 8,000 servers. If what you do is AI, ML, ETL, HPC, DBs, blockchain, or anything significantly different from web apps, what I’m writing here might not be relevant.
3/25
— Daniel Vassallo (@dvassallo) July 25, 2019

有講到 AWS 的業務其實圍繞在 scalability 上發展，但這對 startup 可能反而是扣分，因為暴力法解反而可以大幅簡化架構換得 agile (而讓 startup 存活下來)。

另外從團隊的開發成本來看，這些 scale 的技術增加了開發成本，產生了很多開發上的限制，這些觀點也有點帶到「Premature optimization is the root of all evil」在講的事情：

Step 1: Forget that all these things exist: Microservices, Lambda, API Gateway, Containers, Kubernetes, Docker.
Anything whose main value proposition is about “ability to scale” will likely trade off your “ability to be agile & survive”. That’s rarely a good trade off.
4/25
— Daniel Vassallo (@dvassallo) July 25, 2019

最後的結論可以看到一些列表：

Conclusion: I like to seperate interesting new tech from tech that has survived the test of time. EC2, S3, RDS, DDB, ELB, EBS, SQS definitely have. If you’re considering alternatives, there should be a strong compelling reason for losing all the benefits accrued over time.
25/25
— Daniel Vassallo (@dvassallo) July 25, 2019

除了 DynamoDB 的意見不同外 (這邊提到的 DDB)，其他的我都可以接受...

Amazon EBS 的預設加密機制

EBS 有選項可以預設開加密了：「New – Opt-in to Default Encryption for New EBS Volumes」。

不算太意外的，這個選項要一區一區開：

Per-Region – As noted above, you can opt-in to default encryption on a region-by-region basis.

也不太意外的，無法完全公開共用：

AMI Sharing – As I noted above, we recently gave you the ability to share encrypted AMIs with other AWS accounts. However, you cannot share them publicly, and you should use a separate account to create community AMIs, Marketplace AMIs, and public snapshots. To learn more, read How to Share Encrypted AMIs Across Accounts to Launch Encrypted EC2 Instances.

然後有些服務有自己的 EBS 設定，這次不受影響。而有些底層其實是用 EC2 的服務 (然後開 EBS) 就會直接套用了：

Other AWS Services – AWS services such as Amazon Relational Database Service (RDS) and Amazon WorkSpaces that use EBS for storage perform their own encryption and key management and are not affected by this launch. Services such as Amazon EMR that create volumes within your account will automatically respect the encryption setting, and will use encrypted volumes if the always-encrypt feature is enabled.

AWS 給 EBS 用的 Data Lifecycle Manager 在東京可以用了？

先前在「Amazon EBS Snapshot 支援 Lifecycle Management」這邊提到 AWS 設計了 Data Lifecycle Manager，讓 EBS 磁碟可以自動產生 snapshot 並且管理保留份數，可以當作某種備份機制。

七月公告當時只開放了少數幾區：

Availability – Data Lifecycle Manager is available in the US East (N. Virginia), US West (Oregon), and Europe (Ireland) Regions.

剛剛發現在東京也已經可以用了？但好像沒看到有公告提過... 設下去看看會不會動好了。

Amazon EBS (gp2) 提昇效能...

AWS 宣佈提昇了 Amazon EBS (gp2) 的效能：「Amazon EBS Increases Performance of General Purpose SSD (gp2) Volumes」。

本來上限是 10k IOPS，現在提升到 16k IOPS 了。另外最大傳輸速度也從 160 MB/sec 提升到 250 MB/sec：

Today we are announcing a 60% improvement in performance of General Purpose SSD (gp2) Volumes from 10,000 IOPS to 16,000 IOPS and from 160 MB/s to 250 MB/s of throughput per volume.

應該還是維持 3 IOPS per GB 的設計，但這對於想要用 gp2 堆效能的人來說算是好消息，可以用更大的空間堆出更多 IOPS 了... (像是「Percona 的人接受 AWS 的建議，重新測試了 Percona XtraDB Cluster 在 gp2 上的效能...」這篇)

EC2 支援休眠

AWS 對 EC2 instance 推出了休眠模式：「New – Hibernate Your EC2 Instances」。

休眠模式主要是透過保留記憶體內的內容 (會寫到 EBS 裡)，讓機器開起來比較快，不過這邊有要求 root EBS volume 需要加密：

When an instance is instructed to hibernate, it writes the in-memory state to a file in the root EBS volume and then (in effect) shuts itself down. The AMI used to launch the instance must be encrypted, as must the root EBS volume of the instance. The encryption ensures proper protection for sensitive data when it is copied from memory to the EBS volume.

費用的部分是 EBS volume 與 Elastic IP：

While the instance is in hibernation, you pay only for the EBS volumes and Elastic IP Addresses attached to it; there are no other hourly charges (just like any other stopped instance).

一般對外服務的伺服器好像用不太到... 應該是內部系統或是開發機的需求？

Amazon EBS Snapshot 支援 Lifecycle Management

以往用 Amazon EBS Snapshot 需要用 Lambda 當 cron job 建立 snapshot，以及管理要存的數量 (要刪掉舊的)，現在 AWS 直接提供服務幫你處理：「New – Lifecycle Management for Amazon EBS Snapshots」。文章的截圖就說明了這個新功能：

不收另外的費用，不過目前只有開放三區，東京不在裡面：

You can create and use Data Lifecyle Manager policies at no charge; you pay the usual storage charges for the EBS snapshots that it creates.

Data Lifecycle Manager is available in the US East (N. Virginia), US West (Oregon), and EU (Ireland) Regions.

先繼續用 Lambda，等東京有的時候再換過去...