Home » Posts tagged "instance" (Page 2)

Percona 分析在 AWS 上跑 Percona XtraDB Cluster 的效能 (I/O bound)

Percona 的人分析了在 Amazon EC2 上跑 Percona XtraDB Cluster (PXC) 效能 (I/O bound):「Best Practices for Percona XtraDB Cluster on AWS」。


直接跳到結論的地方。如果資料可以掉,用 i3 本地 storage 的效能是最好的,如果要資料不能掉,用 EBS 的 Provisioned IOPS SSD (io1) 的效能會比 General Purpose (gp2) 好很多。

另外 instance type 的選擇上,避免用 {i3,r4}.large,因為測試出來發現 {i3,r4}.xlarge 的效能好不只一倍。

不過 Aurora 的 Multi-master 已經在 Preview 了啊,如果 Percona 的人拿到帳號的話,應該會有單位成本的效能比較可以看...

Amazon EC2 可以設定 Spread Placement Group 要求打散機器

Amazon EC2 推出的新設計 Spread Placement Group,用來打散 instance:「Introducing Spread Placement Groups for Amazon EC2」。

本來的 Cluster Placement Group 是將機器集中在一起,對於 latency 極為敏感的應用會有幫助 (像是下面有提到 HPC 應用);這次推出的 Spread Placement Group 則是要求跑在不同機器上,打散降低風險:

Spread placement groups help reduce the likelihood of failures within clusters or groups of instances. Amazon EC2 has had cluster placement groups, which enable applications to get the low-latency network performance necessary for tightly-coupled node-to-node communication typical of many HPC applications. Now with spread placement groups, member instances will be placed on distinct hardware, reducing the impact of hardware failures on your applications.


Spread placement groups are available in all AWS regions. To get started, visit the AWS Management Console, AWS Command Line Interface (CLI), and AWS SDKs.

Amazon Aurora 的 Serverless 與 Multi-master

Amazon Aurora 推出了兩包玩意,第一包是 Serverless,讓需要人介入的情況更少:「In The Works – Amazon Aurora Serverless」。

在 Serverless 的第一個重點是支援以秒計費:

Today we are launching a preview (sign up now) of Amazon Aurora Serverless. Designed for workloads that are highly variable and subject to rapid change, this new configuration allows you to pay for the database resources you use, on a second-by-second basis.

然後是極為快速的 auto-scaling:

The endpoint is a simple proxy that routes your queries to a rapidly scaled fleet of database resources. This allows your connections to remain intact even as scaling operations take place behind the scenes. Scaling is rapid, with new resources coming online within 5 seconds

這兩個組合起來,讓使用端可以除了在 Amazon EC2 上可以快速 scale 外,後端的資料庫也能 scale 了...

第二個是 Multi-master 架構:「Sign Up for the Preview of Amazon Aurora Multi-Master」。

Amazon Aurora Multi-Master allows you to create multiple read/write master instances across multiple Availability Zones. This enables applications to read and write data to multiple database instances in a cluster, just as you can read across Read Replicas today.

(話說我一直都誤以為 Aurora 是 R/W master...)

Anyway,這個功能不知道怎麼疊上去的... 不笑得會不會有嚴重的 distributed lock issue,反而推薦大家平常都寫到同一台 (像是 PXC 就會這樣)。

Amazon EC2 再推出兩款新機種:M5 與 H1

Amazon EC2 的新機種發表:「M5 – The Next Generation of General-Purpose EC2 Instances」、「H1 Instances – Fast, Dense Storage for Big Data Applications」。

M5M4 的後續機種 (General Purpose),所以就沒什麼特別好講的了... H1 則是與 D2 接近,而且也應該算是後繼的機種 (Dense Storage),看了看內容感覺只是個升級 (雖然給他一個新的 family type),所以也沒什麼好講...


M5 三區:

You can launch M5 instances today in the US East (Northern Virginia), US West (Oregon), and EU (Ireland) Regions in On-Demand and Spot form (Reserved Instances are also available), with additional Regions in the works.

H1 四區:

H1 instances are available today in the US East (Northern Virginia), US West (Oregon), US East (Ohio), and EU (Ireland) Regions.

產品發表會固定會有的升級 XD

Amazon EC2 推出第一款 Bare Metal 的 Instance

Amazon EC2 直接租整台主機出來了:「Amazon EC2 Bare Metal Instances with Direct Access to Hardware」。

Bare Metal 怎麼翻譯比較好啊?雖然知道是拔掉虛擬化的主機... 裸奔機?

We knew that other customers also had interesting use cases for bare metal hardware and didn’t want to take the performance hit of nested virtualization. They wanted access to the physical resources for applications that take advantage of low-level hardware features such as performance counters and Intel® VT that are not always available or fully supported in virtualized environments, and also for applications intended to run directly on the hardware or licensed and supported for use in non-virtualized environments.

反正這種機器就是要壓榨整台機器的效能,所以不會拿小台機器出來給大家玩。這次推出的是 i3 系列,叫做 i3.metal

Today we are launching a public preview the i3.metal instance, the first in a series of EC2 instances that offer the best of both worlds, allowing the operating system to run directly on the underlying hardware while still providing access to all of the benefits of the cloud. The instance gives you direct access to the processor and other hardware, and has the following specifications:

Processing – Two Intel Xeon E5-2686 v4 processors running at 2.3 GHz, with a total of 36 hyperthreaded cores (72 logical processors).
Memory – 512 GiB.
Storage – 15.2 terabytes of local, SSD-based NVMe storage.
Network – 25 Gbps of ENA-based enhanced networking.

走了十年總算走到這塊了... 不過應該花了不少時間解決各種安全性的問題,像是 network isolation 以及反刷韌體的問題 XD

ElastiCache 支援 r4.* 了

每年 AWS re:Invent 要到的時候就會有很多新的消息出來,Amazon ElastiCache 的團隊應該也是配合著這一波放出消息:「Amazon ElastiCache Now Supports the R4 Node Family」。

包括了 memcachedRedis 都支援了:

Amazon ElastiCache now supports R4 node types. R4 nodes are optimized for latency sensitive and memory intensive workloads. They come in six sizes, providing 12.3GiB to 407GiB of available in-memory capacity. By setting up a 15-shard cluster for Redis, you can scale up to 6.1TiB of in-memory capacity. For Memcached, you can set up a 20-node cluster to support up to 8.14 TiB in-memory workloads. Equipped with the Intel Broadwell processor, and improved networking, R4 node family offers superior performance over the popular R3 node family.


RDS (MySQL/MariaDB) 支援 t2、r4 以及 m4 的新機種

這個大家等好久了,尤其 MySQL 常遇到需要用記憶體換效能的情境:「Amazon RDS for MySQL and MariaDB Supports R4, T2 and M4 Instance Types」。

先前 t2 最大只能開到 t2.large (8GB RAM),對於需要大量記憶體運算的 SQL query,就有機會被 MySQL 使用 filesort 寫到硬碟裡面暫存了。這次支援這些 instance type,開發環境至少有選擇可以開到 t2.2xlarge (32GB RAM) 跟他拼。

r4 應該是正式環境期待已久的 instance type 了。r3 最大是 r3.8xlarge (244GB),跟 r4 最大的 r4.16xlarge (488GB) 剛好差了一倍。

m4 就比較微妙了,順便補上去的感覺... 不過應該還是會有應用會剛好用到。

不過還是期待前陣子出來的 c5,對於寫出很驚人的 SQL query,在 MySQL 內跑大量運算的應用會有幫助,就繼續等吧... :o

EC2 的 X1e 推出更小台的機種

本來 EC2x1e 家族只有 x1e.32xlarge,現在拆小拿出來租了:「Amazon EC2 Update – X1e Instances in Five More Sizes and a Stronger SLA」。

這次算是補足中間的缺口,因為先前的洞有點大:r4.16xlarge 是 488GB RAM,而往上就變成 x1e.32xlarge 的 3904GB,中間都沒有其他選擇可以用。這次 x1e.8xlarge (976GB) 與 x1e.16xlarge (1952GB) 算是補上了這邊的缺口,拿來跑需要大量記憶體,但還不需要到 4TB RAM 等級的程式...

AWS 讓你可以禁止 RI 跨帳號計算了...

現在 AWS 讓你可以設定,是否允許 Reserved Instance (RI) 跨帳號使用:「Customize your organization’s AWS credit and Reserved Instance (RI) discount sharing using new billing preferences」。


Historically, AWS has maximized customer savings by applying credits and RI discounts first to the account that owned the credit or RI lease and then distributing the remainder, if any, to qualifying usage incurred by accounts in the same organization. While this approach had the potential of lowering the overall bill, customers were unable to control if, and how, discounts were applied across organizational lines.


To provide greater flexibility, customers can now disable AWS credit sharing across all accounts in their organization. This ensures that only the account that owns a credit, or has previously redeemed a credit, receives the associated benefit.


You can also designate a set of accounts for which RI discount sharing is disabled, while continuing to share RI discounts among the rest of the accounts in your organization.


在 CLI 下開關以及查詢 EC2 的狀態...

有時候需要開 Ubuntu 測試東西,會在 AWS 上開 EC2 起來測試,但開 console 太麻煩了,寫幾個 function 丟進 shell script 裡面比較乾脆。其中查詢 Ubuntu AMI 的程式出自「How do I know what Ubuntu AMI to launch on EC2?」這邊。

ec2.ls() 裡,我的 jq 版本比較舊,不過不影響我的 copy & paste,所以就沒有 hack 他了。新版的應該可以多加上 | @tsv 變成 tab 隔開 (沒測過,查資料時查到而已)。

ec2.run() 裡,我這邊是先到 console 上查出 security group 與 subnet 的 id,然後這邊 hard code 進去。我的預設是開 t2.medium,臨時要指定的話就 ec2.run t2.nano 就可以改開 t2.nano 了,不過要注意的是,這邊程式在查詢時的條件是 hvm:ebs,換的時候要注意 image 相容性...

# AWS-related
function ec2.ls() {
    aws ec2 describe-instances | \
        jq -c -M '.Reservations[] | .Instances[] | [.InstanceId, .InstanceType, .PublicIpAddress]'

function ec2.rm() {
    local INSTANCE_ID=${1:i-xxxxxxxxxxxxxxxxx}
    aws ec2 terminate-instances --instance-id ${INSTANCE_ID}

function ec2.run() {
    local INSTANCE_TYPE=${1:-t2.medium}
    aws ec2 run-instances --image-id $(ec2.ubuntu_ami()) --key-name gslin --security-group-ids sg-xxxxxxxx --instance-type ${INSTANCE_TYPE} --subnet-id subnet-xxxxxxxx

function ec2.ubuntu_ami() {
    curl -s "https://cloud-images.ubuntu.com/locator/ec2/releasesTable" | \
    sed '$x;$G;/\(.*\),/!H;//!{$!d};$!x;$s//\1/;s/^\n//' | \
    jq -c '.aaData[] | select(contains(["16.04", "us-east-1", "hvm:ebs"]))' | \
    grep -o 'ami-[a-z0-9]\+' | \
    head -1


話說 Ubuntu 網站上的 JSON 居然吐出 malformed data (trailing comma),這是自己 printf() 之類硬幹出來的嗎... XD