Home » Posts tagged "ec2" (Page 3)

Amazon EC2 的 C5 改跑 KVM

前幾天提到的「Amazon EC2 的 C5 家族...」講到 Amazon EC2 推出 C5 系列帶來的效能與價錢的提昇,後來在 The Register 上看到「AWS adopts home-brewed KVM as new hypervisor」這邊提到更多東西,而且有些還頗有趣的...

首先是 Amazon EC2 FAQs 裡面就有提到的,C5 跑的是 KVM,而不像以前是以 Xen 為底:

Q. What is the underlying hypervisor on C5 instances?
C5 instances use a new EC2 hypervisor that is based on core KVM technology.

另外一點比較有疑慮,是 The Register 提到 IntelAmazon 都發出新聞稿,說明 C5 是使用客製化版本的 Intel 處理器,不過看完後讀不出這個感覺...:「Intel Xeon Scalable Processors Supercharge Amazon Web Services’ Most Powerful Compute-Optimized Instances」、「AWS Announces Availability of C5 Instances for Amazon EC2」。

不過先不管這個... 本來 AWS 都一直圍繞在 Xen 上,這次切去 KVM 是個很值得關注的決策,可以找機會問問看為什麼要這樣換。

AWS PrivateLink

AWS 計畫把先前設計的 VPC Endpoint 都併到 AWS PrivateLink 裡,統一管理:「New – AWS PrivateLink for AWS Services: Kinesis, Service Catalog, EC2 Systems Manager, Amazon EC2 APIs, and ELB APIs in your VPC」。

Today we are announcing AWS PrivateLink, the newest generation of VPC Endpoints which is designed for customers to access AWS services in a highly available and scalable manner, while keeping all the traffic within the AWS network. Kinesis, Service Catalog, Amazon EC2, EC2 Systems Manager (SSM), and Elastic Load Balancing (ELB) APIs are now available to use inside your VPC, with support for more services coming soon such as Key Management Service (KMS) and Amazon Cloudwatch.

這樣就不用弄 proxy server 然後在上面管一堆 policy 了... (先不講自己搞 HA 的麻煩事,光是有些程式還得 patch 才能支援 proxy 就會想翻桌了 XD)

在 CLI 下開關以及查詢 EC2 的狀態...

有時候需要開 Ubuntu 測試東西,會在 AWS 上開 EC2 起來測試,但開 console 太麻煩了,寫幾個 function 丟進 shell script 裡面比較乾脆。其中查詢 Ubuntu AMI 的程式出自「How do I know what Ubuntu AMI to launch on EC2?」這邊。

ec2.ls() 裡,我的 jq 版本比較舊,不過不影響我的 copy & paste,所以就沒有 hack 他了。新版的應該可以多加上 | @tsv 變成 tab 隔開 (沒測過,查資料時查到而已)。

ec2.run() 裡,我這邊是先到 console 上查出 security group 與 subnet 的 id,然後這邊 hard code 進去。我的預設是開 t2.medium,臨時要指定的話就 ec2.run t2.nano 就可以改開 t2.nano 了,不過要注意的是,這邊程式在查詢時的條件是 hvm:ebs,換的時候要注意 image 相容性...

# AWS-related
function ec2.ls() {
    aws ec2 describe-instances | \
        jq -c -M '.Reservations[] | .Instances[] | [.InstanceId, .InstanceType, .PublicIpAddress]'
}

function ec2.rm() {
    local INSTANCE_ID=${1:i-xxxxxxxxxxxxxxxxx}
    aws ec2 terminate-instances --instance-id ${INSTANCE_ID}
}

function ec2.run() {
    local INSTANCE_TYPE=${1:-t2.medium}
    aws ec2 run-instances --image-id $(ec2.ubuntu_ami()) --key-name gslin --security-group-ids sg-xxxxxxxx --instance-type ${INSTANCE_TYPE} --subnet-id subnet-xxxxxxxx
}

function ec2.ubuntu_ami() {
    curl -s "https://cloud-images.ubuntu.com/locator/ec2/releasesTable" | \
    sed '$x;$G;/\(.*\),/!H;//!{$!d};$!x;$s//\1/;s/^\n//' | \
    jq -c '.aaData[] | select(contains(["16.04", "us-east-1", "hvm:ebs"]))' | \
    grep -o 'ami-[a-z0-9]\+' | \
    head -1
}

這種工具自己用的順手比較重要,要什麼功能自己改自己加...

話說 Ubuntu 網站上的 JSON 居然吐出 malformed data (trailing comma),這是自己 printf() 之類硬幹出來的嗎... XD

Amazon EC2 的 CRI 支援一年版本了...

Amazon EC2 的 CRI (Convertible Reserved Instance) 支援一年的合約了:「EC2 Convertible Reserved Instance Update – New 1-Year CRI, Merges & Splits」,這樣彈性再多了一些:

Today we are introducing Convertible RIs with a 1-year term, complementing the existing 3-year term.

不過 CRI 主要是用在需要換 family type 的情境下,如果是已知 family type (像是一般性的 worker 會選 C4 或是剛推出的 C5) 那麼就直接選擇 Regional RI 就好...

基本上就是讓財務操作上多個選擇 :o

Amazon EC2 的 C5 家族...

Amazon EC2 推出新的 instance:「Now Available – Compute-Intensive C5 Instances for Amazon EC2」,官方宣稱這次單位價錢的效能與 C4 相比大約提升了 25%,而極端的情況可以到 50%:

The new instances offer a 25% price/performance improvement over the C4 instances, with over 50% for some workloads.

這次比較特別的是切分方式,是 large、xlarge、2xlarge、4xlarge、9xlarge (咦?) 以及 18xlarge (...)。

然後亞洲區都還沒上 XD

You can launch C5 instances today in the US East (Northern Virginia), US West (Oregon), and EU (Ireland) Regions in On-Demand and Spot form (Reserved Instances are also available), with additional Regions in the works.

AWS 推出 NVIDIA Tesla V100 的 GPU 機種 (P3)

這次從 NVIDIA 推出到 AWS 採用比以前快好多,依照維基百科上查到的資料,Tesla V1000 是六月 21 日推出的,這樣算起來四個月後就上線了:「New – Amazon EC2 Instances with Up to 8 NVIDIA Tesla V100 GPUs (P3)」。

Today we are making the next generation of GPU-powered EC2 instances available in four AWS regions. Powered by up to eight NVIDIA Tesla V100 GPUs, the P3 instances are designed to handle compute-intensive machine learning, deep learning, computational fluid dynamics, computational finance, seismic analysis, molecular modeling, and genomics workloads.

P2 是 K80,NVIDIA 推出的時間是 2014 年的十一月了... 這時候拿出新的貨色出來算合理 XD

然後是這幾區有提供 P3:

P3 instances are available in the US East (Northern Virginia), US West (Oregon), EU (Ireland), and Asia Pacific (Tokyo) Regions in On-Demand, Spot, Reserved Instance, and Dedicated Host form.

Amazon RDS 的 RI 也支援彈性計價了

AWS 宣佈 RDS 的 RI 也支援彈性計價了:「Amazon RDS Reserved Instances Offer Instance Size Flexibility」。

也就是說跟 EC2 的 RI 方式切齊,買 db.m4.2xlarge 的 RI 後,可以用在兩台 db.m4.xlarge 上:

For example, let’s say you purchased a db.m4.2xlarge MySQL RI in US East (N. Virginia). The discounted rate of this RI can automatically apply to 2 db.m4.xlarge MySQL instances without you needing to do anything.

包括了相當多種類的 RDS,主要是沒有 license fee 的類型都包括在內了:

Amazon RDS Reserved Instance size flexibility is offered in all regions for the MySQL, MariaDB, PostgreSQL, and Amazon Aurora database engines, as well as the “bring your own license” (BYOL) edition of the Oracle database engine. To learn more about flexible RIs, please visit the Amazon RDS Reserved Instances Page.

但不包括要另外收 license fee 的 SQL Server 與 Oracle LI edition:

Size flexibility does not apply to Microsoft SQL Server and the License Included (LI) edition of Oracle.

Amazon EC2 推出 4TB 的機器

之前 Amazon EC2 記憶體最大的機器是 x1.32xlarge 的 2TB RAM (更精確是 1952GB),現在推出了 4TB RAM 的 x1e.32xlarge (3904GB):「Now Available – EC2 Instances with 4 TB of Memory」。

現在這個時間點在 us-east-1 的價錢是 USD$26.688/hour (一個月 USD$19215.36),用的到的人應該付得起?

另外值得注意的是,x1e.32xlarge 雖然比 x1.32xlarge 多了一倍的記憶體,但 vCPU 不變 (都是 128),而且 ECU 下降了 (從 349 降到 340)。

這個機器目前在 us-east-1us-west-2eu-west-1ap-northeast-1 四區提供服務:

The x1e.32xlarge instances can be launched in On-Demand and Reserved Instance form via the AWS Management Console, AWS Command Line Interface (CLI), AWS SDKs, and AWS Marketplace in the US East (Northern Virginia), US West (Oregon), EU (Ireland), and Asia Pacific (Tokyo) Regions.

EC2 的 Spot Instance 可以「接關」

Amazon EC2Spot Instance 可以「接關」了:「New – Stop & Resume Workloads on EC2 Spot Instances」。

當 Spot Instance 的競價不足以標到機器時,他會先關起來 (Stop),等到價錢低於競價後就會再打開機器,這時候的狀態就會恢復。另外也提到了必須是使用 EBS 的機器才支援:

Amazon EC2 Spot now allows Amazon EBS-backed instances to be stopped in the event of interruption, instead of being terminated when capacity is no longer available at your preferred price. Spot can then fulfill your request by restarting instances from a stopped state when capacity is available within your price and time requirements.

用法是把 Spot Instance 的關機設定設為 Stop:

To use this new feature, choose “stop” instead of “terminate” as the interruption behavior when submitting a persistent Spot request. When you choose “stop”, Spot will shut down your instance upon interruption.

恢復的時候就會儘量保持一樣地開回來 (連 instance id 都相同):

When capacity is available again within your price and time requirements, Spot will restart your instance. Upon restart, the EBS root device is restored from its prior state, previously attached data volumes are reattached, and the instance retains its instance ID.

是個接關的感覺 XD

EC2 與 EBS 十月開始以秒計費

雖然只是 Amazon EC2Amazon EBS 計價模式的改變,但這次 AWS 的改變對於許多開發流程有很大的影響 (重點在 EC2 的部份):「New – Per-Second Billing for EC2 Instances and EBS Volumes」。

10/2 開始改變 (而不是 10/1),低消一分鐘,Windows 機種以及需要額外收費的 Linux 機種不在範圍內:

This change is effective in all AWS Regions and will be effective October 2, for all Linux instances that are newly launched or already running. Per-second billing is not currently applicable to instances running Microsoft Windows or Linux distributions that have a separate hourly charge. There is a 1 minute minimum charge per-instance.

然後 Spot 與買 RI 後也是一樣以秒計價:

List prices and Spot Market prices are still listed on a per-hour basis, but bills are calculated down to the second, as is Reserved Instance usage (you can launch, use, and terminate multiple instances within an hour and get the Reserved Instance Benefit for all of the instances).

這次改變的影響很巨大。馬上可以想到幾個情境...

第一個是對於實踐 Release early, release often 的團隊來說,如果設計成每 deploy 一次就建一個新的 AMI (最乾淨的作法),再開新機器換掉的話,成本就會增加不少。所以對於這樣的團隊,就會偏好朝著替換現有目錄內的東西後重啟...

現在改成以秒計費後,直接透過 Blue-Green Deployment 就可以了 (AWS CodeDeploy 年初也支援了:「AWS CodeDeploy 支援 BlueGreenDeployment」):(如果不熟悉 Blue-Green Deployment 的話,更白話的說法就是「先建後拆」...)

同樣的理由,對於 Auto Scaling 的 policy 也有些改變。之前機器開起來都會想讓他跑一個小時,所以 scale down 的部份都會寫的比較鬆一點。現在就可以重新規劃了...

另外一個影響是對使用 container 的誘因少了不少。很多人用 container 的用法是開大台機器再裡面拆給不同服務用,讓資源利用率變高,現在變成用多少算多少後就不太需要這樣了...

當然也還是有缺點。以前 Spot Instance 如果被 AWS 收回時,最後的那個小時是不計費的。現在因為以秒計費,變成要收費了...

最後是 10/2 生效這件事情頗怪,該不會是財務部門不願意配合 10/1 星期天加班生效,所以只好變成 10/2 生效這種理由吧... XDDD

Archives