Home » Posts tagged "cloud" (Page 50)

Linode 也推出 Load balancer 服務... (剛開始 beta)

Linode 在「NodeBalancers - Managed Load Balancers」這邊公開了 NodeBalancer (lbass) 服務,目前在 beta 中,想要測試的人可以開 ticket 詢問。另外,API 文件也已經先公開,可以在「NodeBalancer API」這邊看到。

以 API 提供的功能看起來還算 okay,這樣就不需要自己用 Linode 提供的 IP Failover 並且在上面架 HAProxy server...

不過有些細節還是得等測試後才知道。像是 trusted IP 問題,我在「ELB 的 IP 信任問題...」有提過 EC2 的同樣問題 (前幾天用 ELB security group 解掉了)。

Amazon 詳細說明 AWS EBS/RDS 故障的原因...

Amazon 在網頁上說明了在 4/21 美東地區 EBS/RDS 故障的原因:「Summary of the Amazon EC2 and Amazon RDS Service Disruption in the US East Region」。

整篇文章相當長,但整個連鎖反應是起於操作時的人為疏失。而 RDS 因為是基於 EBS 上,所以也跟著大爆炸...

另外文章最後面也提到賠償的方式:

For customers with an attached EBS volume or a running RDS database instance in the affected Availability Zone in the US East Region at the time of the disruption, regardless of whether their resources and application were impacted or not, we are going to provide a 10 day credit equal to 100% of their usage of EBS Volumes, EC2 Instances and RDS database instances that were running in the affected Availability Zone.

另外,AWS 的 SLA 中有要求當未滿 SLA 條件時,必須由使用者提出退款要求,但這次則是例外,符合條件的會主動在下次的帳單上,或是系統的頁面上看到:

These customers will not have to do anything in order to receive this credit, as it will be automatically applied to their next AWS bill. Customers can see whether they qualify for the service credit by logging into their AWS Account Activity page.

AWS EC2/EBS 用了三天半時間恢復...

AWS EC2/EBS 從星期四 21 日下午四點開始 (+8 時區),到剛剛在「Service Health Dashboard」上看到宣佈恢復 (亮綠燈),總共花了超過三天半的時間。有些人受到影響時間比較長,有些人比較短,我是在 24 號的凌晨恢復的,受到影響時間約兩天多。這幾天應該會有比較正式的說明...

手上本來沒打算這麼早設計 failover 機制的,現在都要拉到前面做了...

這次的 AWS EC2/EBS 問題...

這次的 AWS EC2/EBS 問題讓很多站台掛掉 (可以看「Who is affected by EC2?」這個站台,在發文的現在 US-East 還是沒完全恢復),但這次並不是每個使用 US-East 的站台都掛掉,像是 Netflix 因為一開始規劃就是「Cloud 也是會爛整片的」,所以服務並沒有中斷。在 Hacker News 上就有被提出來討論

討論裡面也有投影片說明 Netflix 選 AWS 的原因是因為「來不及建立 Data Center」,但在建立時也同時注意到「Cloud 有可能會大規模爛掉」的情況而設計了很多機制防範:「Netflix in the cloud 2011」。

利用 AWS 在多個不同地點都有機房把架構在 AWS 上所能提供的 HA 機制發揮到極致,不過這是建立在「Cheaper than cost of being down」的想法上 (因為 Netflix 已經算是某類型的電子商務公司了)。

話說回來,繼續等美東機房恢復...

目前 AWS EC2 (美東機房) 出問題...

AWS 的「Service Health Dashboard」上可以看到美東機房從 PDT 1:41 AM (台灣時間是 4:41 PM) 開始介入,到現在五個多小時了...

下午本來要跑一堆東西,看起來得晚上做了,明天來凹一下老闆,下午再進公司好了... :o

Update:在 Service Health Dashboard 上面總算有初步的原因了:

8:54 AM PDT We'd like to provide additional color on what were working on right now (please note that we always know more and understand issues better after we fully recover and dive deep into the post mortem). A networking event early this morning triggered a large amount of re-mirroring of EBS volumes in US-EAST-1. This re-mirroring created a shortage of capacity in one of the US-EAST-1 Availability Zones, which impacted new EBS volume creation as well as the pace with which we could re-mirror and recover affected EBS volumes. Additionally, one of our internal control planes for EBS has become inundated such that it's difficult to create new EBS volumes and EBS backed instances. We are working as quickly as possible to add capacity to that one Availability Zone to speed up the re-mirroring, and working to restore the control plane issue. We're starting to see progress on these efforts, but are not there yet. We will continue to provide updates when we have them.

看起來還要再等等...

Archives