AWS CodeBuild 可以管 Secret 了...

AWS CodeBuild 可以管理 secret 了:「AWS CodeBuild Now Provides Ability To Manage Secrets」。

AWS CodeBuild now further enhances securing your build environment. CodeBuild can now store sensitive information as secrets, which can now get directly passed to your build jobs. This can be achieved by modifying the parameter store directly in your buildspec.yml, or via the CodeBuild console.

在文件裡提到:

We strongly discourage using environment variables to store sensitive values, especially AWS access key IDs and secret access keys. Environment variables can be displayed in plain text using tools such as the AWS CodeBuild console and the AWS CLI. For sensitive values, we recommend you use the parameter-store mapping instead, as described later in this section.

這次算是補上其他家已經有蠻久的功能...

不過在找資料的時候,發現 AWS CodeBuild 提供了每個月一百分鐘的 free quota,不論是新帳號還是現有帳號都一直有?(這點是之前沒注意到的...)

The AWS CodeBuild free tier includes 100 build minutes of build.general1.small per month. The CodeBuild free tier does not expire automatically at the end of your 12-month AWS Free Tier term. It is available to new and existing AWS customers.

Amazon EC2 推出 4TB 的機器

之前 Amazon EC2 記憶體最大的機器是 x1.32xlarge 的 2TB RAM (更精確是 1952GB),現在推出了 4TB RAM 的 x1e.32xlarge (3904GB):「Now Available – EC2 Instances with 4 TB of Memory」。

現在這個時間點在 us-east-1 的價錢是 USD$26.688/hour (一個月 USD$19215.36),用的到的人應該付得起?

另外值得注意的是,x1e.32xlarge 雖然比 x1.32xlarge 多了一倍的記憶體,但 vCPU 不變 (都是 128),而且 ECU 下降了 (從 349 降到 340)。

這個機器目前在 us-east-1us-west-2eu-west-1ap-northeast-1 四區提供服務:

The x1e.32xlarge instances can be launched in On-Demand and Reserved Instance form via the AWS Management Console, AWS Command Line Interface (CLI), AWS SDKs, and AWS Marketplace in the US East (Northern Virginia), US West (Oregon), EU (Ireland), and Asia Pacific (Tokyo) Regions.

EC2 的 Spot Instance 可以「接關」

Amazon EC2Spot Instance 可以「接關」了:「New – Stop & Resume Workloads on EC2 Spot Instances」。

當 Spot Instance 的競價不足以標到機器時,他會先關起來 (Stop),等到價錢低於競價後就會再打開機器,這時候的狀態就會恢復。另外也提到了必須是使用 EBS 的機器才支援:

Amazon EC2 Spot now allows Amazon EBS-backed instances to be stopped in the event of interruption, instead of being terminated when capacity is no longer available at your preferred price. Spot can then fulfill your request by restarting instances from a stopped state when capacity is available within your price and time requirements.

用法是把 Spot Instance 的關機設定設為 Stop:

To use this new feature, choose “stop” instead of “terminate” as the interruption behavior when submitting a persistent Spot request. When you choose “stop”, Spot will shut down your instance upon interruption.

恢復的時候就會儘量保持一樣地開回來 (連 instance id 都相同):

When capacity is available again within your price and time requirements, Spot will restart your instance. Upon restart, the EBS root device is restored from its prior state, previously attached data volumes are reattached, and the instance retains its instance ID.

是個接關的感覺 XD

EC2 與 EBS 十月開始以秒計費

雖然只是 Amazon EC2Amazon EBS 計價模式的改變,但這次 AWS 的改變對於許多開發流程有很大的影響 (重點在 EC2 的部份):「New – Per-Second Billing for EC2 Instances and EBS Volumes」。

10/2 開始改變 (而不是 10/1),低消一分鐘,Windows 機種以及需要額外收費的 Linux 機種不在範圍內:

This change is effective in all AWS Regions and will be effective October 2, for all Linux instances that are newly launched or already running. Per-second billing is not currently applicable to instances running Microsoft Windows or Linux distributions that have a separate hourly charge. There is a 1 minute minimum charge per-instance.

然後 Spot 與買 RI 後也是一樣以秒計價:

List prices and Spot Market prices are still listed on a per-hour basis, but bills are calculated down to the second, as is Reserved Instance usage (you can launch, use, and terminate multiple instances within an hour and get the Reserved Instance Benefit for all of the instances).

這次改變的影響很巨大。馬上可以想到幾個情境...

第一個是對於實踐 Release early, release often 的團隊來說,如果設計成每 deploy 一次就建一個新的 AMI (最乾淨的作法),再開新機器換掉的話,成本就會增加不少。所以對於這樣的團隊,就會偏好朝著替換現有目錄內的東西後重啟...

現在改成以秒計費後,直接透過 Blue-Green Deployment 就可以了 (AWS CodeDeploy 年初也支援了:「AWS CodeDeploy 支援 BlueGreenDeployment」):(如果不熟悉 Blue-Green Deployment 的話,更白話的說法就是「先建後拆」...)

同樣的理由,對於 Auto Scaling 的 policy 也有些改變。之前機器開起來都會想讓他跑一個小時,所以 scale down 的部份都會寫的比較鬆一點。現在就可以重新規劃了...

另外一個影響是對使用 container 的誘因少了不少。很多人用 container 的用法是開大台機器再裡面拆給不同服務用,讓資源利用率變高,現在變成用多少算多少後就不太需要這樣了...

當然也還是有缺點。以前 Spot Instance 如果被 AWS 收回時,最後的那個小時是不計費的。現在因為以秒計費,變成要收費了...

最後是 10/2 生效這件事情頗怪,該不會是財務部門不願意配合 10/1 星期天加班生效,所以只好變成 10/2 生效這種理由吧... XDDD

Google 放棄對海外伺服器搜索票的抵抗了...

先前美國政府透過搜索票,要求各雲端廠商提供海外伺服器的資料而引起話題 (像是先前 Microsoft 往上打官司抵抗:「Does US have right to data on overseas servers? We’re about to find out」),而現在看起來 Google 打算放棄掙扎了:「Google stops challenging most US warrants for data on overseas servers」。

Google has quietly stopped challenging most search warrants from US judges in which the data requested is stored on overseas servers, according to the Justice Department.

Microsoft 這邊有些不錯的進展,成功在巡迴庭擋下:

Microsoft convinced the New York-based 2nd US Circuit Court of Appeals—which has jurisdiction over Connecticut, New York, and Vermont—that US search-and-seizure law does not require compliance with a warrant to turn over e-mail stored on its servers in Ireland.

不過沒看到 AWS 相關的消息,感覺不妙...

Cloudflare 的 F-Root

Cloudflare 從三月底開始跟 ISC 簽約合作,服務 F-Root 這個 DNS Service (f.root-servers.net):「Delivering Dot」。

Since March 30, 2017, Cloudflare has been providing DNS Anycast service as additional F-Root instances under contract with ISC (the F-Root operator).

Linode 東京的機器上面可以看出來 www.cloudflare.com 走的路徑跟 f.root-server.net 相同:

gslin@one [~] [22:49] mtr -4 --report www.cloudflare.com
Start: Tue Sep 12 22:49:29 2017
HOST: one.abpe.org                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 139.162.65.2               0.0%    10    0.6   0.6   0.5   0.6   0.0
  2.|-- 139.162.64.5               0.0%    10    2.0   1.1   0.6   2.5   0.5
  3.|-- 139.162.64.8               0.0%    10    0.7   1.0   0.7   2.1   0.3
  4.|-- 218.100.6.62               0.0%    10    0.8   0.8   0.8   1.0   0.0
  5.|-- 198.41.215.162             0.0%    10    0.7   0.7   0.7   0.8   0.0
gslin@one [~] [22:49] mtr -4 --report f.root-servers.net
Start: Tue Sep 12 22:49:46 2017
HOST: one.abpe.org                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 139.162.65.3               0.0%    10    0.5   0.6   0.5   0.6   0.0
  2.|-- 139.162.64.7               0.0%    10    0.7   0.7   0.6   0.8   0.0
  3.|-- 139.162.64.8               0.0%    10    0.7   0.7   0.6   0.8   0.0
  4.|-- 218.100.6.62               0.0%    10    0.8   0.8   0.8   0.8   0.0
  5.|-- f.root-servers.net         0.0%    10    0.8   0.8   0.7   0.8   0.0

而且也可以從監控發現,f.root-servers.net 的效能變好:

Using RIPE atlas probe measurements, we can see an immediate performance benefit to the F-Root server, from 8.24 median RTT to 4.24 median RTT.

DNS query 的量也大幅增加:

而且之後也會隨著 Cloudflare 的 PoP 增加而愈來愈快... 在原文的 comment 也提到了 Cloudflare 也有打算跟其他的 Root Server 合作,所以看起來會讓整個 infrastructure 愈來愈快而且穩定。

另外這也代表台灣在本島也會直接連到 F-Root 了,不過 HiNet 自己也有 F-Root,所以 HiNet 的部份就沒什麼差...

Route 53 的 Query 記錄

Amazon Route 53 可以收 query log 了,會丟到 CloudWatch Logs:「Amazon Route 53 Announces Support For DNS Query Logging」。

If you are using Amazon Route 53 as your public, authoritative DNS, you will now have the capability to easily log DNS queries received by Amazon Route 53 through integration with CloudWatch Logs.

這樣可以拿來分析了...

AWS 推出新的 Load Balancer:NLB (Network Load Balancer)

從一開始推出的 ELB (Elastic Load Balancer),到 ALB (Application),現在則推出了 NLB (Network):「New Network Load Balancer – Effortless Scaling to Millions of Requests per Second」。

有這些特性:

  1. Static IP Addresses
  2. Zonality
  3. Source Address Preservation
  4. Long-running Connections
  5. Failover

雖然不能確定 AWS 用的技術是什麼,但這裡面有好幾個很明顯就是 DSR (Direct Server Return) 架構的特性 (包括了限制與優點)。

另外也因為不用處理 L7 的內容,效能比起 ELB/ALB 好很多,夠大的用量下,價錢也低不少。對於不少非 HTTP/HTTPS 的應用應該很好用,就算是 HTTP/HTTPS,單純一點的應用應該也不錯...

在 EC2 上使用 25Gbps 的網路

Amazon EC2 上許多系列最大台的機器現在都可以透過特殊界面跑到 25Gbps 了:「Announcing improved networking performance for Amazon EC2 instances」。

Amazon EC2 instances now provide a maximum bandwidth of 25 Gbps. This feature is available on the largest instance sizes of the M4, X1, P2, R4, I3, F1, and G3 instance types. Using Elastic Network Adapter (ENA) based Enhanced Networking, customers can utilize up to 25 Gbps of bandwidth.

除了 AWS 提供的的 AMI 外,另外在一些官方作業系統也都有支援:

ENA driver is installed in the latest Amazon Machine Images (AMIs) for the following operating systems: Amazon Linux, Ubuntu 14.04 and 16.04, RHEL 7.4, SLES 12, Windows Server 2008R2, 2012, 2012R2 and 2016. ENA Linux driver source code is also available on Github.com for developers to integrate in their AMIs.

如果不在上面的,也可以透過 GitHub 上的 amzn/amzn-drivers (Official repository of the Elastic Network Adapter (ENA) network adapter for Linux and FreeBSD operating systems) 自己 porting... (如果你一定要想辦法用的話)

KeyCDN 的台灣 PoP...

GCP 推出 Standard Tier 後,KeyCDN 就利用台灣的 GCP 機房建立了 PoP:「KeyCDN Launches New POP in Taiwan」。

GCP 在台灣的 Standard Tier 價錢是:

Source geolocation of traffic 0-10 TB 10-150 TB 150-500 TB
From Asia $0.11 $0.075 $0.07

KeyCDN 亞洲區頻寬的價錢則是 USD$0.12/GB,加上機器費用,有機會不虧?