Home » Computer » Network » Archive by category "Service" (Page 27)

AWS Lambda 可使用的記憶體空間從 1.5GB 變成 3GB

AWS 是說 AWS Lambda 可用的記憶體空間 double 啦,不過 3008MB 這個數字有點怪...:「AWS Lambda Doubles Maximum Memory Capacity for Lambda Functions」。

You can now allocate 3008MB of memory to your AWS Lambda functions. Previously, the maximum amount of memory available to your functions was 1536MB. Now, it's easier to process workloads with higher memory or denser compute requirements, such as big data analysis, large file processing, and statistical computations.

這個就真的全區都生效了,包括一般人不能註冊的 AWS GovCloud (US) 與中國區:

This feature is available in US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), AWS GovCloud (US), Canada (Central), South America (São Paulo), EU (Frankfurt), EU (Ireland), EU (London), Asia Pacific (Mumbai), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), and China (Beijing).

Amazon API Gateway 可以透過 NLB 接進 VPC 內了

AWS 宣佈可以透過 Network Load BalancerAPI Gateway 接進 VPC 內了:「Amazon API Gateway Supports Endpoint Integrations with Private VPCs」。

You can use API Gateway to create an API endpoint that is integrated with your VPC. You create an endpoint to your VPC by setting up a VPC link between your VPC and a Network Load Balancer (NLB), which is provided by Elastic Load Balancing.

基本上是所有的區域都有了,除了美國政府的區域外:

This feature is now available in US East (N. Virginia), US East (Ohio), US West (Oregon), US West (N. California), Canada (Central), South America (São Paulo), EU (Ireland), EU (Frankfurt), EU (London), Asia Pacific (Singapore), Asia Pacific (Tokyo), Asia Pacific (Sydney), Asia Pacific (Seoul), and Asia Pacific (Mumbai) AWS regions.

是透過 NLB 接進去,而不是 ELB Classic 或是 ALB,可以來想像網路架構是怎麼做的...

Microsoft 虛擬化的兩個消息:Azure 被打臉以及 AWS 推出轉移工具

首先是 VMware 發文打臉 Microsoft 說他們所宣稱的轉移工具 (從 VMware 轉到 Azure 上) 並沒有 VMware 原廠支援:「VMware – The Platform of Choice in the Cloud」。

然後 AWS 則是推出了從 Hyper-V 轉移到 AWS 的工具:「Migrate Hyper-V VMs to AWS with AWS Server Migration Service」,這邊倒是沒提到官方支援...

這臉不只是腫腫的而已了,有種連續技的感覺 XD

LinkedIn 忘記續約導致 SSL Certificate 過期

Netcraft 上看到 LinkedIn 出包的消息,這次是 country-mixed 的版本出包:「LinkedIn certificate blunder leaves users LockedOut!」。

在 DNS 上也可以看出來這兩個 CNAME 到不一樣的 load balancer 上:

;; ANSWER SECTION:
www.linkedin.com.       260     IN      CNAME   2-01-2c3e-003c.cdx.cedexis.net.
2-01-2c3e-003c.cdx.cedexis.net. 93 IN   CNAME   pop-ehk1.www.linkedin.com.
pop-ehk1.www.linkedin.com. 3560 IN      A       144.2.3.1
;; ANSWER SECTION:
de.linkedin.com.        86400   IN      CNAME   cctld.linkedin.com.
cctld.linkedin.com.     86400   IN      CNAME   mix.linkedin.com.
mix.linkedin.com.       213     IN      CNAME   pop-ehk1.mix.linkedin.com.
pop-ehk1.mix.linkedin.com. 3546 IN      A       144.2.3.5

SSL Labs 上也看得出來在 Alternative names 的地方是不一樣的:「SSL Server Test: www.linkedin.com (Powered by Qualys SSL Labs)」、「SSL Server Test: de.linkedin.com (Powered by Qualys SSL Labs)」。

然後因為 LinkedIn 有設定 HSTS,所以使用者在界面上完全無法登入:

Google Chrome 上可以用 badidea 繞過 (參考「在 Google Chrome 連上因 HSTS 而無法連線的網站」),但在 Mozilla Firefox 上的話目前沒找到方法可以在界面上 bypass,而是需要改 SiteSecurityServiceState.txt 這個檔案:「HTTP Strict Transport Security prevents me from accessing a server that I'm doing development on」。

不過也因為兩個 cluster 獨立運作,網址改一下應該就會動了...

這幾年比較很少看到大公司出這種包,還蠻有趣的 XD

Etsy 介紹的 Cache Smearing

Etsy 的 engineering blog 上提到了他們怎麼設計 cache 機制:「How Etsy caches: hashing, Ketama, and cache smearing」。

使用 consistent hash 已經是基本款了,文章裡花了一些篇幅介紹為什麼要用 consistent hash。

後半段則是有了 consistent hash 後會遇到的問題,也就是講 hot key 怎麼處理:有些資料非常熱 (常常被存取),就算用 consistent hash 也還是有可能搞爆單一機器。

他們做了幾件事情,第一件事情是設計 cache smearing 機制,把單一資料加上 random key,使得不同的 key 會打散到不同的機器上:

Let’s take an example of a hot key popular_user_data. This key is read often (since the user is popular) and is hashed to pool member 3. Cache smearing appends a random number in a small range (say, [0, 8)) to the key before each read or write. For instance, successive reads might look up popular_user_data3, popular_user_data1, and popular_user_data6. Because the keys are different, they will be hashed to different hosts. One or more may still be on pool member 3, but not all of them will be, sharing the load among the pool.

第二件事情則是監控哪些 key 比較熱門:

We’ve seen this problem many times over the years, using mctop, memkeys, and our distributed tracing tools to track down hot keys.

第三件事情是維護 hot key 的清單 (不是每個 key 都會上 cache smearing):

We manually add cache smearing to only our hottest keys, leaving keys that are less read-heavy efficiently stored on a single host.

是個當規模大到單一 hot key 會讓單台伺服器撐不住時的 workaround...

Amazon API Gateway 支援 Canary Release 了

Amazon API Gateway 支援 Canary Release 了:「Amazon API Gateway Supports Canary Release Deployments」。

Canary Release 重點在於逐步轉移,而不是直接硬切,大致上可以分成三個階段。

首先是一開始的情況:

切到一半的情境:

最後完全使用新版本:

這個方法可以避免新的 code 有效能問題,造成後端壓力過大... 不過這樣就要確定新舊版本的程式碼可以同時跑 (像是後端資料庫的 schema 必須相容這兩個版本)。

前幾天提到的「AWS CodeDeploy 支援在 AWS Lambda 上跑更多奇怪花樣」算是相關的功能,讓 AWS CodeDeploy 參與其中做出各種變化。

Amazon Aurora 的 Serverless 與 Multi-master

Amazon Aurora 推出了兩包玩意,第一包是 Serverless,讓需要人介入的情況更少:「In The Works – Amazon Aurora Serverless」。

在 Serverless 的第一個重點是支援以秒計費:

Today we are launching a preview (sign up now) of Amazon Aurora Serverless. Designed for workloads that are highly variable and subject to rapid change, this new configuration allows you to pay for the database resources you use, on a second-by-second basis.

然後是極為快速的 auto-scaling:

The endpoint is a simple proxy that routes your queries to a rapidly scaled fleet of database resources. This allows your connections to remain intact even as scaling operations take place behind the scenes. Scaling is rapid, with new resources coming online within 5 seconds

這兩個組合起來,讓使用端可以除了在 Amazon EC2 上可以快速 scale 外,後端的資料庫也能 scale 了...

第二個是 Multi-master 架構:「Sign Up for the Preview of Amazon Aurora Multi-Master」。

Amazon Aurora Multi-Master allows you to create multiple read/write master instances across multiple Availability Zones. This enables applications to read and write data to multiple database instances in a cluster, just as you can read across Read Replicas today.

(話說我一直都誤以為 Aurora 是 R/W master...)

Anyway,這個功能不知道怎麼疊上去的... 不笑得會不會有嚴重的 distributed lock issue,反而推薦大家平常都寫到同一台 (像是 PXC 就會這樣)。

Amazon DynamoDB 跨區 Replication 以及備份

Amazon DynamoDB 實做了全球性的 replication,以及備份功能:「Amazon DynamoDB Update – Global Tables and On-Demand Backup」。

跨區 replication 的功能讓每個 region 都可以存取當地機房的 DynamoDB:

Global Tables – You can now create tables that are automatically replicated across two or more AWS Regions, with full support for multi-master writes, with a couple of clicks. This gives you the ability to build fast, massively scaled applications for a global user base without having to manage the replication process.

這有點類似 GoogleCloud Spanner 在前陣子也推出全球性服務,但 DynamoDB 提供的比較偏向 NoSQL 而不是 RDBMS。

另外一個限制是跨區同步是 async,會有 replication lag 的問題:

Updates are propagated to other Regions asynchronously via DynamoDB Streams and are typically complete within one second (you can track this using the new ReplicationLatency and PendingReplicationCount metrics).

不過如果是這樣的機制,conflict 的問題不知道怎麼解決... 文章裡面沒看到。

然後目前支援的區域還是有限:

Global Tables are available in the US East (Ohio), US East (N. Virginia), US West (Oregon), EU (Ireland), and EU (Frankfurt) Regions today, with more Regions in the works for 2018.

另外一個是備份與還原機制,有這樣的功能對很多計畫方便不少:

On-Demand Backup – You can now create full backups of your DynamoDB tables with a single click, and with zero impact on performance or availability. Your application remains online and runs at full speed. Backups are suitable for long-term retention and archival, and can help you to comply with regulatory requirements.

而備份還原機制是陸陸續續開放的,區域也有限:

We are rolling this new feature out on an account-by-account basis as quickly as possible, with initial availability in the US East (Northern Virginia), US East (Ohio), US West (Oregon), and EU (Ireland) Regions.

Amazon Lightsail 推出 Block Storage 與 Load Balancer

Amazon Lightsail 推出了 Block Storage (11/14) 與 Load Balancer (11/29):「Introducing additional block storage for Amazon Lightsail」、「Amazon Lightsail adds load balancers with integrated certificate management」。

兩個是不同時間點發表的,當時懶的寫所以這次一起寫...

Block Storage 有不少 VPS 都有提供了,像是 Linode 的「Linode Block Storage (Fremont beta)」(雖然還在 beta)、DigitalOcean 的「Storage on DigitalOcean | Highly available Block Storage」以及 Vultr 的「High Performance and Cheap Block Storage - Vultr.com」。

AWS 算是很早就有這個服務 (Amazon EBS),這邊應該只是把系統整合進來...

另外一個是這幾天推出的 Load Balancer,目前應該只有 Linode 的「Ensure High-Availability with NodeBalancers - Linode」比較知名。AWS 上的 ELB 有不少選擇可以用 (ELB Classic、ALB 以及 NLB),不過公告裡沒特別提到... 比較特別的是提供免費的 SSL Certificate 吧?這在其他家主要得靠 Let's Encrypt 來做,在 AWS 上應該是整合了 ACM

AWS WAF 提供隨時更新的 Managed Rules

AWS WAF 推出了隨時更新的 Managed Rule:「Ready-to-Use Managed Rules Now Available on AWS WAF」。

這些 ruleset 是由 3rd-party 提供的:

Choose from preconfigured RuleGroups provided in the AWS Marketplace by industry leading security experts: Alert Logic, Fortinet, Imperva, Trend Micro and TrustWave.

然後隨時更新:

Rules are automatically updated as new threats emerge and offer a wide range of protections, including OWASP Top 10 mitigations, bad-bot defenses, and virtual patching against recent CVE’s.

然後是要收費的:

Each RuleGroup is the product of a Seller’s unique expertise, made available to you at an affordable pay-as-you-go price.

AWS Marketplace 的「Managed Rules for AWS WAF - Web Application Firewall」裡拿兩家來看看。

趨勢的「Trend Micro Managed Rules for AWS WAF - WebServer (Apache, Nginx)」與「Trend Micro Managed Rules for AWS WAF - Content Management System (CMS)」都是:

Charge per month in each available region (pro-rated by the hour) $5.00 / unit
Charge per million requests in each available region $0.20 / unit

Imperva 則是提供不一樣的選擇,在「Imperva - Managed Rules for WordPress Protection on AWS WAF」是:

Charge per month in each available region (pro-rated by the hour) $30.00 / unit
Charge per million requests in each available region $0.60 / unit

而「Imperva - Managed Rules for IP Reputation on AWS WAF」則是:

Charge per month in each available region (pro-rated by the hour) $40.00 / unit
Charge per million requests in each available region $0.40 / unit

Archives