Home » Computer » Network » Archive by category "Cloud" (Page 3)

如果你有衛星 (對,衛星),可以玩看看 AWS Ground Station

AWS 推出的新服務,AWS Ground Station:「AWS Ground Station – Ingest and Process Data from Orbiting Satellites」。

AWS 目前在地面上見了兩個基地站 (預定要成長到 12 個),這個服務可以跟衛星溝通:

Ground Stations – As I mentioned earlier, we are launching today with 2 ground stations, and will have a total of 12 in operation by 2019. We will monitor utilization and demand, and will build additional stations and antennas as needed.

收費是按照分鐘計算:

Pricing – Pricing is per-minute of downlink time, with an option to pre-pay for blocks of minutes.

看起來是給租用衛星頻寬的人用的... 不過當初會決定作這個服務是吸了什麼 XDDD

AWS 的 Firecracker 技術 (安全的 Container?)

AWS 放出來的 open source 專案 Firecracker,也就是在 AWS 內打造安裝的 container 環境所使用的技術:「Firecracker – Lightweight Virtualization for Serverless Computing」。

依照說明,看起來是利用 crosvm (KVM-based) 但讓他更輕,啟動 image 的時間更快,達到跟 container 類似的效果:

High Performance – You can launch a microVM in as little as 125 ms today (and even faster in 2019), making it ideal for many types of workloads, including those that are transient or short-lived.

Low Overhead – Firecracker consumes about 5 MiB of memory per microVM. You can run thousands of secure VMs with widely varying vCPU and memory configurations on the same instance.

看起來有機會在自己機器上跑看看 (i.e. 非虛擬環境)?跑之前要注意目前只支援 Intel 的硬體:

Firecracker currently supports Intel CPUs, with planned AMD and Arm support. Firecracker will also be integrated with popular container runtimes.

AWS Transit Gateway:不用再自己處理多個 AWS 帳號之間的 routing 了

AWS 推出 Transit Gateway,讓多個 AWS 帳號之間的 routing 問題總算不用自己處理了:「New – Use an AWS Transit Gateway to Simplify Your Network Architecture」。

這個問題常出現在組織架構大一點的情境下:為了讓各單位自己付自己單位的帳,所以大家都建立了自己的 AWS 帳號。但內部還是有互相連線的需求,雖然把 IP range 都有切開,但還是得搞定彼此之間的 routing...

之前 AWS 還做了「AWS Global Transit Network」這份說明,解釋要怎麼處理 routing,在說明裡官方當時還提供了不少在雲上面建立 router 的方案。像是其中一個方式是用 Cisco 的方案做:「AWS Solution – Transit VPC」、「Overview - Transit Network VPC (Cisco CSR)」。

而現在則是可以直接用 AWS 的服務解決了:

You can attach up to 5000 VPCs to each gateway and each attachment can handle up to 50 Gbits/second of bursty traffic. You can attach your AWS VPN connections to a Transit Gateway today, with Direct Connect planned for early 2019.

每個接點都要收租用費 (美國是 $0.05/hr,一個月大約 $36;日本是 $0.07/hr,一個月大約 $50.4),另外流量要收 USD$0.02/GB,其實就是本來的 in + out 的費用 (在 EC2 同區的流量是要收 in + out 的費用的)。

AWS 推出了 S3 Object Lock,保護資料被刪除的可能性

AWS 推出了 S3 Object Lock,可以設定條件鎖住 S3 上的 object,以保護資料不被刪除:「AWS Announces Amazon S3 Object Lock in all AWS Regions」。

這個功能跟會計做帳的概念很像,也就是寫進去後就不能改,也不能刪除,保留一定時間後才移除掉:

You can migrate workloads from existing write-once-read-many (WORM) systems into Amazon S3, and configure S3 Object Lock at the object- and bucket-levels to prevent object version deletions prior to pre-defined Retain Until Dates or Legal Hold Dates.

AWS 提供有兩種模式,一個是 Governance mode,這個模式下可以設定某些 IAM 權限可以移除 S3 Object Lock。另外一個是 Compliance mode,這個模式下連 root account 都不能刪除:

S3 Object Lock can be configured in one of two modes. When deployed in Governance mode, AWS accounts with specific IAM permissions are able to remove object locks from objects. If you require stronger immutability to comply with regulations, you can use Compliance Mode. In Compliance Mode, the protection cannot be removed by any user, including the root account.

AWS 推出 Global Accelerator,用 AWS 的網路加速

AWS 推出了 Global Accelerator,利用 AWS 的網路加速:「New – AWS Global Accelerator for Availability and Performance」。

這個產品有點像是 GCP 的 Premium Network 的概念,從名稱叫做 Data Transfer-Premium (DT-Premium) 也可以看出來這點。另外 Cloudflare 也有類似的產品,叫做 Spectrum

使用者的連線會先進入最接近使用者的 AWS Edge,然後走 AWS 自己的網路到真正服務所在的 AWS 區域:

AWS 自家的 CloudFront 可以做類似的事情,但是 CloudFront 是 DNS-based service,而且只吃 HTTP 類的連線;這次推出的 Global Accelerator 則是 Anycast-based service,同時支援 TCP 與 UDP。

目前的 edge 只有北美、歐洲與亞洲:

AWS Global Accelerator is available in US East (N. Virginia), US East (Ohio), US West (Oregon), US West (N. California), Europe (Ireland), Europe (Frankfurt), Asia Pacific (Tokyo) and Asia Pacific (Singapore).

這類服務通常也都可以擋下一些 DDoS 攻擊,畢竟是拿大水管在擋...

EC2 推出 ARM 架構的機器...

看到 AWS 推出使用 ARM 架構的 EC2 instance 了:「New – EC2 Instances (A1) Powered by Arm-Based AWS Graviton Processors」。

在 Quick Start 頁面有 Ubuntu 18.04 (ARM) 可以選,開起來後操作跟標準的 Ubuntu 差不多... 連進去後 uname -a 可以看到是 aarch64:

ubuntu@ip-172-30-2-207:~$ uname -a
Linux ip-172-30-2-207 4.15.0-1028-aws #29+nutmeg8-Ubuntu SMP Tue Nov 20 02:59:41 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux

然後來看硬體規格,從最大台的 a1.4xlarge 來看是 16 vCPU + 32 GB RAM,定價是 $0.408/hr (這邊都拿 us-east-1 比較)。

對照 General Purpose 的 m5.4xlarge 是 16 vCPU + 60 GB RAM,定價是 $0.768/hr。如果看記憶體比較接近的 m5.2xlarge 則是 8 vCPU + 31 GB RAM,定價是 $0.384/hr。

對照 Compute Optimized 的 c5.4xlarge 是 16 vCPU + 68 GB RAM,定價是 $0.68/hr。

實際跑一些測試,包括 md5、sha256 與 aes (最後 aes 這個通常都有硬體加速),都用 -mutli 16 跑。

ARM 的 a1.4xlarge

type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
md5            1064103.21k  2484789.44k  4436178.60k  5521370.45k  5943380.65k  5963896.15k
sha256         2059690.93k  5652827.82k 11792656.30k 16108863.15k 18086602.36k 18250851.29k
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128 cbc    1593981.48k  1723960.38k  1752940.20k  1767321.94k  1770212.01k  1768281.43k
aes-192 cbc    1400010.40k  1496414.83k  1516962.99k  1527643.82k  1529834.15k  1527563.26k
aes-256 cbc    1222067.79k  1296972.50k  1313348.18k  1321350.83k  1322947.93k  1321850.20k
blowfish cbc   1384982.01k  1500548.63k  1529793.02k  1540091.22k  1540937.05k  1540767.74k

Intelm5.4xlarge

type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
md5            1370869.17k  3276978.66k  5929591.13k  7441276.93k  8026330.45k  8071796.05k
sha256          592719.47k  1325135.04k  2506009.09k  3184234.50k  3455729.66k  3480365.74k
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128 cbc    1060996.96k  1121951.87k  1135376.21k  1141487.96k  1143270.06k  1143499.43k
aes-192 cbc     890438.97k   934047.98k   943446.44k   947576.49k   948857.51k   949026.82k
aes-256 cbc     768686.53k   800152.85k   806883.93k   809804.12k   810784.09k   810937.00k
blowfish cbc   1735490.97k  1884059.78k  1923876.10k  1932711.94k  1934477.99k  1928680.79k

Intel 的 c5.4xlarge

type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
md5            1501870.92k  3593434.20k  6503591.25k  8162811.90k  8804605.95k  8855147.86k
sha256          650179.22k  1453635.18k  2749318.83k  3492912.13k  3791164.76k  3818105.51k
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128 cbc    1163898.98k  1230539.07k  1244414.63k  1252080.98k  1254110.55k  1254206.12k
aes-192 cbc     976610.23k  1024570.03k  1034886.66k  1039442.26k  1040872.79k  1041143.13k
aes-256 cbc     843184.42k   877695.30k   885125.46k   888408.06k   889503.74k   889766.83k
blowfish cbc   1877162.34k  2056925.74k  2107008.26k  2119893.67k  2121979.22k  2115720.53k

這些數字頗有趣的... 看起來 ARM 上面應該有對某些演算法加速,使得常見的情境會快很多。不過如果是其他應用的話看起來就會比較辛苦了... 目前就價錢來看未必有絕對的優勢,還是得看應用來決定。

Amazon S3 推出了一個自動分析後分類的 Storage Class

Amazon S3 推出了新的 Storage Class,後面直接用演算法分析 access pattern (所以要跑一陣子才會生效),然後決定要放到 Standard 或是 Standard IA 裡:「Announcing S3 Intelligent-Tiering — a New Amazon S3 Storage Class」。

混了 Standard 與 Standard IA:

S3 Intelligent-Tiering stores objects in two access tiers: one tier that is optimized for frequent access and another lower-cost tier that is optimized for infrequent access.

然後連續 30 天沒有被存取的就會被丟到 Standard IA,如果有被存取的話就會被搬回來,而搬移的部份不用收費:

For a small monthly monitoring and automation fee per object, S3 Intelligent-Tiering monitors access patterns and moves objects that have not been accessed for 30 consecutive days to the infrequent access tier. There are no retrieval fees in S3 Intelligent-Tiering. If an object in the infrequent access tier is accessed later, it is automatically moved back to the frequent access tier. No additional tiering fees apply when objects are moved between access tiers within the S3 Intelligent-Tiering storage class.

從費用上可以看到演算法本身是有費用的,換算一下 1M objects 是 USD$2.5/month,好像還可以...

Monitoring and Automation, All storage / Month$0.0025 per 1,000 objects

不過有蠻多要注意的 pattern。像是這邊有提到 128KB 以下的檔案不會搬到 IA 上,但不知道算不算 Monitoring 的費用?

S3 Intelligent-Tiering has a minimum eligible object size of 128KB for auto-tiering. Smaller objects may be stored but will always be charged at the Frequent Access tier rates.

另外這邊講 S3 Intelligent-Tiering 的三十天也不知道是不是 Standard + Standard IA,或是分開算:

S3 Intelligent-Tiering, S3 Standard-IA, and S3 One Zone-IA storage are charged for a minimum storage duration of 30 days.

可以先觀望一下...

Amazon EFS 也要推出 IA 版本了 (Infrequent Access)

Amazon EFS 也要推出 IA (Infrequent Access) 版本了,Infrequent Access 指的是不常存取的資料:「Coming Soon – Amazon EFS Infrequent Access Storage Class」。

這剛好配合上很多人拿 Amazon EFS 來堆 log 的行為... AWS 是說有機會到省到 85%,不過應該是非常大的量才有機會有這個價錢?

EFS IA reduces storage costs for files not accessed every day, with savings up to 85% compared to the EFS Standard storage class.

其實用過 Amazon EFS 的人都對效能抱怨頗嚴重 (透過 NFS 有太多操作沒辦法 cache,於是 network latency issue 就出現了),堆 log 或是當作跨機器的空間大概是目前的主流用法...

Amazon S3 推出 SFTP

Amazon 居然推出對 Amazon S3SFTP 服務了:「Introducing AWS Transfer for SFTP, a Fully Managed SFTP Service for Amazon S3」。

這個服務是掛在 AWS Transfer 的名義下 (AWS Transfer for SFTP),這對老系統來說可以省一些事情,不過目前還不支援固定 IP,這樣不太能直接搬上去 (這種老系統常常都是用 IP firewall 擋著):

Q: Can I use fixed IP addresses to access the SFTP server endpoint?
A: No. Fixed IP addresses that are usually used for firewall whitelisting purposes are currently not supported.

另外認證的部份看起來已經包括了常用的認證:

Q: Which compliance programs does AWS SFTP support?
A: AWS SFTP is PCI-DSS and GDPR compliant, and HIPAA eligible.

不過第一次的 SSH key 部份要怎麼取得啊... 有支援 SSHFP 嗎?

支援的區域蠻多的,對台灣使用常見的區域都有在第一波的清單內:

AWS SFTP is available in AWS Regions worldwide including US East (N. Virginia, Ohio), US West (Oregon, N. California), Canada (Central), Europe (Ireland, Paris, Frankfurt, London), and Asia Pacific (Tokyo, Singapore, Sydney, Seoul).

最後是價錢,上傳與下傳都要另外收費 (USD$0.04/GB),另外服務本身的 endpoint 也要收費 (USD$0.3/hour,一個月大約會是 $216),跟自己弄比起來好像不怎麼便宜,目前看起來主要是整合了 IAM 與其他機制... 不過這就是賣服務,看自己取捨就是了 :o

EC2 推出用 machine learning 協助 auto scaling 控制的功能...

AWSEC2 上推出了用 machine learning 協助 auto scaling 控制的功能:「New – Predictive Scaling for EC2, Powered by Machine Learning」。

最少給他一天的資料 (然後他會每天重新分析一次),接著會預測接下來的 48 小時的使用行為:

The model needs at least one day’s of historical data to start making predictions; it is re-evaluated every 24 hours to create a forecast for the next 48 hours.

所以是個學 pattern 然後預先開好機制等著的概念...

透過預測增加服務穩定性的概念... 如果本來就跑得好好的 (也就是靠 resource-based metric 觸發機器數量的方式跑得很好),就未必需要考慮這個方案了。

目前支援的區域中,東京不在列表內,不過其他常見的區域都支援了:

Predictive scaling is available now and you can starting using it today in the US East (N. Virginia), US East (Ohio), US West (Oregon), Europe (Ireland), and Asia Pacific (Singapore) Regions.

Archives