Home » Posts tagged "bandwidth"

Backblaze 與 Cloudflare 合作,免除傳輸費用

先前知道不少單位會選擇用 CloudFront 的原因就是 S3 到 CloudFront 這段是不需要傳輸費用的。畢竟 CDN 的 hit rate 還是有限,用其他家 CDN 得付這塊費用。

而現在 Backblaze 宣佈跟 Cloudflare 合作,免除掉 Backblaze 到 Cloudflare 的費用:「Backblaze and Cloudflare Partner to Provide Free Data Transfer」。

Today we are announcing that beginning immediately, Backblaze B2 customers will be able to download data stored in B2 to Cloudflare for zero transfer fees.

AWS 這邊會不會有其他動作呢...

AWS 日本區 EC2 與 S3 的傳輸費用降價...

Twitter 看到 Jeff Barr 提到日本區的 EC2S3 傳輸費用降價:

網站的說明文章則是在「AWS Data Transfer Price Reductions – Up to 34% (Japan) and 28% (Australia)」這邊。分成幾個部份降價:

  • 10TB 以下的費用從 USD$0.14/GB 變成 $0.114/GB (約 19%)。
  • 10TB 到 50TB 從 USD$0.135/GB 變成 USD$0.089/GB (約 34%)。
  • 50TB 到 150TB 則是 USD$0.13/GB 變成 USD$0.086/GB (約 34%)。
  • 超過 150TB 的部份從 USD$0.12/GB 變成 USD$0.084/GB (約 30%)。

然後自動回朔到 2018/09/01 開始算:

Effective September 1, 2018 we are reducing prices for data transfer from Amazon Elastic Compute Cloud (EC2), Amazon Simple Storage Service (S3), and Amazon CloudFront by up to 34% in Japan and 28% in Australia.

另外有提到 CloudFront 也有降價,以及澳洲的部份,應該不太會碰到所以就跳過去...

但就文章這樣寫明 EC2 與 S3 的流量有降價,應該表示從 ELB 出去的流量就不算在這次的降價?除非你是直接 S3 裸奔,不然對大多數的人應該沒差?

又一個 TCP BBR 的測試結果

TCP BBRGoogle 發表的 TCP congestion control 演算法,是一個純伺服器端就能夠改善 TCP 壅塞處理的機制。在 Linux Kernel 4.9 之後被納入了。

Spotify 有大量資料要傳到使用者端 (像是音檔),剛好是 TCP BBR 改善的對象之一,實際測試後得到了很不錯的改善數據:「Smoother Streaming with BBR」。

Spotify 公佈的資料沒有提到平台,所以先稍微了解一下他的音質,也就是「Audio settings」這篇。

在 Desktop 是 160kbps/320kbps Ogg (Standard/HQ)。在 Web Player 則是 128kbps/256kbps AAC (Standard/HQ)。

行動平台部份比較複雜,在 iOS 上是 96kbps/160kbps/256kbps Ogg (Normal/High/Extreme),另外有 Automatic 自動調整的設定。在 Android 平台則是 24kbps HE-AACv2 (Low) 與 96kbps/160kbps/320kbps Ogg (Normal/High/Very high) 以及 Automatic。

而最後 Chromecast 則是 128kbps/256kbps (Standard/Premium)。

測試時可以發現 shutter (指跟不上播放速度) 的情況降低了 6%~10%,而且下載速度增加了 5%~7% (對於慢速的裝置改善更多,10%~15%):

Taking daily averages, stutter decreased 6-10% for the BBR group. Bandwidth increased by 10-15% for the slower download cohorts, and by 5-7% for the median. There was no difference in latency between groups.

而各地區的差異也可以看出來改善很多:

另外他們在測試時,剛好遇到秘魯的機房連外發生問題,結果意外發現 BBR 還是可以穩定在這種網路環境下運作:

In Peru, the non-BBR group saw a 400-500% increase in stutter. In the BBR group, stutter only increased 30-50%.

In this scenario, the BBR group had 4x bandwidth for slower downloads (the 10th percentile), 2x higher median bandwidth, and 5x less stutter!

Ubuntu 18.04 上可以直接設定 BBR,在 Ubuntu 16.04 則可以參考「Ubuntu 16.04 用 speedtest-cli 測試 TCP BBR 效能」這篇的方式升級 kernel 後設定 BBR。

Amazon EC2 的可用頻寬提昇

AWSJeff Barr 宣佈了有 ENAEC2 instance 的頻寬提升到 25Gbps:「The Floodgates Are Open – Increased Network Bandwidth for EC2 Instances」。

分成三種,第一種是對 S3 的頻寬提昇:

EC2 to S3 – Traffic to and from Amazon Simple Storage Service (S3) can now take advantage of up to 25 Gbps of bandwidth. Previously, traffic of this type had access to 5 Gbps of bandwidth. This will be of benefit to applications that access large amounts of data in S3 or that make use of S3 for backup and restore.

第二種是 EC2 對 EC2 (內網):

EC2 to EC2 – Traffic to and from EC2 instances in the same or different Availability Zones within a region can now take advantage of up to 5 Gbps of bandwidth for single-flow traffic, or 25 Gbps of bandwidth for multi-flow traffic (a flow represents a single, point-to-point network connection) by using private IPv4 or IPv6 addresses, as described here.

第三種也是 EC2 對 EC2,但是是在同一個 Cluster Placement Group:

EC2 to EC2 (Cluster Placement Group) – Traffic to and from EC2 instances within a cluster placement group can continue to take advantage of up to 10 Gbps of lower-latency bandwidth for single-flow traffic, or 25 Gbps of lower-latency bandwidth for multi-flow traffic.

有 ENA 的有這些,好像沒看到 CentOS

ENA-enabled AMIs are available for Amazon Linux, Ubuntu 14.04 & 16.04, RHEL 7.4, SLES 12, and Windows Server (2008 R2, 2012, 2012 R2, and 2016). The FreeBSD AMI in AWS Marketplace is also ENA-enabled, as is VMware Cloud on AWS.

TP-Link 的 NTP 流量

在「TP-Link repeater firmware squanders 715 MB/month」這邊看到 TP-Link 因為 NTP 的關係而狂吃流量的情況:(這邊是用逗點表示小數點,所以是 715.4 MB/month)

You should probably avoid TP-Link products if you’re on a tight bandwidth budget. By design, TP-Link firmware sends six DNS requests and one NTP query every 5 seconds, for a total of 715,4 MB per month.

如果拿 24 小時都開機的 Windows 相比的話,會發現這數字天差地別:

To put this number in context: an always-on Windows device will use around 1,6 KB per month on NTP.

作者抓出韌體上面的設定,發現裡面寫死了不少伺服器... 那個 aunz 的選擇讓人頗好奇,另外直接把幾個大學的 NTP server 放進去不知道是什麼樣的想法:

TP-Link has hardcoded the following non-configurable NTP servers and server pools in their firmware:

  • time.nist.gov, time-a.nist.gov, time-b.nist.gov, time-nw.nist.gov
  • au.pool.ntp.org, nz.pool.ntp.org
  • 133.100.9.2, 128.138.140.44, 192.36.144.22

The first sets of servers are operated by the US National Institute of Standards and Technology (NIST). The second is the Australian and New Zealand public NTP project time server pools. The IP addresses are owned by universities in Japan, Colorado; US, and Sweden respectively.

而從行為可以看到沒有遵守這些 NTP service 的規範:

The NTP Pool project asks device manufacturers and vendors to register (and optionally sponsor) their own pools through the service (e.g. tplink.pool.ntp.org), and emphasize that they “must absolutely not use the default pool.ntp.org zone names”. They also request that vendors don’t check more often than every 5 minutes at the most.

而且因為沒有地方可以修改這些設定,唯一的解法是不要買 TP-Link 的產品:

You can avoid buying TP-Link products to avoid this problem.

You can’t turn this behavior off in TP-Link’s web administration interface nor in their management app for mobile. You can’t change the NTP server addresses it targets either.

Amazon S3 的流量,以及 S3 與 Glacier 都推出 Select 功能

Twitter 上看到會場的照片,Amazon S3 單一 region 就有 37 Tb/sec 的量:

在這種量下面對 DDoS 沒什麼感覺 XDDD

另外是 Amazon S3 與 Amazon Glacier 都推出了 Select 功能:「S3 Select and Glacier Select – Retrieving Subsets of Objects」。

看示範的程式碼就可以看出用途了,原文中間那段有 sytax error,我這邊就幫忙修掉了:

handler = PrintingResponseHandler()
s3 = boto3.client('s3')
response = s3.select_object_content(
    Bucket="super-secret-reinvent-stuff",
    Key="stuff.csv",
    SelectRequest={
        'ExpressionType': 'SQL',
        'Expression': 'SELECT s._1 FROM S3Object AS s',
        'InputSerialization': {
            'CompressionType': 'NONE',
            'CSV': {
                'FileHeaderInfo': 'IGNORE',
                'RecordDelimiter': '\n',
                'FieldDelimiter': ',',
            }
        },
        'OutputSerialization': {
            'CSV': {
                'RecordDelimiter': '\n',
                'FieldDelimiter': ',',
            }
        }
    }
)

這樣可以大幅降低 I/O,節省成本:

Glacier Select 也是類似的想法,不需要整包拉出來再處理,可以在一開始就設定條件。

PS4 下載速度很慢的原因

在「Why PS4 downloads are so slow」這篇作者花了不少力氣找出原因,發現 PS4 下載速度很慢是故意的... 另外討論了在什麼情況下會變慢,以及要怎麼避免的方式。

懶得看的人可以直接看 Conculsions 那段,主要的原因是 PS4 會因為背景程式而調整 TCP window size (就算背景程式在 idle 也會影響到下載的 TCP window size),進而影響速度:

If any applications are running, the PS4 appears to change the settings for PSN store downloads, artificially restricting their speed. Closing the other applications will remove the limit.

用 TCP window size 來調整速度也算是頗有「創意」的方法...

Anyway,遇到時的解決方法就是把所有在跑的程式都完整關掉,再下載就會正常多了...

Cloudflare 看這次 815 斷電的網路使用變化

Cloudflare 分析了這次 815 停電對網路造成的影響:「Power outage hits the island of Taiwan. Here’s what we learned.」。

以 Cloudflare 在是方機房的 QPS 來看,停電後反而沒有太大變化:

把裝置種類拆開來看,可以看到桌機的使用量下降,但手機的使用量上升:

這點從 HiNet 的使用頻寬也可以看出來,頻寬使用量降了 25% (從光世代與 ADSL/VDSL 換到行動網路上?):

Archives