超慢的 Facebook...

今天的 Facebook 慢到爆炸,看起來像是台灣機房的問題,這邊先提供 workaround:

把 DNS 改用 8.8.8.8,看起來 DNS 會解去洛杉磯的機房,速度就會正常多了。

接下來列出看到的結果,靜態的頁面 (拿 /robots.txt 測試) 沒問題:

Server Software:        
Server Hostname:        www.facebook.com
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-ECDSA-AES128-GCM-SHA256,256,128

Document Path:          /robots.txt
Document Length:        4159 bytes

Concurrency Level:      1
Time taken for tests:   0.792 seconds
Complete requests:      20
Failed requests:        0
Total transferred:      99000 bytes
HTML transferred:       83180 bytes
Requests per second:    25.24 [#/sec] (mean)
Time per request:       39.612 [ms] (mean)
Time per request:       39.612 [ms] (mean, across all concurrent requests)
Transfer rate:          122.03 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       28   30   0.6     30      31
Processing:     9   10   0.3     10      10
Waiting:        9    9   0.2      9      10
Total:         38   39   0.7     39      41

Percentage of the requests served within a certain time (ms)
  50%     39
  66%     40
  75%     40
  80%     40
  90%     40
  95%     41
  98%     41
  99%     41
 100%     41 (longest request)

但動態的頁面就慢到炸:

Server Software:        
Server Hostname:        www.facebook.com
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-ECDSA-AES128-GCM-SHA256,256,128

Document Path:          /
Document Length:        0 bytes

Concurrency Level:      1
Time taken for tests:   20.294 seconds
Complete requests:      20
Failed requests:        0
Non-2xx responses:      20
Total transferred:      6700 bytes
HTML transferred:       0 bytes
Requests per second:    0.99 [#/sec] (mean)
Time per request:       1014.690 [ms] (mean)
Time per request:       1014.690 [ms] (mean, across all concurrent requests)
Transfer rate:          0.32 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       28   30   0.9     29      32
Processing:   151  985 1144.5    567    4939
Waiting:      151  985 1144.5    567    4938
Total:        180 1014 1145.1    597    4971
WARNING: The median and mean for the initial connection time are not within a normal deviation
        These results are probably not that reliable.

Percentage of the requests served within a certain time (ms)
  50%    597
  66%   1226
  75%   1279
  80%   1667
  90%   2267
  95%   4971
  98%   4971
  99%   4971
 100%   4971 (longest request)

用 Intel 網卡上的 Flow Director 過濾封包

在「Traffic filtration using NIC capabilities on wire speed (10GE, 14Mpps)」這邊看到的技巧。

作者建議另外安裝 driver,因為 Linux kernel 內的 driver 功能有限:「Intel Ethernet Drivers and Utilities」。

重點在 ethtool 這個工具,可以看到條件設定:

ethtool --help:
        ethtool -N|-U|--config-nfc|--config-ntuple DEVNAME    Configure Rx network flow classification options or rules
        rx-flow-hash tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r... |
        flow-type ether|ip4|tcp4|udp4|sctp4|ah4|esp4
            [ src %x:%x:%x:%x:%x:%x [m %x:%x:%x:%x:%x:%x] ]
            [ dst %x:%x:%x:%x:%x:%x [m %x:%x:%x:%x:%x:%x] ]
            [ proto %d [m %x] ]
            [ src-ip %d.%d.%d.%d [m %d.%d.%d.%d] ]
            [ dst-ip %d.%d.%d.%d [m %d.%d.%d.%d] ]
            [ tos %d [m %x] ]
            [ l4proto %d [m %x] ]
            [ src-port %d [m %x] ]
            [ dst-port %d [m %x] ]
            [ spi %d [m %x] ]
            [ vlan-etype %x [m %x] ]
            [ vlan %x [m %x] ]
            [ user-def %x [m %x] ]
            [ action %d ]
            [ loc %d]] |
        delete %d

看起來 stateless 的過濾可以在上面做...

Facebook 用 PJPEG 的技巧

Facebook 在「Faster Photos in Facebook for iOS」這篇提到使用 PJPEG 降低流量並且提昇大圖的速度。

實際上我覺得有點詭異,依照說明,降低流量的部份是因為原來有縮圖與大圖:

而現在只要抓一張,所以綜合起來降低了:

但不是每一張都這樣吧?另外是降低了到一定品質所需 image loading 的時間,但這樣看起來用小縮圖應該會更好?雖然 Facebook 的人是以 A/B testing 驗證,但以目前 Facebook 提供的數據並不通,看不太懂他的邏輯...

先丟著吧...

OpenSSL 的 ECDH 中,224 bits 速度比 160/192 bits 快的原因

openssl speed ecdh 的時候發現很特別的現象:

Doing 160 bit  ecdh's for 10s: 40865 160-bit ECDH ops in 9.99s
Doing 192 bit  ecdh's for 10s: 34169 192-bit ECDH ops in 9.99s
Doing 224 bit  ecdh's for 10s: 60980 224-bit ECDH ops in 9.99s
Doing 256 bit  ecdh's for 10s: 34298 256-bit ECDH ops in 10.00s
Doing 384 bit  ecdh's for 10s: 9602 384-bit ECDH ops in 10.00s
Doing 521 bit  ecdh's for 10s: 9127 521-bit ECDH ops in 9.99s

原因是 Google 這篇論文的貢獻:「Fast Elliptic Curve Cryptography in OpenSSL」,開頭就提到:

We present a 64-bit optimized implementation of the NIST and SECG-standardized elliptic curve P-224.

而實際成果:

full TLS handshakes using a 1024-bit RSA certificate and ephemeral Elliptic Curve Diffie-Hellman key exchange over P-224 now run at twice the speed of standard OpenSSL, while atomic elliptic curve operations are up to 4 times faster.

OpenSSLCHANGES 也可以看到對應的修改,不只是 NIST-P224 有被改善,其他的 NIST-P256 與 NIST-P521 也都有被改善:

Add optional 64-bit optimized implementations of elliptic curves NIST-P224, NIST-P256, NIST-P521, with constant-time single point multiplication on typical inputs.

頗特別的...

TLS 的效能分析

看到「Is TLS Fast Yet?」這個站台,開宗明義就講了,這不是問題:

TLS has exactly one performance problem: it is not used widely enough.

Everything else can be optimized.

下面的表格分析把對於效能影響的分析寫得很清楚,除了軟體外,還包括了各家 CDN 支援的分析,可以參考一下這邊的分析。

Wikipedia 換成 HHVM 的成果

維基百科基金會的人發表了將 PHP 換成 HHVM 後的成果:「How we made editing Wikipedia twice as fast」。

可以看到編輯存檔時間大幅改善:

另外是已登入使用者 (通常是經常參與編輯的使用者) 的頁面產生時間也大幅改善:

另外帶來的好處是 CPU 使用率的下降:

再來就是看 PHP 7 能追上多少了...

換到 nginx 1.6 後把 OCSP stapling 與 SPDY 打開...

剛剛把 nginx 換到 ppa 的版本 (目前是 1.6),然後打開 OCSP staplingSPDY,速度明顯有感覺提昇...

OCSP stapling:

SPDY:

目前有想到而且能處理的都做的差不多了 :p

Netflix 與 Comcast 的恩怨

其實就是商業公司之間的勾心鬥角,在包裝後搬到檯面上 :p

Netflix 在美國固網裡吃的流量比 YouTube 還多,可想而知當然就變成各 ISP 找麻煩的對象...


出自 Sandvine 的「Global Internet Phenomena Report 2H 2013」。

Netflix 有多種方式將影片傳遞給使用者。除了早期自建機房外,後來跟不少 CDN 有業務往來 (包括了 AkamaiLimelightLevel3),另外也有 Netflix Open Connect Content Delivery Network 計畫,直接在 ISP 內部機房放設備提供服務。

使用 CDN 的作法成本太高,而 ISP 又不一定會接受 Open Connect 方案 (因為不一定收的到錢),在這種情況下,如果走 transit 線路的速度通常都不會太好。而 Netflix 與 Comcast 之間的狀況就是如此:

在付給 Comcast 錢後速度都都解決了...

除了付錢解決外,上個禮拜 Netflix 就丟出一篇說明的文章發難了:「The Case Against ISP Tolls」,這篇文章除了提到上面的事情外,另外還極力反對 Comcast 與 Time Warner Cable 的併購案 XDDD

然後最近又炒熱的網路中立問題,看起來也這件案子應該會很熱鬧 XDDD