Google Chrome 55 的記憶體改善

前陣子 Google Chrome 55 出了,其中最讓人期待的是對記憶體的改善,有人整理了數據:「Chrome 55 uses ~30% less memory than Chrome 54」。

依照作者拿 測試,前者的記憶體省了 26%,後者省了 30%,都相當明顯。我自己在升到 55 後有明顯感覺到改善,尤其是重開 Chrome 時重新讀取頁面的速度也快了不少...

這些改善主要是出自於「Fall cleaning: Optimizing V8 memory consumption」這邊提到對 V8 engine 的改寫,我感覺到速度變快應該是記憶體用量降低使得 CPU cache rate 提高的關係吧...

Mark Callaghan 講最近的 MySQL 的行銷活動...

Mark Callaghan 這篇倒是沒提到什麼技術的東西,主要是講最近 MySQL 的兩大 conference,一個是 OracleOracle Open World,另外一個是 PerconaPercona Live Amsterdam 2016,然後用了 benchmarketing 這個酸酸的詞 XDDD:「Peak benchmarketing season for MySQL」。


My joke is that each of these makes a different group happy: performance -> marketing, usability -> developers, manageability -> operations, availability -> end users, efficiency -> management.

另外提到了 RocksDB 建出來的 MyRocks 在 memory fit 時可能會比 InnoDB 還要好:

One last disclaimer. If you care about read-mostly/in-memory workloads then InnoDB is probably an excellent choice. MyRocks can still be faster than InnoDB for in-memory workloads. That is more likely when the bottleneck for InnoDB is page write-back performance. So write-heavy/in-memory can still be a winner for MyRocks.


Python 3.6 對 Dict 的改善

Python 3 的 Dict 將會有重大的改變:「[Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered」。

在 3.5 時:

Python 3.5.1 (default, Jun 20 2016, 14:48:22)
>>> def func(**kw): print(kw.keys())
>>> func(a=1, b=2, c=3, d=4, e=5)
dict_keys(['c', 'd', 'e', 'b', 'a'])   # random order

對上目前還在開發的 3.6:

Python 3.6.0a4+ (default:d43f819caea7, Sep  8 2016, 13:05:34)
>>> def func(**kw): print(kw.keys())
>>> func(a=1, b=2, c=3, d=4, e=5)
dict_keys(['a', 'b', 'c', 'd', 'e'])   # expected order

在「Compact and ordered dict」這邊可以看到記憶體的使用量降低:

It seems like the memory usage is between 20% and 25% smaller. Great job!

Memory usage, Python 3.5 => Python 3.6 on Linux x86_64:

./python -c 'import sys; print(sys.getsizeof({str(i):i for i in range(10)}))'

* 10 items: 480 B => 384 B (-20%)
* 100 items: 6240 B => 4720 B (-24%)
* 1000 items: 49248 B => 36984 B (-25%)

Note: the size is the the size of the container itself, not of keys nor values.


3% slowdown in microbench is not surprising.
Compact dict introduces one additional indirection.

Instead, I've added freelist for most compact PyDictKeys.
So I think overall performance is almost same to before compact dict.

不過也有人提到應該拿 3.5 + patch 測,而不是直接拿 3.6 測:

There are a lot of other changes in interpreter core between 3.5 and 3.5 (such as new bytecode and optimized function calls). Could you compare the performance between the version just before adding new dict implementation and the version just after this?


Linode 記憶體升級,以及新的日本機房計畫

Linode 的 13 歲禮物:「Linode’s 13th Birthday – Gifts for All!」。包括了記憶體的升級計畫:

Old Plan New Plan Price
Linode 1 GB Linode 2 GB $10/mo ($0.015/hr)
Linode 2 GB Linode 4 GB $20/mo ($0.03/hr)
Linode 4 GB Linode 8 GB $40/mo ($0.06/hr)
Linode 8 GB Linode 12 GB $80/mo ($0.12/hr)
Linode 16 GB Linode 24 GB $160/mo ($0.24/hr)
Linode 32 GB Linode 48 GB $320/mo ($0.48/hr)
Linode 48 GB Linode 64 GB $480/mo ($0.72/hr)
Linode 64 GB Linode 80 GB $640/mo ($0.96/hr)
Linode 96 GB Linode 120 GB $960/mo ($1.44/hr)

比較小的機器都是 double RAM,比較大的機器就沒那麼明顯了... 但這樣就超越 DigitalOcean 的規格,而且還領先其他 VPS 不少。


Unfortunately, since Tokyo is sold out, the upgrade is not available there. We hope to have our second Tokyo facility online before the end of the year.

是個好消息 XD

AWS 最新的 x1.32xlarge...

Amazon EC2 推出了 x1.32xlarge:「X1 Instances for EC2 – Ready for Your Memory-Intensive Workloads」。


Processor: 4 x Intel™ Xeon E7 8880 v3 (Haswell) running at 2.3 GHz – 64 cores / 128 vCPUs.
Memory: 1,952 GiB with Single Device Data Correction (SDDC+1).
Instance Storage: 2 x 1,920 GB SSD.


If you are ready to start using the X1 instances in the US East (Northern Virginia), US West (Oregon), Europe (Ireland), Europe (Frankfurt), Asia Pacific (Tokyo), Asia Pacific (Singapore), or Asia Pacific (Sydney) Regions, please request access and we’ll get you going as soon as possible.

這樣以後就不能說「用 C4 就對了」...

調整 MySQL 的記憶體用量

Percona 的「Best Practices for Configuring Optimal MySQL Memory Usage」這篇給了個蠻不錯的建議:

Don’t allow the mysqld process VSZ exceed 90% of the system memory

ps ax -O vsz | grep mysqld 可以看到 mysqld 吃了多少 VSZ,然後自己除整台機器的記憶體大小,就可以算出來目前吃了多少,然後調整 innodb_buffer_pool_size 的數字。

另外 performance schema 也會有不少影響:

MySQL is workload related – if you have many connections active at the same time that run heavy selects using a lot of memory for sorting or temporary tables, you might need a lot of memory (especially if Performance Schema is enabled). In other cases this amount of memory is minimal. You’ll generally need somewhere between 1 and 10GB for this purpose.


Another thing you need to account for is memory fragmentation. Depending on the memory allocation library you’re using (glibc, TCMalloc, jemalloc, etc.), the operating system settings such as Transparent Huge Pages (THP) and workload may show memory usage to grow over time (until it reaches some steady state). Memory fragmentation can also account for 10% or more of additional memory usage.


Google Compute Engine 推出 Custom Machine Type

Google Compute Engine 推出了可以自己設定 CPU 與 RAM 的機器種類:「Custom Machine Types - Compute Engine — Google Cloud Platform」。

可以從 1 個 vCPU 到 32 個 vCPU,而記憶體最多是 6.5GB * vCPU 數,所以理論上最高是 208GB?

Create a machine type with as little as 1 vCPU and up to 32 vCPUs, or any even number of vCPUs in between. Memory can be configured up to 6.5 GB of RAM per vCPU.

計價方式就是 vCPU 算一份,記憶體算一份。記得以前有比較小的 Cloud Service 有提供過類似的計價方式,後來都收掉了...

OpenSSH client 的重大安全性更新

CVE-2016-0777 與 CVE-2016-0778 安全性漏洞是關於 OpenSSH client 的部分:(USN-2869-1: OpenSSH vulnerabilities)

It was discovered that the OpenSSH client experimental support for resuming connections contained multiple security issues. A malicious server could use this issue to leak client memory to the server, including private client user keys.

這下除了要更新以外,要重新生 ssh key 然後更新一堆機器了...

PuTTY 安全性問題 (CVE-2015-5309)

雖然很久沒用 PuTTY 了 (因為用 Ubuntu 很久了),不過很難得看到 PuTTY 有安全性問題。

PuTTY 官方發佈了安全性通報 CVE-2015-5309:「PuTTY vulnerability vuln-ech-overflow」:

Versions of PuTTY and pterm between 0.54 and 0.65 inclusive have a potentially memory-corrupting integer overflow in the handling of the ECH (erase characters) control sequence in the terminal emulator.

不過老問題還是沒解啊,透過 HTTPS (i.e. Certificate authority 架構) 雖然有很多問題,但至少還是個靠稽核制度而建立的安全信任機制,在沒有任何可信任環境下可以當作起點下仍然是最好的方案:「如何安全下載軟體...」。

PHP 5.5 的 Generators

在「Save memory by switching to generators」這邊提到了 PHP 5.5 開始提供的 Generators...

由於其他的程式語言有 Generators 的觀念,其實不會太難了解...

不過比較大的問題是,資料庫的查詢操作用 Generators 會把效能壓力壓回資料庫:因為資料庫需要把結果 buffering 在資料庫端,如果不趕快吐出去就是要找記憶體放... 也因此,比較常見到的解法是不要用 Generators。(因為 web 與 application 端相較於資料庫端,比較容易 scale)

後來用 Generators 比較多的印象中還是 filter 類的應用吧,Python 這邊的東西有陣子沒看了 :o