Mark Callaghan 講最近的 MySQL 的行銷活動...

Mark Callaghan 這篇倒是沒提到什麼技術的東西,主要是講最近 MySQL 的兩大 conference,一個是 OracleOracle Open World,另外一個是 PerconaPercona Live Amsterdam 2016,然後用了 benchmarketing 這個酸酸的詞 XDDD:「Peak benchmarketing season for MySQL」。

裡面有些也很有趣的東西:

My joke is that each of these makes a different group happy: performance -> marketing, usability -> developers, manageability -> operations, availability -> end users, efficiency -> management.

另外提到了 RocksDB 建出來的 MyRocks 在 memory fit 時可能會比 InnoDB 還要好:

One last disclaimer. If you care about read-mostly/in-memory workloads then InnoDB is probably an excellent choice. MyRocks can still be faster than InnoDB for in-memory workloads. That is more likely when the bottleneck for InnoDB is page write-back performance. So write-heavy/in-memory can still be a winner for MyRocks.

這就有趣了,找個時間來測試看看...

Python 3.6 對 Dict 的改善

Python 3 的 Dict 將會有重大的改變:「[Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered」。

在 3.5 時:

Python 3.5.1 (default, Jun 20 2016, 14:48:22)
>>> def func(**kw): print(kw.keys())
...
>>> func(a=1, b=2, c=3, d=4, e=5)
dict_keys(['c', 'd', 'e', 'b', 'a'])   # random order

對上目前還在開發的 3.6:

Python 3.6.0a4+ (default:d43f819caea7, Sep  8 2016, 13:05:34)
>>> def func(**kw): print(kw.keys())
...
>>> func(a=1, b=2, c=3, d=4, e=5)
dict_keys(['a', 'b', 'c', 'd', 'e'])   # expected order

在「Compact and ordered dict」這邊可以看到記憶體的使用量降低:

It seems like the memory usage is between 20% and 25% smaller. Great job!

Memory usage, Python 3.5 => Python 3.6 on Linux x86_64:

./python -c 'import sys; print(sys.getsizeof({str(i):i for i in range(10)}))'

* 10 items: 480 B => 384 B (-20%)
* 100 items: 6240 B => 4720 B (-24%)
* 1000 items: 49248 B => 36984 B (-25%)

Note: the size is the the size of the container itself, not of keys nor values.

不過效能上似乎慢了一些:

3% slowdown in microbench is not surprising.
Compact dict introduces one additional indirection.

Instead, I've added freelist for most compact PyDictKeys.
So I think overall performance is almost same to before compact dict.

不過也有人提到應該拿 3.5 + patch 測,而不是直接拿 3.6 測:

There are a lot of other changes in interpreter core between 3.5 and 3.5 (such as new bytecode and optimized function calls). Could you compare the performance between the version just before adding new dict implementation and the version just after this?

看起來後續還在進行中...

Linode 記憶體升級,以及新的日本機房計畫

Linode 的 13 歲禮物:「Linode’s 13th Birthday – Gifts for All!」。包括了記憶體的升級計畫:

Old PlanNew PlanPrice
Linode 1 GBLinode 2 GB$10/mo ($0.015/hr)
Linode 2 GBLinode 4 GB$20/mo ($0.03/hr)
Linode 4 GBLinode 8 GB$40/mo ($0.06/hr)
Linode 8 GBLinode 12 GB$80/mo ($0.12/hr)
Linode 16 GBLinode 24 GB$160/mo ($0.24/hr)
Linode 32 GBLinode 48 GB$320/mo ($0.48/hr)
Linode 48 GBLinode 64 GB$480/mo ($0.72/hr)
Linode 64 GBLinode 80 GB$640/mo ($0.96/hr)
Linode 96 GBLinode 120 GB$960/mo ($1.44/hr)

比較小的機器都是 double RAM,比較大的機器就沒那麼明顯了... 但這樣就超越 DigitalOcean 的規格,而且還領先其他 VPS 不少。

不過由於東京機房已經滿了,這次升級不包括在內,但也透漏了東京的新機房將會在今年年底前啟用:

Unfortunately, since Tokyo is sold out, the upgrade is not available there. We hope to have our second Tokyo facility online before the end of the year.

是個好消息 XD

AWS 最新的 x1.32xlarge...

Amazon EC2 推出了 x1.32xlarge:「X1 Instances for EC2 – Ready for Your Memory-Intensive Workloads」。

看這精美的規格:

Processor: 4 x Intel™ Xeon E7 8880 v3 (Haswell) running at 2.3 GHz – 64 cores / 128 vCPUs.
Memory: 1,952 GiB with Single Device Data Correction (SDDC+1).
Instance Storage: 2 x 1,920 GB SSD.

不過需要申請才能用,目前也只有其中幾區有提供:

If you are ready to start using the X1 instances in the US East (Northern Virginia), US West (Oregon), Europe (Ireland), Europe (Frankfurt), Asia Pacific (Tokyo), Asia Pacific (Singapore), or Asia Pacific (Sydney) Regions, please request access and we’ll get you going as soon as possible.

這樣以後就不能說「用 C4 就對了」...

調整 MySQL 的記憶體用量

Percona 的「Best Practices for Configuring Optimal MySQL Memory Usage」這篇給了個蠻不錯的建議:

Don’t allow the mysqld process VSZ exceed 90% of the system memory

ps ax -O vsz | grep mysqld 可以看到 mysqld 吃了多少 VSZ,然後自己除整台機器的記憶體大小,就可以算出來目前吃了多少,然後調整 innodb_buffer_pool_size 的數字。

另外 performance schema 也會有不少影響:

MySQL is workload related – if you have many connections active at the same time that run heavy selects using a lot of memory for sorting or temporary tables, you might need a lot of memory (especially if Performance Schema is enabled). In other cases this amount of memory is minimal. You’ll generally need somewhere between 1 and 10GB for this purpose.

記憶體碎裂問題應該是在調整時就會考慮進去:

Another thing you need to account for is memory fragmentation. Depending on the memory allocation library you’re using (glibc, TCMalloc, jemalloc, etc.), the operating system settings such as Transparent Huge Pages (THP) and workload may show memory usage to grow over time (until it reaches some steady state). Memory fragmentation can also account for 10% or more of additional memory usage.

其實就是沒有一定的定論,在不同的系統上會有不同的反應...

Google Compute Engine 推出 Custom Machine Type

Google Compute Engine 推出了可以自己設定 CPU 與 RAM 的機器種類:「Custom Machine Types - Compute Engine — Google Cloud Platform」。

可以從 1 個 vCPU 到 32 個 vCPU,而記憶體最多是 6.5GB * vCPU 數,所以理論上最高是 208GB?

Create a machine type with as little as 1 vCPU and up to 32 vCPUs, or any even number of vCPUs in between. Memory can be configured up to 6.5 GB of RAM per vCPU.

計價方式就是 vCPU 算一份,記憶體算一份。記得以前有比較小的 Cloud Service 有提供過類似的計價方式,後來都收掉了...

OpenSSH client 的重大安全性更新

CVE-2016-0777 與 CVE-2016-0778 安全性漏洞是關於 OpenSSH client 的部分:(USN-2869-1: OpenSSH vulnerabilities)

It was discovered that the OpenSSH client experimental support for resuming connections contained multiple security issues. A malicious server could use this issue to leak client memory to the server, including private client user keys.

這下除了要更新以外,要重新生 ssh key 然後更新一堆機器了...

PuTTY 安全性問題 (CVE-2015-5309)

雖然很久沒用 PuTTY 了 (因為用 Ubuntu 很久了),不過很難得看到 PuTTY 有安全性問題。

PuTTY 官方發佈了安全性通報 CVE-2015-5309:「PuTTY vulnerability vuln-ech-overflow」:

Versions of PuTTY and pterm between 0.54 and 0.65 inclusive have a potentially memory-corrupting integer overflow in the handling of the ECH (erase characters) control sequence in the terminal emulator.

不過老問題還是沒解啊,透過 HTTPS (i.e. Certificate authority 架構) 雖然有很多問題,但至少還是個靠稽核制度而建立的安全信任機制,在沒有任何可信任環境下可以當作起點下仍然是最好的方案:「如何安全下載軟體...」。

PHP 5.5 的 Generators

在「Save memory by switching to generators」這邊提到了 PHP 5.5 開始提供的 Generators...

由於其他的程式語言有 Generators 的觀念,其實不會太難了解...

不過比較大的問題是,資料庫的查詢操作用 Generators 會把效能壓力壓回資料庫:因為資料庫需要把結果 buffering 在資料庫端,如果不趕快吐出去就是要找記憶體放... 也因此,比較常見到的解法是不要用 Generators。(因為 web 與 application 端相較於資料庫端,比較容易 scale)

後來用 Generators 比較多的印象中還是 filter 類的應用吧,Python 這邊的東西有陣子沒看了 :o

Facebook 推出靜態分析工具:Facebook Infer

Facebook 推出了靜態分析工具 Facebook Infer,可以事先找出 AndroidiOS 上的 bug:Open-sourcing Facebook Infer: Identify bugs before you ship

從官方給的操作動畫中就可以看出來怎麼跑了。目前看起來支援三種程式語言,C、Objective-C、Java:

Facebook Infer is a static analysis tool - if you give Infer some Objective-C, Java, or C code, it produces a list of potential bugs.

在 Android 上 (Java) 會找出的類型:

Infer reports null pointer exceptions and resource leaks in Android and Java code.

iOS 上則只找 memory leak:

In addition to this, it reports memory leak problems in iOS and C code.

比較特別的是,這個工具是用 OCaml 寫:

Infer is a static analysis tool for Java, Objective-C and C, written in OCaml.