用 Go 寫的 Badger

Dgraph 在推銷自家發展出來的 Badger:「Introducing Badger: A fast key-value store written natively in Go」。

標靶是 RocksDB,號稱比 RocksDB 快好幾倍:

Based on benchmarks, Badger is at least 3.5x faster than RocksDB when doing random reads. For value sizes between 128B to 16KB, data loading is 0.86x - 14x faster compared to RocksDB, with Badger gaining significant ground as value size increases. On the flip side, Badger is currently slower for range key-value iteration, but that has a lot of room for optimization.

不過我覺得有些重要的功能在 Badger 不提供,這比起來有種橘子比蘋果的感覺... 像是 RocksDB 提供了 Transaction,而 Badger 則是直接講明他們不打算支援 Transaction:

Keep it simple, stupid. No support for transactions, versioning or snapshots -- anything that can be done outside of the store should be done outside.

Linux 上跑電腦圍棋程式 (CGOS)

這邊講的不是對人下的,而是電腦之間的對弈。

目前大多數的對弈軟體會到「CGOS - A Go Server Just for Computers」這邊對弈,當然目前最紅的是一般 (人類) 正式比賽用的 19x19 棋盤,數子採用中國規則 (相當於貼 7.5 目)。

不過與常見的正式比賽比較不一樣的是採用包干制,每方限時 15 分鐘,超時就直接裁定敗,不過有個小例外:

CGOS silently adds a fraction of a second to each players clock for each move played.

要把圍棋程式接到 CGOS 上面需要兩段程式,一段是 Go Engine 本身要支援 Go Text Procotol (GTP),另外一段是把 GTP 接到 CGOS。

前面 Go Engine 的部份,目前不少圍棋軟體都有支援 GTP,像是 Leela 或是 Ray

後者一般會用 Python CGOS Client

其中比較特別的是 CGOS 的帳號密碼,帳號只允許 18 個字,另外沒有帳號申請系統,第一次用什麼帳號他就自動記錄起來,之後就要用這組。

# config.cfg
Common:
  KillFile = kill.txt

# First engine
GTPEngine:
  Name = Leela090-test
  CommandLine = ./leela090-gtp.sh

  ServerHost = yss-aya.com
  ServerPort = 6819
  ServerUser = Leela090-test
  ServerPassword = mypassword

  NumberOfGames = 1

  SGFDirectory = sgf

然後建立 sgf 目錄存棋譜後,用 python bin/cgosclient.py config.cfg 跑起來。當你 touch kill.txt 後,下一盤棋就會自己結束 (預設是一直下)。

再來是裡面提到的 leela090-gtp.sh

#!/bin/bash
exec /usr/bin/nice -n 20 $(dirname $0)/leela_090_linux_x64_opencl -g -t 8 -b 50 -q -l /tmp/leela090-gtp.log

這邊跑的是 OpenCL 版本,你也可以跑 CPU 版。其中 -g 是 GTP mode,-t 是 CPU thread 數量,-b 是 network latency penalty (避免超時),-q 是 quiet mode,-l 是 log。

AlphaGo 又要重出江湖了

DeepMind 家的 AlphaGo 又要重出江湖了:「Exploring the mysteries of Go with AlphaGo and China's top players」。這次將會跟中國的頂尖職業棋手對弈,裡面提到三種賽制:

The summit will feature a variety of game formats involving AlphaGo and top Chinese players, specifically designed to explore the mysteries of the game together. The games will include:

  • “Pair Go” — A game where one Chinese pro will play against another...except they will both have their own AlphaGo teammate, alternating moves, to take the concept of ‘learning together’ quite literally.
  • “Team Go” — A game between AlphaGo and a five-player team consisting of China’s top pro players, working together to test AlphaGo’s creativity and adaptability to their combined style.
  • “Ke Jie vs AlphaGo” — Of course, the centerpiece of the event will be a classic 1:1 match of three games between AlphaGo and the world’s number one player, Ke Jie, to push AlphaGo to (...perhaps beyond!) its limits.

有傳言是這樣打 (目前大家都抄來抄去...),但目前還沒看到正式的公告:

(一)5月23,25,27日 柯潔與AlphaGo三番棋:每方3小時,5次1分鐘讀秒;
(二)5月26日 時越、羋昱廷、唐韋星、陳耀燁和周睿羊5人團隊賽:每方2小時30分,保留3次1分鐘讀秒;
(三)5月26日 古力、連笑人機配對賽:每方1小時,1次1分鐘讀秒。

另外有一篇「Innovations of AlphaGo」講了不少年初 AlphaGo 連勝六十盤的對局給人類棋手的啟發。

另外 DeepMind 的 CEO,Demis Hassabis 前幾天剛好有給了一個演講,也是講了不少東西 (不過都是之前就提過的):

這樣來安排那幾天空時間下來...

Cloudflare 因為閏秒炸掉...

Cloudflare 這次閏秒炸掉:「How and why the leap second affected Cloudflare DNS」,影響範圍包括了 DNS query 與 HTTP request:

At peak approximately 0.2% of DNS queries to Cloudflare were affected and less than 1% of all HTTP requests to Cloudflare encountered an error.

主要的原因在於 Gotime.Now() 不保證遞增:

RRDNS is written in Go and uses Go’s time.Now() function to get the time. Unfortunately, this function does not guarantee monotonicity. Go currently doesn’t offer a monotonic time source (see issue 12914 for discussion).

修正的方式是針對時間回朔時處理:

In this patch we allowed RRDNS to forget about current upstream performance, and let it normalize again if time skipped backwards.

應該是因為 Cloudflare 這段程式還沒遇過 leap second 造成的...

Golang 本身也出字型了...

不愧是 Google 在後面撐腰,Golang 找人設計了一整套字型,叫做「Go fonts」:

The Go fonts are divided into two sets, Go proportional, which is sans-serif, and Go Mono, which is slab-serif.

不過看了一下 monospace 的部份,有點微妙:

其他兩個也許哪天換心情來用看看好了 :o

Go 1.9 的 GC 改善

Update:被提醒後仔細看了一下,是 1.8 預設生效 (但保留選項切回來 debug),如果沒問題的話 1.9 把舊的方式拔乾淨:

Assuming things go smoothly, we will remove stack re-scanning support when the tree opens for Go 1.9 development.

標題就不改了... 以下原文。

在「Sub-millisecond GC pauses」這邊看到的。Golang 想辦法將 GC 造成的影響降低:「Proposal: Eliminate STW stack re-scanning」。

目標是解決最大的 GC pause 來源:

As of Go 1.7, the one remaining source of unbounded and potentially non-trivial stop-the-world (STW) time is stack re-scanning.

然後拿新的解法來戰,目前初步的測試看起來可以降到 50µs (== 0.05ms):

We propose to eliminate the need for stack re-scanning by switching to a hybrid write barrier that combines a Yuasa-style deletion write barrier [Yuasa '90] and a Dijkstra-style insertion write barrier [Dijkstra '78]. Preliminary experiments show that this can reduce worst-case STW time to under 50µs, and this approach may make it practical to eliminate STW mark termination altogether.

在「runtime: eliminate stack rescanning · Issue #17503 · golang/go」這邊可以看到進度,現在已經在 master branch 上了,看起來會在 1.9 的時候被放出來... 不過 worst case 的時間上修了 XDDD

The high level summary is that this reduces worst-case STW time to about 100 µs and typical 95%ile STW time to 50 µs (assuming, of course, that the OS doesn't get in the way and that the system isn't otherwise overloaded).

但看起來應該還是很大的效能改善,尤其是 CPU bound 的應用?

玩 Pokémon Go 抓怪用的 iPhone 手機殼...

看到「This iPhone thong case will help you catch 'em all in 'Pokémon Go'」這個,光看圖就知道意思了:

應該是用 3D Printer 做出來的:

If this sounds familiar, it's basically the commercialized, rubberized version of the 3D-printed Poké Ball "aimer" case that one guy made.

不過這不能丟 Curveball 啊 XDDD

Golang 1.7

Golang 1.7 主打更小的 binary size:「Smaller Go 1.7 binaries」:

Typical programs, ranging from tiny toys to large production programs, are about 30% smaller when built with Go 1.7.

還附了一張經典的「Hello, world」程式的分析:

由於現代 CPU 的速度與 L1/L2/... cache 有緊密關係,當 binary size 變小時,常常會伴隨著 memory access 變快 (因為 hitrate 提昇),所以 binary size 也是效能指數蠻重要的一環。

Go 上面的白箱安全性檢查

HP 的 open source 專案「Go AST Scanner」,分析 Go 的原始程式碼拉出 AST 進行分析 (Static program analysis),再找出可能的安全性問題。

雖然是 alpha 階段,但看起來是個好東西啊... 至少寫的太誇張的 SQL injection 可以掃出來。

舊 bug 新名字:httpoxy

依照慣例,security issue 都會取個名字,這次叫做 httpoxy:「A CGI application vulnerability for PHP, Go, Python and others」。

事情發生在兩個命名變數上的衝突:

  • RFC 3875 (The Common Gateway Interface (CGI) Version 1.1) 定義了 CGI 環境會把 Header 裡的 Proxy 欄位放到環境變數裡的 HTTP_PROXY
  • 而很多程式會拿環境變數裡的 HTTP_PROXY 當作 proxy 設定。

這件事情 2001 年在 libwww-perl 就有發生過 (並且修正),curl 也發生過 (然後修正),2012 年在 Ruby 的 Net::HTTP 也發生過 (也修正了)。

然後在 2016 年還是被發現有很多應用程式會中獎... 這頭好痛啊 :o