zetcd:用 etcd 跑 ZooKeeper 架構

在「zetcd: running ZooKeeper apps without ZooKeeper」這邊介紹了用 etcd 當作 ZooKeeper 伺服器。程式碼在「Serve the Apache Zookeeper API but back it with an etcd cluster」這邊可以看到。

不過可以看到有不少 overhead:

但 etcd 用 Go 寫 (省下 JVM tuning?),可能是個不錯的誘因...

用 Go 寫的 Badger

Dgraph 在推銷自家發展出來的 Badger:「Introducing Badger: A fast key-value store written natively in Go」。

標靶是 RocksDB,號稱比 RocksDB 快好幾倍:

Based on benchmarks, Badger is at least 3.5x faster than RocksDB when doing random reads. For value sizes between 128B to 16KB, data loading is 0.86x - 14x faster compared to RocksDB, with Badger gaining significant ground as value size increases. On the flip side, Badger is currently slower for range key-value iteration, but that has a lot of room for optimization.

不過我覺得有些重要的功能在 Badger 不提供,這比起來有種橘子比蘋果的感覺... 像是 RocksDB 提供了 Transaction,而 Badger 則是直接講明他們不打算支援 Transaction:

Keep it simple, stupid. No support for transactions, versioning or snapshots -- anything that can be done outside of the store should be done outside.

Google 弄出來的 Grumpy:把 Python 2.7 的程式碼轉成 Go...

Google 放出 Grumpy,可以把 Python 2.7 的程式碼轉成 Go:「Grumpy: Go running Python!」。

下面看到一個留言頗有趣的:

This sad to see that Grumpy is mean to be a replacement of CPython 2.7 instead of CPython 3.x . I presume the code from youtube was written in python 2.x hence the reason but I hope we'll see Grumpy supporting python 3.x :)

回到原文,這次的需求主要是出自 YouTube 的需求:

The front-end server that drives youtube.com and YouTube’s APIs is primarily written in Python, and it serves millions of requests per second! YouTube’s front-end runs on CPython 2.7, so we’ve put a ton of work into improving the runtime and adapting our application to work optimally within it.

然後 Python 的 GIL 又被拿出來鞭屍:

除了 C extension 不支援外,還是有些「過於動態」的語法不支援:

exec, eval and compile: These dynamic features of CPython are not supported by Grumpy because Grumpy modules consist of statically compiled Go code. Supporting dynamic execution would require bundling Grumpy programs with the compilation toolchain which would be unwieldy and impractically slow.

這樣可用的範圍少不少,這個專案可以當作 YouTube 這種規模的網站所做的改善,而不是什麼可以拿來用的工具 :o

Cloudflare 因為閏秒炸掉...

Cloudflare 這次閏秒炸掉:「How and why the leap second affected Cloudflare DNS」,影響範圍包括了 DNS query 與 HTTP request:

At peak approximately 0.2% of DNS queries to Cloudflare were affected and less than 1% of all HTTP requests to Cloudflare encountered an error.

主要的原因在於 Gotime.Now() 不保證遞增:

RRDNS is written in Go and uses Go’s time.Now() function to get the time. Unfortunately, this function does not guarantee monotonicity. Go currently doesn’t offer a monotonic time source (see issue 12914 for discussion).

修正的方式是針對時間回朔時處理:

In this patch we allowed RRDNS to forget about current upstream performance, and let it normalize again if time skipped backwards.

應該是因為 Cloudflare 這段程式還沒遇過 leap second 造成的...

Golang 本身也出字型了...

不愧是 Google 在後面撐腰,Golang 找人設計了一整套字型,叫做「Go fonts」:

The Go fonts are divided into two sets, Go proportional, which is sans-serif, and Go Mono, which is slab-serif.

不過看了一下 monospace 的部份,有點微妙:

其他兩個也許哪天換心情來用看看好了 :o

Go 1.9 的 GC 改善

Update:被提醒後仔細看了一下,是 1.8 預設生效 (但保留選項切回來 debug),如果沒問題的話 1.9 把舊的方式拔乾淨:

Assuming things go smoothly, we will remove stack re-scanning support when the tree opens for Go 1.9 development.

標題就不改了... 以下原文。

在「Sub-millisecond GC pauses」這邊看到的。Golang 想辦法將 GC 造成的影響降低:「Proposal: Eliminate STW stack re-scanning」。

目標是解決最大的 GC pause 來源:

As of Go 1.7, the one remaining source of unbounded and potentially non-trivial stop-the-world (STW) time is stack re-scanning.

然後拿新的解法來戰,目前初步的測試看起來可以降到 50µs (== 0.05ms):

We propose to eliminate the need for stack re-scanning by switching to a hybrid write barrier that combines a Yuasa-style deletion write barrier [Yuasa '90] and a Dijkstra-style insertion write barrier [Dijkstra '78]. Preliminary experiments show that this can reduce worst-case STW time to under 50µs, and this approach may make it practical to eliminate STW mark termination altogether.

在「runtime: eliminate stack rescanning · Issue #17503 · golang/go」這邊可以看到進度,現在已經在 master branch 上了,看起來會在 1.9 的時候被放出來... 不過 worst case 的時間上修了 XDDD

The high level summary is that this reduces worst-case STW time to about 100 µs and typical 95%ile STW time to 50 µs (assuming, of course, that the OS doesn't get in the way and that the system isn't otherwise overloaded).

但看起來應該還是很大的效能改善,尤其是 CPU bound 的應用?

Golang 1.7

Golang 1.7 主打更小的 binary size:「Smaller Go 1.7 binaries」:

Typical programs, ranging from tiny toys to large production programs, are about 30% smaller when built with Go 1.7.

還附了一張經典的「Hello, world」程式的分析:

由於現代 CPU 的速度與 L1/L2/... cache 有緊密關係,當 binary size 變小時,常常會伴隨著 memory access 變快 (因為 hitrate 提昇),所以 binary size 也是效能指數蠻重要的一環。

Go 上面的白箱安全性檢查

HP 的 open source 專案「Go AST Scanner」,分析 Go 的原始程式碼拉出 AST 進行分析 (Static program analysis),再找出可能的安全性問題。

雖然是 alpha 階段,但看起來是個好東西啊... 至少寫的太誇張的 SQL injection 可以掃出來。

Go 1.6 把 HTTP/2 變成預設支援的功能

Go 的官方公告「Go 1.6 is released」提到了把 net/http 的 HTTP/2 預設啟用了:

In Go 1.6, support for HTTP/2 is enabled by default for both servers and clients when using HTTPS, bringing the benefits of the new protocol to a wide range of Go projects, such as the popular Caddy web server.

另外值得一提的是 sort 演算法的效能改善:

The algorithm inside sort.Sort was improved to run about 10% faster, but the change may break programs that expect a specific ordering of equal but distinguishable elements.

這應該算還蠻基本常用到的東西,會改善很多程式效能...

Google 的 Load balancer:Seesaw

前幾天因為流感而睡太多,來消化一些文章。

上個星期 Google 放出一套用 Go 寫的 Load balancer,叫 Seesaw:「Seesaw: scalable and robust load balancing」。

比較有趣的是 BGP 與 anycast VIP 的能力:

Seesaw v2 provides full support for anycast VIPs - that is, it will advertise an anycast VIP when it becomes available and will withdraw the anycast VIP if it becomes unavailable.

Google 這個規模玩的是不同 scale 的花樣...