Tag: kv

Akamai 也推出了 Key-Value 服務 EdgeKV

沒介紹過 Akamai 的一些架構，先講到 Akamai 的 Edge 端 Serverless 架構是 EdgeWorkers，跑的是 JavaScript：

EdgeWorkers lets developers just code — integrating into existing CI/CD workflows and enabling multiple teams to work in parallel using JavaScript. EdgeWorkers eliminates the hassle of managing compute resources and building for scale.

然後這次推出的是 EdgeKV，目前還在 Beta 版：「Serverless Storage at the Edge (EdgeKV Beta)」。

如同名字所說的，架構上 Key-Value 架構，放棄了 CAP theorem 裡面的 C，改走 Eventual Consistency：

EdgeKV uses what is known in distributing computing as an eventual consistency model to perform writes and updates. This model achieves high availability with low read latency by propagating data writes globally. The period of time it takes the system to distribute data globally is called the “inconsistency window”.

隔壁 Cloudflare Workers KV 也是 Eventual Consistency (出自「How KV works」這邊)：

KV achieves this performance by being eventually-consistent. Changes are immediately visible in the edge location at which they're made, but may take up to 60 seconds to propagate to all other edge locations.

看起來算是補上競爭對手的產品線...

Cloudflare 的 Workers KV

Cloudflare 推出了 Workers KV 服務：「Building With Workers KV, a Fast Distributed Key-Value Store」。

是個 key-value 結構服務 (全球性，eventually consistent，約 10 秒的同步時間)，key 的限制是 2KB，value 是 64KB，一個 namespace 最多 10 億筆資料。

讀取可以到 100k+ read/s，但寫入是 1 write/s/key，可以看出來主要是為了讀取資料而設計的。

現有的 worker ($5/month) 會送一些量，包括 1GB 的空間與一千萬次的讀取：

Your $5 monthly Workers compute minimum includes 1 GB of KV storage and up to 10 million KV reads. If you use less than the 10 million included Worker requests now, you can use KV without paying a single cent more.

超過的部份另外再收：

Beyond the minimums, Workers KV is billed at $0.50 per GB-month of additional storage and $0.50 per million additional KV reads.

這樣好像整個 blog 的基本功能都可以直接在上面跑了... 而搜尋靠外部服務，圖片與影音也可以靠外部空間協助？

GitHub 重新定位 Redis 的功能...

GitHub Engineering 說明了他們為什麼改變 Redis 的使用情境：「Moving persistent data out of Redis」。

在 GitHub 裡面，Redis 有兩種不同的情境，一種叫做 transient Redis，只用做 cache：

We used it as an LRU cache to conveniently store the results of expensive computations over data originally persisted in Git repositories or MySQL. We call this transient Redis.

另外一種則是打開 persistence 功能，叫做 persistent Redis：

We also enabled persistence, which gave us durability guarantees over data that was not stored anywhere else. We used it to store a wide range of values: from sparse data with high read/write ratios, like configuration settings, counters, or quality metrics, to very dynamic information powering core features like spam analysis. We call this persistent Redis.

這邊講的是 persistent Redis 被換成用 MySQL (InnoDB) 儲存：

Recently we made the decision to disable persistence in Redis and stop using it as a source of truth for our data. The main motivations behind this choice were to:

  • Reduce the operational cost of our persistence infrastructure by removing some of its complexity.
  • Take advantage of our expertise operating MySQL.
  • Gain some extra performance, by eliminating the I/O latency during the process of writing big changes on the server state to disk.

For the majority of callsites, we replaced persistent Redis with GitHub::KV, a MySQL key/value store of our own built atop InnoDB, with features like key expiration. We were able to use GitHub::KV almost identically as we used Redis: from trending repositories and users for the explore page, to rate limiting to spammy user detection.

後面講了不少轉換的過程 (還包含了某些功能的改寫)，但沒有講的太清楚為什麼不繼續使用 Redis。

目前只能就提到的三點問題來看，persistent 的 i/o 成本可能太高？而且難以再壓榨效能出來？而相反的，InnoDB 已經花了很多力氣在上面，直接拿來用反而可以解決問題？

不過看得出來這個轉換還是花了不少力氣，看得出來有些 application 使用 Redis 的模式不能直接搬到 InnoDB 上，花了時間改寫...