DNSFilter 使用 InfluxDB 與 TimescaleDB 的過程

DNSFilter 這篇講 InfluxDBTimescaleDB 的文章頗有趣的:「Towards 3B time-series data points per day: Why DNSFilter replaced InfluxDB with TimescaleDB」。

在沒有實際用過之前,其實都只能算是一方之詞... 另外這種轉換其實也跟每個公司內的組織組成有關,像是熟悉 PostgreSQL 的單位就比較有機會用 TimescaleDB 解決 time series data 的問題。

不過有個地方倒是讓我想記錄起來:

Comparing TimescaleDB to InfluxDB at the same time — we realized we were losing data. InfluxDB relied on precisely timed execution of rollup commands to process the last X minutes of data into rollups. Combined with our series of rollups, we realized that some slow queries were causing us to lose data. The TimescaleDB data had 1–5% more entries! Also we no longer had to deal with cardinality issues, and could show our customers every last DNS request, even at a monthly rollup.

會掉資料等於是跟 InfluxDB 的使用者發出警訊,要大家確認自己手上的資料是否正確... 這對於正確性要求 100% 的應用就不是開玩笑了 @_@