Home » Posts tagged "time" (Page 3)

Facebook 在 MySQL 裡存時間的型態

MySQL at Facebook這邊說明提到了,Facebook 內部是使用 INT UNSIGNED 儲存時間:

Which gets us to the point that it is no different than storing INT (hello 2038?) or UNSIGNED INT (a bit later) or BIGINT (till the end of time) and possibly passing binary values in efficient protocols eventually.

If you got that far of this post, your likes in Facebook graph are stored with 'INT UNSIGNED' time field.

順道一提,INT 是 2038 年問題,INT UNSIGNED 是 2106 年問題。

而 Facebook 在 MySQL 上會選擇不使用 DATETIMETIMESTAMP 的原因其實跟技術搭不上太多關係,主因是因為 MySQL 根本沒打算修 XDDD

It is my favorite MySQL bug, simply because it forces any reasonable mind not to use TIMESTAMP, and MySQL is never going to fix it (nor will ever understand time). I lost my temper a bit on that bug: https://bugs.mysql.com/bug.php?id=38455

我的猜測是已經爛成一團了,而且大家都有 workaround (呃,其實就是 Facebook 推薦用 INT UNSIGNED 的方法),再考慮到有一票現有程式,在上面狂用 side effect 讓執行結果正確,不如就不要修這種吃力不討好的東西了 XDDD

另外一方面 timezone 資訊其實常常變化,常常需要更新 MySQL 的 timezone database (而這對於維運來說不是什麼開心的事情):

There're few ways around that. One of them is side-load and maintain timezone data inside MySQL itself - it has support for internal timezone database and tracks obscure time shifts like ones for "Pacific War Time" and "Pacific Peace Time". That is operationally feasible (you have to remind yourself to update the database whenever time rules change, and they do change a lot, if you consider every timezone in the world), but has limited value.

這就是為什麼大家遇到 MySQL 時都會推薦用 INT UNSIGNED 了...

另外可以參考三年前的文章「MySQL 裡儲存時間的方式...」,裡面引用了 Baron Schwartz 的說明:

All date and time columns shall be INT UNSIGNED NOT NULL, and shall store a Unix timestamp in UTC.

其實這已經是個 best practice 了...

除了 DNS 的 TTL 外,還有瀏覽器本身的 cache time...

在看「Reviewing Fastly’s New Approach To Load Balancing In The Cloud」這篇的時候被提醒:

However, most browsers have implemented their own caching layer that can override the TTL specified by the server. In fact, some browsers cache for 5-10 minutes, which is an eternity when a region or data center fails and you need to route end users to a different location.


結果 IE 在「How Internet Explorer uses the cache for DNS host entries」直接說三十分鐘 XDDD 這篇文章是 2011 年更新的,所以至少到 IE9 都是對的?

Internet Explorer 4.x and later versions modify how DNS host entries are cached by decreasing the default time-out value to 30 minutes.

Firefox 的值可以從 Mozilla networking preferences 這邊對 network.dnsCacheExpiration 的說明看到是 60 秒。

Google Chrome 沒找到官方的說明...

不過這可以知道當你要換 IP address 時,如果可以讓新舊 IP 都提供服務的話,至少規劃半個小時會比較保險。如果有其他理由而沒辦法同時提供服務的話,至少公告步驟裡要有「重開瀏覽器」這塊。

而作業系統自己的 cache 又是另外要計算進去的事了...

Amazon ECS 可以跑 cron job 了...

Amazon ECS 上面固定時間跑某些東西,以前得自己用 AWS Lambda 帶 (或是自己架,不過這樣就要自己考慮 High Availability 架構了),現在則是直接支援:「Amazon ECS Now Supports Time and Event-Based Task Scheduling」。

Previously, you could start and stop Amazon ECS tasks manually, but running tasks on a schedule required writing and integrating an external scheduler with the Amazon ECS API.

Now you can schedule tasks through the Amazon ECS console on fixed time intervals (e.g.: number of minutes, hours, or days). Additionally, you can now set Amazon ECS as a CloudWatch Events target, allowing you to launch tasks by using CloudWatch Events.

微軟的 Time Service 回應錯誤的時間...

看起來會有不少災情 (像是 SQL Server 遇到使用 server side 的時間的 SQL query):「Windows Time Service is sending out wrong times and that’s a big problem」,報導裡引用了 Reddit 上「PSA: time.windows.com NTP server seems to be sending out wrong time」這邊的討論串。

為了避免這種情況,不同單位會用不同方法解決。像是財力充足的 Google 就自己搞了原子鐘,然後還放 Google Public NTP 出來給大家用。可以不倚靠外部裝置確保自家時間的正確性。

另外是有人用 Raspberry Pi 收 GPS 訊號轉成 NTP service (像是「The Raspberry Pi as a Stratum-1 NTP Server」這邊介紹的方式),不過之前有發生過 GPS 送出來的時間差了 13ms 的事情,也不是完全可靠 (不過相較起來應該還是可以接受):「GPS error caused '12 hours of problems' for companies」。另外可能的方案有 GLONASS (俄羅斯的系統)。


Amazon EC2 上的 gettimeofday 與 clock_gettime 的效能

看到「Two frequently used system calls are ~77% slower on AWS EC2」這篇在講 gettimeofdayclock_gettime 的效能,另外搜資料時發現應該也是作者提問的「gettimeofday() not using vDSO?」這篇。

EC2 比較新的機器上用 tsc 應該是沒問題的 (在 2015 的時候官方就這樣建議了):

it seems tsc support in Xen has improved with version 4.0 and with improved CPU support in Sandy Bridge+ platforms. Modern EC2 machines should be okay with tsc. Check Xen version using dmesg | grep "Xen version". Amazon recommended the tsc clocksource already in re:Invent 2015 (https://www.slideshare.net/AmazonWebServices/cmp402-amazon-ec2-instances-deep-dive). I'm not yet running to production with this, but the situation doesn't seem as bad as implied by packagecloud.

開了一台 t2.micro 看 /sys/devices/system/clocksource/clocksource0/current_clocksource 看起來目前是設成 xen

ubuntu@ip-172-31-22-165:~$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource

在「(CMP402) Amazon EC2 Instances Deep Dive」這邊也可以看到一些資料 (page 24 與 page 25):

DynamoDB 推出 TTL 功能

DynamoDB 推出了依照時間自動刪除的功能:「New – Manage DynamoDB Items Using Time to Live (TTL)」。

You can enable this feature on a table-by-table basis, specifying an item attribute that contains the expiration time for the item.

這個功能比較特別的是,刪除的 scan 是不收取費用的:

There is no charge for the internal scan operation or for the deletion. You will pay for storage until the item is actually deleted.

Cloudflare 因為閏秒炸掉...

Cloudflare 這次閏秒炸掉:「How and why the leap second affected Cloudflare DNS」,影響範圍包括了 DNS query 與 HTTP request:

At peak approximately 0.2% of DNS queries to Cloudflare were affected and less than 1% of all HTTP requests to Cloudflare encountered an error.

主要的原因在於 Gotime.Now() 不保證遞增:

RRDNS is written in Go and uses Go’s time.Now() function to get the time. Unfortunately, this function does not guarantee monotonicity. Go currently doesn’t offer a monotonic time source (see issue 12914 for discussion).


In this patch we allowed RRDNS to forget about current upstream performance, and let it normalize again if time skipped backwards.

應該是因為 Cloudflare 這段程式還沒遇過 leap second 造成的...

Google 的 time.google.com

看到這張圖在講不同 server (service) 如何處理今年的 leap second (UTC 的跨年,台灣時間早上八點),出自 leap smear 這邊:

在很早前就有 time.google.com 這個 domain,但是當時 Google 的人有跳出來說明這個服務不是公開服務 (當時),不保證這個服務的正確性與穩定性:「timeX.google.com provide non standard time」。

不過一個月前公佈出來的 Google Public NTP 服務算是把整個系統搞定了。

其中在 Configuring Clients 這邊直接推薦用 iburst 參數,不愧是家大業大的 Google XDDD:

When the server is unreachable and at each poll interval, send a burst of eight packets instead of the usual one. As long as the server is unreachable, the spacing between packets is about 16s to allow a modem call to complete. Once the server is reachable, the spacing between packets is about 2s. This is designed to speed the initial synchronization acquisition with the server command and s addresses and when ntpd is started with the -q option.

回到原來的 leap smear 的比較圖,可以看出 Google 對 leap second 的解法是往前十二小時與往後十二小時各拉緩衝時間來避開,有些是沒在管,另外有些有種來亂的感覺 XDDD