youtube-dl 的恢復與後續 GitHub 的處理模式

youtube-dl 被下架的事情後續又有不少進展了 (先前寫過「youtube-dl 被 RIAA 用 DMCA 打下來的事件」這篇),主要是 GitHub 恢復了 youtube-dl 的 Git Repository,然後丟出了一篇文章解釋過程,以及後續的作法:「Standing up for developers: youtube-dl is back」。


GitHub 的這篇文章是由 Abby Vollmer 發表的,從 LinkedIn 上的資料也可以看到他以前的經歷都是法律相關,現在在 GitHub 的頭銜是「Director of Platform Policy and Counsel」,而這篇的用字也可以看出來很小心。

首先 GitHub 認為下架的原因是 Section 1201:

Section 1201 dates back to the late 1990s and did not anticipate the various implications it has for software use today. As a result, Section 1201 makes it illegal to use or distribute technology (including source code) that bypasses technical measures that control access or copying of copyrighted works, even if that technology can be used in a way that would not be copyright infringement. Circumvention was the core claim in the youtube-dl takedown.

而 GitHub 後續恢復 youtube-dl 的原因是收到 EFF 代表 youtube-dl 開發者所做的 counter notice:「2020-11-16-RIAA-reversal-effletter.pdf」。

照以往這種把事情搞超大的慣例,RIAA 應該是不會有後續的動作,所以就 youtube-dl 這件事情來說應該是差不多告一段落。

GitHub 後續針對 Section 1201 提出了兩個改善措施,一個是重新設計 Section 1201 的處理方式,另外一個是 GitHub 自己投入資金,降低 developer 的負擔。

但整件事情背後的問題主要是在 DMCA 的設計,讓濫發 takedown notice 的人很難受罰,在這點沒有改善之前,同樣的劇碼應該都還是會繼續上演。

ScyllaDB 1.7 支援 Counters 了

在「Scylla release: version 1.7」這邊看到 ScyllaDB 支援 Counters 的消息了 (雖然剛出來,掛著 Experimental 的消息):

Scylla now supports Counters as a native type. A counter column is a column whose value is a 64-bit signed integer and on which two operations are supported: incrementing and decrementing.

這其實是 Cassandra 其中一個強項,針對 counter 這種應用特化的資料型態。

ScyllaDB 1.7 要支援 Cassandra 的 Counter 了

Counter 是 Cassandra 裡一個特別的資料型態,總算是要 porting 進 ScyllaDB 了:「Scylla 1.7 introduces experimental support for counters」。


  • once deleted, counter column value cannot be used again—this is a consequence of the fact that counters can only be incremented or decremented, they cannot be set to any specific value
  • a table cannot contain both counter and regular columns—without this limitation it wouldn’t be possible to provide proper atomicity and isolation guarantees for updates that modify both counters and regular columns in a single row
  • counter columns cannot be members of primary key
  • updates are not idempotent – in case of a write failure the client cannot safely retry the request
    counter columns cannot have time-to-live set

跟 Cassandra 一樣是透過 CRDT 實做的,行為上會是 eventually consistency:

Scylla implements counters using state-based conflict-free replicated data types (CRDT), which allow atomic operations, like increment or decrement, to be performed locally without the need for synchronization between nodes.

在「Scylla Status and Roadmap」這邊可以看到其他會在 1.7 出現的功能。

Amazon EC2 上的 gettimeofday 與 clock_gettime 的效能

看到「Two frequently used system calls are ~77% slower on AWS EC2」這篇在講 gettimeofdayclock_gettime 的效能,另外搜資料時發現應該也是作者提問的「gettimeofday() not using vDSO?」這篇。

EC2 比較新的機器上用 tsc 應該是沒問題的 (在 2015 的時候官方就這樣建議了):

it seems tsc support in Xen has improved with version 4.0 and with improved CPU support in Sandy Bridge+ platforms. Modern EC2 machines should be okay with tsc. Check Xen version using dmesg | grep "Xen version". Amazon recommended the tsc clocksource already in re:Invent 2015 ( I'm not yet running to production with this, but the situation doesn't seem as bad as implied by packagecloud.

開了一台 t2.micro 看 /sys/devices/system/clocksource/clocksource0/current_clocksource 看起來目前是設成 xen

ubuntu@ip-172-31-22-165:~$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource

在「(CMP402) Amazon EC2 Instances Deep Dive」這邊也可以看到一些資料 (page 24 與 page 25):

PostgreSQL 9.5 釋出,UPSERT!

PostgreSQL 9.5 正式發行,這次新增了大家期待已久的 UPSERT 功能:「PostgreSQL 9.5: UPSERT, Row Level Security, and Big Data」。

SQL:2003 正式定義出 UPSERT,被稱為 Merge,不過看網路上一般還是比較習慣 UPSERT 這個用法:

A relational database management system uses SQL MERGE (also called upsert) statements to INSERT new records or UPDATE existing records depending on whether condition matches.

也就是當沒資料的時候就 INSERT,有資料的時候就 UPDATE 的語法。常見的使用情境是拿來當 counter 用 (雖然這很傷資料庫的效能)。

沒有 UPSERT 的時候只能用 transaction 或是 store procedure 搭出來,效能上會比在 database engine 裡實作來的差,所以 UPSERT 還是被實作出來了。