Brave 出手檢舉 Google 沒有遵守 GDPR

Brave (從 Chromium 分支出來的瀏覽器) 檢舉 Google 沒有遵守 GDPR 的規定:「Formal GDPR complaint against Google’s internal data free-for-all」。

主要是「purpose limitation」這個部份,出自「REGULATION (EU) 2016/679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 27 April 2016」:

1. Personal data shall be:


collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes; further processing for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes shall, in accordance with Article 89(1), not be considered to be incompatible with the initial purposes (‘purpose limitation’);

比較重要的是 specified 與 explicit 這兩個詞,GDPR 規定必須明確指明用途,而可以從整理出來的文件「Inside the black box」裡的「Purported processing purpose」看到大量的極為廣泛的說明。

Google 應該會就這塊反擊認為這樣的描述就夠用,就看歐盟決定要怎麼做了...

Google 用 x-client-data 追蹤使用者的問題

前陣子 Chromium 團隊在研究要移除 User-Agent 字串的事情 (參考「User-Agent 的淘汰提案」),結果 kiwibrowser 就直接炸下去,Google 很久前就會針對自家網站送出 x-client-data 這個 HTTP header,裡面足以辨識使用者瀏覽器的單一性:「Partial freezing of the User-Agent string#467」。

Google 的白皮書裡面是說用在 server 的試驗:

We want to build features that users want, so a subset of users may get a sneak peek at new functionality being tested before it’s launched to the world at large. A list of field trials that are currently active on your installation of Chrome will be included in all requests sent to Google. This Chrome-Variations header (X-Client-Data) will not contain any personally identifiable information, and will only describe the state of the installation of Chrome itself, including active variations, as well as server-side experiments that may affect the installation.

The variations active for a given installation are determined by a seed number which is randomly selected on first run. If usage statistics and crash reports are disabled, this number is chosen between 0 and 7999 (13 bits of entropy). If you would like to reset your variations seed, run Chrome with the command line flag “--reset-variation-state”. Experiments may be further limited by country (determined by your IP address), operating system, Chrome version and other parameters.

但因為這個預設值開啟的關係,就算關掉後也足以把使用者再分類到另外一個區塊,仍然具有高度辨識性,不是你 Google 說無法辨識就算數。

另外如果看 source code 裡的說明:

    // Note the criteria for attaching client experiment headers:
    // 1. We only transmit to Google owned domains which can evaluate
    // experiments.
    //    1a. These include hosts which have a standard postfix such as:
    //         * or * or
    //         exactly or
    //         international TLD domains *.google. or *.youtube..
    // 2. Only transmit for non-Incognito profiles.
    // 3. For the X-Client-Data header, only include non-empty variation IDs.

可以看到 ** 全部都是廣告相關,另外 Google 自家搜尋引擎是直接提供廣告 (不透過前面提到的網域),YouTube 也是一樣的情況,所以完全可以猜測 x-client-data 這個資料就是用在廣告相關的系統上。

The Register 在「Is Chrome really secretly stalking you across Google sites using per-install ID numbers? We reveal the truth」這邊用粗體的 Update 提到了 GDPR 的問題,不確定是不是開始有單位在調查了:

Updated Google is potentially facing a massive privacy and GDPR row over Chrome sending per-installation ID numbers to the mothership.

在這個問題沒修正之前,只能暫時用操作 HTTP header 的 extension 移掉這個欄位。

Python 3.7+ 保證 dict 內容的順序

在「Dicts are now ordered, get used to it」這邊看到的,因為 Python 官方 (也就是 CPython) 實做 dict 的方式改變,然後決定把這個特性當作是 social contract,而不是當作 side effect 的特性 (也就是不保證之後版本會有相同特性)。

Changed in version 3.7: Dictionary order is guaranteed to be insertion order. This behavior was an implementation detail of CPython from 3.6.

作者裡面的兩張圖清楚表示出來以前的版本怎麼實做,與 3.7+ 的版本怎麼實做:


不過考慮到還是有些系統用 Python 3.5 (像是 Ubuntu 16.04 內建的 python3) 與 Python 3.6 (Ubuntu 18.04 內建的 python3,雖然沒問題,但當時還沒有寫出來),也許還是先不要依賴這個行為會比較好。


Avast 與 Jumpshot 販賣使用者瀏覽記錄與行為


報導可以看 PCMag 的「The Cost of Avast's Free Antivirus: Companies Can Spy on Your Clicks」與 Motherboard (VICE) 的「Leaked Documents Expose the Secretive Market for Your Web Browsing Data」這兩篇,大綱先把重點列出來了,Avast 在賣使用者的瀏覽記錄與行為:

Avast is harvesting users' browser histories on the pretext that the data has been 'de-identified,' thus protecting your privacy. But the data, which is being sold to third parties, can be linked back to people's real identities, exposing every click and search they've made.

Avast 利用免費的防毒軟體,蒐集使用者的瀏覽記錄與行為,然後透過 Jumpshot 這家子公司販賣出去:

The Avast division charged with selling the data is Jumpshot, a company subsidiary that's been offering access to user traffic from 100 million devices, including PCs and phones.


Who else might have access to Jumpshot's data remains unclear. The company's website says it's worked with other brands, including IBM, Microsoft, and Google. However, Microsoft said it has no current relationship with Jumpshot. IBM, on the other hand, has "no record" of being a client of either Avast or Jumpshot. Google did not respond to a request for comment.

Microsoft 說「現在沒有關係」,IBM 說「沒有 client 的記錄」,Google 則是不回應。


For instance, a single click can theoretically look like this:

Device ID: abc123x Date: 2019/12/01 Hour Minute Second: 12:03:05 Domain: Product: Apple iPad Pro 10.5 - 2017 Model - 256GB, Rose Gold Behavior: Add to Cart

At first glance, the click looks harmless. You can't pin it to an exact user. That is, unless you're, which could easily figure out which Amazon user bought an iPad Pro at 12:03:05 on Dec. 1, 2019. Suddenly, device ID: 123abcx is a known user. And whatever else Jumpshot has on 123abcx's activity—from other e-commerce purchases to Google searches—is no longer anonymous.

所以,如果 Google 手上有這個資料,就可以交叉比對自家的記錄,然後得到使用者完整的記錄。

在消息一公開後沒多久後,Avast 就宣佈關閉 Jumpshot,感覺連被抓包後的反應動作都超流暢,一臉就是排練過:「A message from Avast CEO Ondrej Vlcek」。

看了一下,Avast 旗下還有 AVG,還有個 VPN 服務...

比 Bloom filter 與 Cuckoo filter 再更進一步的 Xor filter

Bloom filter 算是教科書上的經典演算法之一,在實際應用上有更好的選擇,像是先前提到的 Cuckoo filter:「Cuckoo Filter:比 Bloom Filter 多了 Delete」。

現在又有人提出新的資料結構,號稱又比 Bloom filter 與 Cuckoo filter 好:「Xor Filters: Faster and Smaller Than Bloom Filters」。

不過並不是完全超越,其中馬上可以看到的差異就是不支援 delete:

Deletions are generally unsafe with these filters even in principle because they track hash values and cannot deal with collisions without access to the object data: if you have two objects mapping to the same hash value, and you have a filter on hash values, it is going to be difficult to delete one without the other.

論文的預印本可以在 arXiv 上下載:「Xor Filters: Faster and Smaller Than Bloom and Cuckoo Filters」。


看到「Three ways to reduce the costs of your HTTP(S) API on AWS」這邊介紹在 AWS 上省頻寬費用的方法,看了只能一直笑 XD

第一個是降低 HTTP response 裡沒有用到的 header,因為每天有五十億個 HTTP request,所以只要省 1byte 就是省下 USD$0.25/day:

Since we would send this five billion times per day, every byte we could shave off would save five gigabytes of outgoing data, for a saving of 25 cents per day per byte removed.

然後調了一些參數後省下 USD$1,500/month:

Sending 109 bytes instead of 333 means saving $56 per day, or a bit over $1,500 per month.

第二個是想辦法在 TLS 這邊下手,一開始其中一個方向是利用 TLS session resumption 降低第二次連線的成本,但他們發現沒有什麼參數可以調整:

One thing that reduces handshake transfer size is TLS session resumption. Basically, when a client connects to the service for the second time, it can ask the server to resume the previous TLS session instead of starting a new one, meaning that it doesn’t have to send the certificate again. By looking at access logs, we found that 11% of requests were using a reused TLS session. However, we have a very diverse set of clients that we don’t have much control over, and we also couldn’t find any settings for the AWS Application Load Balancer for session cache size or similar, so there isn’t really anything we can do to affect this.

所以改成把 idle 時間拉長 (避免重新連線):

That leaves reducing the number of handshakes required by reducing the number of connections that the clients need to establish. The default setting for AWS load balancers is to close idle connections after 60 seconds, but it seems to be beneficial to raise this to 10 minutes. This reduced data transfer costs by an additional 8%.

再來是 AWS 本身發的 SSL certification 太肥,所以他們換成 DigiCert 發的,大幅降低憑證本身的大小,反而省下 USD$200/day:

So given that the clients establish approximately two billion connections per day, we’d expect to save four terabytes of outgoing data every day. The actual savings were closer to three terabytes, but this still reduced data transfer costs for a typical day by almost $200.

這些方法真的是頗有趣的 XDDD

不過這些方法也是在想辦法壓榨降低與 client 之間的傳輸量啦,比起成本來說反而是提昇網路反應速度...

AWS Outposts 總算要開始出貨了

去年 AWSre:Invent 喊的 AWS Outposts 總算是有東西要出貨了:「AWS Outposts Now Available – Order Yours Today!」。

放在自家實體的機櫃,然後掛到 AWS 上變成一個特殊的 region。目前一個特殊的 region 只能放 16 個機櫃,但預期之後可以更多:

Capacity Expansion – Today, you can group up to 16 racks into a single capacity pool. Over time we expect to allow you to group thousands of racks together in this manner.

不過要注意的是,需要有 AWS Enterprise Support 才能下單,而且看起來硬體的維修也包在內了:

Support – You must subscribe to AWS Enterprise Support in order to purchase an Outpost. We will remotely monitor your Outpost, and keep it happy & healthy over time. We’ll look for failing components and arrange to replace them without disturbing your operations.

看了一下價錢的頁面,如果以北美的 upfront 來算,最便宜的是 OR-L8IF4WFOR-I0OGL02 的 USD$225,504.81,最貴的是 OR-HSZHMMF 的 USD$898,129.52,暫時應該用不到 XDDD

Amazon Redshift 會自動在背景重新排序資料以增加效能

Amazon Redshift 的新功能,會自動在背景重新排序資料以增加效能:「Amazon Redshift introduces Automatic Table Sort, an automated alternative to Vacuum Sort」。

版本要到更新到 1.0.11118,然後預設就會打開:

This feature is available in Redshift 1.0.11118 and later.

Automatic table sort is now enabled by default on Redshift tables where a sort key is specified.


Redshift runs the sorting in the background and re-organizes the data in tables to maintain sort order and provide optimal performance. This operation does not interrupt query processing and reduces the compute resources required by operating only on frequently accessed blocks of data. It prioritizes which blocks of table to sort by analyzing query patterns using machine learning.

算是丟著讓他跑就好的東西,升級上去後可以看一下 CloudWatch 的報告,這邊沒有特別講應該是還好... XD

Amazon Redshift 可以處理座標資料了

這一個月 AWS 因為舉辦一年一度的 AWS re:Invent,會開始陸陸續續放出各種消息... 這次是 Amazon Redshift 宣佈支援 spatial data,這樣一來就能夠方便的處理座標資料了:「Using Spatial Data with Amazon Redshift」。

支援的種類與使用的限制可以在官方的文件裡面看到,也就是「Querying Spatial Data in Amazon Redshift」與「Limitations When Using Spatial Data with Amazon Redshift」這兩篇。


Data types for Python user-defined functions (UDFs) don't support the GEOMETRY data type.


PostgreSQL 上去識別化的套件

在「PostgreSQL Anonymizer 0.5: Generalization and k-anonymity」這邊看到的套件,看起來可以做到一些常見而且簡單的去識別化功能:

The extension supports 3 different anonymization strategies: Dynamic Masking, In-Place Anonymization and Anonymous Dumps. It also offers a large choice of Masking Functions: Substitution, Randomization, Faking, Partial Scrambling, Shuffling, Noise Addition and Generalization.

看起來可以把欄位轉成 range 這件事情半自動化處理掉 (還是需要 SQL 本身呼叫這些函數),之後遇到 PII 的時候也許會用到...