CloudFront 支援 JA3 資訊 (SSL/TLS fingerprint)

看到 CloudFront 宣佈支援帶入 JA3 資訊了:「Amazon CloudFront now supports JA3 fingerprint headers」:

Details: Amazon CloudFront now supports Cloudfront-viewer-ja3-fingerprint headers, enabling customers to access incoming viewer requests’ JA3 fingerprints. Customers can use the JA3 fingerprints to implement custom logic to block malicious clients or allow requests from expected clients only.

JA3 的頁面上可以看到說明,針對 SSL/TLS 這個複雜的 handshake 過程中,就可以從中得知不同的 client 實做,比起容易偽造的 User-agent 有效:

JA3 is a method for creating SSL/TLS client fingerprints that should be easy to produce on any platform and can be easily shared for threat intelligence.

像是 Tor 的 SSL/TLS 連線就會有對應的 fingerprint 可以偵測:

JA3 fingerprint for the standard Tor client:
e7d705a3286e19ea42f587b344ee6865

先前在「修正 Curl 的 TLS handshake,避開 bot 偵測機制」裡面提到的 curl-impersonate 就是反制這類偵測的方式。

cURL 的 TLS fingerprint

看到「curl's TLS fingerprint」這篇,cURL 的作者 Daniel Stenberg 提到 TLS fingerprint。

先前在「修正 Curl 的 TLS handshake,避開 bot 偵測機制」與「curl 的 TLS fingerprint 偽裝專案 curl-impersonate 支援 Chrome 了」這邊有提到 curl-impersonate 這個專案,試著在 cURL 裡模擬市面上常見的瀏覽器的 TLS fingerprint。

在 Daniel Stenberg 的文章裡面也有提到這件事情,另外也提到了對 curl-impersonate 的態度:

I cannot say right now if any of the changes done for curl-impersonate will get merged into the upstream curl project, but it will also depend on what users want and how the use of TLS fingerprinting spread or changes going forward.

看起來短期內他是沒打算整,跟當初 curl-impersonate 的預期差不多...

利用字型來判斷使用者是否有安裝特定軟體

Hacker News 上的「TeamViewer installs suspicious font only useful for web fingerprinting (ctrl.blog)」這邊看到的技巧,原文在「TeamViewer installs suspicious font only useful for web fingerprinting」這邊,但文章標題本身可以忽略。

這別提到的方法是,在安裝軟體時額外安裝一個特別的字型,然後網頁就可以透過 javascript 判斷這個字型存不存在,來得知使用者是否有安裝自己的軟體,接下來就可以走到不同的 flow:可以導引使用者下載軟體,或是透過 handler 拉起應用程式。

不過這也透漏出了隱私問題,代表廣告商可以利用這點取得 fingerprint,而不只是軟體自家的網站。

看討論串裡面說 Firefox 上可以用 privacy.resistFingerprinting 擋住:「Firefox's protection against fingerprinting」,但 Firefox 本身也沒有說明的太清楚到底會放行哪些字型:

Not all fonts installed on your computer are available to webpages

在「Security/Fingerprinting」這頁則是:

We only allow specific system fonts to be used, and we ship them to the user using kinto

直接試著找 Bugzilla 與 source code 的資料可以翻到「Restrict CSS font visibility to standard fonts only when privacy.resistFingerprinting is true」這個討論,裡面有提到「https://searchfox.org/mozilla-central/search?path=StandardFonts*.inc」這個,可以看到有 LinuxmacOSWindows 10 下的清單。

不過 Chromium-based browser 下目前好像沒看到方案...

curl 的 TLS fingerprint 偽裝專案 curl-impersonate 支援 Chrome 了

先前在「修正 Curl 的 TLS handshake,避開 bot 偵測機制」這邊提到 curl-impersonate 這個專案,試著修改 curl 的 TLS handshake fingerprint 讓偽裝的更好,本來只支援 Firefox,現在則是支援 Google Chrome 的 fingerprint 了...

作者寫的兩篇說明文章也可以看看:「Making curl impersonate Firefox」、「Impersonating Chrome, too」。

看起來愈來愈完整了,連 LD_PRELOAD 的用法也出現了,然後在 Arch Linux 上也出現 AUR 可以用了...

修正 Curl 的 TLS handshake,避開 bot 偵測機制

利用 TLS handshake 的 pattern 可以當作是某種 fingerprint,就可以知道你是用 Curl,這個方式在蠻多 CDN 都會用在 anti-bot 機制 (像是 Cloudflare),而剛剛看到有人投稿自己的 patch,試著將 Curl 修改成 Firefox 的 pattern:「curl-impersonate」,Hacker News 上的討論在這邊可以看到:「Show HN: Curl modified to impersonate Firefox and mimic its TLS handshake (github.com/lwthiker)」。

作者有提到這次的 patch 偏 hack,不太可能整進上游,但希望未來改的乾淨一點,然後整進上游:

I hope to do so in the future, for now the implementation is extremely hacky so I doubt it can get accepted into curl.

另外有人提出來說應該要用 Firefox ESR 版本的 pattern 而非 stable channel,也有人提出來說用 Google Chrome 的更好,不過我覺得有人開始做就已經很棒了 XD

更多瀏覽器內 GPU fingerprint 的技巧

在「Your device's GPU may be used for fingerprinting purposes」這邊看到有研究單位找出更多 GPU fingerprint 的技巧,論文在「DRAWNAPART: A Device Identification Technique based on Remote GPU Fingerprinting」這邊可以翻到,另外一些 source code 可以在 GitHub 上的「Drawn Apart」這邊可以翻到。

看起來是透過 WebGL 去建立模型:

The researchers ran experiments on 2500 unique devices using the technique. They developed two methods, both of which use the Web Graphics Library (WebGL), which is supported by all modern web browsers.

接下來瀏覽器端應該會在研究後有更多反制機制被放進來...

SSH 的 StrictHostKeyChecking=accept-new

OpenSSH 在連到新的 host 時會跳出 key fingerprint 的資訊讓使用者確認,有時候為了自動化會用 StrictHostKeyChecking=no 避開,在 Lobsters Daily 上則看到了新的選項可以用,StrictHostKeyChecking=accept-new

就如同選項的名字所描述的,查了一下 OpenSSH Release Notes 可以看到這是在 OpenSSH 7.5 導入的參數,是在 March 20, 2017 引入的:

* ssh(1): expand the StrictHostKeyChecking option with two new settings. The first "accept-new" will automatically accept hitherto-unseen keys but will refuse connections for changed or invalid hostkeys. This is a safer subset of the current behaviour of StrictHostKeyChecking=no. The second setting "off", is a synonym for the current behaviour of StrictHostKeyChecking=no: accept new host keys, and continue connection for hosts with incorrect hostkeys. A future release will change the meaning of StrictHostKeyChecking=no to the behaviour of "accept-new". bz#2400

對於一些自動化的流程應該夠用了,不需要到用 no 完全關掉。

翻了「Ubuntu – Package Search Results -- openssh-client」可以看到 18.04 之後都是 7.5 之後的版本了,支援度應該是沒什麼太大問題...

跨瀏覽器追蹤的方式

看到「Exploiting custom protocol handlers for cross-browser tracking in Tor, Safari, Chrome and Firefox」這個方式,跨瀏覽器收集 fingerprint 追蹤。

這次用的方式是透過 handler 追:

The scheme flooding vulnerability allows an attacker to determine which applications you have installed. In order to generate a 32-bit cross-browser device identifier, a website can test a list of 32 popular applications and check if each is installed or not. On average, the identification process takes a few seconds and works across desktop Windows, Mac and Linux operating systems.

最近大家比較常使用到的應該就是 Zoom 從網頁把應用程式帶起來的方式:

而要怎麼偵測的部份,用到了不同瀏覽器的 side channel。

Chromium 系列的部份對應的 ticket 在「Issue 1096610: External Protocol handler anti-flood protection is ineffective and flaky」這邊有被提出來。主要用到的方法是,在遇到有 handler 時,連打兩次時會被擋下:

被擋下後再打都會失敗,所以需要一個方式重設 flag,而內建的 Chrome PDF Viewer 剛好可以重設 flag:

The built-in Chrome PDF Viewer is an extension, so every time your browser opens a PDF file it resets the scheme flood protection flag. Opening a PDF file before opening a custom URL makes the exploit functional.

Firefox 的 side channel 則是可以透過 same-origin policy 測試當作 side channel,對應的 ticket 在「Scheme flooding technique for reliable cross-browser fingerprinting」這邊:

Every time you navigate to an unknown URL scheme, Firefox will show you an internal page with an error. This internal page has a different origin than any other website, so it is impossible to access it because of the Same-origin policy limitation. On the other hand, a known custom URL scheme will be opened as about:blank, whose origin will be accessible from the current website.

Safari 上的問題與 Firefox 一樣,不過沒登入看不到 ticket (也懶的註冊了):

You are not authorized to access bug #225769. To see this bug, you must first log in to an account with the appropriate permissions.

另外,雖然 Tor Browser 底層是 Firefox,但因為有改變預設值,所以攻擊者也得換方法:

Tor Browser is based on the Firefox source code, so the Same-origin policy trick was used here as well. But because Tor Browser does not show pop-ups, we used the same-origin policy trick with iframe elements instead.

這個方法還蠻暴力的...

Privacy Badger 預設關閉學習功能

Privacy Badger 是一個自動學習的 extension,可以學習 tracker 並且予以阻擋:

Privacy Badger automatically learns to block invisible trackers.

而這個自動學習功能在剛剛看到公告說明預設會關閉:「Privacy Badger Is Changing to Protect You Better」,主要是因為這個自動學習功能可以變成 fingerprint 資訊的一環:

The team also alerted us to a class of attacks that were enabled by Privacy Badger’s learning. Essentially, since Privacy Badger adapts its behavior based on the way that sites you visit behave, a dedicated attacker could manipulate the way Privacy Badger acts: what it blocks and what it allows. In theory, this can be used to identify users (a form of fingerprinting) or to extract some kinds of information from the pages they visit. This is similar to the set of vulnerabilities that Safari’s Intelligent Tracking Prevention feature disclosed and patched late last year.

所以現在變成會固定更新 pre-train ruleset 了:

From now on, Privacy Badger will rely solely on its “Badger Sett” pre-trained list of tracking domains to perform blocking by default. Furthermore, Privacy Badger’s tracker database will be refreshed periodically with the latest pre-trained definitions. This means, moving forward, all Privacy Badgers will default to relying on the same learned list of trackers for blocking.

當然這個功能還是可以手動開,但就有可能會被拿去 fingerprint 了,要開的人可以自己想一下... 不過不開的話就只是一個 ruleset 了 XD

有用的人可以自己考慮一下 XD

Google 用 x-client-data 追蹤使用者的問題

前陣子 Chromium 團隊在研究要移除 User-Agent 字串的事情 (參考「User-Agent 的淘汰提案」),結果 kiwibrowser 就直接炸下去,Google 很久前就會針對自家網站送出 x-client-data 這個 HTTP header,裡面足以辨識使用者瀏覽器的單一性:「Partial freezing of the User-Agent string#467」。

Google 的白皮書裡面是說用在 server 的試驗:

We want to build features that users want, so a subset of users may get a sneak peek at new functionality being tested before it’s launched to the world at large. A list of field trials that are currently active on your installation of Chrome will be included in all requests sent to Google. This Chrome-Variations header (X-Client-Data) will not contain any personally identifiable information, and will only describe the state of the installation of Chrome itself, including active variations, as well as server-side experiments that may affect the installation.

The variations active for a given installation are determined by a seed number which is randomly selected on first run. If usage statistics and crash reports are disabled, this number is chosen between 0 and 7999 (13 bits of entropy). If you would like to reset your variations seed, run Chrome with the command line flag “--reset-variation-state”. Experiments may be further limited by country (determined by your IP address), operating system, Chrome version and other parameters.

但因為這個預設值開啟的關係,就算關掉後也足以把使用者再分類到另外一個區塊,仍然具有高度辨識性,不是你 Google 說無法辨識就算數。

另外如果看 source code 裡的說明:

    // Note the criteria for attaching client experiment headers:
    // 1. We only transmit to Google owned domains which can evaluate
    // experiments.
    //    1a. These include hosts which have a standard postfix such as:
    //         *.doubleclick.net or *.googlesyndication.com or
    //         exactly www.googleadservices.com or
    //         international TLD domains *.google. or *.youtube..
    // 2. Only transmit for non-Incognito profiles.
    // 3. For the X-Client-Data header, only include non-empty variation IDs.

可以看到 *.doubleclick.net*.googlesyndication.comwww.googleadservices.com 全部都是廣告相關,另外 Google 自家搜尋引擎是直接提供廣告 (不透過前面提到的網域),YouTube 也是一樣的情況,所以完全可以猜測 x-client-data 這個資料就是用在廣告相關的系統上。

The Register 在「Is Chrome really secretly stalking you across Google sites using per-install ID numbers? We reveal the truth」這邊用粗體的 Update 提到了 GDPR 的問題,不確定是不是開始有單位在調查了:

Updated Google is potentially facing a massive privacy and GDPR row over Chrome sending per-installation ID numbers to the mothership.

在這個問題沒修正之前,只能暫時用操作 HTTP header 的 extension 移掉這個欄位。