AWS 推出 Amazon Route 53 Resolver DNS Firewall

長久以來的洞總算有比較好的方法補上了,AWS 推出了 Amazon Route 53 Resolver DNS Firewall:「Introducing Amazon Route 53 Resolver DNS Firewall」。

Route 53 Resolver 是 AWS 官方提供的 DNS Resolver,沒有特殊的設定的話通常會在 x.x.x.2 (/24 或是更大的網段),先前一直沒有辦法解決 data leak 的問題,也就是透過 DNS 把敏感資料從 private network 裡丟出去。

以前的作法是透過 security group 擋掉對 Route 53 Resolver 的流量 (或是透過 VPC 的 Firewall 擋),然後自己架設兩台 DNS resolver 過濾,現在 Route 53 Resolver 支援 DNS Firewall,提供 allowlist 與 blocklist 這兩個功能使用,總算是把這件事情解的比較乾淨了:

Route 53 Resolver DNS Firewall lets you create “blocklists” for domains you don’t want your VPC resources to communicate with via DNS. You can also take a stricter, “walled-garden” approach by creating “allowlists” that permit outbound DNS queries only to domains you specify. You can also create alerts for when outbound DNS queries match certain firewall rules, allowing you to test your rules before deploying for production traffic.

另外這次的 DNS Firwall 提供了兩組由 AWS 維護的清單讓人使用,包括了 malware 與 botnet:

Route 53 Resolver DNS Firewall offers two managed domain lists—malware domains and botnet command and control domains—enabling you to get started quickly with managed protections against common threats.

這樣省事多了...

Facebook 5.33 億筆個資外洩,以及 Mark Zuckerberg 的電話...

這兩天應該已經有很多其他媒體報導了 Facebook 5.33 億筆資料外洩的事情:「533 million Facebook users' phone numbers and personal data have been leaked online」。

據稱是用 2019 年的洞撈出來的,不過這份資料看起來也不是完整資料,舉個例子來說,台灣只有 73 萬筆左右,而且也少了很多地區,像是泰國就不在列表內...

另外一個比較特別的是,Mark Zuckerberg 的電話也在這次外洩資料裡面:「Mark Zuckerberg's phone number appeared among the leaked data of Facebook users, according to a researcher」,應該會換號碼了。

這包資料看起來會這陣子會很熱門...

Mapbox GL JS 的授權改變,以及 MapLibre GL 的誕生

看到「MapLibre GL is a free and open-source fork of mapbox-gl-JS (github.com/maplibre)」這篇,翻了一下資料發現年初時 Mapbox GL JS 的軟體授權從 v2.0.0 開始變成不是 open source license (本來是 BSD license),而社群也馬上 fork 最後一個 open source 版本並且投入開發,變成 MapLibre GL

MapTiler 在年初的時候有提到這件事情:「MapLibre: Mapbox GL open-source fork」。

The community reacted swiftly: forks of the latest open-source version were made almost immediately by multiple parties. In another positive development, the community came together the next day and agreed to make this a joint effort, rather than splitting energies. A video call was organized and the MapLibre coalition was formed. It includes people working for MapTiler, Elastic, StadiaMaps, Microsoft, Ceres Imaging, WhereGroup, Jawg, Stamen Design, etc.

MapLibre GL 目前與本來 v1.13.0 相容,可以直接抽換過去 (後來在二月的時候有出一個 v1.13.1,不過那是在 v2.0.0 改 license 之後的事情了):

  "dependencies": {
-    "mapbox-gl": "^1.13.0"
+    "maplibre-gl": ">=1.14.0"
  }

記錄一下,以後要在網站上用的話,得注意到 Mapbox GL JS 在沒有註冊的情況下不能使用,而且 SDK 會強制蒐集資料:

Mapbox gl-js version 2.0 or higher (“Mapbox Web SDK”) must be used according to the Mapbox Terms of Service. This license allows developers with a current active Mapbox account to use and modify the Mapbox Web SDK. Developers may modify the Mapbox Web SDK code so long as the modifications do not change or interfere with marked portions of the code related to billing, accounting, and anonymized data collection. The Mapbox Web SDK only sends anonymized usage data, which Mapbox uses for fixing bugs and errors, accounting, and generating aggregated anonymized statistics. This license terminates automatically if a user no longer has an active Mapbox account.

不過如果是抓 OpenStreetMap 資料的話,Leaflet 應該還是目前的首選...

AWS 推出 CloudWatch Metric Streams

AWS 推出了 CloudWatch Metric Streams,把 CloudWatch Metric 的資料往 Kinesis Data Firehose 裡面丟:「CloudWatch Metric Streams – Send AWS Metrics to Partners and to Your Apps in Real Time」。

其中一個賣點是即時性比用 API 去拉好很多:

In order to make it easier for AWS Partners and others to gain access to CloudWatch metrics faster and at scale, we are launching CloudWatch Metric Streams. Instead of polling (which can result in 5 to 10 minutes of latency), metrics are delivered to a Kinesis Data Firehose stream.

格式上可以是 JSON 或是 Open Telemetry

When you set up a stream you choose between the binary Open Telemetry 0.7 format, and the human-readable JSON format.

另外一個賣點是價位,每千次 $0.003:

Pricing – You pay $0.003 for every 1000 metric updates, and for any charges associated with the Kinesis Data Firehose. To learn more, check out the pricing page.

另外算一下 Kinesis Data Firehose 的價錢,是以資料量的大小計費,不過最小計價單位是 5KB (一筆應該是不會到),單價是 $0.029/GB (us-east-1) 或是 $0.037/GB (ap-southeast-1),算了一下跟 CloudWatch Metrics Streams 比起來只是零頭...

之前如果要自己拉出來的話是透過 API call 抓,每 1000 次是 USD$0.01,這個方法相較起來便宜不少,不過數量多的時候還是一筆費用 (而且有不少 metrics 是一分鐘更新一次)。

如果只是要備份起來或是跑分析的話,也許先前用 API 拉的作法可能還是比較好?一個小時拉一次對於備份與分析應該都很夠了,而 alarm 的機制還是掛在 CloudWatch 上。

這次產品的定位看起來是要把 ecosystem 做起來:

We designed this feature with the goal of making it easier & more efficient for AWS Partners including Datadog, Dynatrace, New Relic, Splunk, and Sumo Logic to get access to metrics so that the partners can build even better tools.

Zoom 在分享螢幕時會有資料安全性問題

看到「Zoom Screen-Sharing Glitch ‘Briefly’ Leaks Sensitive Data」這篇,講 Zoom 在分享螢幕時的安全性問題,發現問題的德國資安團隊所寫的 security advisory 可以在「SYSS-2020-044.txt」這邊看到,對應的 CVE 號碼是 CVE-2021-28133

安全問題出在,分享者要分享某個應用程式的畫面,但有可能會先分享到其他畫面,再切回來:

However, “under certain conditions” if a Zoom presenter chooses to share one application window, the share-screen feature briefly transmits content of other application windows to meeting participants, according to German-based SySS security consultant Michael Strametz, who discovered the flaw, and researcher Matthias Deeg, in a Thursday disclosure advisory (which has been translated via Google).

而目前的版本還是有一樣的問題:

The current Zoom client version, 5.5.4 (13142.0301), for Windows is still vulnerable to the issue, Deeg told Threatpost.

這個問題是去年十二月就通報了,過了三個月沒有回應,所以團隊公開出來:

Disclosure Timeline:

2020-12-02: Vulnerability reported to manufacturer
2020-12-02: Manufacturer acknowledges receipt of security advisory
2020-12-02: Manufacturer asks for more information
2020-12-03: SySS provides more information concerning the security issue
2020-12-03: Manufacturer confirms reproducing the security issue in both the Windows and the Linux client and asks further questions
2020-12-04: SySS answers open questions
2020-12-04: Manufacturer responds and will look into the reported security issue
2021-01-21: SySS asks for status update
2021-02-01: SySS asks for status update
2021-03-18: Public release of security advisory

不知道什麼時候會修正

Airtable 的 Open Source 替代方案 Baserow

Hacker News Daily 上看到的產品 Baserow,另外在 Hacker News 的討論也有蠻多有趣的東西:「Baserow.io – Self-hosted Airtable alternative (baserow.io)」。

Baserow 的後端是用 Django 寫的,前端則主要是用 Vue.js

可以看到討論裡 Airtable 的問題,先不講定價問題,只講其他的部份... (因為就如同討論裡面有提到的,有可能是公司故意把定價策略訂成讓不願意花大錢的人使用)

首先是每個月都有「穩定的 downtime」,不過這邊的描述到的數字明顯有問題,一個月當了七個小時也有 99% uptime (兩個九)。

不過先撇開這點,付費服務一般都可以看到 99.9% 的 Monthly SLA (三個九,一個月只允許 43.2 分鐘),這個情況的確是不太符合業界的標準:

Reliable 1-5 hours a month of downtime. There are months where it's literally 1 nine of uptime. Their status page was actually down for a while (I think they forgot to renew it?)

另外一個有很趣的是很奇耙的設計:

Booleans in the API are `true` or `undefined`. Seriously.

看起來要有蠻多背景知識才有可能知道為什麼這樣搞 XDDD

先丟給 data team 看看,如果他們有興趣的話再找機會架起來...

Akamai 也推出了 Key-Value 服務 EdgeKV

沒介紹過 Akamai 的一些架構,先講到 Akamai 的 Edge 端 Serverless 架構是 EdgeWorkers,跑的是 JavaScript:

EdgeWorkers lets developers just code — integrating into existing CI/CD workflows and enabling multiple teams to work in parallel using JavaScript. EdgeWorkers eliminates the hassle of managing compute resources and building for scale.

然後這次推出的是 EdgeKV,目前還在 Beta 版:「Serverless Storage at the Edge (EdgeKV Beta)」。

如同名字所說的,架構上 Key-Value 架構,放棄了 CAP theorem 裡面的 C,改走 Eventual Consistency:

EdgeKV uses what is known in distributing computing as an eventual consistency model to perform writes and updates. This model achieves high availability with low read latency by propagating data writes globally. The period of time it takes the system to distribute data globally is called the “inconsistency window”.

隔壁 Cloudflare Workers KV 也是 Eventual Consistency (出自「How KV works」這邊):

KV achieves this performance by being eventually-consistent. Changes are immediately visible in the edge location at which they're made, but may take up to 60 seconds to propagate to all other edge locations.

看起來算是補上競爭對手的產品線...

T-Mobile US 打算要賣使用者的瀏覽記錄了

全美第二大的 T-Mobile US 打算要賣使用者的瀏覽記錄了,除非你登入進去選擇退出:「T-Mobile to Step Up Ad Targeting of Cellphone Customers」,Hacker News 上的討論則可以看「T-Mobile to share customers' data with advertisers unless they opt out (thehill.com)」這邊。

The No. 2 U.S. carrier by subscribers said in a recent privacy-policy update that unless they opt out it will share customers’ web and mobile-app data with advertisers starting April 26.

這次的改變包括了 2020 年併購 Sprint 的使用者:

T-Mobile’s new policy will also cover Sprint customers acquired through the carriers’ 2020 merger. Sprint had previously shared similar data only from customers who opted into its third-party ad program.

所以連在美國都有 DNS over HTTPS (或是 DNS over TLS) 與 ESNI 需求了...

OVH 法國機房 SBG2 火災全毀

OVH 算是國際上很大的 Hosting 公司,昨天在法國史特拉斯堡 (Strasbourg) 的 SBG2 機房發生火災,這邊的 Octave Klaba 是 OVH 的創辦人與老闆,另外在 Hacker News 上的「Fire declared in OVH SBG2 datacentre building (ovh.net)」這邊也有討論可以看:

可以在 Threadreader 上面讀整個 thread,Octave Klaba 有一直有在 Twitter 上 update 進度與後續的計畫:「https://threadreaderapp.com/thread/1369478732247932929.html」。

新聞媒體也有一些當時的空拍圖放出來了:

出自「Strasbourg: important incendie chez OVHcloud, de nombreux sites internet indisponibles partout dans le monde」。

另外更重要的是伺服器裡面資料的部份,其中 SBG2 全毀,SBG1 毀了四間 (SBG1 總共 12 間),這些資料看起來都沒辦法救了。而 SBG3 與 SBG4 的機器還在,但目前沒有電力。

接下來的會花時間重建 SBG{1,3,4} 的電力系統與重建對外連線,看起來 20KV 的線路與 240V 的線路都有受損需要重弄。

然後也已經有廠商丟災情出來了,線上遊戲的 Rust 一開始說他們受到影響:

但更慘的是官方後續更新,直接說資料無法恢復,聽起來像是沒有備份資料,或是備份資料也在同一個機房內:

除了重建外,現在應該是等後續看起火原因,理論上機房的消防設備應該要能擋下全毀... 等原因出來後,來看看是不是會改變整個機房產業的消防設計架構。

GTA 的啟動讀取效能問題

這件事情也已經過了一個禮拜,來整理一下發生什麼事情...

起因是 GTA Online 的遊戲開啟速度很慢,而有人一路 reverse engineering 找出問題並且解決:「How I cut GTA Online loading times by 70%」,對應的 Hacker News 討論有提到其他有趣的事情也可以看看:「How I cut GTA Online loading times by 70% (nee.lv)」。

作者的電腦不算太差,但光開啟 GTA Online 就需要六分鐘,網路上甚至有辦投票蒐集大家的等待時間,發現也有很多人反應類似的問題:

接下來就開始 reverse engineering 了,先觀察各種狀態後發現是卡在 CPU,而不是網路或 Disk I/O,然後就拿出 Luke Stackwalker 這個工具 profiling,不過因為沒有 debug symbol 幫忙 group,所以只能人工判斷後,可以看到兩個問題:

第一個問題發現效能是卡在 strlen(),而 call stack 可以看出來是從 sscanf() 一路打進去的:

反追發現是在處理 10MB 的 JSON 檔造成的,裡面 sscanf() 因為拉出 strlen(),於是就造成把整個 10MB 的 JSON 掃過很多次 (一開始是 10MB,掃到後面會愈來愈少,平均下來應該是 5MB):

第二個問題產生的時間會在第一個問題跑完後,另外看問題的性質,應該跟第一個 JSON 處理有關,他會把 JSON 處理過的資料丟進 array,每個 entry 長這樣:

struct {
    uint64_t *hash;
    item_t   *item;
} entry;

丟進 array 是 OK 的,但問題在於他需要判斷 entry 是否重複,卻沒有用 hash 或是 tree 的結構,而這邊大約有 63k 筆資料,用 array 實做就產生了 O(n^2) 的演算法:

But before it’s stored? It checks the entire array, one by one, comparing the hash of the item to see if it’s in the list or not. With ~63k entries that’s (n^2+n)/2 = (63000^2+63000)/2 = 1984531500 checks if my math is right. Most of them useless. You have unique hashes why not use a hash map.

作者在 PoC 的章節裡面描述他怎麼解這兩個問題。

第一個問題比較好的解法是修正 JSON Parser,但這太複雜,所以他用 workaround 解:把 strlen() 包起來,針對長字串加上一層 cache:

  • hook strlen
  • wait for a long string
  • “cache” the start and length of it
  • if it’s called again within the string’s range, return cached value

而第二個問題他直接把檢查是否有重複的跳過,因為資料本身不重複:

And as for the hash-array problem, it’s more straightforward - just skip the duplicate checks entirely and insert the items directly since we know the values are unique.

整個開啟的速度從六分鐘降到一分五十秒,還是偏慢,但算是大幅緩解的 GTA Online 啟動速度的問題了。

不過故事到這邊還沒結束,有人一路去挖,發現其實 sscanf() 的效能地雷已經不是第一次了:YAML 的 Parser 也中過一樣的問題:「Parsing can become accidentally quadratic because of sscanf」,這篇也一樣上了 Hacker News:「Parsing can become accidentally quadratic because of sscanf (github.com/biojppm)」。

然後這又帶出了六年前在 StackOverflow 上就有人問過這個問題:「Why is glibc's sscanf vastly slower than fscanf on Linux?」。

另外也有人整理出來,應該是大家把同樣的演算法拿來實做:

JdeBP 3 days ago

I found this while making a collection of what C implementation does what at https://news.ycombinator.com/item?id=26298300.

There are two basic implementation strategies. The BSD (FreeBSD and OpenBSD and more than likely NetBSD too), Microsoft, GNU, and MUSL C libraries use one, and suffer from this; whereas the OpenWatcom, P.J. Plauger, Tru64 Unix, and my standard C libraries use another, and do not.

The 2002 report in the comp.lang.c Usenet newsgroup (listed in that discussion) is the earliest that I've found so far.

後續的更新動作可以再追一下進度 (包括 GTA Online 與各家的 libc)。