AWS 推出 Amazon Route 53 Resolver DNS Firewall

長久以來的洞總算有比較好的方法補上了,AWS 推出了 Amazon Route 53 Resolver DNS Firewall:「Introducing Amazon Route 53 Resolver DNS Firewall」。

Route 53 Resolver 是 AWS 官方提供的 DNS Resolver,沒有特殊的設定的話通常會在 x.x.x.2 (/24 或是更大的網段),先前一直沒有辦法解決 data leak 的問題,也就是透過 DNS 把敏感資料從 private network 裡丟出去。

以前的作法是透過 security group 擋掉對 Route 53 Resolver 的流量 (或是透過 VPC 的 Firewall 擋),然後自己架設兩台 DNS resolver 過濾,現在 Route 53 Resolver 支援 DNS Firewall,提供 allowlist 與 blocklist 這兩個功能使用,總算是把這件事情解的比較乾淨了:

Route 53 Resolver DNS Firewall lets you create “blocklists” for domains you don’t want your VPC resources to communicate with via DNS. You can also take a stricter, “walled-garden” approach by creating “allowlists” that permit outbound DNS queries only to domains you specify. You can also create alerts for when outbound DNS queries match certain firewall rules, allowing you to test your rules before deploying for production traffic.

另外這次的 DNS Firwall 提供了兩組由 AWS 維護的清單讓人使用,包括了 malware 與 botnet:

Route 53 Resolver DNS Firewall offers two managed domain lists—malware domains and botnet command and control domains—enabling you to get started quickly with managed protections against common threats.

這樣省事多了...

Facebook 5.33 億筆個資外洩,以及 Mark Zuckerberg 的電話...

這兩天應該已經有很多其他媒體報導了 Facebook 5.33 億筆資料外洩的事情:「533 million Facebook users' phone numbers and personal data have been leaked online」。

據稱是用 2019 年的洞撈出來的,不過這份資料看起來也不是完整資料,舉個例子來說,台灣只有 73 萬筆左右,而且也少了很多地區,像是泰國就不在列表內...

另外一個比較特別的是,Mark Zuckerberg 的電話也在這次外洩資料裡面:「Mark Zuckerberg's phone number appeared among the leaked data of Facebook users, according to a researcher」,應該會換號碼了。

這包資料看起來會這陣子會很熱門...

利用 Cloudflare Workers 繞過 Cloudflare 自家的阻擋機制

Hacker News 首頁上看到「How to bypass Cloudflare bot protection (jychp.medium.com)」這則,裡面的文章是「How to bypass CloudFlare bot protection ?」這篇,利用 Cloudflare Workers 繞過 Cloudflare 自家的 CAPTCHA 機制。

這個漏洞有先被送給 Cloudflare,但被認為不是問題,所以作者就決定公開:

Several months ago I submitted what appeared to be a security flaw to CloudFalre’s bugbounty program. According to them, this is not a problem, it’s up to you to make up your own mind.

技術上就是透過 Cloudflare Workers 當作 proxy server,只是看起來 Cloudflare 對自家 IP 有特別處理,在設定妥當後,用 Cloudflare Workers 的 IP address 去連 Cloudflare 的站台,幾乎不會觸發 Cloudflare 的阻擋機制。

不過 free tier 還是有限制,主要就是數量:

The first 100,000 requests each day are free and paid plans start at just $5/10 million requests, making Workers as much as ten-times less expensive than other serverless platforms.

作者也有提到這點:

So let’s enjoy the 100 000 request/day for your free Cloudflare account and go scrape the world !

但這是個有趣的方法,加上信用卡盜刷之類的方式,這整包看起來就很有威力...

Mapbox GL JS 的授權改變,以及 MapLibre GL 的誕生

看到「MapLibre GL is a free and open-source fork of mapbox-gl-JS (github.com/maplibre)」這篇,翻了一下資料發現年初時 Mapbox GL JS 的軟體授權從 v2.0.0 開始變成不是 open source license (本來是 BSD license),而社群也馬上 fork 最後一個 open source 版本並且投入開發,變成 MapLibre GL

MapTiler 在年初的時候有提到這件事情:「MapLibre: Mapbox GL open-source fork」。

The community reacted swiftly: forks of the latest open-source version were made almost immediately by multiple parties. In another positive development, the community came together the next day and agreed to make this a joint effort, rather than splitting energies. A video call was organized and the MapLibre coalition was formed. It includes people working for MapTiler, Elastic, StadiaMaps, Microsoft, Ceres Imaging, WhereGroup, Jawg, Stamen Design, etc.

MapLibre GL 目前與本來 v1.13.0 相容,可以直接抽換過去 (後來在二月的時候有出一個 v1.13.1,不過那是在 v2.0.0 改 license 之後的事情了):

  "dependencies": {
-    "mapbox-gl": "^1.13.0"
+    "maplibre-gl": ">=1.14.0"
  }

記錄一下,以後要在網站上用的話,得注意到 Mapbox GL JS 在沒有註冊的情況下不能使用,而且 SDK 會強制蒐集資料:

Mapbox gl-js version 2.0 or higher (“Mapbox Web SDK”) must be used according to the Mapbox Terms of Service. This license allows developers with a current active Mapbox account to use and modify the Mapbox Web SDK. Developers may modify the Mapbox Web SDK code so long as the modifications do not change or interfere with marked portions of the code related to billing, accounting, and anonymized data collection. The Mapbox Web SDK only sends anonymized usage data, which Mapbox uses for fixing bugs and errors, accounting, and generating aggregated anonymized statistics. This license terminates automatically if a user no longer has an active Mapbox account.

不過如果是抓 OpenStreetMap 資料的話,Leaflet 應該還是目前的首選...

YAML 的問題 (挪威問題)

Hacker News 首頁上看到的,YAML 寫多的人都遇過類似的問題:「The Norway Problem - why StrictYAML refuses to do implicit typing and so should you」,對應的討論「The Norway Problem (hitchdev.com)」也可以看一下。

第一個「經典」是字串不需要包起來就很容易出事,這邊提到的例子是因為保留字而中槍,挪威的簡碼 NO 變成 False 了:

countries:
- GB
- IE
- FR
- DE
- NO
>>> from pyyaml import load
>>> load(the_configuration)
{'countries': ['GB', 'IE', 'FR', 'DE', False]}

同樣的是字串問題,「看起來」是數字的就會變成數字:

python: 3.5.3
postgres: 9.3

然後還是字串,人名遇到保留字:

first name: Christopher
surname: Null

這種問題都是碰過一次學一次...

作者另外提到的 StrictYAML 改變了 YAML 規格,我會看看就好,能用 JSON 也許還是會偏好先用 JSON,不是完全解決,但踩雷的機率會少很多。

用 vttest 測試 terminal 程式實做的相容性

看到「Wez's Terminal」,又一個用 GPU 加速的 terminal 專案,跑去翻一下 Hacker News 上的討論,意外的翻到有趣的東西:「Wezterm – A GPU-accelerated cross-platform terminal emulator and multiplexer (github.com/wez)」。

目前最前面的討論是這個:

I just saw this and after a quick checkout of the latest release (which is what I can quickly install), I feel a bit disappointed. Here is yet another GPU-accelerated terminal that claims to be a VT500 (judging by its Device Attributes report), sets TERM to xterm-256color, while having holes in basic VT100 support (just ran VTTEST in it and saw glaring issues, need not look further than basic VT100 tests, TAB setting/resetting, cursor save/restore).

比較有趣的是裡面提到了 vttest 這套測試軟體可以測 terminal 的標準相容性,在 vttest 的官網上有提到可以測 VT100VT220VT520

Vttest tests the compatibility (demonstrates the non-compatibility) of so-called “VT100-compatible” terminals. In conformance of the good old hacker traditions, the only documentation of this program is the source code itself. To understand it, you also need a copy of the original VT100 manual from DEC.

Additional tests (past version 1.7) are provided for analysis of vt220 through vt520 terminals, as well as variants of xterm.

另外比較驚訝的是,terminal 下可以作到這樣的效果 (這邊看官方的截圖應該是 VT420?):

回到原來講標準相容性的問題,對於泡在 terminal 下的人來說超級重要,有工具可以測試幫助不少... vttest 這套程式在 Ubuntu 內可以直接裝來用:「Ubuntu – Package Search Results -- vttest」。

AWS 推出 CloudWatch Metric Streams

AWS 推出了 CloudWatch Metric Streams,把 CloudWatch Metric 的資料往 Kinesis Data Firehose 裡面丟:「CloudWatch Metric Streams – Send AWS Metrics to Partners and to Your Apps in Real Time」。

其中一個賣點是即時性比用 API 去拉好很多:

In order to make it easier for AWS Partners and others to gain access to CloudWatch metrics faster and at scale, we are launching CloudWatch Metric Streams. Instead of polling (which can result in 5 to 10 minutes of latency), metrics are delivered to a Kinesis Data Firehose stream.

格式上可以是 JSON 或是 Open Telemetry

When you set up a stream you choose between the binary Open Telemetry 0.7 format, and the human-readable JSON format.

另外一個賣點是價位,每千次 $0.003:

Pricing – You pay $0.003 for every 1000 metric updates, and for any charges associated with the Kinesis Data Firehose. To learn more, check out the pricing page.

另外算一下 Kinesis Data Firehose 的價錢,是以資料量的大小計費,不過最小計價單位是 5KB (一筆應該是不會到),單價是 $0.029/GB (us-east-1) 或是 $0.037/GB (ap-southeast-1),算了一下跟 CloudWatch Metrics Streams 比起來只是零頭...

之前如果要自己拉出來的話是透過 API call 抓,每 1000 次是 USD$0.01,這個方法相較起來便宜不少,不過數量多的時候還是一筆費用 (而且有不少 metrics 是一分鐘更新一次)。

如果只是要備份起來或是跑分析的話,也許先前用 API 拉的作法可能還是比較好?一個小時拉一次對於備份與分析應該都很夠了,而 alarm 的機制還是掛在 CloudWatch 上。

這次產品的定位看起來是要把 ecosystem 做起來:

We designed this feature with the goal of making it easier & more efficient for AWS Partners including Datadog, Dynatrace, New Relic, Splunk, and Sumo Logic to get access to metrics so that the partners can build even better tools.

EC2 總算支援透過 Serial Console 操作了...

以往 Amazon EC2 的機器爛到開不起來時只能「看」到 Console 的輸出,然後要把 root volume 掛到其他機器上修正,接著再掛回來 (然後沒修好就要再重複...),現在總算可以透過 EC2 Serial Console 來操作了:「Troubleshoot Boot and Networking Issues with New EC2 Serial Console」。

不過裡面有一些限制,首先機器必須是基於 AWS Nitro System,這個部份在「Amazon EC2 Instance Types」這邊可以翻到是不是 Nitro,比較新的 family type 應該都是 (像是 t3/t3a/t4g 都是 Nitro,但 t2 不是):

EC2 Serial Console access is available for EC2 instances based on the AWS Nitro System. It supports all major Linux distributions, FreeBSD, NetBSD, Microsoft Windows, and VMWare.

另外是支援的區域目前只有幾個主力區域,不過公司在用的主力區 (新加坡) 也進去了,之後遇到問題的時候可以測試看看:

  • US East (N. Virginia), US West (Oregon), US East (Ohio)
  • Europe (Ireland), Europe (Frankfurt)
  • Asia Pacific (Tokyo), Asia Pacific (Sydney), Asia Pacific (Singapore)

Twitter 上看到有些人有提到「總算啊...」之類的感想...

路透社的 RSS feed

翻資料才發現路透社的 feed 已經不見了,大概是去年 2020 年六月的事情:「Returning the "killed" RSS of Reuters from the dead」。

不過文章裡面提到的替代方案還蠻有趣的,Google News 上可以透過 filter 條件輸出:

https://news.google.com/rss/search?q=when:24h+allinurl:reuters.com&ceid=US:en&hl=en-US&gl=US

不過測了一下台灣自家的新聞媒體網址,看起來不會動... 把網站改成 news.google.com.tw 也不行,如果沒有提供的還是得自己寫...