Amazon EC2 Auto Scaling 支援 Warm Pools

EC2 推出的新功能:「Amazon EC2 Auto Scaling introduces Warm Pools to accelerate scale out while saving money」。

重點只有這個,這個作法是先把機器準備好,然後關掉放在 stopped 狀態:

Additionally, Warm Pools offer a way to save compute costs by placing pre-initialized instances in a stopped state.

理論上可以快到 30 秒:

Now, these applications can start pre-initialized, stopped instances to serve traffic in as low as 30 seconds.

不過考慮到就算是 stopped 的機器,啟動時還是得去確認有沒有新版程式... 目前可以理解的部份,應該是加快 EBS 的準備時間吧?

架了一台 News Server

前陣子跟朋友聊天的時候,才想到好像沒有公開提這件事情...

學術網路上的 news server 似乎都掛差不多了,就花了一些時間用 INN 架了一台 news server,然後找了兩個 peering 對接,給自己的 BBS 站台用:「newsfeed.hasname.com」。

一般目前比較常用的是 news.aioe.org,不過有限制每天最多只能發 40 則:

In order to avoid mass abuses, every IP address is authorized to post no more than 40 messages per day.

有架設 BBS 站又想要弄轉信的朋友可以來戳一下,需要有固定 IP address 就是了。

Google 與 Oracle 對 Java API 爭議的案子

前幾天應該很多媒體都有報導了,這邊算是整理一下看到的資料。

美國最高法院公佈的全文在「18-956_d18f.pdf」這邊可以看到,算是最重要的資料。

另外很多地方也有更新,像是維基百科上面的條目「Google LLC v. Oracle America, Inc.」。

這次的案件在軟體產業也很關注,難得可以在 Hacker News 上看到 upvote 超過四千的新聞:「Google’s copying of the Java SE API was fair use [pdf] (supremecourt.gov)」,不過裡面的討論我覺得就是鄉民拿著爆米花的感覺...

第一個重要的消息當然是 6-2 認定 fair use,並且讓聯邦法院重審 (但最高法院已經把最重要的部份拍板定案了),不過要注意的是,對於更基本的問題「API 是否有著作權」並沒有定案:

In April 2021, the Supreme Court ruled in a 6–2 decision that Google's use of the Java APIs fell within the four factors of fair use, bypassing the question on the copyrightability of the APIs. The decision reversed the Federal Circuit ruling and remanded the case for further review.

判決全文 PDF 的前面三頁多算是簡介說明這次的重點,Page 44 到 Page 62 則是反對的兩位大法官 (Clarence ThomasSamuel Alito) 所提出的異議,可以看到兩位大法官批評了 copyrightability 與 fair-use analysis 的問題。

這次的結果對軟體與網路產業影響超級大,舉個例子來說,一堆公司都有推出與 Amazon S3 相容 API 的產品 (這邊是 Network-based API)。另外 Firefox 直接拿 Chromium 的 Manifest 格式來相容降低開發者開發 extension 的成本。

之後應該可以看到大家用的更爽了...

Django 3.2 LTS

Django 3.2 LTS 出了:「Django 3.2 released」,對應的 release note 可以在「Django 3.2 release notes」這邊看到。

這個版本比較特別,一般版本提供大約 15 個月的支援,LTS 版本則提供 36 個月的支援,不過目前用起來的感覺還是鼓勵大家平常就要安排升級計畫,不然 3.2 升級到 4.2 鐵定是個超痛苦的過程。

昨天把 Django 3.1 的專案升級上 3.2 後,跑 test case 就遇到「Customizing type of auto-created primary keys」這邊描述的問題,目前先用 DEFAULT_AUTO_FIELD = 'django.db.models.AutoField' 解,其他的倒是沒什麼問題...

幾個我覺得有趣的,像是 JSONL 格式:

The new JSONL serializer allows using the JSON Lines format with dumpdata and loaddata. This can be useful for populating large databases because data is loaded line by line into memory, rather than being loaded all at once.

然後不支援 PostgreSQL 9.5 與 MySQL 5.6 了:

Upstream support for PostgreSQL 9.5 ends in February 2021. Django 3.2 supports PostgreSQL 9.6 and higher.

The end of upstream support for MySQL 5.6 is April 2021. Django 3.2 supports MySQL 5.7 and higher.

很久前寫 Django 1.x,然後就荒廢很久,最近是從 3.0 開始寫,有些東西熟一點以後覺得怪怪的,之後要找時間測一些東西,修正對 Django 的用法。

Amazon EC2 提供跨區直接複製 AMI (Image) 的功能

Amazon EC2AMI 可以跨區複製了:「Amazon EC2 now allows you to copy Amazon Machine Images across AWS GovCloud, AWS China and other AWS Regions」。

如同公告提到的,在這個功能出來以前,想要產生一樣的 image 得重新在 build 一份:

Previously, to copy AMIs across these AWS regions, you had to rebuild the AMI in each of them. These partitions enabled data isolation but often made this copy process complex, time-consuming and expensive.

有一些限制,image 大小必須在 1TB 以下,另外需要存到 S3 上,不過這些限制應該是還好:

This feature provides a packaged format that allows AMIs of size 1TB or less to be stored in AWS Simple Storage Service (S3) and later moved to any other region.

然後目前只有透過 cli 操作的方式,或是直接用 SDK 呼叫 API,看起來 web console 還沒提供:

This functionality is available through the AWS Command Line Interface (AWS CLI) and the AWS Software Development Kit (AWS SDK). To learn more about copying AMIs across these partitions, please refer to the documentation.

GitHub 的 API Token 換格式

GitHub 前幾天宣佈更換 API token 的格式:「Authentication token format updates are generally available」,在今年三月初的時候有先公告要換:「Authentication token format updates」。

另外昨天也解釋了換成這樣的優點:「Behind GitHub’s new authentication token formats」。

首先是 token 的字元集合變大了:

The character set changed from [a-f0-9] to [A-Za-z0-9_]

另外是增加了 prefix 直接指出是什麼種類的 token:

The format now includes a prefix for each token type:

  • ghp_ for Personal Access Tokens
  • gho_ for OAuth Access tokens
  • ghu_ for GitHub App user-to-server tokens
  • ghs_ for GitHub App server-to-server tokens
  • ghr_ for GitHub App refresh tokens

另外官方目前先不會改變 token 長度 (透過字元變多增加 entropy),但未來有打算要增加:

The length of our tokens is remaining the same for now. However, GitHub tokens will likely increase in length in future updates, so integrators should plan to support tokens up to 255 characters after June 1, 2021.

看起來當初當作 hex string 而轉成 binary 會有問題,不過就算這樣做應該也是轉的回來的。

回到好處的部份,這個作法跟 SlackStripe 類似,讓開發者或是管理者更容易辨識 token 的類型:

As we see across the industry from companies like Slack and Stripe, token prefixes are a clear way to make tokens identifiable. We are including specific 3 letter prefixes to represent each token, starting with a company signifier, gh, and the first letter of the token type.

另外這也讓 secret scanning 的準確度更高,本來是 40 bytes 的 hex string,有機會撞到程式碼內的 SHA-1 string:

Many of our old authentication token formats are hex-encoded 40 character strings that are indistinguishable from other encoded data like SHA hashes. These have several limitations, such as inefficient or even inaccurate detection of compromised tokens for our secret scanning feature.

另外官方也建議現有的 token 換成新的格式,這樣如果真的發生洩漏,可以透過 secret scanning 偵測並通知:

We strongly encourage you to reset any personal access tokens and OAuth tokens you have. These improvements help secret scanning detection and will help you mitigate any risk to compromised tokens.

AWS 推出 Amazon Route 53 Resolver DNS Firewall

長久以來的洞總算有比較好的方法補上了,AWS 推出了 Amazon Route 53 Resolver DNS Firewall:「Introducing Amazon Route 53 Resolver DNS Firewall」。

Route 53 Resolver 是 AWS 官方提供的 DNS Resolver,沒有特殊的設定的話通常會在 x.x.x.2 (/24 或是更大的網段),先前一直沒有辦法解決 data leak 的問題,也就是透過 DNS 把敏感資料從 private network 裡丟出去。

以前的作法是透過 security group 擋掉對 Route 53 Resolver 的流量 (或是透過 VPC 的 Firewall 擋),然後自己架設兩台 DNS resolver 過濾,現在 Route 53 Resolver 支援 DNS Firewall,提供 allowlist 與 blocklist 這兩個功能使用,總算是把這件事情解的比較乾淨了:

Route 53 Resolver DNS Firewall lets you create “blocklists” for domains you don’t want your VPC resources to communicate with via DNS. You can also take a stricter, “walled-garden” approach by creating “allowlists” that permit outbound DNS queries only to domains you specify. You can also create alerts for when outbound DNS queries match certain firewall rules, allowing you to test your rules before deploying for production traffic.

另外這次的 DNS Firwall 提供了兩組由 AWS 維護的清單讓人使用,包括了 malware 與 botnet:

Route 53 Resolver DNS Firewall offers two managed domain lists—malware domains and botnet command and control domains—enabling you to get started quickly with managed protections against common threats.

這樣省事多了...

Facebook 5.33 億筆個資外洩,以及 Mark Zuckerberg 的電話...

這兩天應該已經有很多其他媒體報導了 Facebook 5.33 億筆資料外洩的事情:「533 million Facebook users' phone numbers and personal data have been leaked online」。

據稱是用 2019 年的洞撈出來的,不過這份資料看起來也不是完整資料,舉個例子來說,台灣只有 73 萬筆左右,而且也少了很多地區,像是泰國就不在列表內...

另外一個比較特別的是,Mark Zuckerberg 的電話也在這次外洩資料裡面:「Mark Zuckerberg's phone number appeared among the leaked data of Facebook users, according to a researcher」,應該會換號碼了。

這包資料看起來會這陣子會很熱門...

利用 Cloudflare Workers 繞過 Cloudflare 自家的阻擋機制

Hacker News 首頁上看到「How to bypass Cloudflare bot protection (jychp.medium.com)」這則,裡面的文章是「How to bypass CloudFlare bot protection ?」這篇,利用 Cloudflare Workers 繞過 Cloudflare 自家的 CAPTCHA 機制。

這個漏洞有先被送給 Cloudflare,但被認為不是問題,所以作者就決定公開:

Several months ago I submitted what appeared to be a security flaw to CloudFalre’s bugbounty program. According to them, this is not a problem, it’s up to you to make up your own mind.

技術上就是透過 Cloudflare Workers 當作 proxy server,只是看起來 Cloudflare 對自家 IP 有特別處理,在設定妥當後,用 Cloudflare Workers 的 IP address 去連 Cloudflare 的站台,幾乎不會觸發 Cloudflare 的阻擋機制。

不過 free tier 還是有限制,主要就是數量:

The first 100,000 requests each day are free and paid plans start at just $5/10 million requests, making Workers as much as ten-times less expensive than other serverless platforms.

作者也有提到這點:

So let’s enjoy the 100 000 request/day for your free Cloudflare account and go scrape the world !

但這是個有趣的方法,加上信用卡盜刷之類的方式,這整包看起來就很有威力...

Mapbox GL JS 的授權改變,以及 MapLibre GL 的誕生

看到「MapLibre GL is a free and open-source fork of mapbox-gl-JS (github.com/maplibre)」這篇,翻了一下資料發現年初時 Mapbox GL JS 的軟體授權從 v2.0.0 開始變成不是 open source license (本來是 BSD license),而社群也馬上 fork 最後一個 open source 版本並且投入開發,變成 MapLibre GL

MapTiler 在年初的時候有提到這件事情:「MapLibre: Mapbox GL open-source fork」。

The community reacted swiftly: forks of the latest open-source version were made almost immediately by multiple parties. In another positive development, the community came together the next day and agreed to make this a joint effort, rather than splitting energies. A video call was organized and the MapLibre coalition was formed. It includes people working for MapTiler, Elastic, StadiaMaps, Microsoft, Ceres Imaging, WhereGroup, Jawg, Stamen Design, etc.

MapLibre GL 目前與本來 v1.13.0 相容,可以直接抽換過去 (後來在二月的時候有出一個 v1.13.1,不過那是在 v2.0.0 改 license 之後的事情了):

  "dependencies": {
-    "mapbox-gl": "^1.13.0"
+    "maplibre-gl": ">=1.14.0"
  }

記錄一下,以後要在網站上用的話,得注意到 Mapbox GL JS 在沒有註冊的情況下不能使用,而且 SDK 會強制蒐集資料:

Mapbox gl-js version 2.0 or higher (“Mapbox Web SDK”) must be used according to the Mapbox Terms of Service. This license allows developers with a current active Mapbox account to use and modify the Mapbox Web SDK. Developers may modify the Mapbox Web SDK code so long as the modifications do not change or interfere with marked portions of the code related to billing, accounting, and anonymized data collection. The Mapbox Web SDK only sends anonymized usage data, which Mapbox uses for fixing bugs and errors, accounting, and generating aggregated anonymized statistics. This license terminates automatically if a user no longer has an active Mapbox account.

不過如果是抓 OpenStreetMap 資料的話,Leaflet 應該還是目前的首選...