Amazon DocumentDB 推出相容 MongoDB 4.0 的版本

在「Amazon DocumentDB (with MongoDB compatibility) adds support for MongoDB 4.0 and transactions」這邊看到 AWSAmazon DocumentDB 上推出相容 MongoDB 4.0 的版本。

把年初在 Ptt 上寫的「Re: [請益] 選擇mongoDB或是relational database ??」這篇拿出來講一下,MongoDB 4.0 最大的改進就是 multi-document transactions 了。

不過 AWS 先前推出 DocumentDB (MongoDB) 時看到的限制,大家都猜測是用 PostgreSQL 當底層 (「AWS 推出 MongoDB 服務:Amazon DocumentDB」與「大家在猜 Amazon DocumentDB 的底層是不是 PostgreSQL...」),雖然目前還是不太清楚,但如果這個猜測屬實的話,要推出各種 transaction 的支援完全不是問題 XDDD

RFC 裡面的 MUST/SHOULD/MAY

讀 RFC 文件時常看到會使用這組定義,甚至有些非 RFC 文件也會使用:

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

這邊提到的 RFC 2119 就是 1997 年訂的「Key words for use in RFCs to Indicate Requirement Levels」這篇,以 2020 年的現在來說,這組定義已經被人熟知,用這組定義可以讓閱讀的人很輕鬆的了解條件的強制性。

剛剛在讀新的文件時發現這段文字有更新,往回查發現是針對大小寫的差異提出更新:「Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words」,主要是這兩條:

  • The words have the meanings specified herein only when they are in all capitals.
  • When these words are not capitalized, they have their normal English meanings and are not affected by this document.

然後也更新了引用的部份:

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

不過就算是 2017 年之前應該也是這樣讀就是了...

AWS 推出 MongoDB 服務:Amazon DocumentDB

AWS 推出了 Amazon DocumentDB 服務,相容於 MongoDB 3.4 3.6 的界面:「New – Amazon DocumentDB (with MongoDB Compatibility): Fast, Scalable, and Highly Available」。

這個新聞其實引起不少關注,不單純是 AWS 支援了 MongoDB service,而是 AWS 對去年一系列 license issue 的態度。

先講 license 的事情,後面再提技術上的差異。

背景是 MongoDB 在去年十月的時候決定換 license,決定從 GNU AGPL 換成他們自己定義的 SSPL:「MongoDB now released under the Server Side Public License」。

相關的報導可以參考 TechCrunch 當時寫的「MongoDB switches up its open-source license」,主要的重點在於:

[T]he SSPL explicitly states that anybody who wants to offer MongoDB as a service — or really any other software that uses this license — needs to either get a commercial license or open source the service to give back the community.

而 AWS 在三個月後的回應也意外的清楚,他直接照著 MongoDB 3.6 版的 API 刻一個出來,不需要用你的軟體提供服務 (所以就不用照你的 license 走):

Amazon DocumentDB implements the Apache 2.0 open source MongoDB 3.6 API by emulating the responses that a MongoDB client expects from a MongoDB server, allowing you to use your existing MongoDB drivers and tools with Amazon DocumentDB.

TechCrunch 下的標題也頗直接,認為 AWS 對這套搞法不怎麼認同:「AWS gives open source the middle finger」。

回到技術上的層面來看,可以看到 Amazon DocumentDB 提供的技術資料看起來跟 Amazon Aurora 很像,都是六份三區:

Amazon DocumentDB uses a purpose-built SSD-based storage layer, with 6x replication across 3 separate Availability Zones.

連 read replica 的限制也都是 15 份,可以「猜測」後面應該是用同一套技術在運作...:

In Amazon DocumentDB, the storage and compute are decoupled, allowing each to scale independently, and developers can increase the read capacity to millions of requests per second by adding up to 15 low latency read replicas in minutes, regardless of the size of your data.

看了一下價錢,最小台是 db.r4.large,需要 USD$0.277/hr,相當於一個月要 USD$200 左右,而且 storage 與 i/o 要另外計算,門檻不算低。

目前主要還是歐美區先上:

Amazon DocumentDB (with MongoDB compatibility) is available now and you can start using it today in the US East (N. Virginia), US East (Ohio), US West (Oregon), and Europe (Ireland) Regions.

隔壁棚的 Redis 不知道有什麼感想...

SQL 的設計與寫作規範

看到「SQL Style Guide」這個網站,把 SQL 常見的行為都列出來,寫了一份規範... 每個團隊未必都要照這個規範走,可以透過他條列的項目思考,再改成自己團隊的規範。

附註一下,最底下有繁體中文的翻譯版本,如果懶的看英文的版本可以看這份:「SQL樣式指南 · SQL Style Guide」。

SSL Certificate 的認證方式限縮

在「Ballot 218 - Remove validation methods 1 and 5 - CAB Forum」看到「Ballot 218: Remove validation methods #1 and #5」這則議案以 78% 的同意票通過,限縮 SSL Certificate 的認證方式。眼睛瞄到中華電信投下反對票:

14 Yes votes: CFCA, Cisco, Comodo CA, D-TRUST, DigiCert, GDCA, GlobalSign, GoDaddy, Izenpe, Let’s Encrypt, Logius PKIoverheid, SSL.com, TrustCor, Trustwave

4 No votes: Buypass, Chunghwa Telecom, Entrust Datacard, SwissSign

4 Abstain: Actalis, Disig, HARICA, OATI

78% of voting CAs voted in favor

找了一下在 BR (Baseline Requirements) 的 3.2.2.4.1 與 3.2.2.4.5,其中前者是透過註冊商認證:

3.2.2.4.1 Validating the Applicant as a Domain Contact

Confirming the Applicant's control over the FQDN by validating the Applicant is the Domain Contact directly with the Domain Name Registrar.

後者是透過文件認證:

3.2.2.4.5 Domain Authorization Document

Confirming the Applicant's control over the FQDN by relying upon the attestation to the authority of the Applicant to request a Certificate contained in a Domain Authorization Document.

在想投下反對的原因,會不會是因為中華自己的 domain 應該都是透過後者方式發的?透過內部公文系統...

在 AWS 上跑 Consul 與 Vault 的介紹

HashiCorp 這邊看到在 AWS 上跑 ConsulVault 的介紹文章:「Consul and Vault on AWS: Quick Start Guides」。

Consul 負責 service discovery 與 health check (還有簡單的 key-value 功能);Vault 則負責管理各種 secret (像是資料庫的帳號密碼之類的資訊)。

這些資料分別可以在「HashiCorp Consul on AWS」與「HashiCorp Vault on AWS」看到,打開 PDF 後可以發現是 AWS 與 HashiCorp 的人合作生出來的文件,要在上面實作的人可以看一看,應該是可以少走冤枉路...

另外一篇講文件掃描的...

在「Page dewarping」這篇看到講文件掃描的技術,以及 open source 的程式,對比之前提到的「Dropbox 的文件掃描功能」與「Dropbox 的 Document Detecting」的時間點,有種淡淡的惡意 XD

這篇作者是為了未婚妻的需求而寫出來的,本來是作者收到學生的作業時手動在跑,後來未婚妻也拿去用,但量愈來愈大,決定自動化處理:

A while back, I wrote a script to create PDFs from photos of hand-written text. It was nothing special – just adaptive thresholding and combining multiple images into a PDF – but it came in handy whenever a student emailed me their homework as a pile of JPEGs. After I demoed the program to my fiancée, she ended up asking me to run it from time to time on photos of archival documents for her linguistics research. This summer, she came back from the library with a number of images where the text was significantly warped due to curled pages.

So I decided to write a program that automatically turns pictures like the one on the left below to the one on the right:

程式都可以在 GitHub 上翻到:「Text page dewarping using a "cubic sheet" model」。跟 Dropbox 互別苗頭的感覺 XDDD

Dropbox 的文件掃描功能

算是講 Dropbox 的「Dropbox 的 Document Detecting」這篇的續集,在抓出文件位置後講顏色的校準:「Fast Document Rectification and Enhancement」。

要怎麼把左邊的原始圖轉換成右邊的圖,包括了座標轉換以及顏色校準:

顏色校準的部份講到了這張很有名的圖。在圖片上,A 與 B 的區塊顏色是相同的,但你校準出來的時候必須跟人腦的感覺相同:

Here’s a great illustration of this “illusion,” in which the two tiles marked A and B have the same pixel values, but appear to be very different:

這是最後的成果,左邊是原始圖,中間是將背景改成白色,其他顏色保留,而右邊則是試著修正顏色:

Left: the original image. Middle: an enhanced image, with the background becoming white and the foreground preserved in exact R, G, B values. Note that the colors appear faded. Right: an enhanced image that tries to correct for the perceptual discrepancy.

應該是在 Dropbox 裡面的專案,是個有不少數學可以玩的專案...