prime

另外一則跟 Prime Video 有關的有趣留言

在 Hacker News 上的「Even Amazon can't make sense of serverless or microservices (world.hey.com)」這邊看到 DHH 抓到機會剛好戳了 AWS 的費用問題，不過讓我注意到的不是 DHH 的文章，而是在 Hacker News 上的留言 35823366，自稱是當時設計這個 serverless 架構的人出來解釋當時的確有壓力測試過，看起來也還 OK：

I actually designed the original serverless system (a few years back when I was still at Prime Video), and yes we did and it did sort of look like it could work until it didn't. Obviously wasn't the right solution for the scale we had in mind (or rather the type of problem we were working on) but it's sad to see the mistake of one team be used to justify shitting on serverless as a general solution.

但這是全新帳號的發言，拿這個 id 去找是可以翻到 TimboKZ 這個 GitHub 帳號，然後一路找也可以看到 Timur Kuzhagaliyev 的 LinkedIn 帳號，這個帳號裡面有提到 2019 到 2020 的時候在 Amazon 裡面當 SDE：

Worked on various computer vision projects as a part of Video Quality Analysis team at Prime Video.

是都符合沒錯，但還是不能完全肯定，目前看八卦的心態還是比較多...

Amazon.com CTO 解釋 Prime Video 的那篇文章

先前在「Amazon Prime Video 捨棄 AWS Step Functions 回頭用 EC2 與 ECS 省錢的文章」這篇裡面有提到 Prime Video 團隊寫的文章，有點負面 PR 的感覺。

而讓人意外的是 Amazon.com 的 CTO Werner Vogels 也寫了一篇文章提到這件事情：「Monoliths are not dinosaurs」。

從標題可以看到他是針對 monolith 架構以及其他的架構在討論，而不是在講 AWS 成本問題 (這反而是原本 Prime Video 那篇文章在標題宣傳的)，另外也講了一些管理面上的東西，也就是文章裡面的副標題，以及兩段粗體字：

Building evolvable software systems is a strategy, not a religion. And revisiting your architectures with an open mind is a must.

If you hire the best engineers, you should trust them to make the best decisions.

there is not one architectural pattern to rule them all.

至於算不算是提油... 就見仁見智了。

Amazon Prime Video 捨棄 AWS Step Functions 回頭用 EC2 與 ECS 省錢的文章

昨天在 Hacker News 上熱烈討論的文章，是一篇三月就放出來，但昨天被丟上來意外的熱烈討論，在講 Amazon Prime Video 的團隊改寫程式，把 AWS Step Functions 拔掉，並且回頭用 EC2 與 ECS 而省下大量 AWS 費用的文章討論：「Scaling up the Prime Video audio/video monitoring service and reducing costs (primevideotech.com)」，原文在「Scaling up the Prime Video audio/video monitoring service and reducing costs by 90%」，Internet Archive 的備份、Archive Today 的備份。

先看文章的部分，裡面提到了他們用 AWS Step Functions，但意外的貴：

The initial version of our service consisted of distributed components that were orchestrated by AWS Step Functions. The two most expensive operations in terms of cost were the orchestration workflow and when data passed between distributed components.

然後改寫程式把所有東西都放在單一 process 裡面跑就好，用標準的 EC2 或是 ECS 就可以 scale 很好，而且也省錢：

To address this, we moved all components into a single process to keep the data transfer within the process memory, which also simplified the orchestration logic. Because we compiled all the operations into a single process, we could rely on scalable Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Elastic Container Service (Amazon ECS) instances for the deployment.

可以看出起因是一開始設計時的 overdesign，把可以簡單處理的東西拆開，另外加上雲端在這塊收費特別貴而導致成本爆增... 這件事情偶而會發生，尤其是比較新的東西會沒注意到成本，通常在上線發現不太對的時候就會安排 refactor 掉。

但如果是 Amazon 自家集團的其他團隊出來抱怨，就有很棒的 PR 效果了，所以 Hacker News 上就看到有人在猜可能過不久後文章就會不見 XD (但文章紅了以後應該就不會不見 XD)：

My word. I'm sort of gob smacked this article exists.

I know there are nuances in the article, but my first impression was it's saying "we went back to basics and stopped using needless expensive AWS stuff that caused us to completely over architect our application and the results were much better". Which is good lesson, and a good story, but there's a kind of irony it's come from an internal Amazon team. As another poster commented, I wouldn't be surprised if it's taken down at some point.

很政治不正確的文章 XD

以之前的經驗來說，AWS 上類似的東西還包括了 NAT Gateway，這東西只適合在有強資安需求 (像是法規要求)，而且需要連外的流量很少的時候適合。

NAT Gateway 在新加坡 ap-southeast-1 要 $0.059/hr (美金，所以大約是 $42.48/mo)，以及 US$0.045/GB 的處理費用，所以假設你每天只有 100GB (平均 10Mbps)，就等於是 3TB/mo，要 $135/mo。這樣整包就 $172.48/mo 了。

如果讓 EC2 機器直接連去 internet 抓資料的話，這些費用就是 $0，你只要付無論是有 NAT Gateway 或是沒有 NAT Gateway 的 outbound traffic 費用部分 (大多是各種 TCP/TLS/HTTP header)。

比較省成本的解法是用 security group 對 outbound traffic 開放特定的流量來解。

另外一種方式還是 NAT，但是是自己架設 HA 的 NAT service，像是 2015 年的文章「The Right Way to set up NAT in EC2」提到的方法。

這個方法以現在的機種來說，兩台 t4g.nano 的機器加上 EBS 不到 $10/mo，唯一要注意的應該是網路頻寬雖然可以 burst 到 5Gbps，但他的網路頻寬是 credit 機制，當 credit 用完的時候 t4g.nano 記得是剩下 100Mbps 左右？不過真的有這個量的時候機器也可以往上開大一點...

另外還有很多「好用」的雲端服務，但看到帳單後就變得「不好用」的雲端服務... 在用之前先算一下成本就會發現了。

OpenSSH 9.0 的 scp 與 sntrup761x25519 (Streamlined NTRU Prime)

OpenSSH 9.0 發行了，從 release-9.0 這邊可以看到重點，首先是 scp 的底層換掉，改用 SFTP 了，這點在先前「OpenSSH 的 scp 改用 SFTP 協定」這邊也有提到過：

This release switches scp(1) from using the legacy scp/rcp protocol to using the SFTP protocol by default.

另外一個是將 OpenSSH 8.9 引入的 post-quantum cipher 放入 default：

ssh(1), sshd(8): use the hybrid Streamlined NTRU Prime + x25519 key exchange method by default ("sntrup761x25519-sha512@openssh.com"). The NTRU algorithm is believed to resist attacks enabled by future quantum computers and is paired with the X25519 ECDH key exchange (the previous default) as a backstop against any weaknesses in NTRU Prime that may be discovered in the future. The combination ensures that the hybrid exchange offers at least as good security as the status quo.

We are making this change now (i.e. ahead of cryptographically-relevant quantum computers) to prevent "capture now, decrypt later" attacks where an adversary who can record and store SSH session ciphertext would be able to decrypt it once a sufficiently advanced quantum computer is available.

查了一下 Streamlined NTRU Prime 發現有 djb 參與，而且發現頭銜上面有掛中研院：

Daniel J. Bernstein, University of Illinois at Chicago, USA, and Ruhr University Bochum, Germany, and Academia Sinica, Taiwan

清單裡面有不少中研院的人：

Ming-Shing Chen, Academia Sinica, Taiwan

Bo-Yuan Peng, National Taiwan University and Academia Sinica, Taiwan

裡面還看到魯教授：

Bo-Yin Yang, Academia Sinica, Taiwan

後面也有看到計畫相關的資訊：

Taiwanese authors were supported by Taiwan Ministry of Science and Technology Grants 108-2221-E-001-008 and 109-2221-E-001-009-MY3, Sinica Investigator Award AS-IA-109-M01, Executive Yuan Data Safety and Talent Cultivation Project (AS-KPQ-109-DSTCP).

第 48 個梅森質數確認

在「Great Internet Mersenne Prime Search - PrimeNet」這邊看到 GIMPS 確認了 $M_{57885161}$ (也就是 $2^{57885161}-1$ ) 是第 48 個梅森質數：

All tests smaller than the 48th Mersenne Prime, M(57885161), have been verified

第 47 個梅森質數是 $M_{43112609}$ ，但一直都沒有確認完到 $M_{57885161}$ 中間是否還有梅森質數，這次宣佈全部都算完，而且雙重確認過，所以可以確認第 48 個梅森質數。

下一個要確認的是第 49 個梅森質數，目前已知的下一個是 $M_{74207281}$ ，也就是要看到 $M_{74207281}$ 中間是否還有其他的梅森質數...

還原被碼掉的 PEM 資訊 (SSH RSA key)

在「Recovering a full PEM Private Key when half of it is redacted」這邊看到的，起因是 _SaxX_ 幫客戶做滲透測試時找到客戶公開在網路上的 SSH key，然後他就碼掉一部分貼出來：

This morning, I began another #pentest for a client. After some google-fu dorking combine with the major search engines, I found the id_rsa key that gave me access to the server and a bunch of others 😬😬

another mistake under their radar... pic.twitter.com/K99UyjKTzW

— S. A. X. X. (@_SaxX_) March 24, 2021

原圖是這樣，接下來就開始被還原 XD

首先是 OCR 的過程，被稱為是整個還原過程最難的一部分 (哭爸啊)：

Ironically, this was the hardest part of the challenge. It took the longest time of all the steps and was the easiest to make errors in.

接下來就是解讀 PEM 檔的格式，可以藉此得到裡面的參數。

然後是套公式，窮舉運算裡面的值，可以看到迴圈 kp 只算了 3 到 65537，就推算出可能的 p：

e = 65537
q = 0xc28871e8714090e0a33327b88acc57eef2eb6033ac6bc44f31baeac33cbc026c3e8bbe9e0f77c8dbc0b4bfed0273f6621d24bc1effc0b6c06427b89758f6d433a02bf996d42e1e2750738ac3b85b4a187a3bcb4124d62296cb0eecaa5b70fb84a12254b0973797a1e53829ec59f22238eab77b211664fc2fa686893dda43756c895953e573fd52aa9bb41d22306135c81174a001b32f5407d4f72d80c5de2850541de5d55c19c1f817eea994dfa534b6d941ba204b306225a9e06ddb048f4e34507540fb3f03efeb30bdd076cfa22b135c9037c9e18fe4fa70cf61cea8c002e9c85e53c1eaac935042d00697270f05b8a7976846963c933dadd527227e6c45e1
dp = 0x878f7c1b9b19b1693c1371305f194cd08c770c8f5976b2d8e3cf769a1117080d6e90a10aef9da6eb5b34219b71f4c8e5cde3a9d36945ac507ee6dfe4c146e7458ef83fa065e3036e5fbf15597e97a7ba93a31124d97c177e68e38adc4c45858417abf8034745d6b3782a195e6dd3cf0be14f5d97247900e9aac3b2b5a89f33a3f8f71d27d670401ca185eb9c88644b7985e4d98a7da37bfffdb737e54b6e0de2004d0c8c425fb16380431d7de40540c02346c98991b748ebbc8aac73dd58de6f7ff00a302f4047020b6cd9098f6ba686994f5e043e7181edfc552e18bce42b3a42b63f7ccb7729b74e76a040055d397278cb939240f236d0a2a79757ba7a9f09

for kp in range(3, e):
    p_mul = dp * e - 1
    if p_mul % kp == 0:
        p = (p_mul // kp) + 1
        if isPrime(p):
            print(f"Possible p: {p}")

後面就是跑驗證確認，就被打出來了...

RSA-240 (十進位 240 位數) 成功分解

在「RSA-240 Factored」這邊看到的，RSA-240 前幾天被解開了：

RSA-240 = 124620366781718784065835044608106590434820374651678805754818788883289666801188210855036039570272508747509864768438458621054865537970253930571891217684318286362846948405301614416430468066875699415246993185704183030512549594371372159029236099 = 509435952285839914555051023580843714132648382024111473186660296521821206469746700620316443478873837606252372049619334517 * 244624208838318150567813139024002896653802092578931401452041221336558477095178155258218897735030590669041302045908071447

有效長度是 795 bits，相較於十年前解出來的 RSA-768 (768 bits) 又多了「一些」。看了一下，算法上沒有太多突破，主要是硬體的發展與軟體的最佳化有進展...

ROCA：Infineon Technologies 的 RSA 實做問題

最近的另外一個大包，不過這包是 Infineon Technologies 在實做 RSA 算法時的問題，倒不是 RSA 算法本身有問題。之所以會「大」是因為有太多人用了：「ROCA: Vulnerable RSA generation (CVE-2017-15361)」。

起因於 Infineon Technologies 在產生 key 時的組合有限，於是要猜測的 keyspace 小很多。

以研究者的估算，可以看出 CPU year 都被大幅減少了，都是屬於「可行」的範圍：

The time complexity and cost for the selected key lengths (Intel E5-2650 v3@3GHz Q2/2014):

512 bit RSA keys - 2 CPU hours (the cost of $0.06);
1024 bit RSA keys – 97 CPU days (the cost of $40-$80);
2048 bit RSA keys – 140.8 CPU years, (the cost of $20,000 - $40,000).

而且這邊是用 CPU year 估算，如果考慮 FPGA 加速計算，應該會短更多...

另外從發現到公開的時間線也拉得很長，可以看到中間一直在找解決方案：

2nd of November 2017 - Presentation of all details at the ACM CCS conference (to come)
16th of October 2017 - The initial version of the public disclosure published
May to October 2017 - Cooperation with the manufacturer and other affected parties to help evaluate and mitigate the vulnerability
1st of February - The vulnerability disclosed to Infineon Technologies AG
End of January - The vulnerability found

過一陣子就會去 conference 上報告了...

Amazon 決定在自家的購物網站上停售 Google Chromecast 與 Apple TV

這大概是這幾天最刺激的事情... Amazon 決定在自家購物往站上停售 Google Chromecast 與 Apple TV：「Amazon to Ban Sale of Apple, Google Video-Streaming Devices」。

引用新聞的報導說明，因為 Chromecast 與 Apple TV 不相容於 Amazon 自家的 Prime Video，所以下令所有的店家停售：

The Seattle-based Web retailer sent an e-mail to its marketplace sellers that it will stop selling the Apple TV and Google’s Chromecast since those devices don’t "interact well" with Prime Video. No new listings for the products will be allowed and posting of existing inventory will be removed Oct. 29, Amazon said. Prime Video doesn’t run easily on its rival’s hardware.

至於會不會引起反壟斷調查呢？站在看好戲的立場當然是很希望，但這兩個產品就算不在 Amazon 上買，也有一卡車的競爭對手有賣... 不曉得後面會怎麼發展。

Amazon 在巴爾的摩與邁阿密推出「一個小時到貨」服務...

從「Amazon Launches One-Hour Delivery Service In Baltimore and Miami」這邊看到的，引用的報導是「Amazon launches one-hour delivery service in Baltimore」。

有付 Amazon Prime 費用的人在這兩個地區將提供兩小時到貨 (free)，或是一小時到貨 (多收 USD$7.99)：

The service will be available in select zip codes to Amazon Prime subscribers, who pay $99 a year for unlimited free two-day delivery on more than 20 million items. The one-hour service, available through the Prime Now mobile app, costs $7.99, while two-hour delivery will be free.

去年在曼哈頓的實驗算是成功吧... 只是 FAA 同意無人小型貨運機的實驗授權一下來就這樣公佈啊... (Amazon Gets Experimental Airworthiness Certificate)