Home » Archive by category "Murmuring" (Page 4)

Amazon Aurora (MySQL) 提供 Parallel Query 讓人申請使用

AWS 宣佈了 Amazon Aurora (MySQL) 支援 Parallel Query:「Amazon Aurora Parallel Query is Available for Preview」。

這邊提到的 Parallel Query 比較像是 Amazon Athena,直接把單一 Query 打散到多台機器上跑:

Amazon Aurora Parallel Query improves the performance of large analytic queries by pushing processing down to the Aurora storage layer, spreading processing across hundreds of nodes.

也就是說,這算是單一 SQL Query 平行運算的進階版本。

在這之前,AWS 都已經支援單一 Query 在單台機器上利用多 CPU 平行運算。其中 PostgreSQL 是 9.6+ 本身就有支援。Amazon Aurora (MySQL) 則是在 2016 時透過 Parallel Read Ahead 支援某些情境下的的單一 Query 多 CPU 運算了 (發現之前沒寫到...):「Amazon Aurora Update – Parallel Read Ahead, Faster Indexing, NUMA Awareness」。

這個功能目前是 Preview 階段,然後開在這些地區讓大家測試使用:

The preview is available for the MySQL-compatible edition of Amazon Aurora, and is currently available in the US East (N. Virginia), US East (Ohio), US West (Oregon), and Europe (Ireland) Regions. Sign up to get access.

這個功能提供了想要提昇效能,但懶得改架構的人可以用錢直接硬換出來...

EnterpriseDB 打算推出的 zheap,想要解 VACUUM 問題...

前天被問到「DO or UNDO - there is no VACUUM」這篇,回家後仔細看一看再翻了一些資料,看起來是要往 InnoDB 的解法靠...

PostgreSQL 與 InnoDB 都是透過 MVCC 的概念實做 transaction 之間的互動,但兩者實際的作法不太一樣。其中帶來一個明顯的差異就是 PostgreSQL 需要 VACUUM。這點在同一篇作者八年前 (2011) 的文章就有提過兩者的差異以及優缺點:「MySQL vs. PostgreSQL, Part 2: VACUUM vs. Purge」。

UPDATE 時,InnoDB 會把新資料寫到表格內,然後把可能會被 rollback 的舊資料放到表格外:

In InnoDB, only the most recent version of an updated row is retained in the table itself. Old versions of updated rows are moved to the rollback segment, while deleted row versions are left in place and marked for future cleanup. Thus, purge must get rid of any deleted rows from the table itself, and clear out any old versions of updated rows from the rollback segment.

而被 DELETE 清除的資料則是由 purge thread 處理:

All the information necessary to find the deleted records that might need to be purged is also written to the rollback segment, so it's quite easy to find the rows that need to be cleaned out; and the old versions of the updated records are all in the rollback segment itself, so those are easy to find, too.

所以可以在 InnoDB 看到 purge thread 相關的設定:「MySQL :: MySQL 5.7 Reference Manual :: 14.6.11 Configuring InnoDB Purge Scheduling」,負責處理這些東西。

而在 PostgreSQL 的作法則是反過來,舊的資料放在原來地方,新資料另外存:

PostgreSQL takes a completely different approach. There is no rollback tablespace, or anything similar. When a row is updated, the old version is left in place; the new version is simply written into the table along with it.

新舊資料的位置其實還好,主要是因為沒有類似的地方可以記錄哪些要清:

Lacking a centralized record of what must be purged, PostgreSQL's VACUUM has historically needed to scan the entire table to look for records that might require cleanup.

這也使得 PostgreSQL 裡需要 autovacuum 之類的程序去掃,或是手動跑 vacuum。而在去年 (2017) 的文章裡也有提到目前還是類似的情況:「MVCC and VACUUM」。

而在今年 (2018) 的文章裡,EnterpriseDB 就提出了 zheap 的想法,在 UPDATE 時寫到 table 裡,把可能被 rollback 的資料放到 undo log 裡。其實就是把 InnoDB 那套方法拿過來用,只是整篇都沒提到而已 XD:

That brings me to the design which EnterpriseDB is proposing. We are working to build a new table storage format for PostgreSQL, which we’re calling zheap. In a zheap, whenever possible, we handle an UPDATE by moving the old row version to an undo log, and putting the new row version in the place previously occupied by the old one. If the transaction aborts, we retrieve the old row version from undo and put it back in the original location; if a concurrent transaction needs to see the old row version, it can find it in undo. Of course, this doesn’t work when the block is full and the row is getting wider, and there are some other problem cases as well, but it covers many useful cases. In the typical case, therefore, even bulk updates do not force a zheap to grow. Instead, the undo grows. When a transaction commits, all row versions that will become dead are in the undo, not the zheap.

不過馬上就會想到問題,如果要改善問題,不是個找地方記錄哪些位置要回收就好了嗎?順便改變方法是為了避免 fragment 嗎?

等著看之後變成什麼樣子吧...

Ubuntu 18.04 LTS Minimal Image 的大小

看到「RFC: Ubuntu 18.04 LTS Minimal Images」這篇,在蒐集將來要出的 Ubuntu 18.04 LTS Minimal Image 的意見...

The Ubuntu Minimal Image is the smallest base upon which a user can apt install any package in the Ubuntu archive.

雖然應該還會有改變,不過以目前的版本來看,可以看出壓縮前後兩種版本都比 16.04 小了不少:

對需要這些 image 的人來說 (像是當作 Docker 的 base image),小一點操作起來也比較開心...

Firefox 對 HTTPS 網站中 "Referer" 的保護

Firefox 從 59 之後,在開啟 Private Browsing 的情況下,不會送出完整的 Referer:「Preventing data leaks by stripping path information in HTTP Referrers」。

這篇吸引到我的是 EFF 的研究員發現的事情:

EFF researchers discovered this leak of personal health data from healthcare.gov to DoubleClick.

其中 EFF 研究員的文章是「HealthCare.gov Sends Personal Data to Dozens of Tracking Websites」這篇。

更好的作法應該是平常就完全阻擋,像是 Firefox 可以用 Referrer Control 設定,或是 Chrome 裡用 Referer Control 設定。

GitHub 停用過時加密演算法的計畫

先前有提到 GitHub 廢除 SSH 中的弱演算法 (參考「GitHub 明年關閉 SSH 上 SHA1 相關的 Kx (Key Exchange) 演算法」),現在宣佈詳細作法了:「Weak cryptographic standards removal notice」。

包括 HTTPS 的 TLSv1/TLSv1.1 以及 SSH 的 diffie-hellman-group1-sha1/diffie-hellman-group14-sha1 都會被廢止。而作法跟其他家不太一樣:

  • February 8, 2018 19:00 UTC (11:00 am PST): Disable deprecated algorithms for one hour
  • February 22, 2018 19:00 UTC (11:00 am PST): Permanently disable deprecated algorithms

先關閉一個小時讓沒看公告但是有注意到的人可以發現,然後過兩個禮拜後才完全關閉。跟其他家不太一樣的作法...

V8 version 6.5 (Chrome 65) 的改變

V8 version 6.5 將會有不少改變:「V8 release v6.5」。

其中因為 Spectre 的關係,新的 V8 設計了 Untrusted code mode,拿來跑不信任的程式,裡面會設計反制措施。而且這在新版的 Chrome 將會預設開啟:

In response to the latest speculative side-channel attack called Spectre, V8 introduced an untrusted code mode. If you embed V8, consider leveraging this mode in case your application processes user-generated, not-trustworthy code. Please note that the mode is enabled by default, including in Chrome.

另外是針對 WebAssembly 提供邊下載邊 compile 的能力,這讓速度大幅提昇。在原文是拿一個比較大包的 WebAssembly 來測試:

For the graph below we measure the time it takes to download and compile a WebAssembly module with 67 MB and about 190,000 functions. We do the measurements with 25 Mbit/sec, 50 Mbit/sec, and 100 Mbit/sec download speed.

可以看到網路不夠快的使用者就會直接被 compile 速度跟上,讓瀏覽器在下載時就做一些事情。

另外在某些情況下對 Array 的操作會有大幅改善:

這些新功能與改善都會在 Chrome 65 推出。依照「Chrome Platform Status」這邊的資料,stable 版預定在三月初,beta 版應該是要出了... (雖然上面寫著 2/1,但目前好像還沒更新)

Stripe 宣佈 TLS 1.0/1.1 的退場時間表

Stripe 宣佈了今年的 2/19 會停用測試環境的 TLS 1.0/1.1,並且在 6/13 全面停用:「Completing an upgrade to TLS 1.2」。

  • Monday, February 19: All servers using older versions of TLS will be blocked from the Stripe API in test mode.
  • Wednesday, June 13: All servers using older versions of TLS will be blocked from the Stripe API in live mode.

這喊好久了,總算是開槍了...

隔壁棚 PayPal 反而早就把 Sandbox 環境上 TLS 1.2-only 了,而六月底也會強制所有連線都必須是 TLS 1.2:「TLS 1.2 and HTTP/1.1 Upgrade - PayPal」。

Cloudflare Worker 進入 Open Beta 讓大家玩了...

去年 Cloudflare 宣佈了 Cloudflare Worker,讓使用者可以在 Edge 端跑 JavaScript (參考「Cloudflare 也能在各端點跑 JavaScript 了」),也就是可以在 Cloudflare 節點上面對 HTTP request 與 HTTP response 做更多事情,類似於 AWSLambda@Edge

不過去年公佈的當時需要申請才有機會用,算是 Private Beta。現在則是開放讓大家玩 (Open Beta) 讓大家幫忙測試了:「Cloudflare Workers is now on Open Beta」。

文件在「Cloudflare Workers Docs」這邊可以取得,就如同去年 Cloudflare 所提到的,程式的撰寫上是透過 Service Worker 的界面,這樣就不用再學一套:

Cloudflare Workers are modeled on the Service Workers available in modern web browsers, and use the same API whenever possible.

現階段 Cloudflare Worker 是免費的,看起來是用這段時間的用量與用法來看要怎麼設計收費機制:

Cloudflare Workers is completely free during the open beta. We do intend on charging for Workers, but we will notify you of our plans at least thirty days before any changes are made.

Telegram 推出新的 Client:Telegram X

Telegram 推出新的 client,叫做 Telegram X:「Telegram X: Progress through Competition」。

全新打造:

The Telegram X project features apps written from scratch, with an entirely new code base and without all the legacy components that our older apps have accumulated through the years.

然後包括 iOS 版本與 Android 版本都有對應的版本:

The goal of Telegram X is to reinvent Telegram and explore new frontiers in speed, ease of use, quality of animations and all other aspects. Today we are glad to present two new official apps – Telegram X for Android and iOS.

就裝起來玩看看,不知道實際的感覺會有什麼差異...

Twitter 推出 Full-archive search API

在先前的「Twitter 要推出 Premium API」這篇文章裡有提到 Twitter 打算在 Standard 與 Enterprise 兩個層級中間推出 Premium API,算是補產品線的概念,提供 Startup 有中間階段的服務可以使用。

而在昨天,Twitter 推出了 Full-archive search API:「Introducing the premium full-archive search endpoint」,從 Rate limit 就可以看出來對 Enterprise 不夠用,但對 Startup 應該有機會使用:

台灣用 Twitter 的量偏低,也許對專注在台灣的應用來說還好,但對國外的單位來說應該是多了不少變化可以玩...

Archives