Home » Posts tagged "rdbms" (Page 3)

Amazon Aurora (MySQL) 提供 Parallel Query 讓人申請使用

AWS 宣佈了 Amazon Aurora (MySQL) 支援 Parallel Query:「Amazon Aurora Parallel Query is Available for Preview」。

這邊提到的 Parallel Query 比較像是 Amazon Athena,直接把單一 Query 打散到多台機器上跑:

Amazon Aurora Parallel Query improves the performance of large analytic queries by pushing processing down to the Aurora storage layer, spreading processing across hundreds of nodes.

也就是說,這算是單一 SQL Query 平行運算的進階版本。

在這之前,AWS 都已經支援單一 Query 在單台機器上利用多 CPU 平行運算。其中 PostgreSQL 是 9.6+ 本身就有支援。Amazon Aurora (MySQL) 則是在 2016 時透過 Parallel Read Ahead 支援某些情境下的的單一 Query 多 CPU 運算了 (發現之前沒寫到...):「Amazon Aurora Update – Parallel Read Ahead, Faster Indexing, NUMA Awareness」。

這個功能目前是 Preview 階段,然後開在這些地區讓大家測試使用:

The preview is available for the MySQL-compatible edition of Amazon Aurora, and is currently available in the US East (N. Virginia), US East (Ohio), US West (Oregon), and Europe (Ireland) Regions. Sign up to get access.


EnterpriseDB 打算推出的 zheap,想要解 VACUUM 問題...

前天被問到「DO or UNDO - there is no VACUUM」這篇,回家後仔細看一看再翻了一些資料,看起來是要往 InnoDB 的解法靠...

PostgreSQL 與 InnoDB 都是透過 MVCC 的概念實做 transaction 之間的互動,但兩者實際的作法不太一樣。其中帶來一個明顯的差異就是 PostgreSQL 需要 VACUUM。這點在同一篇作者八年前 (2011) 的文章就有提過兩者的差異以及優缺點:「MySQL vs. PostgreSQL, Part 2: VACUUM vs. Purge」。

UPDATE 時,InnoDB 會把新資料寫到表格內,然後把可能會被 rollback 的舊資料放到表格外:

In InnoDB, only the most recent version of an updated row is retained in the table itself. Old versions of updated rows are moved to the rollback segment, while deleted row versions are left in place and marked for future cleanup. Thus, purge must get rid of any deleted rows from the table itself, and clear out any old versions of updated rows from the rollback segment.

而被 DELETE 清除的資料則是由 purge thread 處理:

All the information necessary to find the deleted records that might need to be purged is also written to the rollback segment, so it's quite easy to find the rows that need to be cleaned out; and the old versions of the updated records are all in the rollback segment itself, so those are easy to find, too.

所以可以在 InnoDB 看到 purge thread 相關的設定:「MySQL :: MySQL 5.7 Reference Manual :: 14.6.11 Configuring InnoDB Purge Scheduling」,負責處理這些東西。

而在 PostgreSQL 的作法則是反過來,舊的資料放在原來地方,新資料另外存:

PostgreSQL takes a completely different approach. There is no rollback tablespace, or anything similar. When a row is updated, the old version is left in place; the new version is simply written into the table along with it.


Lacking a centralized record of what must be purged, PostgreSQL's VACUUM has historically needed to scan the entire table to look for records that might require cleanup.

這也使得 PostgreSQL 裡需要 autovacuum 之類的程序去掃,或是手動跑 vacuum。而在去年 (2017) 的文章裡也有提到目前還是類似的情況:「MVCC and VACUUM」。

而在今年 (2018) 的文章裡,EnterpriseDB 就提出了 zheap 的想法,在 UPDATE 時寫到 table 裡,把可能被 rollback 的資料放到 undo log 裡。其實就是把 InnoDB 那套方法拿過來用,只是整篇都沒提到而已 XD:

That brings me to the design which EnterpriseDB is proposing. We are working to build a new table storage format for PostgreSQL, which we’re calling zheap. In a zheap, whenever possible, we handle an UPDATE by moving the old row version to an undo log, and putting the new row version in the place previously occupied by the old one. If the transaction aborts, we retrieve the old row version from undo and put it back in the original location; if a concurrent transaction needs to see the old row version, it can find it in undo. Of course, this doesn’t work when the block is full and the row is getting wider, and there are some other problem cases as well, but it covers many useful cases. In the typical case, therefore, even bulk updates do not force a zheap to grow. Instead, the undo grows. When a transaction commits, all row versions that will become dead are in the undo, not the zheap.

不過馬上就會想到問題,如果要改善問題,不是個找地方記錄哪些位置要回收就好了嗎?順便改變方法是為了避免 fragment 嗎?


Percona 版本的 MySQL 對於 Meltdown/Spectre 漏洞修復造成的效能損失 (Intel 平台)


PerconaUbuntu 16.04 上測試 MeltdownSpectre 這兩個安全漏洞的修正對於效能的影響。在原文標題就講了結論,為了修正 Meltdown 與 Spectre 兩個安全漏洞,效能的損失很明顯:「20-30% Performance Hit from the Spectre Bug Fix on Ubuntu」。

這邊測的結果發現,在 CPU bound 時的損失大約是 20%~25% (甚至到 30%),而 I/O bound 會輕一些,大約是 15%~20%:

We can see that in CPU-bound workloads the overhead is 20-25%, reaching up to 30% in point select queries. In IO-bound (25G buffer pool) workloads, the observed overhead is 15-20%.

在 comment 的地方 Percona 的人被問到 AMD 平台上效能會損失多少的問題,但因為他們手上目前沒有 AMD 平台的新機器所以不知道會有多少:

I do not have modern AMD servers on my hands right now

理論上 AMD 平台不需要處理 Meltdown 問題,損失應該會少一些,但沒測過也不曉得會是什麼情況... (像是 Spectre 的修正損失會不會比 Intel 還重,這之類的...)

另外補上早些時候的文章,當時 Ubuntu 上的 kernel 只有對 Meltdown 攻擊的修正,當時 Percona 的人也測了一次:「Does the Meltdown Fix Affect Performance for MySQL on Bare Metal?」,看起來對 Meltdown 攻擊的修正對效能的影響不太大,不過文裡有測試到 syscall 的效率的確如同預期掉很多。

Amazon Aurora (PostgreSQL) 也支援 Read Replica 了

Amazon Aurora (PostgreSQL) 支援 Read Replica 了:「Announcing Amazon Aurora PostgreSQL Read Replica for Amazon RDS for PostgreSQL」。

馬上想到的用途是量爆增時,如果當初有作 R/W split (讀寫分離) 就可以直接用錢撐住,不過官方給的範例是降低 RDS 轉移到 Aurora 的 downtime,這點就有點微妙...:

You can now create an Amazon Aurora PostgreSQL read replica for an Amazon RDS for PostgreSQL instance, allowing you to continuously replicate to Amazon Aurora PostgreSQL. This helps you minimize downtime when migrating a live workload from Amazon RDS for PostgreSQL to Amazon Aurora PostgreSQL, by keeping the instances in sync until you're ready to move your applications and users to Amazon Aurora PostgreSQL.

所以這次算是陸陸續續把功能補上來,在 Amazon Aurora (MySQL) 有的一般性功能,這邊就跟著先實作...

Amazon Aurora (MySQL) 的 Stored Procedure 可以跑 AWS Lambda...

查了資料才發現去年十月 Amazon Aurora (MySQL-Compatible Edition) 就支援用 AWS Lambda 當 stored procedure 了,只是當時只支援 async mode,能做的事情比較有限:「Amazon Aurora New Features: AWS Lambda Integration and Data Load from Amazon S3 to Aurora Tables」。

Now you can invoke Lambda functions directly from within an Aurora database via stored procedures or user-defined functions. Lambda integration allows you to extend the capabilities of the database and invoke external applications to act upon data changes. For example, you can create a Lambda function that sends emails to customers whenever their address in the database is updated.

前幾天發表的則是支援 sync mode,可以等到:「Amazon Aurora with MySQL Compatibility Natively Supports Synchronous Invocation of AWS Lambda Functions」。

Starting with version 1.16, we are extending this feature to be able to able to synchronously invoke Lambda functions.

Use the native function lambda_sync when you must know the result of the execution before moving on to another action.

這解掉了 MySQL 的 stored procedure 一直很殘的問題...

Amazon Aurora 的 MySQL-Compatible Edition 開始提供相容於 MySQL 5.7 的服務

Amazon AuroraMySQL-Compatible Edition 開始支援相容 MySQL 5.7 的服務:「Announcing Preview of Amazon Aurora with MySQL 5.7 Compatibility」。

Amazon Aurora with MySQL 5.7 compatibility offers enhancements such as JSON support, spatial indexes, generated columns and performance improvement of 5x over MySQL 5.7, and up to 10x for spatial datasets.

雖然目前還在 Preview (需要申請才能用),而且開放區域有限,但總算是有消息了:

The preview is currently available in US East (N. Virginia), US East (Ohio), US West (Oregon), and Canada (Montreal).

會一定要 5.7 的,主要的需求應該是在 spatial index 上吧... JSON 反倒還好。

Percona 分析在 AWS 上跑 Percona XtraDB Cluster 的效能 (I/O bound)

Percona 的人分析了在 Amazon EC2 上跑 Percona XtraDB Cluster (PXC) 效能 (I/O bound):「Best Practices for Percona XtraDB Cluster on AWS」。


直接跳到結論的地方。如果資料可以掉,用 i3 本地 storage 的效能是最好的,如果要資料不能掉,用 EBS 的 Provisioned IOPS SSD (io1) 的效能會比 General Purpose (gp2) 好很多。

另外 instance type 的選擇上,避免用 {i3,r4}.large,因為測試出來發現 {i3,r4}.xlarge 的效能好不只一倍。

不過 Aurora 的 Multi-master 已經在 Preview 了啊,如果 Percona 的人拿到帳號的話,應該會有單位成本的效能比較可以看...

Amazon Aurora 的 Serverless 與 Multi-master

Amazon Aurora 推出了兩包玩意,第一包是 Serverless,讓需要人介入的情況更少:「In The Works – Amazon Aurora Serverless」。

在 Serverless 的第一個重點是支援以秒計費:

Today we are launching a preview (sign up now) of Amazon Aurora Serverless. Designed for workloads that are highly variable and subject to rapid change, this new configuration allows you to pay for the database resources you use, on a second-by-second basis.

然後是極為快速的 auto-scaling:

The endpoint is a simple proxy that routes your queries to a rapidly scaled fleet of database resources. This allows your connections to remain intact even as scaling operations take place behind the scenes. Scaling is rapid, with new resources coming online within 5 seconds

這兩個組合起來,讓使用端可以除了在 Amazon EC2 上可以快速 scale 外,後端的資料庫也能 scale 了...

第二個是 Multi-master 架構:「Sign Up for the Preview of Amazon Aurora Multi-Master」。

Amazon Aurora Multi-Master allows you to create multiple read/write master instances across multiple Availability Zones. This enables applications to read and write data to multiple database instances in a cluster, just as you can read across Read Replicas today.

(話說我一直都誤以為 Aurora 是 R/W master...)

Anyway,這個功能不知道怎麼疊上去的... 不笑得會不會有嚴重的 distributed lock issue,反而推薦大家平常都寫到同一台 (像是 PXC 就會這樣)。

AWS 推出可以在 Red Hat Enterprise Linux 上跑 Microsoft SQL Server 的 AMI

自從 Microsoft SQL Server 宣佈可以在 Linux 上跑後 (參考「Microsoft SQL Server 出 Linux 版...」),就沒看到什麼 Linux 上跑 SQL Server 的消息了... 結果在這波 AWS 的活動上推出了 RHEL 上跑 SQL Server 的消息:「Amazon EC2 now offers SQL Server 2017 with Red Hat Enterprise Linux 7.4」。

SQL Server 2017 is now available for Amazon EC2 instances running Red Hat Enterprise Linux (RHEL) 7.4 as an Amazon Machine Image (AMI) from the AWS Marketplace. With this release, you can now launch RHEL instances on-demand using SQL Server 2017 Enterprise License Included AMIs without having to bring your own license. SQL Server 2017 on RHEL 7.4 AMI is available in all public AWS regions starting today.


Amazon Aurora (PostgreSQL 版本) 開放到其他區域了

上個月月底才宣佈 Amazon Aurora 推出 PostgreSQL 版本 (參考「Amazon Aurora 也支援 PostgreSQL 了」),不過當時只有少數的四個區域開放 (us-east-1us-east-2us-west-2eu-west-1),還不到一個月的時間就擴張了:「Amazon Aurora with PostgreSQL Compatibility Region Expansion」。

不過還是沒有東京與新加坡這兩個台灣比較常用的區域... 看起來都是比較新的 region,但首爾應該也是新的區域啊,不知道怎麼選擇的:

The PostgreSQL-compatible edition of Amazon Aurora is now available in four additional AWS regions: Canada (Central), EU (Frankfurt), Asia Pacific (Sydney), and Asia Pacific (Mumbai). This increases the number of available regions to eight, as the service launched on October 24, 2017 in US East (N. Virginia), US East (Ohio), US West (Oregon), and EU (Ireland).