Home » Computer » Software » Database » Archive by category "PostgreSQL"

Amazon RDS 宣佈支援 PostgreSQL 10

Amazon RDS 宣佈支援 PostgreSQL 10 了:「PostgreSQL 10 now Supported in Amazon RDS」。而且 AWS 這次推出的還包括了 10.1 的 patch:

As of version 10, PostgreSQL no longer uses three-part version numbers, and is shifting to two-part version numbers. This release includes all patches from the PostgreSQL 10.1 minor version.

10 的第一個版本是去年十月初 (在「PostgreSQL 10 Released」這邊可以看到),10.1 是去年十一月初 (在「PostgreSQL 10.1, 9.6.6, 9.5.10, 9.4.15, 9.3.20, and 9.2.24 released!」),現在二月底,所以延遲大約是三個多月的時間...

10.2 是二月初,不知道會多久...

用 Percona Monitoring and Management (PMM) 蒐集 PostgreSQL 的數據

難得在 Percona 的 blog 上看到專門談 PostgreSQL 的文章:「Collect PostgreSQL Metrics with Percona Monitoring and Management (PMM)」。

其實是透過 Prometheus 疊出來的:

Starting from PMM 1.4.0. it’s possible to add monitoring for any service supported by Prometheus.


3. In the next dialog, choose Prometheus as a data source and continue.


AWS Tokyo 也有 Amazon Aurora (PostgreSQL) 可以用了

剛剛翻到 AWS 宣佈 Amazon Aurora (PostgreSQL) 在東京開放使用了:「Amazon Aurora with PostgreSQL Compatibility is Available in the Asia Pacific (Tokyo) Region」。

The PostgreSQL-compatible edition of Amazon Aurora is now available in 10 regions. With the addition of the AWS Asia Pacific (Tokyo) region, you have a new option for database placement, availability, and scalability.

不過 Region Table 裡面還沒更新,亞洲區裡面的東京還沒勾起來,應該過幾天就會更新了...

EnterpriseDB 打算推出的 zheap,想要解 VACUUM 問題...

前天被問到「DO or UNDO - there is no VACUUM」這篇,回家後仔細看一看再翻了一些資料,看起來是要往 InnoDB 的解法靠...

PostgreSQL 與 InnoDB 都是透過 MVCC 的概念實做 transaction 之間的互動,但兩者實際的作法不太一樣。其中帶來一個明顯的差異就是 PostgreSQL 需要 VACUUM。這點在同一篇作者八年前 (2011) 的文章就有提過兩者的差異以及優缺點:「MySQL vs. PostgreSQL, Part 2: VACUUM vs. Purge」。

UPDATE 時,InnoDB 會把新資料寫到表格內,然後把可能會被 rollback 的舊資料放到表格外:

In InnoDB, only the most recent version of an updated row is retained in the table itself. Old versions of updated rows are moved to the rollback segment, while deleted row versions are left in place and marked for future cleanup. Thus, purge must get rid of any deleted rows from the table itself, and clear out any old versions of updated rows from the rollback segment.

而被 DELETE 清除的資料則是由 purge thread 處理:

All the information necessary to find the deleted records that might need to be purged is also written to the rollback segment, so it's quite easy to find the rows that need to be cleaned out; and the old versions of the updated records are all in the rollback segment itself, so those are easy to find, too.

所以可以在 InnoDB 看到 purge thread 相關的設定:「MySQL :: MySQL 5.7 Reference Manual :: 14.6.11 Configuring InnoDB Purge Scheduling」,負責處理這些東西。

而在 PostgreSQL 的作法則是反過來,舊的資料放在原來地方,新資料另外存:

PostgreSQL takes a completely different approach. There is no rollback tablespace, or anything similar. When a row is updated, the old version is left in place; the new version is simply written into the table along with it.


Lacking a centralized record of what must be purged, PostgreSQL's VACUUM has historically needed to scan the entire table to look for records that might require cleanup.

這也使得 PostgreSQL 裡需要 autovacuum 之類的程序去掃,或是手動跑 vacuum。而在去年 (2017) 的文章裡也有提到目前還是類似的情況:「MVCC and VACUUM」。

而在今年 (2018) 的文章裡,EnterpriseDB 就提出了 zheap 的想法,在 UPDATE 時寫到 table 裡,把可能被 rollback 的資料放到 undo log 裡。其實就是把 InnoDB 那套方法拿過來用,只是整篇都沒提到而已 XD:

That brings me to the design which EnterpriseDB is proposing. We are working to build a new table storage format for PostgreSQL, which we’re calling zheap. In a zheap, whenever possible, we handle an UPDATE by moving the old row version to an undo log, and putting the new row version in the place previously occupied by the old one. If the transaction aborts, we retrieve the old row version from undo and put it back in the original location; if a concurrent transaction needs to see the old row version, it can find it in undo. Of course, this doesn’t work when the block is full and the row is getting wider, and there are some other problem cases as well, but it covers many useful cases. In the typical case, therefore, even bulk updates do not force a zheap to grow. Instead, the undo grows. When a transaction commits, all row versions that will become dead are in the undo, not the zheap.

不過馬上就會想到問題,如果要改善問題,不是個找地方記錄哪些位置要回收就好了嗎?順便改變方法是為了避免 fragment 嗎?


Amazon Aurora (PostgreSQL) 也支援 Read Replica 了

Amazon Aurora (PostgreSQL) 支援 Read Replica 了:「Announcing Amazon Aurora PostgreSQL Read Replica for Amazon RDS for PostgreSQL」。

馬上想到的用途是量爆增時,如果當初有作 R/W split (讀寫分離) 就可以直接用錢撐住,不過官方給的範例是降低 RDS 轉移到 Aurora 的 downtime,這點就有點微妙...:

You can now create an Amazon Aurora PostgreSQL read replica for an Amazon RDS for PostgreSQL instance, allowing you to continuously replicate to Amazon Aurora PostgreSQL. This helps you minimize downtime when migrating a live workload from Amazon RDS for PostgreSQL to Amazon Aurora PostgreSQL, by keeping the instances in sync until you're ready to move your applications and users to Amazon Aurora PostgreSQL.

所以這次算是陸陸續續把功能補上來,在 Amazon Aurora (MySQL) 有的一般性功能,這邊就跟著先實作...

Amazon RDS 支援更大的硬碟空間與更多的 IOPS

Amazon RDS 的升級:「Amazon RDS Now Supports Database Storage Size up to 16TB and Faster Scaling for MySQL, MariaDB, Oracle, and PostgreSQL Engines」。

空間上限從 6TB 變成 16TB,而且可以無痛升。另外 IOPS 上限從 30K 變成 40K:

Starting today, you can create Amazon RDS database instances for MySQL, MariaDB, Oracle, and PostgreSQL database engines with up to 16TB of storage. Existing database instances can also be scaled up to 16TB storage without any downtime.

The new storage limit is an increase from 6TB and is supported for Provisioned IOPS and General Purpose SSD storage types. You can also provision up to 40,000 IOPS for Provisioned IOPS storage volumes, an increase from 30,000 IOPS.

不過隔壁的 Amazon Aurora 還是大很多啊 (64TB),而且實際上不用管劃多大,他會自己長大:

Q: What are the minimum and maximum storage limits of an Amazon Aurora database?

The minimum storage is 10GB. Based on your database usage, your Amazon Aurora storage will automatically grow, up to 64 TB, in 10GB increments with no impact to database performance. There is no need to provision storage in advance.

Amazon Aurora (PostgreSQL 版本) 開放到其他區域了

上個月月底才宣佈 Amazon Aurora 推出 PostgreSQL 版本 (參考「Amazon Aurora 也支援 PostgreSQL 了」),不過當時只有少數的四個區域開放 (us-east-1us-east-2us-west-2eu-west-1),還不到一個月的時間就擴張了:「Amazon Aurora with PostgreSQL Compatibility Region Expansion」。

不過還是沒有東京與新加坡這兩個台灣比較常用的區域... 看起來都是比較新的 region,但首爾應該也是新的區域啊,不知道怎麼選擇的:

The PostgreSQL-compatible edition of Amazon Aurora is now available in four additional AWS regions: Canada (Central), EU (Frankfurt), Asia Pacific (Sydney), and Asia Pacific (Mumbai). This increases the number of available regions to eight, as the service launched on October 24, 2017 in US East (N. Virginia), US East (Ohio), US West (Oregon), and EU (Ireland).

Amazon Aurora 也支援 PostgreSQL 了

AWS 宣佈 Amazon Aurora 也支援 PostgreSQL 了,相容於 9.6.3 的版本 (應該就是改自這個版本):「Now Available – Amazon Aurora with PostgreSQL Compatibility」。


On the performance side, you can expect up to 3x the throughput that you’d get if you ran PostgreSQL on your own (you can read Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases to learn more about how we did this).

架構上也是採用 6-way replication 的方式:

It is compatible with PostgreSQL 9.6.3 and scales automatically to support up to 64 TB of storage, with 6-way replication behind the scenes to improve performance and availability.


You can use Amazon Aurora with PostgreSQL Compatibility today in the US East (Northern Virginia), EU (Ireland), US West (Oregon), and US East (Ohio) Regions, with others to follow as soon as possible.

Amazon RDS 的 RI 也支援彈性計價了

AWS 宣佈 RDS 的 RI 也支援彈性計價了:「Amazon RDS Reserved Instances Offer Instance Size Flexibility」。

也就是說跟 EC2 的 RI 方式切齊,買 db.m4.2xlarge 的 RI 後,可以用在兩台 db.m4.xlarge 上:

For example, let’s say you purchased a db.m4.2xlarge MySQL RI in US East (N. Virginia). The discounted rate of this RI can automatically apply to 2 db.m4.xlarge MySQL instances without you needing to do anything.

包括了相當多種類的 RDS,主要是沒有 license fee 的類型都包括在內了:

Amazon RDS Reserved Instance size flexibility is offered in all regions for the MySQL, MariaDB, PostgreSQL, and Amazon Aurora database engines, as well as the “bring your own license” (BYOL) edition of the Oracle database engine. To learn more about flexible RIs, please visit the Amazon RDS Reserved Instances Page.

但不包括要另外收 license fee 的 SQL Server 與 Oracle LI edition:

Size flexibility does not apply to Microsoft SQL Server and the License Included (LI) edition of Oracle.