Django 3.2 LTS

Django 3.2 LTS 出了:「Django 3.2 released」,對應的 release note 可以在「Django 3.2 release notes」這邊看到。

這個版本比較特別,一般版本提供大約 15 個月的支援,LTS 版本則提供 36 個月的支援,不過目前用起來的感覺還是鼓勵大家平常就要安排升級計畫,不然 3.2 升級到 4.2 鐵定是個超痛苦的過程。

昨天把 Django 3.1 的專案升級上 3.2 後,跑 test case 就遇到「Customizing type of auto-created primary keys」這邊描述的問題,目前先用 DEFAULT_AUTO_FIELD = 'django.db.models.AutoField' 解,其他的倒是沒什麼問題...

幾個我覺得有趣的,像是 JSONL 格式:

The new JSONL serializer allows using the JSON Lines format with dumpdata and loaddata. This can be useful for populating large databases because data is loaded line by line into memory, rather than being loaded all at once.

然後不支援 PostgreSQL 9.5 與 MySQL 5.6 了:

Upstream support for PostgreSQL 9.5 ends in February 2021. Django 3.2 supports PostgreSQL 9.6 and higher.

The end of upstream support for MySQL 5.6 is April 2021. Django 3.2 supports MySQL 5.7 and higher.

很久前寫 Django 1.x,然後就荒廢很久,最近是從 3.0 開始寫,有些東西熟一點以後覺得怪怪的,之後要找時間測一些東西,修正對 Django 的用法。

Eventbrite 的 MySQL 升級計畫

在 2021 年看到 EventbiteMySQL 升級計畫:「MySQL High Availability at Eventbrite」。

看起來是 2019 年年初的時候 MySQL 5.1 出問題,後續決定安排升級,在 2019 年年中把系統升級到 MySQL 5.7 (Percona Server 版本):

Our first major hurdle was to get current with our version of MySQL. In July, 2019 we completed the MySQL 5.1 to MySQL 5.7 (v5.7.19-17-log Percona Server to be precise) upgrade across all MySQL instances.

然後看起來是直接在 EC2 上跑,不過這邊提到的空間問題就不太確定了,是真的把 EBS 的空間上限用完嗎?比較常使用的 gp2gp3 上限都是 16TB,不確定是不是真的用到接近爆掉了:

Not only was support for MySQL 5.1 at End-of-Life (more than 5 years ago) but our MySQL 5.1 instances on EC2/AWS had limited storage and we were scheduled to run out of space at the end of July. Our backs were up against the wall and we had to deliver!

另外在升級到 5.7 的時候,順便把本來是 INT 的 primary key 都換成 BIGINT

As part of the cut-over to MySQL 5.7, we also took the opportunity to bake in a number of improvements. We converted all primary key columns from INT to BIGINT to prevent hitting MAX value.

然後系統因為舊版的 Django 沒辦法配合 MySQL 5.7,得升級到 Django 1.6 (要注意 Django 1 系列的最新版是 1.11,看起來光是升級到 1.6 勉強會動就升不上去了?):

In parallel with the MySQL 5.7 upgrade we also Upgraded Django to 1.6 due a behavioral change in MySQL 5.7 related to how transactions/commits were handled for SELECT statements. This behavior change was resulting in errors with older version of Python/Django running on MySQL 5.7

然後採用了 GitHub 家研發的 gh-ost 當作改變 schema 的工具:

In December 2019, the Eventbrite DBRE successfully implemented a table ALTER via gh-ost on one of our larger MySQL tables.

看起來主要的原因是有遇到 pt-online-schema-change 的限制 (在「GitHub 發展出來的 ALTER TABLE 方式」這邊有提到):

Eventbrite had traditionally used pt-online-schema-change (pt-osc) to ALTER MySQL tables in production. pt-osc uses MySQL triggers to move data from the original to the “duplicate” table which is a very expensive operation and can cause replication lag. Matter of fact, it had directly resulted in several outages in H1 of 2019 due to replication lag or breakage.

另外一個引入的技術是 Orchestrator,看起來是先跟 HAProxy 搭配,不過他們打算要再換到 ProxySQL

Next on the list was implementing improvements to MySQL high availability and automatic failover using Orchestrator. In February of 2020 we implemented a new HAProxy layer in front of all DB clusters and we released Orchestrator to production!

Orchestrator can successfully detect the primary failure and promote a new primary. The goal was to implement Orchestrator with HAProxy first and then eventually move to Orchestrator with ProxySQL.

然後最後題到了 Square 研發的 Shift,把 gh-ost 包裝起來變成有個 web UI 可以操作:

2021 還可以看到這類文章還蠻有趣的...

Django 3.1

看到「Django 3.1 Released」這邊的公告,比較完整的改變可以在「Django 3.1 release notes」這邊翻到。

這應該是第一次要把手上的專案跳 minor version 升級,看起來應該是還好,但天曉得會有什麼狀況... 看起來主要會是 sha1 要被換成 sha256 會有影響到。

另外看到這個:

SimpleTestCase now implements the debug() method to allow running a test without collecting the result and catching exceptions. This can be used to support running tests under a debugger.

看起來應該也蠻有用的,可以玩看看...

django 的 LTS

看到「Django 1.11 released」這篇才發現 django 也有設計 LTS 機制... 在「Django’s release process」這邊有介紹:

Certain feature releases will be designated as long-term support (LTS) releases. These releases will get security and data loss fixes applied for a guaranteed period of time, typically three years.

所以是提供三年的維護期...

Facebook 的「Augmented Traffic Control」模擬網路環境

Facebook 推出 Augmented Traffic Control,模擬網路環境:「Augmented Traffic Control: A tool to simulate network conditions」。

可以測這五種變數:

  • bandwidth
  • latency
  • packet loss
  • corrupted packets
  • packets ordering

Facebook 的成果是 Python + Django 寫的前端管理界面,實際運作還是透過 Linuxiptables

也有提供 Vagrant 的操作方式讓人「試用」,主要是讓人在本機上就可以用吧?