OpenRsync 專案

看到「(open)rsync gains include/exclude support」這篇才注意到有 OpenRsync 專案...

在 OpenRsync 的網站上是指到 OpenBSD 的 cvsweb 上:「src/usr.bin/rsync/」,不過在 GitHub 上也有一個 repository:「kristapsdz/openrsync」,裡面有提到目前應該是以 OpenBSD 內的 source code 為主:

This system has been merged into OpenBSD base. If you'd like to contribute to openrsync, please mail your patches to tech@openbsd.org. This repository is simply the OpenBSD version plus some glue for portability.

然後有也提到 OpenRsync 主要就是 license 的關係 (rsync 目前是 GPLv3):

This is an implementation of rsync with a BSD (ISC) license. It's compatible with a modern rsync (3.1.3 is used for testing, but any supporting protocol 27 will do), but accepts only a subset of rsync's command-line arguments.

不過在一開始的報導裡面,有人反應軟體與 rsync 的相容性不太好,會搞爆 rsync:

By grey (grey) on 2021-08-31 05:17

Nice!

Albeit, the last time I was testing openrsync, I discovered I could use openrsync to reproducibly crash rsync on FreeBSD13-CURRENT on a Raspberry Pi 3 and decided rather than try to debug rsync, I would wait for openrsync to mature a bit, I'm grateful to see that it continues to progress!

就當作個記錄...

檔案壓縮順序造成壓縮率的差異

Hacker News Daily 上看到「Why are tar.xz files 15x smaller when using Python's tar library compared to macOS tar?」這篇,作者問了為什麼他用 Pythontarfile 壓出來比起用 tar 壓出來小了 15 倍,檔案都是 JSON 檔壓成 XZ 格式:

I'm compressing ~1.3 GB folders each filled with 1440 JSON files and find that there's a 15-fold difference between using the tar command on macOS or Raspbian 10 (Buster) and using Python's built-in tarfile library.

看到 1440 個檔案應該會有直覺是一分鐘一個檔案,跑一天的量...

隔天他把原因找出來了,在裝了 GNU Tar 並且加上 --sort='name' 參數後,壓出來的大小就跟 Python 的 tarfile 差不多了:

Ok, I think I found the issue: BSD tar and GNU tar without any sort options put the files in the archive in an undefined order.

After installing GNU tar on my Mac with:

brew install gnu-tar

And then tarring the same folder, but with the --sort option:

gtar --sort='name' -cJf zsh-archive-sorted.tar.xz /Users/user/Desktop/temp/tar/2021-03-11

I get a .tar.xz archive of 1.5 MB, equal to the archive created by the Python library.

底層的原因是檔名與檔案內容有正相關的相似度 (因為裡面都是 sensor 資料),依照檔名排序壓縮就等於把類似的 JSON 檔案放在一起壓,使得 xz 可以利用這點急遽拉高壓縮率:

My JSON files contain measurements from hundreds of sensors. Every minute I read out all sensors, but only a few of these sensors have a different value from minute to minute.

By sorting the files by name (which has the creation unixtime at the beginning of it), two subsequent files have very little different characters between them. Apparently this is very favourable for the compression efficiency.

遇到類似的情境可以當作 tuning 的一種,測試看看會不會變小很多...

WireGuard 的 OpenBSD porting

在「WireGuard patchset for OpenBSD」這邊看到有人試著把 WireGuard 放入 OpenBSD 的消息。

整包 patchset 包括了 kernel 與 userland 的實做,可以在 mailing list 上「WireGuard patchset for OpenBSD」這邊可以看到,整串討論可以在「'WireGuard patchset for OpenBSD' thread - MARC」這邊看到,目前看起來還在 code review 的階段,有看到討論提到應該用 OpenBSD 內已經實做的 Chacha20-Poly1305,所以可能還會需要一些時間...

看起來慢慢的在滲進每個作業系統中,蠻有希望在幾年後成為業界標準...

勸人從 Linux 跳到 FreeBSD 的戰文...

前幾天在 Facebook 上看到正妹朋友的貼文,然後在 Hacker News Daily 上也看到了:「Technical reasons to choose FreeBSD over GNU/Linux」。

標準的戰文 XDDD

在這篇講的是 FreeBSD 在技術上的優勢 (雖然我覺得很多不是優勢...),作者在一月的時候有另外兩篇在講 BSD (不只是 FreeBSD) 與 Linux,主題偏政治面 (包括了社群的運作方式) 與 license 之類的問題,也是戰文:「Why you should migrate everything from Linux to BSD」與「Why you should migrate everything from Linux to BSD - part 2」。

不過畢竟就是戰文,拿爆米花來吃就好,還是用的順手比較重要。當初 FreeBSD 的 issue tracking system 換掉以後,覺得太卡就抽手了,不然還蠻愛 Ports 的架構的...

AWS 提供程式,可以掃出有哪些 RDS 的憑證需要更新

先前在「AWS 的 CA 更新 (CA-2019),將會影響與資料庫相關的服務」這邊提到了 AWSRDS 因為 Root Certificate 快要過期,需要更新到 CA-2019。

剛剛在 Twitter 上看到 Jeff Barr 提到了一個可以直接列出有哪些機器需要更新的工具:

程式放在 GitHub 上的 aws-samples/rds-ssl-update,另外看到一個有趣的授權條款,叫做 MIT-0 License,是 AWS 自己改的一個授權版本,拿掉了 MIT License 裡面的這段:

subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

也就是要求要引用授權條款的部份被拿掉,不過我還是比較喜歡 BSD license,尤其是 3-clause 的版本...

Ken Thompson 的密碼

剛剛看到這串還蠻歡樂的...

起因於 BSD 3 的程式碼裡面有個 /etc/passwd,而且是帶有 crypt 的版本:「unix-history-repo/etc/passwd」。

裡面有蠻多密碼都已經被解出來了,但還是有些還沒解出來... 而最近的消息是 ken (Ken Thompson) 的密碼被解了出來:「Ken Thompson's Unix password」。

From: Nigel Williams <nw@retrocomputingtasmania.com>
Cc: TUHS main list <tuhs@minnie.tuhs.org>
Subject: Re: [TUHS] Recovered /etc/passwd files
Date: Wed, 9 Oct 2019 16:49:48 +1100

ken is done:

ZghOT0eRm4U9s:p/q2-q4!

took 4+ days on an AMD Radeon Vega64 running hashcat at about 930MH/s
during that time (those familiar know the hash-rate fluctuates and
slows down towards the end).

另外解出來的人也發現了這組密碼是一組西洋棋的 Descriptive notation,跟 Ken Thompson 的背景也相符:

From: Nigel Williams <nw@retrocomputingtasmania.com>
Cc: TUHS main list <tuhs@minnie.tuhs.org>
Subject: Re: [TUHS] Recovered /etc/passwd files
Date: Wed, 9 Oct 2019 16:52:00 +1100

On Wed, Oct 9, 2019 at 4:49 PM Nigel Williams
<nw@retrocomputingtasmania.com> wrote:
> ZghOT0eRm4U9s:p/q2-q4!

BTW, is that a chess move?

不過我覺得最好玩的是這個,不確定是不是本尊就是了:

From: Ken Thompson via TUHS <tuhs@minnie.tuhs.org>
To: Andy Kosela <akosela@andykosela.com>
Cc: TUHS main list <tuhs@minnie.tuhs.org>
Subject: Re: [TUHS] Recovered /etc/passwd files
Date: Wed, 9 Oct 2019 01:53:25 -0700

congrats.

On Wed, Oct 9, 2019 at 1:16 AM Andy Kosela <akosela@andykosela.com> wrote:
>
> On 10/9/19, Warner Losh <imp@bsdimp.com> wrote:
> > On Tue, Oct 8, 2019, 11:52 PM Nigel Williams
> > <nw@retrocomputingtasmania.com>
> > wrote:
> >
> >> On Wed, Oct 9, 2019 at 4:49 PM Nigel Williams
> >> <nw@retrocomputingtasmania.com> wrote:
> >> > ZghOT0eRm4U9s:p/q2-q4!
> >>
> >> BTW, is that a chess move?
> >>
> >
> > Most common opening.
> >
>
> Descriptive chess notation is not as popular today as it was back in
> the 70s, but it actually makes perfect sense as Ken is a long time
> chess enthusiast.
>
> --Andy

還有 Rob Pike 對這件事情不怎麼贊同的看法:

From: Rob Pike <robpike@gmail.com>
To: Nigel Williams <nw@retrocomputingtasmania.com>
Cc: TUHS main list <tuhs@minnie.tuhs.org>
Subject: Re: [TUHS] Recovered /etc/passwd files
Date: Wed, 9 Oct 2019 09:59:43 -1000

I coulda told you that. One tends to learn passwords (inadvertently) when
they're short and typed nearby often enough. (Sorry, ken.)

If I remember right, the first half of this password was on a t-shirt
commemorating Belle's first half-move, although its notation may have been
different.

Interesting though it is, though, I find this hacking distasteful. It was
distasteful back when, and it still is. The attitudes around hackery have
changed; the position nowadays seems to be that the bad guys are doing it
so the good guys should be rewarded for doing it first. That's disingenuous
at best, and dangerous at worst.

-rob


On Tue, Oct 8, 2019 at 7:50 PM Nigel Williams <nw@retrocomputingtasmania.com>
wrote:

> ken is done:
>
> ZghOT0eRm4U9s:p/q2-q4!
>
> took 4+ days on an AMD Radeon Vega64 running hashcat at about 930MH/s
> during that time (those familiar know the hash-rate fluctuates and
> slows down towards the end).
>

意外的引誘到一群人跑出來...

openrsync

在「openrsync imported into the tree」這邊看到 openrsync 專案進入到 OpenBSD 的 source tree 內。

rsync 是使用 GPLv3 授權,這個授權剛出來的時候,幾個比較大的 BSD 的團隊都有找律師研究過,最後都是做出不要把 GPLv3 的軟體放進 source tree 的建議,但 rsync 算是很好用的工具 (尤其是在效率上)。

看起來 openrsync 這個專案主要的目的就是重新實做出 ISC license 版本的 rsync:

This is an implementation of rsync with a BSD (ISC) license. It's compatible with a modern rsync (3.1.3 is used for testing, but any supporting protocol 27 will do), but accepts only a subset of rsync's command-line arguments.

然後目前只有設計在 OpenBSD 上跑,其他平台可能需要花些時間 porting 修正相容性:

At this time, openrsync runs only on OpenBSD. If you want to port to your system (e.g. Linux, FreeBSD), read the Portability section first.

GitHub 上的 Git repository 只是個 mirror,真正在管理程式碼的部份還是使用 CVS

This repository is a read-only mirror of a private CVS repository. I use it for issues and pull requests. Please do not make feature requests: I will simply close out the issue.

React 的專利授權議題

ASF (Apache Software Foundation) 全面禁止 Facebook 的 BSD+PATENTS 後 (「Apache Foundation 宣佈禁止使用 Facebook BSD+Patents 的軟體」),整件事情開始熱起來了...

簡單來說,Facebook 有意為之,而且不打算撤回這個有攻擊性的授權模式,參考「Explaining React's license」這邊官方的說明以及有人寫了一篇解讀:「If you’re a startup, you should not use React (reflecting on the BSD + patents license)」。

Facebook 內的意見其實也不一樣,像是 Yarn 之所以沒有 PATENTS 是因為爭取出來的:

接下來應該會有更多爭議討論了...

Apache Foundation 宣佈禁止使用 Facebook BSD+Patents 的軟體

在「RocksDB Integrations」這邊討論到 RocksDBFacebook 所使用的 Facebook BSD+Patents License。

不過因為 RocksDB 最近在換 license (從 Facebook BSD+Patents 換到 Apache License, Version 2.0),移除了 PATENTS 內的限制,需要看 PATENTS 的舊檔案可以在 PATENTS 這邊看到。

Chris Mattmann 正式發出決議禁用 Facebook BSD+Patents License。(參考最後)

另外也提到了 Facebook 是故意埋下這些限制:

Note also Roy's comment that he has discussed the matter with FB's counsel and the word is that the FB license is intentionally incompatible. It is hard to make the argument that it is compatible after hearing that. Pragmatically speaking, regardless of any semantic shaving being done, having a statement like that from the source of the license is very daunting. If they think it is incompatible, we need to not try to wheedle and convince ourselves it is not.

這個 license 之後應該會有更多挑戰...

Hi,

As some of you may know, recently the Facebook BSD+patents license has been
moved to Category X (https://www.apache.org/legal/resolved#category-x).
Please see LEGAL-303 [1] for a discussion of this. The license is also referred
to as the ROCKSDB license, even though Facebook BSD+patents is its more
industry standard name.

This has impacted some projects, to date based on LEGAL-303
and the detective work of Todd Lipcon:

Samza, Flink, Marmotta, Kafka and Bahir

(perhaps more)

Please take notice of the following policy:

o No new project, sub-project or codebase, which has not
  used Facebook BSD+patents licensed jars (or similar), are allowed to use
  them. In other words, if you haven't been using them, you
  aren't allowed to start. It is Cat-X.

o If you have been using it, and have done so in a *release*,
  you have a temporary exclusion from the Cat-X classification thru
  August 31, 2017. At that point in time, ANY and ALL usage
  of these Facebook BSD+patents licensed artifacts are DISALLOWED. You must
  either find a suitably licensed replacement, or do without.
  There will be NO exceptions.

o Any situation not covered by the above is an implicit
  DISALLOWAL of usage.

Also please note that in the 2nd situation (where a temporary
exclusion has been granted), you MUST ensure that NOTICE explicitly
notifies the end-user that a Facebook BSD+patents licensed artifact exists. They
may not be aware of it up to now, and that MUST be addressed.

If there are any questions, please ask on the legal-discuss@a.o
list.

Thanks.

Cheers,
Chris Mattmann
VP Legal Affairs

[1] https://issues.apache.org/jira/browse/LEGAL-303

OpenBSD 將 ACME Client (Let's Encrypt Client) 納入系統

看到 OpenBSD 直接把 ACME 協定的 client 放進系統內,而 ACME 也就是 Let's Encrypt 所使用的協定:「Let's Encrypt client imported into -current」:

CVSROOT:	/cvs
Module name:	src
Changes by:	florian@cvs.openbsd.org	2016/08/31 16:01:42

Added files:
	usr.sbin/acme-client: ChangeLog Makefile acctproc.c base64.c 
	                      certproc.c chngproc.c dbg.c dnsproc.c 
	                      extern.h fileproc.c http.c http.h jsmn.c 
	                      jsmn.h json.c keyproc.c letskencrypt.1 
	                      main.c netproc.c revokeproc.c rsa.c rsa.h 
	                      sandbox-pledge.c util-pledge.c util.c 

Log message:
Import Kristaps' letskencrypt and call it acme-client in tree.
OK to get it in deraadt@ (and probably beck@)

At least deraadt@, beck@ and otto@ are fine with the name and the
disagreements stopped.

用的是 acme-client,先前叫做 letskencrypt,以 C 開發的 ACME client。