用 zrepl over ZFS 每十分鐘做一次 incremental backup 的設計

前陣子在 Hacker News 上看到「I only lost 10 minutes of data, thanks to ZFS (mastodon.social)」這篇,講他的硬碟故障,但是靠著 zrepl 每十分鐘將本地的 ZFS filesystem 同步一次到 NAS 上,所以他只掉了十分鐘的資料的故事...

Hacker News 上最熱的討論居然是在討論 WDSanDisk 的 SSD disk issue,反倒不是這個想法或是 zrepl 這個工具...

看了一下這個方法還蠻有趣的,有需求的人好像是可以這樣搞沒錯...

Anyway,想當初 OpenZFS 剛出的時候,因為 license 是 CDDL 而被 FSF 認為無法與 GPLv2 相容,所以 Linux 這邊無法內建或是散佈 binary,想玩 ZFS 就得用 OpenSolaris 或是 porting 到 FreeBSD 的版本。

結果後來 Ubuntu 的法律顧問認為可以透過 kernel module (binary) 的方式散佈相容,在 Ubuntu 16.04 包進去後就開始盛行了...

而且當年記憶體 overhead (GB 等級) 要求對於 desktop 是個不能忽略的問題,現在回頭來看也不是大問題了,桌機與筆電常常都是 16GB+ 在跑...

OpenSSH 加入了 noise (keystroke timing obfuscation) 功能

Hacker News 上看到在 OpenSSH 裡加入 keystroke timing obfuscation 的功能:「Keystroke timing obfuscation added to ssh(1) (undeadly.org)」。

如同 commit log 裡面提到的,這個功能會想要故意沒事就送一些沒用的資料 (增加一些噪音),降低從 side channel 被判讀的資訊量:

This attempts to hide inter-keystroke timings by sending interactive traffic at fixed intervals (default: every 20ms) when there is only a small amount of data being sent. It also sends fake "chaff" keystrokes for a random interval after the last real keystroke. These are controlled by a new ssh_config ObscureKeystrokeTiming keyword/

基於 OpenSSH 算是 SSH 這塊的 de-factor standard 了,接下來看其他家像是 Dropbear 會不會也實作?

GitLab 想要支援 ActivityPub

看到「Support ActivityPub for merge requests」這則消息,這個 epic 的作者 Derek Ferguson 可以看到是 GitLab 家的「Group Manager, Product」,看起來是產品團隊的主管職 (不是很確定)。

這張 epic 想建立跨 GitLab 服務之間的 ecosystem:

There already has been several very popular discussions around this (see here, here and the epic here). The gist of it is: what people really want is to have one global "Gitlab network" to be able to interact between various projects without having to register on each of their hosts.

不過目前像是在討論階段?但既然是由內部提出來的,目前的討論看起來也還算... 正面?應該是有機會看到後續的更新...

Amazon SES 寄到 Gmail 受到阻擋的情況

我自己沒遇過,但是 Hacker News 上看到有人有遇到,所以記錄起來:「Tell HN: Gmail rate limiting emails from AWS SES」。

Amazon SES 預設是共用 IP pool,所以遇到這種情況不算太意外,但應該是暫時性的,不過發問的作者有提到後來的解法是花 US$25/mo 使用 Dedicated IP 解決 IP reputation 的問題 (在 id=37177533 這邊):

Thanks you all for comments. I have made a decision to subscribed to dedicated IPs (credits: @slau).

The differentiating factor between our current AWS SES plan and the competitors (mentioned in the comments) is having a dedicated IP. With our current volume, none of the competitors are anyway near AWS SES costs. So, moving to a dedicated IPs thats cost 25$ extra not only solves our issue, but also no change in code/infrastructure.

記得以前另外一個教訓是,寄信還是儘量用 IPv4 address 去寄,因為 IPv6 address 的 reputation 得養頗久... 不過這個也是很久前的事情了。

AWS 弄出了 AWS Dedicated Local Zones,很像 AWS Outposts...

AWS 推出了 AWS Dedicated Local Zones:「Announcing AWS Dedicated Local Zones」。

先講 AWS Outposts,他就是提供 AWS 自己的硬體,放到用戶的機房裡面,所以依照需求有不同大小的機器,甚至是整個機櫃:

AWS Outposts is a family of fully managed solutions delivering AWS infrastructure and services to virtually any on-premises or edge location for a truly consistent hybrid experience. Outposts solutions allow you to extend and run native AWS services on premises, and is available in a variety of form factors, from 1U and 2U Outposts servers to 42U Outposts racks, and multiple rack deployments.

在「What is AWS Outposts?」這邊有詳細列出有哪些服務可以跑在上面,可以看到主要就是基礎服務,以及一些吃 local 特性的服務。

另外在「How AWS Outposts works」這邊可以看出架構上會在同一個 VPC 裡面,但是不屬於同一個 AZ 下:

而這次推出的 Dedicated Local Zones 還是有些地方沒看懂跟 AWS Outposts 差在哪裡,看起來很像是重新包裝而已...

首先是首頁提到的,這邊有提到 AWS Nitro System,所以猜測這是 AWS 的硬體,而不是自己的硬體:

Build with AWS managed secure cloud infrastructure

Benefit from the same AWS security standards that apply to AWS Regions and AWS Local Zones and are delivered with the security of the AWS Nitro System to help ensure confidentiality and integrity of customer data.

另外在公告裡面提到的服務,跟 Outposts 有些差異:

AWS services, such as Amazon EC2, Amazon Virtual Private Cloud (Amazon VPC), Amazon Elastic Block Store (Amazon EBS), Elastic Load Balancing (ELB), Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS), and AWS Direct Connect are available in Dedicated Local Zones.

另外在「AWS Dedicated Local Zones FAQs」這邊則試著說明兩者差異,但就這些句子看起來,只是不同面向的東西:

Q: How are AWS Dedicated Local Zones different from AWS Outposts?

AWS Outposts is designed for workloads that need to remain on-premises due to latency requirements, where customers want those workloads to run seamlessly with their other workloads in AWS. AWS Outposts racks are fully managed and configurable compute and storage racks built with AWS-designed hardware that allow customers to run compute and storage on-premises, while seamlessly connecting to AWS’s broad array of services in the cloud.

AWS Dedicated Local Zones are designed to eliminate the operational overhead of managing on-premises infrastructure at scale. Some customers have long-term, complex cloud migration projects and need infrastructure that seamlessly scales to support their large-scale demand. Some of these customers represent the interests of a customer community and also need multi-tenancy features to efficiently coordinate across their stakeholders. Dedicated Local Zones enable these customers to reduce the administrative burden of managing their own infrastructure on-premises with scalable, resilient, and multitenant cloud infrastructure that is fully AWS-managed and built exclusively for their use.

另外回到首頁看使用單位,目前是 GovTech Singapore,看起來就是重新包裝?

另外一個猜測是在客戶的機器上面裝 AWS Nitro System,然後裝 AWS 的軟體?這就有點怪了,而且這樣相容性之類的問題也頗麻煩,也許要指定配合的機種?

等有機會遇到的時候再跟 AWS 的人問問看好了,目前也還用不到...

Backblaze 宣佈漲價

Backblaze 宣佈漲價:「Backblaze Product and Pricing Updates」。

其中 B2 Cloud Storage 這邊最主要的改變在 Storage 的部分,這次漲了 20%,從 $5/TB 變成 $6/TB:

Storage Price: Effective October 3, 2023, we are increasing the monthly pay-as-you-go storage rate from $5/TB to $6/TB. The price of B2 Reserve will not change.

頻寬的部分增加了一些 free quota,不過在意頻寬成本的人都會用 Cloudflare 之類的方式避開了,這個其實沒有什麼差... (因為 Backblaze 流出到 Cloudflare 的流量是不計費的)

Backblaze Computer Backup 的部分沒有什麼在碰,但看起來最主要的改變是從現有的 $7/mo 漲到 $9/mo,大約 28.57%:

Computer Backup Pricing: Effective October 3, new purchases and renewals will be $9/month, $99/year, and $189 for two-year subscription plans, and Forever Version History pricing will be $0.006/GB/month.

漲幅其實頗高的,但漲完後還是市場上比較低價的產品...

Tor 的 Onion 導入防禦機制,在遭受 DoS 的時候要求用戶端執行 PoW 任務

在「Introducing Proof-of-Work Defense for Onion Services」這邊看到 0.4.8 的新機制,當 Onion 服務受到 DoS 時,會需要 client 提供 PoW 證明,有證明的會優先處理:

Tor's PoW defense is a dynamic and reactive mechanism, remaining dormant under normal use conditions to ensure a seamless user experience, but when an onion service is under stress, the mechanism will prompt incoming client connections to perform a number of successively more complex operations. The onion service will then prioritize these connections based on the effort level demonstrated by the client.

主要原因是傳統遇到 DoS 時可以透過 IP address 之類的資訊設計阻擋機制,但在 Onion 服務裡面沒有這個資訊,所以需要其他方式阻擋:

The inherent design of onion services, which prioritizes user privacy by obfuscating IP addresses, has made it vulnerable to DoS attacks and traditional IP-based rate limits have been imperfect protections in these scenarios. In need of alternative solutions, we devised a proof-of-work mechanism involving a client puzzle to thwart DoS attacks without compromising user privacy.

這個 PoW 機制的說明可以在「torspec/proposals/327-pow-over-intro.txt」這邊看到,看起來是三年前 (2020/04/02) 就提出來了,直到 0.4.8 才推出。

裡面有提到 PoW 的演算法是用 Equi-X

For our proof-of-work function we will use the Equi-X scheme by tevador [REF_EQUIX].

看起來是個方法,而且從 cryptocurrency 後大家對 PoW 的用法愈來愈熟悉了,在這邊用還不錯...

SSH 的各種好用的功能

在「An Excruciatingly Detailed Guide To SSH (But Only The Things I Actually Find Useful)」這篇看到在介紹 SSH 的各種好用的功能。

其中當然會提到 tunnel,這部份裡面提到了一張圖,原始文章是「A Visual Guide to SSH Tunnels: Local and Remote Port Forwarding」這邊,圖解 SSH tunnel 的功能 (不過只有 -L-R 的):

回到原來這篇文章,這篇講的東西比較多一點,關於 tunnel 相關的還包括了 -D-J

另外提到了 -A-t-g~? 的用法,以及其他各種跟 SSH 有關的工具。

翻了一輪後應該就 -g 還不熟,另外發現 -J (ProxyJump) 居然可以用逗號 , 指定一串跳板機,一路跳進去... 翻了 manpage 發現有寫:

Multiple jump hops may be specified separated by comma characters.

OpenTF 宣佈從 Terraform 最後一個 Open Source 版本 fork 出來

先前在「HashiCorp 將放棄 Open Source License,改採用 BSL 1.1」這邊提到的,HashiCorp 決定將所有產品線從現有的 open source license 換成非開源的 BSL 1.1 後,OpenTF 先丟出了「呼籲」希望 HashiCorp 可以撤回這個決定:「The OpenTF Manifesto」。

想當然的,HashiCorp 沒有回應,所以 OpenTF 宣佈了要把 Terraform 的最後一個 open source 版本 fork 出來:「OpenTF Announces Fork of Terraform」。

有幾個比較重要的資訊,第一個是申請 Linux Foundation 資格,希望成為 CNCF 的一環:

We completed all documents required for OpenTF to become part of the Linux Foundation with the end goal of having OpenTF as part of Cloud Native Computing Foundation.

另外一個是首頁上的 Co-signed 的部分,翻了一下有三家公司 (Spacelift、env0、Scalr) 有提出支援五年五位的 Full time engineer 的經費 (Cover the cost of 5 FTEs for at least 5 years),另外一家公司 (Sailorcloud) 則是提出支援兩年一位的經費 (Cover the cost of 1 FTE for at least 2 years)。

接下來就是看這些能量到底有多少效果了...

llama.cpp 官方支援 Falcon

先前有提過採用 Apache License 2.0Falcon 40B,少數能跟 LLaMA (第一代) 打對台的版本,而且是真正的 open source license:「Falcon 40B 超越 LLaMA 65B 成為目前 Open LLM 的領頭」,當時有提到 llama.cpp 還沒有支援。

過了一陣子,社群自己先 fork 了一版,想辦法支援 Falcon 40B:「cmp-nct/ggllm.cpp」,但這也導致沒有跟到很多 llama.cpp 的新功能 (尤其是各種透過硬體加速的支援)。

剛剛刷了一下,發現前幾天 llama.cpp 官方支援 Falcon 的 model 了:「llm : add Falcon support」。

看起來是個開始,可以看到還有列出一些項目要實作的,但看起來可以跑了。