C++ 實作高頻交易程式的技巧

看到「C++ patterns for low-latency applications including high-frequency trading (arxiv.org)」這篇,原文是 2023 年九月上傳到 arXiv 的 paper:「C++ Design Patterns for Low-latency Applications Including High-frequency Trading」。

有點 LLM 的文字感,在 Hacker News 上有人有提到這點,另外有一些圖表的錯誤,像是這兩份資料可以發現對不上,label 有標錯:

主要還是看列出的方法在自家專案上嘗試,不能直接把他們的數據拿來參考,在自家專案還是得在自家專案上面 benchmark 才知道有多少效益。

不過裡面提到蠻多有趣的作法,很多都是「知道」但沒有放在心上的...

FTC 出手告 Adobe 的退租機制

在「FTC sues Adobe for hiding fees and inhibiting cancellations (ftc.gov)」這邊看到的,FTC 的稿子在這邊:「FTC Takes Action Against Adobe and Executives for Hiding Fees, Preventing Consumers from Easily Cancelling Software Subscriptions」。

FTC 的標題就講差不多了,然後第一段再更細節一點:

The Federal Trade Commission is taking action against software maker Adobe and two of its executives, Maninder Sawhney and David Wadhwani, for deceiving consumers by hiding the early termination fee for its most popular subscription plan and making it difficult for consumers to cancel their subscriptions.

後面有提到法源依據 Restore Online Shoppers' Confidence Act

The complaint charges that Adobe’s practices violate the Restore Online Shoppers’ Confidence Act.

然後 FTC 內是 3-0 通過,然後在加州北區聯邦地院打官司:

The Commission vote to refer the civil penalty complaint to the DOJ for filing was 3-0. The Department of Justice filed the complaint in the U.S. District Court for the Northern District of California.

這個也是值得期待的案子,會是 dark pattern 在法律上的攻防戰...

修改 booking.com 的 dark pattern

Hacker News Daily 上看到修改 booking.comdark pattern 的套件:「De-Stressing Booking.com (alexcharlton.co)」,原文連結到「De-stressing Booking.com」,這是 2019 的文章,在介紹他寫的套件。

裡面講的是像這樣的東西:

這個例子裡面是故意用有壓力的顏色 (這邊是紅色) 去推動使用者趕快下單,算是蠻經典的 dark pattern,作者有舉個 Airbnb 的類似例子,比較起來就好很多:

在 comment 也有人提到其他種類的 dark pattern,故意把一些飯店標成已經售出,製造你不趕快訂就會釘不到的假象。不過下面有人提到,在有些法律制度比較完整的國家裡面,這會牽扯到不實宣傳之類的行為:

After browsing hotels for some time I've seen booking.com show several hotels start to sell out of rooms. That usually causes me to hurry up and book, but after several hotels showed full at once I got suspicious and checked my partners phone. The hotels still showed as available there. Dark stuff. Their website is otherwise pretty good though and I still use them.

在「Online hotel booking」這邊有英國對這些線上訂房網站的調查與裁罰。

然後在 Hacker News 上的 comment 有看到一個有趣的方法,是 PresidentObama 提到的方法 (這 id XDDD),用 uBlock Origin 來擋:

From the last time booking.com was discussed I picked up some ublock origin filters that make the website more bearable.

You can copy and paste them directly in your ublock config (ublock options -> My filters)

  ! https://news.ycombinator.com/item?id=21860328
  booking.com##.soldout_property
  booking.com##.sr_rooms_left_wrap.only_x_left
  booking.com##.lastbooking
  booking.com##.sr--x-times-booked
  booking.com##.in-high-demand-not-scarce
  booking.com##.top_scarcity
  booking.com##.hp-rt-just-booked
  booking.com##.cheapest_banner_content > *
  booking.com##.hp-social_proof
  booking.com##.fe_banner__red.fe_banner__w-icon.fe_banner__scale_small.fe_banner
  booking.com##.urgency_message_x_people.urgency_message_red
  booking.com##.rackrate
  booking.com##.urgency_message_red.altHotels_most_recent_booking
  booking.com##.fe_banner__w-icon-large.fe_banner__w-icon.fe_banner
  booking.com##.smaller-low-av-msg_wrapper
  booking.com##.small_warning.wxp-sr-banner.js-wxp-sr-banner
  booking.com##.lock-price-banner--no-button.lock-price-banner.bui-u-bleed\@small.bui-alert--large.bui-alert--success.bui-alert

另外還有擋一些追蹤的 url parameter:

Apart from these, I use some additional ublock filters to block some of their tracking that I am not ok with.

$removeparam=/^(error_url|ac_suggestion_theme_list_length|ac_suggestion_list_length|search_pageview_id|ac_click_type|ac_langcode|ac_position|ss_raw|from_sf|is_ski_area|src|sb_lp|sb|search_selected|srpvid|click_from_logo|ss|ssne|ssne_untouched|b_h4u_keep_filters|aid|label|all_sr_blocks|highlighted_blocks|ucfs|arphpl|hpos|hapos|matching_block_id|from|tpi_r|sr_order|srepoch|sr_pri_blocks|atlas_src|place_types)/,domain=booking.com
  $removeparam=/sid=.\*;BBOX/,domain=booking.com
  ||www.booking.com/c360/v1/track
  ||www.booking.com/fl/exposed
  ||booking.com/personalisationinfra/track_behaviour_property
  ||booking.com/has_seen_review_list

不過好像很少用 booking.com 了...

美國 FTC 提案要阻擋退訂的 Dark Pattern

2021 年的時候寫過「最近很熱鬧的 New York Times 退訂截圖」這篇,在講紐約時報在退訂這塊的 dark pattern,這個方式後來被許多報社的網路服務使用 (像是 WSJ)。

後來加州政府通過法律阻擋這樣的 dark pattern,所以就有 Reddit 上面這樣的討論,教大家直接把 billing address 改到加州後就可以網路上退訂:「WSJ Subscription policy makes it easy to subscribe (online), but hard to unsubscribe (via phone).」。

現在看起來 FTC 打算推動變成全國性的法案,而且不只是網路服務,也包括了像是健身房與第四台的服務都必須提供對稱的方法 (訂閱與退訂):「The FTC wants to ban those tough-to-cancel gym and cable subscriptions」。

來繼續追進度,看看什麼時候通過...

風扇的出風孔的遮罩,不同形狀產生噪音的差異

前幾天在 Hacker News Daily 上看的文章:「Effects of grill patterns on fan performance/noise (2011) (pugetsystems.com)」,原文在「Effects of Grill Patterns on Fan Performance/Noise」。文章在講電腦風扇的出風口擋板的樣式對噪音的影響。這是一篇 2011 的老文章,但看了一下好像沒有什麼新資料...

出風口的遮罩在有些地區似乎是法律要求,主要是怕小朋友手指戳進去,以及成年人不小心碰到,所以就會有專門的測試項目在跑:

I remember back in 2000 ish I worked in R&D of a PC manufacturer and had to check the new PSUs and cases with a 'Test Finger' to make sure all the holes were small enough. The test finger was really expensive IIRC.

但回到這篇的重點,噪音與風量的差異,所以基準值是拔掉遮罩的部份先測試 (所以可以知道最低的噪音與最大的風量),然後上各種遮罩測差異。

Mesh 增加的噪音是最少,但風量有點差:

Wire 增加的噪音也很少,風量也達到最高:

從倒數的方向來看,Turbine 是最吵風量也最少的:

發現比較差的遮罩增加的噪音不算少,把很多風的動能變成聲音的能量了...

公平會對創業家兄弟與松果公司的 SEO 誘導轉向開罰

好像很少提到國內的新聞,但這則應該是這兩天蠻熱門的一個新聞,創業家兄弟與松果公司 (也是創業家兄弟公司) 被公平會開罰:「操作SEO搜尋關鍵字誤導消費者 創業家兄弟、松果公司挨罰」,相關的備份先留起來:Internet Archivearchive.today

公平會官方的新聞稿則可以在「利用程式設計引誘消費者「逛錯街」,公平會開罰」這邊看到,對應的網頁備份:Internet Archivearchive.today

用的是公平交易法第 25 條:

公平會於4月12日第1594次委員會議通過,創業家兄弟股份有限公司及松果購物股份有限公司利用「搜尋引擎優化 (Search Engine Optimization,簡稱SEO)」技術,並在搜尋引擎的顯示結果上不當顯示特定品牌名稱,使消費者誤認該賣場有販售特定品牌產品,藉以增進自身網站到訪率,違反公平交易法第25條規定,處創業家兄弟公司200萬元、松果公司80萬元罰鍰。

這條的條文可以從「公平交易法§25-全國法規資料庫」這邊看到:

除本法另有規定者外,事業亦不得為其他足以影響交易秩序之欺罔或顯失公平之行為。

主要的原因是點進去後卻沒有該項商品:

公平會發現,消費者在Google搜尋引擎打上特定品牌名稱,例如「悅夢床墊」時,搜尋結果會出現「悅夢床墊的熱銷搜尋結果│生活市集」、「人氣熱銷悅夢床墊口碑推薦品牌整理─松果購物」等搜尋結果,消費者被前述搜尋結果吸引點選進入「生活市集」、「松果購物」網站後,卻發現該賣場並無「悅夢床墊」之產品,此係生活市集及松果購物之經營者創業家兄弟公司及松果公司分別利用SEO技術所產生的現象。

而且會透過使用者在往站上搜尋的關鍵字產生對應的頁面:

公平會進一步調查後發現,創業家兄弟公司及松果公司對其所經營之「生活市集」及「松果購物」網頁進行設計,只要網路使用者在該2網站搜尋過「悅夢床墊」,縱然該2網站賣場並沒有賣「悅夢床墊」,其網站程式也會主動生成行銷文案網頁,以供搜尋引擎攫取。若有消費者之後在Google搜尋引擎查詢「悅夢床墊」時,搜尋結果便會帶出「悅夢床墊的熱銷搜尋結果│生活市集」、「人氣熱銷悅夢床墊口碑推薦品牌整理─松果購物」等搜尋結果項目,經消費者點選後即會導向「生活市集」、「松果購物」之網站。

然後判罰的部份:

公平會過往即曾就事業使用競爭對手事業名稱作為關鍵字廣告,並在關鍵字廣告併列競爭對手事業名稱之行為,認定違反公平交易法第25條規定。本案雖非創業家兄弟公司及松果公司直接使用「悅夢床墊」等他人商品品牌作為關鍵字廣告,但最終呈現之結果,本質上都是「誘導/轉向」(bait-and-switch)的欺罔行為,除了打斷消費者正常的商品搜尋與購買過程,也對其他販售該等品牌商品之經營者形成不公平競爭的效果。若任由發生而不予規範,未來將可能導致其他競爭者之競相仿效,消費者將更難以分辨搜尋結果呈現資訊之真偽,進而威脅電商市場之競爭秩序及消費者利益。故公平會認為違反公平交易法第25條「足以影響交易秩序之欺罔及顯失公平行為」,並分別處創業家兄弟公司200萬元、松果公司80萬元罰鍰。

所以這算是對 Dark pattern SEO 的部份開罰...

最近 Linux 核心安全性問題的 Dirty Pipe 故事很有趣...

Hacker News 上看到「The Dirty Pipe Vulnerability」這個 Linux kernel 的安全性問題,Hacker News 上相關的討論在「The Dirty Pipe Vulnerability (cm4all.com)」這邊可以看到。

這次出包的是 splice() 的問題,先講他寫出可重製 bug 的程式碼,首先是第一個程式用 user1 放著跑:

#include <unistd.h>
int main(int argc, char **argv) {
  for (;;) write(1, "AAAAA", 5);
}
// ./writer >foo

然後第二個程式也放著跑 (可以是不同的 user2,完全無法碰到 user1 的權限):

#define _GNU_SOURCE
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char **argv) {
  for (;;) {
    splice(0, 0, 1, 0, 2, 0);
    write(1, "BBBBB", 5);
  }
}
// ./splicer <foo |cat >/dev/null

理論上不會在 foo 裡面看到任何 BBBBB 的字串,但卻打穿了... 透過 git bisect 的檢查,他也確認了是在「pipe: merge anon_pipe_buf*_ops」這個 commit 時出的問題。

不過找到問題的過程拉的頗長,一開始是有 web hosting 服務的 support ticket 說 access log 下載下來發現爛掉了,無法解壓縮:

It all started a year ago with a support ticket about corrupt files. A customer complained that the access logs they downloaded could not be decompressed. And indeed, there was a corrupt log file on one of the log servers; it could be decompressed, but gzip reported a CRC error.

然後他先手動處理就把票關起來了:

I fixed the file’s CRC manually, closed the ticket, and soon forgot about the problem.

接下來過幾個月後又發生,經過幾次的 support ticket 後他手上就有一些「資料」可以看:

Months later, this happened again and yet again. Every time, the file’s contents looked correct, only the CRC at the end of the file was wrong. Now, with several corrupt files, I was able to dig deeper and found a surprising kind of corruption. A pattern emerged.

然後因為發生的頻率也不是很高,加上邏輯上卡到死胡同,所以他也沒有辦法花太多時間在上面:

None of this made sense, but new support tickets kept coming in (at a very slow rate). There was some systematic problem, but I just couldn’t get a grip on it. That gave me a lot of frustration, but I was busy with other tasks, and I kept pushing this file corruption problem to the back of my queue.

後來真的花時間下去找,利用先前的 pattern 掃了一次系統 log,發現有規律在:

External pressure brought this problem back into my consciousness. I scanned the whole hard disk for corrupt files (which took two days), hoping for more patterns to emerge. And indeed, there was a pattern:

  • there were 37 corrupt files within the past 3 months
  • they occurred on 22 unique days
  • 18 of those days have 1 corruption
  • 1 day has 2 corruptions (2021-11-21)
  • 1 day has 7 corruptions (2021-11-30)
  • 1 day has 6 corruptions (2021-12-31)
  • 1 day has 4 corruptions (2022-01-31)

The last day of each month is clearly the one which most corruptions occur.

然後就試著寫各種 reproducible code,最後成功的版本就是開頭提到的,然後他發現這個漏洞可以是 security vulnerability,就回報出去了,可以看到前後從第一次的 support ticket 到最後解決花了快一年的時間,不過 Linux kernel 端修正的速度蠻快的:

  • 2021-04-29: first support ticket about file corruption
  • 2022-02-19: file corruption problem identified as Linux kernel bug, which turned out to be an exploitable vulnerability
  • 2022-02-20: bug report, exploit and patch sent to the Linux kernel security team
  • 2022-02-21: bug reproduced on Google Pixel 6; bug report sent to the Android Security Team
  • 2022-02-21: patch sent to LKML (without vulnerability details) as suggested by Linus Torvalds, Willy Tarreau and Al Viro
  • 2022-02-23: Linux stable releases with my bug fix (5.16.11, 5.15.25, 5.10.102)
  • 2022-02-24: Google merges my bug fix into the Android kernel
  • 2022-02-28: notified the linux-distros mailing list
  • 2022-03-07: public disclosure

整個故事還蠻精彩的 XD

修正 Curl 的 TLS handshake,避開 bot 偵測機制

利用 TLS handshake 的 pattern 可以當作是某種 fingerprint,就可以知道你是用 Curl,這個方式在蠻多 CDN 都會用在 anti-bot 機制 (像是 Cloudflare),而剛剛看到有人投稿自己的 patch,試著將 Curl 修改成 Firefox 的 pattern:「curl-impersonate」,Hacker News 上的討論在這邊可以看到:「Show HN: Curl modified to impersonate Firefox and mimic its TLS handshake (github.com/lwthiker)」。

作者有提到這次的 patch 偏 hack,不太可能整進上游,但希望未來改的乾淨一點,然後整進上游:

I hope to do so in the future, for now the implementation is extremely hacky so I doubt it can get accepted into curl.

另外有人提出來說應該要用 Firefox ESR 版本的 pattern 而非 stable channel,也有人提出來說用 Google Chrome 的更好,不過我覺得有人開始做就已經很棒了 XD

透過 Interview 的管道推銷產品

Hacker News 上看到的 dark pattern:「NDA Expired - let's spill the beans on a weird startup」,對應的討論可以在「NDA expired, let’s spill the beans on a weird startup (shkspr.mobi)」這邊翻到。

整個行銷手法大概是這樣,首先是客戶會想要推廣某個產品 (文章內的 AWS 想要推廣 InfiniDash 這個產品),所以找上這個行銷公司。而行銷公司會找一群人去各公司 interview,然後在 interview 過程中問到各種問題的時候都回答用 InfiniDash 這個產品解決:

"Let's say in the interview you're asked for a time when you handled a difficult situation. We want you to talk about how using InfiniDash made life easier for your team."

這個行銷公司會跟某些獵頭合作,把人塞進去 (而這些獵頭也會拿到錢):

"Oh, we work with loads of recruitment consultants. They get paid for every decent candidate who gets interviewed, so they give us a cut of their commission. And we get paid by our tech clients. It's win-win!"

後來作者沒有加入這家公司,而這家公司後來也消失了:

I did not take the job. I had serious concerns about the ethics and legality of their business model. I also didn't think that it would work. Evidently, neither did their investors. Fronk bumbled along for a few months - and I'd occasionally meet them at conferences - but then they seemed to disappear.

但作者最後提到了他之後在 interview 時會聽到面試者講一些特定的產品,讓他懷疑這家公司是不是還存在 XDDD

Every once in a while, I'll be interviewing a candidate who starts waxing lyrical about how rewriting everything in today's flavour of JavaScript really helped their last company. Or how their bosses were impressed with what this cool new bit of tech can do. Or why they could never work anywhere which didn't use this specific code editor.

Reddit 的前 EM 提到 Reddit 用的 Dark Pattern

Hacker News Daily 上看到「Reddit's disrespectful design」這個,在講 Reddit 用到的 dark pattern, 不過我覺得更棒的是在 Hacker News 上的討論:「Reddit’s disrespectful design (ognjen.io)」。

討論裡面出現了之前在 Reddit 裡面負責這塊的 EM (engineer manager),wting

I was the EM for Reddit's Growth team around this time. I am responsible for / contributed to a few features like the current signup flow, AMP pages, push notifications, email digests, app download interstitials, etc.

然後他提到他當初拒絕實做其中一些 dark pattern,然後在經過 [0] 的事情後,他決定換團隊:

There was a new product lead who joined with many good ideas, but some of them were dark patterns that I heavily protested. After a few months of this, it was obvious that I was going to be reigned in or let go[0]; I immediately transferred to a different org.

其中 [0] 的部份是,公司把他 reporting line 改成到 product lead XDDD

0: They changed it so I would report to the product lead, which is odd for an EM to report into a product chain and the only instance within the company ever.

另外他也講了一個事情,就是強姦使用者是有用的,而且超級有用:

Now let me explain the other side of the story. 4 years later, Reddit's DAU, MAU, and revenue have all grown at ridiculous rates[1]. Yes, power users complain—and still continue using the site—but the casual user does not. These dark patterns have been normalized on other websites.

These practices are done because it works.

1: Many friends are startup founders and I've been at a few startups myself—a byproduct of being in the Bay Area—and Reddit's growth numbers are impressive. As a former employee, I am quite happy about my equity growth.

這幾乎是共識了,dark pattern 看起來很邪惡沒錯,但他就是有用...