Tag Archives: engineering

Algolia 從 Heroku 搬到 GKE 的故事

Algolia 是一個搜尋引擎服務,他可以幫你 index 資料後,你直接 query 他取得結果。

在這篇文章裡 Algolia 決定從 Heroku 搬到 GKE:「The Challenging Migration from Heroku to Google Kubernetes Engine」。

在文章只單純就產品與技術面上的需求在討論,像是一開始討論 IP 白名單的問題:

A good example of this complexity is with IP Whitelisting. One of our customers wanted us to crawl from a fixed IP address so that they could whitelist that IP for high-rate crawling without being throttled by their load balancer. Only two engineers were developing the crawler, so we asked other colleagues to set up an HTTP proxy with a fixed IP address. Yet, as the number of customers grew, many more started asking for the same thing, and our infrastructure team told us it was time for us to take care of it ourselves.

不過我更想知道搬過去後的各類成本差異... 省了多少平台費用,以及多少維護人力的差異,不過看起來沒提到 XD

一堆軟體工程的定律...

看到「Famous Laws Of Software Development」這篇文章,2017 年的整理,裡面包括了很多軟體工程上的定律:

  • Murphy's Law
  • Brook's Law
  • Hofstadter's Law
  • Conway's Law
  • Postel's Law
  • Pareto Principle
  • The Peter Principle
  • Kerchkhoff's Principle
  • Linus's Law
  • Moore's Law
  • Wirth's law
  • Ninety-ninety rule
  • Knuth's optimization principle
  • Norvig's Law

開頭就放 Murphy's Law... XD

裡面的概念大多數都聽過了,有些知道名字,但有些只知道概念...

IDA 免費版

Update:被 comment 提醒,找了一下資料,看起來有段歷史了,所以說 RetDec 的影響就未必是這樣了。下面的文章內容就不修正了...:「IDA Support: Evaluation Version」。

IDA 居然也提供免費版了,雖然是比較舊的版本,而且不提供技術支援:「IDA Support: Freeware Version」。IDA 是個可以反組譯以及當 debugger 的工具:

IDA is a Windows, Linux or Mac OS X hosted multi-processor disassembler and debugger that offers so many features it is hard to describe them all. Just grab an evaluation version if you want a test drive.

我猜是 Avast 放出 MIT 授權版本的 RetDec 的關係 (參考「Avast 放出他們的 Decompiler,RetDec」這篇),導致 IDA 這邊要做一些動作推廣試用...

不過我覺得有了 open source 的工具後,會看到 open source 工具慢慢成長...

GitHub 上的 Continuous Integration (CI)

GitHub 寫了一篇「GitHub welcomes all CI tools」,不過對我來說比較有趣的是各家 CI 在 GitHub 上的市占率:

可以看到 Travis CI 還是擁有壓倒性的佔有率 (大約一半),這跟他提供免費的 public repository 方案有關。但以另外一個角度來看,這也讓使用商用版本 (travis-ci.com) 的人有大量的範例可以參考,而不需要擔心資源太少。

從 IndieHackers.com 上掃出各種「成功的方法」

Indie Hackes 是個放各種「成功案例」的網站:

Indie Hackers is a place where the founders of profitable businesses and side projects can share their stories transparently, and where entrepreneurs can come to read and learn from those examples. It's also a community where individual "indie hackers" can come together to share their experiences, give and receive feedback, and rely on one another for support.

而文章的作者把上面的文章拿出來分析,得到了一些有趣的資訊:「Reverse Engineering A Successful Lifestyle Business: Here’s Everything I’ve Learned From Reading IndieHackers.com」。

在「Books」的地方列出了提到的書籍,被提超過一次的有:

The 4-Hour Workweek by Tim Ferriss (5x)
The Hard Thing About Hard Things by Ben Horowitz (5x)
Zero to One by Peter Thiel (4x)
The Lean Startup by Eric Ries (4x)
Four Steps to the Epiphany by Steve Blank (3x)
Steve Jobs by Walter Isaacson (2x)
High Output Management by Andy Groove of Intel (2x)
Good to Great by Jim Collins (2x)
Traction by Gabriel Weinberg and Justin Mares (2x)
Built to Sell by John Warrillow (2x)
The Personal MBA by Josh Kaufman (2x)
Start Small, Stay Small by Rob Walling (2x)
The Power Of The Subconscious Mind by Joseph Murphy (2x)

然後是各種技巧 (?):

On Raising Prices (19x)
This was the most common advice. The easiest way to increase revenue: raise your prices! Don’t be scared to ask for money.

Real Artists Ship (9x)
Our first idea is a grand opening, a big launch, a press release, or major media coverage. We default to thinking we need an advertising budget. Our delusion is that we should be Transformers and not The Blair Witch Project. – Ryan Holiday

KISS (9x)
Automation (8x)
Start Charging Straight Away (8x)

可以拿來讀一讀吸收進去...

GitHub 引入 Code Owner 的概念

GitHub 推出了 Code Owner 的概念:「Introducing code owners」。也很直接說這個能是向 Chromium「致敬」出來的:

The code owners feature was inspired by Chromium's use of OWNERS files.

檔案名稱是 CODEOWNERS,可以放在根目錄或是 .github/ 下,可以針對不同的目錄設不同的人:

To specify code owners, create a file named CODEOWNERS in the repository's root directory (or in .github/ if you prefer) with the following format[.]

這樣一來,在 pull request 的時候就會跳出來:

另外也可以設定需要 code owner 同意才能 merge:

Microsoft Blogs 上 TDD 的戰文...

文章標題就直接寫「#NoTDD」的戰文 XDDD

列了 Pros (一行) 跟 Cons (超長 XDDD):

Pros

  • We end up with tests that verify the behavior of the code and help prevent regressions

這個是 TDD 的目的。而 Cons:

Cons

  • It takes us longer to write code using TDD
  • The tests get in the way. Because my design does not have low coupling, I end up with tests that also do not have low coupling. This means that if I change the behavior of how class works, I often have to fix tests for other classes.
  • Because I don’t have low coupling, I need to use mocks or other tests doubles often. Tests are good to the extent that the tests use the code in precisely the same way the real system uses the code. As soon as I introduce mocks, I now have a test that only works as long as that mock faithfully matches the behavior of the real system. If I have lots of mocks – and since I don’t have low coupling, I need lots of mocks – then I’m going to have cases where the behavior does not match. This will either show up as a broken test, or a missed regression.
  • Design on the fly is a learned skill. If you don’t have the refactoring skills to drive it, it is possible that the design you reach through TDD is going to be worse than if you spent 15 minutes doing up-front design.

這四個問題講的是時間與能力兩個因子的作用,轉一個角度來討論其實是:如果 junior engineer 可以寫出好的測試,他們就不叫 junior engineer 了... 而 senior engineer 是稀缺資源,讓他們多花時間寫出「好的測試」未必是划算的。

反而是另外一種常見的方式常常跟 TDD 在對抗:透過 QA team 在完成後測試,尤其是用手動測試的 QA team XDDD

這個方法很簡單,而且行之有年。而且很無奈的,跟一般軟體工程所期望的相反,junior engineer 就可以做的不錯,所以人力的部份相當好 scale,而品質也有不錯的水準 (畢竟是直接測實際的功能了)。

剛好前陣子有提到另外一篇論文也在討論 TDD 的效果 (參考「又一篇戰文:討論 TDD 的過程」這邊),也有類似的反思。

前陣子在 Twitter 上看到這則也是很有趣,拿來當結尾 XDDD:

重設密碼 + Social Engineering

在「The password reset MitM attack」這邊看到 PRMitM (Password Reset Man-in-the-Middle) 這樣的攻擊,原始論文在「The Password Reset MitM Attack」這邊可以取得。

用圖說明基本版的攻擊方式:

另外列出了各大站台的狀態:

以及各家簡訊的文字,可以發現不是每一家都有把產品的名稱寫上去:

這方法好有趣啊... XD

又一篇戰文:討論 TDD 的過程

最近在 39th International Conference on Software Engineering 上受邀參加說明的論文,在 The Morning Paper 上看到的:「A dissection of the test-driven development process: does it really matter to test-first or test-last?」。

論文本身在「A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?」這邊可以下載,是去年 2016 就發出來的論文。

論文拆解 TDD 的行為,分析到底是哪些階段才是有正面幫助的:

Background: Test-driven development (TDD) is a technique that repeats short coding cycles interleaved with testing. The developer first writes a unit test for the desired functionality, followed by the necessary production code, and refactors the code. Many empirical studies neglect unique process characteristics related to TDD iterative nature. Aim: We formulate four process characteristic: sequencing, granularity, uniformity, and refactoring effort. We investigate how these characteristics impact quality and productivity in TDD and related variations. Method: We analyzed 82 data points collected from 39 professionals, each capturing the process used while performing a specific development task. We built regression models to assess the impact of process characteristics on quality and productivity. Quality was measured by functional correctness.

比較特別的是作者指出 refactoring 的負面效果:

Result: Quality and productivity improvements were primarily positively associated with the granularity and uniformity. Sequencing, the order in which test and production code are written, had no important influence. Refactoring effort was negatively associated with both outcomes. We explain the unexpected negative correlation with quality by possible prevalence of mixed refactoring. Conclusion: The claimed benefits of TDD may not be due to its distinctive test-first dynamic, but rather due to the fact that TDD-like processes encourage fine-grained, steady steps that improve focus and flow.

在 The Morning Paper 上的註解也可以看看 (反駁論文的意見),像是對論文使用自評系統的批評。

Stripe 的 Increment 雜誌

Stripe 推出了 Increment 雜誌,講團隊合作時的各種議題:「Introducing Increment」。

And so we've decided to start Increment, a software engineering magazine dedicated to providing practical and useful insight into what effective teams are doing so that the rest of us can learn from them more quickly.

雜誌網站上也有類似的描述:

A digital magazine about how teams build and operate software systems at scale.

Increment is dedicated to covering how teams build and operate software systems at scale, one issue at a time.

可以看一看 Stripe 對團隊合作的想法...