Tag: sre

Google SRE 團隊整理出過去二十年的十一條心得

Google 的 SRE 團隊整理出過去二十年的心得，當看故事的心態在看的：「Lessons Learned from Twenty Years of Site Reliability Engineering」，在 Hacker News 上也有討論：「Lessons Learned from Twenty Years of Site Reliability Engineering (sre.google)」。

裡面的項目大多都會在公司成長時不斷的導入，都是夠大就會遇到的。

比較有趣的是第六條，這是唯一一條全部都用大寫字母列出來的：

COMMUNICATION CHANNELS! AND BACKUP CHANNELS!! AND BACKUPS FOR THOSE BACKUP CHANNELS!!!

到 Google 這個規模的架構，這邊就會規劃找完全獨立於 Google 架構的方案來用；我猜應該是傳統的 colocation 機房 (像是 AT&T 之類的)，上面跑 IRC server 之類的？

在 Hacker News 上面也有其他人提到 Netflix 也有類似的規劃，需要有一個備援的管道是完全獨立於 AWS 的；另外同一則 comment 裡也有提到 Reddit 的作法是在辦公室裡面放 IRC server 備援：

Yes! At Netflix, when we picked vendors for systems that we used during an outage, we always had to make sure they were not on AWS. At reddit we had a server in the office with a backup IRC server in case the main one we used was unavailable.

IRC 還是很好用的 XD

Ansible 的爭論

前幾天在 Hacker News Daily 上看到「Five Ansible Techniques I Wish I’d Known Earlier」這篇，裡面提到了一些 Ansible 的用法還蠻有用的，算是開始用 Ansible 後應該都會有幫助的用法... 不過 Hacker News 上的討論「Ansible Techniques I Wish I’d Known Earlier (zwischenzugs.com)」比較精彩...

目前在頂端的留言對 Ansible 幹到不行，尤其是那個 YAML 格式：

Ansible is abysmal. I don't know why anyone still chooses it. It's a mess of yaml and what feels like a million yaml files that is always extremely hard to follow. Honestly writing some python, or bash is a lot easier to follow, read, and understand. The only thing it has going for it is the inventory system. I wish ansible would die already.

然後講到 bash 與 python 之類的工具時有人提到 idempotent：

>bash and python
Neither of those of idempotent.

馬上就有人幹勦，大多數人在寫 Ansible playbook 時根本沒人在注意 idempotent，而且一堆 shell script 的東西被塞進 YAML 只能說痛苦 XDDD

Most of the ansible roles I come across written by my team are not idempotent either, its a huge lie that Ansible is idempotent. Its idempotent if you put the effort into make it be but if I see tons of shell or command module invocations without prerequisite checks to see if the work should be done. Most devs I see using Ansible treat it like a shell script written in YAML and to that purpose it sucks.

我自己目前會挑 Ansible 主要還是因為 server 不需要另外裝軟體，是個 production 為導向的設計，再更大的時候就要想一下要怎麼繼續搞下去了...