stanford

Alpaca.cpp 有 13B 與 30B 的 model 可以玩了

在「Stanford Alpaca 與 Alpaca.cpp」這邊有提到 Alpaca 有放出 7B 的 model 可以用，後來 community 也照著同樣的方式訓練出對應的 13B 與 30B 的 model 了，在 Alpaca.cpp 專案的 README.md 有寫怎麼取得。

看起來比較大的限制是記憶體，13B 與 30B 的記憶體建議是 >10GB RAM 與 >32GB RAM，不知道剛好 32GB RAM 跑不跑得動，先拿 32GB RAM 的機器跑看看，如果真的不行的話家裡桌機是 64GB RAM，應該是不會有大問題...

不過 13B 的 model 檔案位置是放在 IPFS 上，透過這幾個 gateway 下載起來都有點慢，有需要的人用 BitTorrent 可能會快一點？

Update：測了一下 13B 的版本：

> Explain FAANG.
FAANG refers to five of America's largest technology companies - Facebook, Apple Inc., Amazon, Netflix and Google (Alphabet). These are some of the most valuable brands in today’s marketplace with a combined value that exceeded $3 trillion as at 2018.

以及 30B 的版本：

> Explain FAANG.
FAANG stands for Facebook, Amazon, Apple, Netflix and Google - five of the most powerful technology companies in today's world. These tech giants have become increasingly influential over recent years due to their dominance in various markets such as social media platforms (Facebook), e-commerce websites (Amazon) or streaming services (Netflix).

Stanford Alpaca 與 Alpaca.cpp

先前提到的「玩最近 Facebook Research (Meta) 放出來的 LLaMA」，在 2/24 公佈後，過沒多久 Stanford CRFM 團隊就丟出 fine-tuned 過的版本：「Alpaca: A Strong, Replicable Instruction-Following Model」，對應的討論在「Alpaca: A strong open-source instruction-following model (stanford.edu)」。

這個 tuning 是基於 LLaMA-7B：

We are releasing our findings about an instruction-following language model, dubbed Alpaca, which is fine-tuned from Meta’s LLaMA 7B model. We train the Alpaca model on 52K instruction-following demonstrations generated in the style of self-instruct using text-davinci-003. On the self-instruct evaluation set, Alpaca shows many behaviors similar to OpenAI’s text-davinci-003, but is also surprisingly small and easy/cheap to reproduce.

後來就再被包成一個專案 (改自 llama.cpp)，讓大家可以很方便的跑：「Alpaca.cpp」。

剛好可以比較，用 llama.cpp 跑的話是：

$ ./main -m ./models/7B/ggml-model-q4_0.bin -t 12 -n 128 -p 'Explain FAANG. '

Explain FAANG. 🙂
The acronym “FAAN” is used for the five tech giants (Facebook, Apple, Amazon and Netflix) that are
 taking over our lives:
15 November 2018 / by Amaraksha S [end of text]

而用 alpaca.cpp 跑的話回答品質好不少：

> Explain FAANG.
FAANG (or FANNG) refers to five technology companies, Facebook Inc., Apple Inc., Amazon.com Inc., Netflix Inc and Alphabet's Google division that have become some of the most influential in recent years due to their rapid growth rates as well as their market capitalization values: 
Facebook (FB), $596 billion; Apple (AAPL) ,$1,084.72 billion; Amazon (AMZN), $394.5 billion ; Netflix( NFLX ), $154.8 billion and Google's parent company Alphabet Inc., ($GOOG,$GOOGL).
These companies have become some of the most influential in recent years due to their rapid growth rates as well as their market capitalization values:  Facebook (FB),$307.15 billion; Apple(AAPL) , $924.86 billion ; Amazon (AMZN ), $783.

可以看到 instruction-following 後的差異，很像是 GPT-3.5 API 與 ChatGPT API 的輸出結果差異。

而這只是 LLaMA-7B 的成果，好像可以期待有人對 LLaMA-65B 做些事情？

加快 ls 的速度

看到「When setting an environment variable gives you a 40x speedup」這篇在講 ls 的速度。

文章是由 Stanford 的 Sherlock 發出來的，不過看起來跟電視劇沒關係，從網站上的標語「The HPC cluster for all your computing needs」可以看出是 HPC 相關的單位。

在 HPC 環境裡面可以預期單一目錄裡會有很多檔案，所以使用者跑來抱怨 ls 的速度就不算太意外了。不過這次使用者有提到在他自己的 laptop 上跑 ls 反而很快：

It all started from a support question, from a user reporting a usability problem with ls taking several minutes to list the contents of a 15,000+ entries directory on $SCRATCH.

Having thousands of files in a single directory is usually not very file system-friendly, and definitely not recommended. The user knew this already and admitted that wasn’t great, but when he mentioned his laptop was 1,000x faster than Sherlock to list this directory’s contents, of course, it stung. So we looked deeper.

直接跳到後面的結論... 原因是出自於因為需要顯示不同顏色，而需要透過 lstat() 查詢額外的檔案性質 (可執行、setuid 以及 setgid 這些資料)，導致速度變慢：

From 13s with the default settings, to 0.3s with a small LS_COLORS tweak, that’s a 40x speedup right there, for the cheap price of not having setuid/setgid or executable files colorized differently.

Of course, this is now setup on Sherlock, for every user’s benefit.

透過設定 LS_COLORS='ex=00:su=00:sg=00:ca=00:'，可以讓 lstat() 消失，所以被放進 Sherlock 的預設值了... 而沒有遇到這個問題的環境 (像是有設計好對應的目錄結構)，或是想要維持原來的樣子的人，則可以 unset 掉這個值讓輸出還是有色彩差異 :o