Home » Posts tagged "test"

Google 發表新的 TTS (Text-to-Speech) 技術 Tacotron 2

Tacotron 是 Google 發表的 TTS 技術 (i.e. 輸入文字,請電腦發音),而前一版的 Tacotron 的錄音可以參考「Audio samples from "Tacotron: Towards End-to-End Speech Synthesis"」,論文則是在「Tacotron: Towards End-to-End Speech Synthesis」這邊可以看到。

這一版的則是在 Twitter 上看到有人提到:

這一版叫做 Tacotron 2,錄音可以參考「Audio samples from "Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions"」,論文在「Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions」。

這次在錄音頁面的最下面提供了盲測 (人類與 Tacotron 2 的錄音),基本上已經分不出哪個是真人了...

原來 Oracle 與 Microsoft 裡的條款是這樣來的...

看到「That time Larry Ellison allegedly tried to have a professor fired for benchmarking Oracle」這篇文章的講古,想起很久前就有聽過 Microsoft 有這樣的條款 (禁止未經原廠同意公開 benchmark 結果),原來是 Oracle 在三十幾年前創出來的?而且這種條款還有專有名詞「DeWitt Clauses」,出自當初被搞的教授 David DeWitt...

Microsoft 的條款是這樣:

You may not disclose the results of any benchmark test … without Microsoft’s prior written approval

Oracle 的則是:

You may not disclose results of any Program benchmark tests without Oracle’s prior consent

IBM 的反而在 license 裡面直接允許:

Licensee may disclose the results of any benchmark test of the Program or its subcomponents to any third party provided that Licensee (A) publicly discloses the complete methodology used in the benchmark test (for example, hardware and software setup, installation procedure and configuration files), (B) performs Licensee’s benchmark testing running the Program in its Specified Operating Environment using the latest applicable updates, patches and fixes available for the Program from IBM or third parties that provide IBM products (“Third Parties”), and © follows any and all performance tuning and “best practices” guidance available in the Program’s documentation and on IBM’s support web sites for the Program…

用 Makefile 跑測試的 Makefile.test

Box 放出來的東西,用 Makefile 跑各種測試:「Introducing Makefile.test: A Generic Makefile to Run Test Executables」,專案在 GitHub 上的 box/Makefile.test

Makefile.test can be used to run any type of test executables. It is not language specific nor it requires any changes to your code. Parallel, serial execution, various platforms and make versions are supported. The executables can be organized in any desired way. The user only lists the test files, the rest is taken care of Makefile.test.

程式碼很短,看程式碼其實比看說明容易理解:「Makefile.test」。看完後感覺沒有很實用... 就當作趣味看一看 XD

Microsoft Blogs 上 TDD 的戰文...

文章標題就直接寫「#NoTDD」的戰文 XDDD

列了 Pros (一行) 跟 Cons (超長 XDDD):

Pros

  • We end up with tests that verify the behavior of the code and help prevent regressions

這個是 TDD 的目的。而 Cons:

Cons

  • It takes us longer to write code using TDD
  • The tests get in the way. Because my design does not have low coupling, I end up with tests that also do not have low coupling. This means that if I change the behavior of how class works, I often have to fix tests for other classes.
  • Because I don’t have low coupling, I need to use mocks or other tests doubles often. Tests are good to the extent that the tests use the code in precisely the same way the real system uses the code. As soon as I introduce mocks, I now have a test that only works as long as that mock faithfully matches the behavior of the real system. If I have lots of mocks – and since I don’t have low coupling, I need lots of mocks – then I’m going to have cases where the behavior does not match. This will either show up as a broken test, or a missed regression.
  • Design on the fly is a learned skill. If you don’t have the refactoring skills to drive it, it is possible that the design you reach through TDD is going to be worse than if you spent 15 minutes doing up-front design.

這四個問題講的是時間與能力兩個因子的作用,轉一個角度來討論其實是:如果 junior engineer 可以寫出好的測試,他們就不叫 junior engineer 了... 而 senior engineer 是稀缺資源,讓他們多花時間寫出「好的測試」未必是划算的。

反而是另外一種常見的方式常常跟 TDD 在對抗:透過 QA team 在完成後測試,尤其是用手動測試的 QA team XDDD

這個方法很簡單,而且行之有年。而且很無奈的,跟一般軟體工程所期望的相反,junior engineer 就可以做的不錯,所以人力的部份相當好 scale,而品質也有不錯的水準 (畢竟是直接測實際的功能了)。

剛好前陣子有提到另外一篇論文也在討論 TDD 的效果 (參考「又一篇戰文:討論 TDD 的過程」這邊),也有類似的反思。

前陣子在 Twitter 上看到這則也是很有趣,拿來當結尾 XDDD:

又一篇戰文:討論 TDD 的過程

最近在 39th International Conference on Software Engineering 上受邀參加說明的論文,在 The Morning Paper 上看到的:「A dissection of the test-driven development process: does it really matter to test-first or test-last?」。

論文本身在「A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?」這邊可以下載,是去年 2016 就發出來的論文。

論文拆解 TDD 的行為,分析到底是哪些階段才是有正面幫助的:

Background: Test-driven development (TDD) is a technique that repeats short coding cycles interleaved with testing. The developer first writes a unit test for the desired functionality, followed by the necessary production code, and refactors the code. Many empirical studies neglect unique process characteristics related to TDD iterative nature. Aim: We formulate four process characteristic: sequencing, granularity, uniformity, and refactoring effort. We investigate how these characteristics impact quality and productivity in TDD and related variations. Method: We analyzed 82 data points collected from 39 professionals, each capturing the process used while performing a specific development task. We built regression models to assess the impact of process characteristics on quality and productivity. Quality was measured by functional correctness.

比較特別的是作者指出 refactoring 的負面效果:

Result: Quality and productivity improvements were primarily positively associated with the granularity and uniformity. Sequencing, the order in which test and production code are written, had no important influence. Refactoring effort was negatively associated with both outcomes. We explain the unexpected negative correlation with quality by possible prevalence of mixed refactoring. Conclusion: The claimed benefits of TDD may not be due to its distinctive test-first dynamic, but rather due to the fact that TDD-like processes encourage fine-grained, steady steps that improve focus and flow.

在 The Morning Paper 上的註解也可以看看 (反駁論文的意見),像是對論文使用自評系統的批評。

用 Machine Learning 調校資料庫

AWS AI Blog 在月初上放出來的消息:「Tuning Your DBMS Automatically with Machine Learning」。

Carnegie Mellon Database Group 做的研究,除了預設值以外,另外跟四種不同的參數做比較,分別是 OtterTune (也就是這次的研究)、Tuning script (對於不熟資料庫的人,常用的 open source 工具)、DBA 手動調整,以及 RDS

MySQL

PostgreSQL

比較明顯的結論是:

  • Default 值在所有的 case 下都是最差的 (無論是 MySQL 與 PostgreSQL 平台,以及包括 99% 的 Latency 與 QPS,這樣二乘二的四個結果)。而且 Default 跑出來的數字與其他的差距都很明顯。
  • OtterTune 在所有 case 下跑出來都比 Tuning script 的好。這也是合理的結果,本來就是想要取代其他機器跑出來的結果。

至於有些討論 DBA 會失業的事情,我是樂見其成啦... 這些繁瑣的事情可以自動化就想交給自動化吧 XD

Bash 裡處理 PID file 的方式...

看到「Age comparison in Bash for files and processes」後查了一些資料,如果在不使用外部程式處理的話,的確是多做了不少事情。

這是 Bash-ot 的說明:

file1 -ot file2
       True if file1 is older than file2, or if file2 exists and file1 does not.

而這是 test (也就是 [ 這隻程式) 對 -ot 的說明:

FILE1 -ot FILE2
       FILE1 is older than FILE2

多了檔案是否存在的檢查...

另外可以參考「What's the difference between [ and [[ in Bash?」這邊的說明,Bash 的 [[ 是 Bash 特有的加強版,而 [ 則是用系統的 test 運算。

Heroku CI 正式推出

月初才提到「Heroku 也要進入 CI 市場...」,前幾天就開放使用了:「Heroku CI Is Now Generally Available: Fast, Low Setup CI That’s Easy to Use」。

收費方式是依照 Heroku Pipelines 收費,一個 Heroku Pipelines 是 USD$10/month:

Each CI-enabled Heroku Pipeline is charged just $10/month for an unlimited number of test runs.

比起其他棚是無限量 Travis CI 的 $69/month (1 concurrent job,不限時間),或是 CircleCI 的 Free 方案 (1 concurrent job,限時每個月 1500 分鐘) 或是 USD$50/month (2 concurrent job,不限時間),感覺有些微妙...

感覺還是先繼續用 Travis CI 好了,目前 Open Source Project 的標準,路邊可以隨便檢到很多範例用... (像是用 Google".travis.yml" "language:php" site:github.com 就可以找到很多 PHP 的範例)

Archives