Tag: platform

在 Hacker News Daily 上看到「How big is YouTube? (ethanzuckerman.com)」這篇，原文在「How Big is YouTube?」。

算是個老問題了，而且應該是統計學上比較簡單的方法。先列出作者最後的成果：「TubeStats」。

作者用的方法是觀察 YouTube 的 vid：

Here’s how this works: YouTube URLs look like this: https://www.youtube.com/ watch?v=vXPJVwwEmiM

可以分析出來 vid 包括了 64-bit 的資訊，這個資料型態對工程師來說，看起來就很像是 uniformly distributed：

That bit after “watch?v=” is an 11 digit string. The first ten digits can be a-z,A-Z,0-9 and _-. The last digit is special, and can only be one of 16 values. Turns out there are 2^64 possible YouTube addresses, an enormous number: 18.4 quintillion. There are lots of YouTube videos, but not that many. Let’s guess for a moment that there are 1 billion YouTube videos – if you picked URLs at random, you’d only get a valid address roughly once every 18.4 billion tries.

然後就是隨機去產生 vid 去掃，這個方法跟 drunk dialing 的行為很像，算是 random sampling 的方式：

We refer to this method as “drunk dialing”, as it’s basically as sophisticated as taking swigs from a bottle of bourbon and mashing digits on a telephone, hoping to find a human being to speak to. Jason found a couple of cheats that makes the method roughly 32,000 times as efficient, meaning our “phone call” connects lots more often. Kevin Zheng wrote a whole bunch of scripts to do the dialing, and over the course of several months, we collected more than 10,000 truly random YouTube videos.

另外在 2011 年就有提出來利用 autocomplete 機制去算：

By comparing our results to other ways of generating lists of YouTube videos, we can declare them “plausibly random” if they generate similar results. Fortunately, one method does – it was discovered by Jia Zhou et. al. in 2011, and it’s far more efficient than our naïve method. (You generate a five character string where one character is a dash – YouTube will autocomplete those URLs and spit out a matching video if one exists.) Kevin now polls YouTube using the “dash method” and uses the results to maintain our dashboard at Tubestats.

目前他們的預估大約是 13B 左右的影片，換算大約是用掉 33.63 bits 了 (2^33.6)：

In our case, our drunk dials tried roughly 32k numbers at the same time, and we got a “hit” every 50,000 times or so. Our current estimate for the size of YouTube is 13.325 billion videos – we are now updating this number every few weeks at tubestats.org.

而這邊提到的 32768 * 50k 會中一次的部分，這邊的大約是 30.61 bits，這樣加起來是差不多 64 bits 沒錯。

不過要注意的是，他們沒有給出 interval，所以 13B 的上下可能是一倍左右的差距 (6.5B～26B 之類的)，這邊的數字當作概念比較好...