Home » Posts tagged "search"

Googlebot 的 Web rendering service 的細節

在「Polymer 2 and Googlebot」這邊文章裡面才看到 Google 官方在今年八月就有公開 Googlebot 所使用的 Web rendering service (WRS) 的細節:「Rendering on Google Search」。可以想像到是基於 Google Chrome 的修改:

Googlebot uses a web rendering service (WRS) that is based on Chrome 41 (M41). Generally, WRS supports the same web platform features and capabilities that the Chrome version it uses — for a full list refer to chromestatus.com, or use the compare function on caniuse.com.

裡面提到一些值得注意的事情,像是不支援 WebSocket,所以對於考慮 Google 搜尋結果的頁面來說,就要注意錯誤處理了...

用 PublicWWW 分析網站

在「Keylogger Found on Nearly 5,500 Infected WordPress Sites」這邊看到的網站服務 PublicWWW

雖然原文是說 WordPress 被感染的情況,但注意到的反而是他提到的網站 PublicWWW。

在 PublicWWW 上面目前收錄了兩億個網站的資料,有些東西頗不賴的,像是可以搜尋有哪些是使用同樣的 Google Analytics 帳號:

Sites with the same analytics id: "UA-19778070-"

這拿來找誰是內容容場後面的人超棒的啊,而且可以拿來補內容農場的清單,像是「UA-31425034 - 19 Websites - PublicWWW.com」這個 XD

免費版只能搜 Top 3M 的部份,付費版 (USD$49/month) 則是可以搜所有的資料。

Yahoo! 與 Mozilla 針對預設搜尋引擎的事情戰起來了...

Mozilla 先前終止與 Yahoo! 的合作後 (在 Firefox 內預設使用 Yahoo! 的搜尋引擎),Yahoo! 提告以及 Mozilla 還手的消息在最近被 Mozilla 揭露:「Mozilla Files Cross-Complaint Against Yahoo Holdings and Oath」。

Yahoo! 提告的檔案 (PDF) 在「2017-12-01-Yahoo-Redacted-Complaint.pdf」,Mozilla 還手的檔案 (PDF) 則是在「2017-12-05-Mozilla-Redacted-X-Complaint-with-Exhibits-and-POS.pdf」這邊。

Firefox 57 釋出時,Mozilla 就把預設的搜尋引擎改回 Google (參考「Mozilla terminates its deal with Yahoo and makes Google the default in Firefox again」),不過當時 Firefox 57 更大的消息是推出了 Quantum,讓瀏覽器的速度拉到可以跟目前的霸主 Google Chrome 競爭的程度,所以就沒有太多人注意到這件事情...

過了幾個禮拜消息比較退燒後,被告以及反過來告的消息出來後,才注意到原來換了搜尋引擎... XD

旁邊搖旗吶喊沒什麼用,就拉板凳出來看吧...

Google 的 .search 網域

Netcraft 的「November 2017 Web Server Survey」這篇看到關於 Google 弄到的 .search 網域:

This month the controversial new .search gTLD being run by Google’s Charleston Road Registry subsidiary was found for the first time, with www.nic.search responding to the survey. Google hopes it will be able to run .search as a dotless domain which will automatically redirect users to their search engine of choice. This proposal has been criticised for going against ICANN’s own rules, which prohibits this functionality due to the potential for conflicts with existing names on internal networks. This feature could also cause confusion for users who have come to expect that typing words into their address bar will perform a search query for that term.

It is currently uncertain whether or not Google will be allowed to run the .search TLD as a dotless domain, however with the launch of the first site on this TLD this month Google is one step closer to the provision of this service.

找了一下資料,ICANNWiki 上的「.search」有些資料,另外也有新聞資訊 (2013 年的):「Google Wants To Operate .Search As A “Dotless” Domain, Plans To Open .Cloud, .Blog And .App To Others」。

不太妙...

MySQL 上的全文搜尋引擎:Mroonga

算是無意間翻到的資料,MySQL 上的全文搜尋引擎:「Mroonga」。

看起來後面主要是日本社群?從 2010 年就開始發展了,號稱 CJK 都支援,而且各大作業系統也都有預先包好的版本 (像是 Ubuntu 上有 PPA)。

雖然現在社群音量最大的應該還是 Elasticsearch,但看起來頗有趣的,對於只是想要架個小東西玩的專案,說不定是個有趣的方案?

在 MOPCON 2017 的 Unconference「MySQL to NoSQL & Search Engine」

把投影片傳到 Speaker Deck 上了:「MySQL to NoSQL & Search Engine」。

這是在介紹 noplay/python-mysql-replication 這個軟體,我在示範時用的 python script 有增加 blocking 參數讓他保持一直讀取 MySQL replication stream:

from pymysqlreplication import BinLogStreamReader

mysql_settings = {'host': '127.0.0.1', 'port': 3306, 'user': 'root', 'passwd': ''}

stream = BinLogStreamReader(connection_settings = mysql_settings, server_id=100, blocking=True)

for binlogevent in stream:
    binlogevent.dump()

stream.close()

利用這樣的工具可以做很多事情,像是當 post 表格更新時自動更新 search engine,並且清空 memcached 內的資料。這可以避免使用 library 時有可能會漏掉忘記做 (因為有些程式不用 library 處理),可靠度比較高。

另外一方面 replication protocol 本身就有考慮重連的問題,重新接上時是可以從上一次處理完的資料繼續處理 (只要不要隔太久),這讓寫應用的人不需要用太複雜的方式確保他不會漏掉。

Google 放棄對海外伺服器搜索票的抵抗了...

先前美國政府透過搜索票,要求各雲端廠商提供海外伺服器的資料而引起話題 (像是先前 Microsoft 往上打官司抵抗:「Does US have right to data on overseas servers? We’re about to find out」),而現在看起來 Google 打算放棄掙扎了:「Google stops challenging most US warrants for data on overseas servers」。

Google has quietly stopped challenging most search warrants from US judges in which the data requested is stored on overseas servers, according to the Justice Department.

Microsoft 這邊有些不錯的進展,成功在巡迴庭擋下:

Microsoft convinced the New York-based 2nd US Circuit Court of Appeals—which has jurisdiction over Connecticut, New York, and Vermont—that US search-and-seizure law does not require compliance with a warrant to turn over e-mail stored on its servers in Ireland.

不過沒看到 AWS 相關的消息,感覺不妙...

直接從 IMDb 編號看影片

看到「Now Anyone Can Embed a Pirate Movie in a Website」這邊介紹的東西,直接輸入 IMDb 的編號 (包括 tt 開頭的那串編號),他就自動拉出 embed code:

然後可以直接線上觀看:

然後還支援字幕 (唔):

Interestingly, should one of those sources be Google Video, Vodlocker says its player offers Chromecast and subtitle support.

官網有寫來源是到處找:

VoDLocker searches all general video hosters like youtube, google drive, openload...

看起來整塊技術其實都是現成的。透過 search engine 加上定期的檢查機制與回報機制就可以做完 @_@

Google 與 Facebook 都在建立消息驗證系統

Google 的在「Fact Check now available in Google Search and News around the world」這,Facebook 的在「Working to Stop Misinformation and False News」這。

Google 是針對搜尋與新聞的部份給出建議,透過第三方的網站確認,像是這樣:

後面的機制是透過公開的協定進行:

For publishers to be included in this feature, they must be using the Schema.org ClaimReview markup on the specific pages where they fact check public statements (documentation here), or they can use the Share the Facts widget developed by the Duke University Reporters Lab and Jigsaw.

但也是透過演算法判斷提供的單位是否夠權威:

Only publishers that are algorithmically determined to be an authoritative source of information will qualify for inclusion.

而 Facebook 是針對 Timeline 上的新聞判斷,但是是透過與 Facebook 合作的 partner 判斷,而且會針對判斷為假的消息降低出現的機率:

We’ve started a program to work with independent third-party fact-checking organizations. We’ll use the reports from our community, along with other signals, to send stories to these organizations. If the fact-checking organizations identify a story as false, it will get flagged as disputed and there will be a link to a corresponding article explaining why. Stories that have been disputed also appear lower in News Feed.

我不是很喜歡 Facebook 的方法,變相的在控制言論自由 (不過也不是第一天了)。

Archives