GoogleとIf Modified Since

GoogleとIf Modified Since

『GoogleのクローラGooglebotは訪問したページが更新されたかどうかをサーバの返すIf Modified Sinceヘッダによって判断する。だからIf Modified Sinceが適切に応答して更新したことをGooglebotに知らせてくれれば、あなたのWebページにも更新に応じて訪問するようになる』ということでサーバがIf Modified Sinceに対応していることは大切だよというのがGoogleGuyのお話です。

I wanted to urge people to configure their server to support the "If Modified Since" header? Why should you care about "IMS"? When a smart spider like Googlebot comes around, IMS lets you tell the spider that a page hasn't changed. Then Googlebot can use the old copy of the page. That frees up the bot to download more pages while saving bandwidth. Because of the bandwidth savings, IMS hits are almost "free" in terms of server load. Plain apache can serve _lots_ of IMS queries per second before slowing a machine down.

IMS can work for dynamically generated pages too. Someone posted how to do it for PHP-generated pages, for example. The bottom line is that if your server supports IMS correctly, you can tell Googlebot about more pages without as much server load or bandwidth on your part. As Google crawls more often to make the web a fresher place, adding this flag will help you and search engines.
[GoogleGuy, "Are you using If Modified Since?", WebmasterWorld, 5:57 pm on Oct 8, 2002]