Romantic Robot Working on it.

10:10 Tue 5 Jul 2005

Googlebot: Proof

Filed under: Google — 10:10 Tue 5 Jul 05

Googlebot has free rein here, just like other bots.
It has crawled ~20meg of data.

Go search for “codingvisions” – Tamba2 is #1
Go search for “freshlypressed” – Tamba2 is #1
Go search for “tamba2” – this domain does not exist.

Both of those domains have a PR of 0, yet this blog has a PR of –

The links to those other two domains exist in only one place – here in the sidebar. Now if Google can crawl my site and put those two search results at #1, yet also deny that this domain exists that must surely show that my domain is blacklisted. I have asked Google for clarification (though I do not expect them to acknowledge anything and to use the bot as an excuse).Now … if they are taking my data but not using it, why should I let them ?


  1. It’ll probably take at least a month or two after you allow them in robots.txt to end up back in search results.

    Comment by Matt — 13:26 Tue 5 Jul 2005 @ 13:26 Tue 5 Jul 05

  2. And this is what I do not understand.
    They have been in, and all they have used is two links. Now if those two links did not get #1 in Google last week but they do today, why is my site not listed anywhere else at all ?
    If they want to take 20meg of data just to return 2 url’s, that seems just….. odd / wrong.

    Comment by Mark — 13:30 Tue 5 Jul 2005 @ 13:30 Tue 5 Jul 05

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by WordPress