Romantic Robot Working on it.

19:52 Fri 25 Mar 2005

Removing Google

Filed under: Google,T2 — 19:52 Fri 25 Mar 05

The robots.txt for the domain now only allows bots to index the /wordpress directory and subdirs of that (and a couple of other minor dirs). Good bots are banned from accessing every other directory. Careful – if you go looking at the robots.txt and start poking around where you should not, you could set a trap off, and I will ban every IP that comes through. So why have I banned the bots ?
It’s got little to do with my current attitude to Google, and more to wondering just how my blog (and other directories) benefit the world at large. I don’t believe they do, so why clutter ? I have had the #1 search result for “Mark”, “WordPress help” “XML Parse Error” “Enabled parking” and several others and yes, it’s cool the first time you see your site as #1 (that is a result from within your site – we can all create stupid domain names then go WOW! KOOL! as we get #1 in a field of … 1) but what does it DO exactly ? Apart from bandwidth, very very little. I carry no ads on the hidden pages, I have nothing unique to offer and as I have yet to discover and continue to read a blog found through a search engine, why should I think that being in the G’s results will benefit me then ?
If comments are then considered, I can recall about 6 comments that have been made by what I would call random visitors. Each blog that I read tends to have it’s own circle of commentors and again, why does being on google help this ? I do not see that it does.
So there was this thing that was costing me, yet providing no tangible benefit. Doesn’t make sense to me, so I closed the door.


  1. Phu wrote this @ 6:17 pm Mark recently decided to exclude his blog from being indexed by search engines and the arguments he puts forward have got me thinking about whether or not I should contemplate doing something similar. For me however, it

    Pingback by Fully Independent — 16:19 Sun 5 Jun 2005 @ 16:19 Sun 5 Jun 05

  2. […] dependent”>   Styles: [m] [s] [c] Saying goodbye to search engines Mark recently decided to exclude his blog from being indexed by search engines a […]

    Pingback by Saying goodbye to search engines - Fully Independent — 18:17 Sat 26 Mar 2005 @ 18:17 Sat 26 Mar 05

  3. So that why Bloglines stopped picking up you’re feed. ^_- That’s ok, it’s time for me to grow up and use a real feed reader.

    Comment by MacManX — 19:34 Sat 26 Mar 2005 @ 19:34 Sat 26 Mar 05

  4. […] Remember a few days ago I had a pop at Google because they were permitting someone to steal my content ? ( Google steals ) and I then followed that up by banning Google from all areas of my site except the WordPress help area ( Removing Google ), well, I read (through Feeddemon) an article on Joi Ito’s blog which also mentions Google. The NYTimes is reporting on the increasing amount of lawsuits which face the big G, with the article focusing on ad revenue. It IS good that companies are starting to realise that Google (and the others) are blindly taking whatever their bots get their hands on and they are then not caring what they do with what is in effect not even their property. Let’s say you take a picture, add a © to it (not that you need to do that) and put it on your site. If I come along, copy it and place it on my site you can action against me. Yet Google, by it’s very nature gives me access to more pictures and more choice, regardless of the legal status of an image. Go read the Google procedure on what to do if someone steals your content and sticks it on blogger – it is convoluted and designed purely to protect them from the inevitable furore. I probably sound like I’m whinging, but unless companies hammer through some form of protection, it’s bloggers like us ( and those with some ISP space for a gallery) who will find our images and our work in places we never meant it to be. Most people reading this will be savvy enough about the net, but do people new to this really understand that google can and will copy that image of the family around the world when all they wanted to do was show a distant relative ? Aren’t the search engines at some level guilty of at least not being straight ? And you just know that Google will decide that some existing tag will be used to demarcate intellectual property don’t you ? They won’t help at all. And yes, I am perfectly aware of the new Yahoo engine – – and as a source of guilt / lawsuit / full copyright free work it’s unreliable. » Blogroll Dive: 3/28/05 ¤ Read (192) […]

    Pingback by What makes you happy ? » More on Google — 15:15 Mon 11 Jul 2005 @ 15:15 Mon 11 Jul 05

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by WordPress