Hi Rob I thought you may find this interesting.
RewriteCond %{THE_REQUEST} ^(GET|HEAD) /([^/]+/)*index.(php|html|htm) HTTP
RewriteRule ^(([^/]+/)*)index.(php|html|htm)$ http://website.com/$1 [R=301,L]
RewriteCond %{HTTP_HOST} ^www.website.com [nc]
RewriteRule (.*) http://website.com/$1 [R=301,L]
The first rule will redirect index.php, index.html, and index.htm to ./
The second rule will redirect xxhttp://www.website.com to http://website.com
This code works well but the Googlebot does not like it. See following error message:
URLs not followed
When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot because they contained too many redirects. Please change the URLs in your Sitemap that redirect and replace them with the destination URL (the redirect target). All valid URLs will still be submitted.
I also found this on Google Groups Webmaster Help:
Adam
> > The requirements are specified clearly.
> > All URLs in your sitemap and all internal URLs on your site should be
> > to the correct final destination without ANY form of redirection.
> > Why not just do that?
> > THe phrase “some URLs were not accessible to Googlebot because they
> > contained too many redirects,” should be interpreted to mean “… not
> > accessible and/or less accessible and/or need correction to be
> > consistent and not waste resources for crawlers and for your
> > server…”
> > I guess if you checked in at a hotel and they sent you to Room 301 and
> > there you found a note saying go to Room 607 and there a note saying
> > go 301a it could be argued that a room was still accessible to you but
> > it would still be bang out of order to mess people around like that.
> > You’d be dischuffed wouldn’t you?
> > Robbo
Dischuffed??? Thats a new one. Needles to say I will be removing the code supplied.