Lots of reports are out today about how effectively Dell is using social media marketing and especially Twitter to generate revenue, $6.5M in sales are the headlines, I wonder what that translates to in margins.
However yesterday I read a post ranking cloud computing vendors based on mind share using a points system based on Google ranking for the term [cloud computing]
The winner was Rackspace, but we have some idea how they achieved that
But I was more interested in the result for Dell
First off they are doing some funny redirects so the displyed URL in the snippet
www.dell.com/cloudcomputing
Redirects to their cloud computing page on a subdomain which I assume is some kind of CDN
http://content.dell.com/us/en/enterprise/cloud-computing.aspx?c=us&cs=555&l=en&s=biz
Notice that there is only a URL displayed in the snippet and not an excerpt from the page or meta description. Google is just displaying the page as a reference, but for some reason couldn’t crawl the page.
Now before I show you what they have done wrong I did try to contact Dell about this yesterday. I sent a tweet to the guy who maintains at least one of their social media profiles, and since this tweet he has been tweeting on his account so I assume he has had a chance to read the tweet I sent.
I didn’t receive a response, and as this is a significant problem I am blogging about it, mainly because Google seemed to be doing something contrary to what I previously understood (they weren’t if you read more) and have seen written by authoritative sources such as Vanessa Fox.
Here is the robots.txt of that Dell subdomain
Allow: /at/de/home/
Allow: /au/en/business/
Allow: /ca/en/business/
Allow: /de/de/corp/
Allow: /de/de/home/
Allow: /hk/en/business/
Allow: /ie/en/business/
Allow: /ie/en/home/
Allow: /in/en/business/
Allow: /my/en/business/
Allow: /nz/en/business/
Allow: /sg/en/business/
Allow: /uk/en/business/
Allow: /uk/en/home/
Allow: /us/en/business/
Allow: /us/en/corp/
Allow: /us/en/enterprise/
Allow: /us/en/home/
Allow: http://content.dell.com
Allow: http://content.dell.com/
Allow: http://content.dell.com/au.sitemap.txt
Allow: http://content.dell.com/hk.sitemap.txt
Allow: http://content.dell.com/in.sitemap.txt
Allow: http://content.dell.com/jp.sitemap.txt
Allow: http://content.dell.com/my.sitemap.txt
Allow: http://content.dell.com/nz.sitemap.txt
Allow: http://content.dell.com/sg.sitemap.txt
Disallow: /
The page concerned us under /us/en/enterprise/ so should be controlled by this directive.
Allow: /us/en/enterprise/
If conflicts exist in the file, the robot obeys the longest (and therefore generally more specific) line.
So somthing is going wrong in between the URL displayed in the search results, and the final landing page.
Server Header Analysis
Search result
>> [302] http://www.dell.com/cloudcomputing
>> [302] http://content.dell.com/cloud-computing.aspx?c=us&cs=555&l=en&s=biz
>> [301] /us/en/enterprise/cloud-computing.aspx?c=us&cs=555&l=en&s=biz
GET /cloudcomputing HTTP/1.1
Host: www.dell.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: UTF-8,*
Keep-Alive: 300
Connection: keep-alive
Referer: http://www.google.com/search?hl=en&pws=0&gl=US&q=cloud+computing&start=20&sa=N
Cookie: SITESERVER=ID=6596edc940b547a0afe7a6a1db4e40ee; s_vi=[CS]v1|258EA06C05011703-600001124062905C[CE]; StormPCookie=pl=pl&pc=pl&bandwidth=NA
HTTP/1.x 302 Found
Content-Type: text/html; charset=utf-8
Location: http://content.dell.com/cloud-computing.aspx?c=us&cs=555&l=en&s=biz
Server: Microsoft-IIS/7.0
Set-Cookie: stop_mobi=; path=/
X-Powered-By: ASP.NET
x-ua-compatible: IE=7
Date: Tue, 08 Dec 2009 19:42:34 GMT
Content-Length: 196
http://content.dell.com/cloud-computing.aspx?c=us&cs=555&l=en&s=biz
GET /cloud-computing.aspx?c=us&cs=555&l=en&s=biz HTTP/1.1
Host: content.dell.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: UTF-8,*
Keep-Alive: 300
Connection: keep-alive
Referer: http://www.google.com/search?hl=en&pws=0&gl=US&q=cloud+computing&start=20&sa=N
Cookie: SITESERVER=ID=6596edc940b547a0afe7a6a1db4e40ee; s_vi=[CS]v1|258EA06C05011703-600001124062905C[CE]; StormPCookie=pl=pl&pc=pl&bandwidth=NA
HTTP/1.x 301 Moved Permanently
Date: Tue, 08 Dec 2009 19:42:35 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: /us/en/enterprise/cloud-computing.aspx?c=us&cs=555&l=en&s=biz
Cache-Control: private
Content-Length: 0
http://content.dell.com/us/en/enterprise/cloud-computing.aspx?c=us&cs=555&l=en&s=biz
GET /us/en/enterprise/cloud-computing.aspx?c=us&cs=555&l=en&s=biz HTTP/1.1
Host: content.dell.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: UTF-8,*
Keep-Alive: 300
Connection: keep-alive
Referer: http://www.google.com/search?hl=en&pws=0&gl=US&q=cloud+computing&start=20&sa=N
Cookie: SITESERVER=ID=6596edc940b547a0afe7a6a1db4e40ee; s_vi=[CS]v1|258EA06C05011703-600001124062905C[CE]; StormPCookie=pl=pl&pc=pl&bandwidth=NA
HTTP/1.x 200 OK
Cache-Control: private,max-age=0
Date: Tue, 08 Dec 2009 19:42:36 GMT
Content-Type: text/html; charset=utf-8
Expires: Wed, 01 Jan 1997 12:00:00 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
X-Awesomed-By: Thundera RE-TJN
Set-Cookie: lwp=c=us&l=en&s=biz&cs=555; domain=.dell.com; path=/
Set-Cookie: dus=ci=cloud-computing&th=Default; path=/
Content-Encoding: gzip
Vary: Accept-Encoding
Transfer-Encoding: chunked
Problem – Robots.txt Disallow Match
The inbetween URL redirect is blocked by robots.txt
http://content.dell.com/cloud-computing.aspx?c=us&cs=555&l=en&s=biz
It is actually matched by
Allow: http://content.dell.com/
Disallow: /
Google has decided to take the most restrictive version, to disallow the URL.
I haven’t checked how many links are pointing to that URL both from internal and external sources, but I would guess it is a fair amount and I am assuming at this time that is a page they would want to rank as in their line of business it is probably quite a competitive term.
At a guess without that one URL blocked by robots.txt, they would have been much higher on the comparison charts and ranking higher for cloud computing – most likely that is a 1st page result thrown away by robots.txt
I haven’t checked how many other key landing pages are blocked in a similar manner – I bet it is quite a few
This is just chalking up one more reason why robots.txt should be avoided as it is so easy to mess things up.
p.s. My niece is 18 in 2 months & I am sure she would love a Dell
Tags: 301 redirect, dell, http header, robots.txt, SEO Blog
Related posts
- Are Blogger | Blogspot Blogs SEO Friendly? (83)
- SEO Linking Gotchas Even The Pros Make (98)
- Optimizing HTML Links In The Aftermath Of A Blog Storm (25)
- Wordpress SEO Masterclass For Competitive Niches (131)
- Wordpress Plugin: Nofollow Those Dupes (Wordpress SEO Masterclass Continued) (13)
- Wordpress Plugin: Antisocial (24)
- WordPress Plugin – Custom Query String – UTW Edition (31)
- WordPress Hacked? Total Security Lockdown (4)
- Wikipedia NoFollow Plugin? WikiDigg? (30)
- Wikipedia NoFollow Plugin (4)