Page 2 of 6
UPDATE #2: September 6, 2010
At noontime Eastern, I heard from Naoko again and she confirmed robots.txt has been upgraded site-wide on WordPress.com!
I had to change the privacy settings on all my blogs — and then back again to “public” — to force the new robots.txt file to update. The plan worked.
All 13 public blogs are now set and updated and ready for Google and the rest of the indexed search world to remove our proprietary Movable Type search results.
Here’s a screenshot of the new robots.txt file disallowing the “/cgi-bin/” directory. I highlighted the new addition:
THANK YOU from prying us from the rock!
The moment I published this article today at 12:24pm Easter time, I followed up with WordPress.com support and gave them the link to this article in an attempt to better explain — with screenshots — the problem I was trying to solve.
At 2:12pm — less than two hours after I wrote to WordPress.com — Naoko replied:
I was waiting for this to actually go live, but a change has been made in our code.
Will be added to robots.txt (not visible yet, I need to check back with the developer).
Fantastic news! That solves my proprietary Movable type search results problem across all 13 of my public WordPress.com blogs! Here is my reply:
Oh, that’s great news! Is this change on a per-blog basis, or is it site wide?
If it’s side wide, are there plans to include robots.txt proprietary search disallows for the other blogging services?
I will update my article to reflect the information you provide.
I will keep you updated!
I don’t see the “/cgi-bin/” disallow yet on any of my blogs in robots.txt, but the moment it goes live, I will go back to Webmaster Tools and specifically ask that the “/cgi-bin/” directory be removed now and forever from all my blogs.
As well, because of this robots.txt disallow addition, I will now be able to effectively venture into Yahoo! and Bing to see if I can get the same directory deleted in those services for all my blogs.
Thank you WordPress.com Gods!
I recently discovered a terrible Movable Type artifact that still remains festering and alive within me — via Google Search Returns — six months after I became a Six Apart refugee and gave up my expensive, self-hosted, standalone, blog hosting and returned to my first blogging home: WordPress.com. You can see an example of the problem below in the third search return in the screenshot. That “Memeingful: Search Results” link takes you to a proprietary Movable Type search return that has been dead for six months. Click on that link, and you’ll be taken to a “Not Found” error page on WordPress.com.
United States Cyber Command is now in action — proactively protecting you by thwarting the ethereal outsider threats from terrorists, rogue actors, and other nations wishing to do us harm — but that lockdown protection travels two ways: In and out. We know the in — but the out is you — and your local internet connectivity.
See the Black Man in the top image? Ooops! Now you don’t see him in the same image below. Both images are Microsoft advertising. The grinning “Black Man” is for the American market, while the goofy “White Faced Dude with the Black Hand” is for advertising market in Poland.
I’m not a tremendous fan of Microsoft Surface. The technology feels 20 years old to me and if I want to move things around in space in time, I want to do it in a 3D hologram and not on my kitchen table.
Total Information Awareness is now possible thanks to Microsoft and the NSA’s new Texas Cryptology Center in San Antonio, Texas:
“No longer able to store all the intercepted phone calls and e-mail in its secret city, the agency has now built a new data warehouse in San Antonio, Texas,” writes author James Bamford in the Shadow Factory, his third book about the NSA. “Costing, with renovations, upwards of $130 million, the 470,000-square-foot facility will be almost the size of the Alamodome. Considering how much data can now be squeezed onto a small flash drive, the new NSA building may eventually be able to hold all the information in the world.” …