A year or so ago, Google dropped a bomb on all website designers, publishers and online content authors: Your websites had better not only be SSL-secure, but also “mobile-friendly” — and while the first edict is easy to solve with money, the second command costs you a lot of time and money and energy — especially if you’ve been publishing live content on the web for a long time.
Yesterday, while I was editing Gordon Davidescu’s fine article about Mitt Romney appearing on television in Brownface — I had a devil of a time trying to find an article I swore I wrote a long while ago about Ted Danson appearing in public in Blackface while he was dating Whoopi Goldberg. I wanted to link that article in Gordon’s article.
I can usually find anything I’ve written on the internet in a refined Google search — though it’s harder now that all the search engines have re-tuned their return results to reflect newer articles instead of older artifacts — but I could not find any article like that in Google. I also searched my online Google Drive, too. Nothing. I searched all my local hard drive archives. Nothing. I still couldn’t let go of the notion that I’d written about Ted’s Blackface.
As a last gasp effort, I decided to give Microsoft Bing a try — just to see if they held something I hadn’t been able to find that might provide a breadcrumb of a link back to my Danson in Blackface article that didn’t seem to exist anywhere.
When I landed on the Bing homepage, I saw the “Bing it On” challenge I’d heard about en passant, and I decided my missing Whoopi Goldberg and Ted Danson Blackface article was a perfect test of the two search engine giants to see if they could help me find an article that didn’t appear to exist.
In March 2011, I read a story online about the debut of Greplin, a new way of indexing the online nuggets of your life into a fast and easy to sift-through search interface. I immediately signed up for the service on the first payment tier because I believe in paying to support what you love, and I really loved Greplin — for the first day or so — until I began to realize I had a lot of stuff to index and I was already running out of space on the service even thought I just got started.
It started with a simple request — “Sir, could you please stand over there.” — and ended in a gunshot.
Continue reading → It Ends in Sheep: Ten Sentence Story #104
UPDATE #2: September 6, 2010
At noontime Eastern, I heard from Naoko again and she confirmed robots.txt has been upgraded site-wide on WordPress.com!
I had to change the privacy settings on all my blogs — and then back again to “public” — to force the new robots.txt file to update. The plan worked.
All 13 public blogs are now set and updated and ready for Google and the rest of the indexed search world to remove our proprietary Movable Type search results.
Here’s a screenshot of the new robots.txt file disallowing the “/cgi-bin/” directory. I highlighted the new addition:
Yay, WordPress.com!
THANK YOU from prying us from the rock!
UPDATE:
The moment I published this article today at 12:24pm Easter time, I followed up with WordPress.com support and gave them the link to this article in an attempt to better explain — with screenshots — the problem I was trying to solve.
At 2:12pm — less than two hours after I wrote to WordPress.com — Naoko replied:
Hi there,
I was waiting for this to actually go live, but a change has been made in our code.
User-agent: *
Disallow: /cgi-bin/Will be added to robots.txt (not visible yet, I need to check back with the developer).
Fantastic news! That solves my proprietary Movable type search results problem across all 13 of my public WordPress.com blogs! Here is my reply:
Hi Naoko!
Oh, that’s great news! Is this change on a per-blog basis, or is it site wide?
If it’s side wide, are there plans to include robots.txt proprietary search disallows for the other blogging services?
I will update my article to reflect the information you provide.
Thanks!
Best,
db
I will keep you updated!
I don’t see the “/cgi-bin/” disallow yet on any of my blogs in robots.txt, but the moment it goes live, I will go back to Webmaster Tools and specifically ask that the “/cgi-bin/” directory be removed now and forever from all my blogs.
As well, because of this robots.txt disallow addition, I will now be able to effectively venture into Yahoo! and Bing to see if I can get the same directory deleted in those services for all my blogs.
Thank you WordPress.com Gods!
ORIGINAL ARTICLE:
I recently discovered a terrible Movable Type artifact that still remains festering and alive within me — via Google Search Returns — six months after I became a Six Apart refugee and gave up my expensive, self-hosted, standalone, blog hosting and returned to my first blogging home: WordPress.com. You can see an example of the problem below in the third search return in the screenshot. That “Memeingful: Search Results” link takes you to a proprietary Movable Type search return that has been dead for six months. Click on that link, and you’ll be taken to a “Not Found” error page on WordPress.com.
Continue reading → Dead Search Returns: Caught Between a WordPress.com Rock and a Google Hard Place
Are we forever at the end of the Know-It-All? You know the sort of person I’m describing, right? The kind who always know the right answer to everything when asked and, if not asked, will offer up blind bits of trivia on their own accord from the dark recesses of history while simultaneously adding living citations and quotes from books, politicians and serving up sharp shards of microinformation that only encyclopedia writers need to know.
We are quite familiar with the young woman who had her laptop “killed” with three bullets by Israeli security forces on a recent trip to Israel — but are you aware there is still a Bush-Era standing search and data
seizure policy that gives any U.S. Customs official or Border patrol agent the right to take your laptop and papers without a warrant or even a suspicion of illegality?
Continue reading → No Warrant Required for Seizing Your Laptop
You must be logged in to post a comment.