(UPDATE:  UrbanSemiotic.com is no longer hosted on WordPress.com.  We now run on Movable Type 4.1.  We are keeping this article in publication to preserve the record.)

Over the past few days I have noticed something curious happening with this Urban Semiotic Blog and Google.
For some strange reason a lot of our articles are no longer being returned in a basic Google web search and that is killing our readership because Google usually sends a lot of traffic our way.

Those articles used to appear in Google — they still appear in Yahoo! and Microsoft Live Search as the top returns — but they have been removed from Google view. We’ll see if this article you’re reading right now ever appears in Google and remains there or if, it too, eventually mysteriously disappears from Google’s search returns.


Yesterday, I sent the following inquiry to WordPress.com support:

I am doing some searches on published articles on my
blog and I am not finding them in a Google search. They used to appear
there. They still appear in Yahoo! and MSFT searches.
Is WordPress.com in any way editing/blocking/content spidering by
disallowing Google access to certain posts with tags/topics/categories
that you — or AdSense — find objectionable?

The articles that are missing deal with death, murder,
censorship, critical articles, killings and child abuse, etc. You can
see some of the articles in the general Google tag indexing but the
full articles themselves are missing and I’m wondering why.

I received a crisp reply from WordPress.com that they do not censor content from Google Searches.
I followed up with this reply:

I appreciate your response, however, my experience is
proving very interesting in the ongoing discovery of my blog articles
that have been deleted from only Google Search returns and it seems
they are being removed because of specific title words and
WordPress.com tags and categories.
While I realize there is no such thing as a coincidence — does your
AdSense deal with Google allow them to disallow placing ads on content
they find offensive for their advertising program?

If so, is it then
equally possible that your deal with Google AdSense and the special
server Matt spoke about online, is silently and automatically
blocking/blacklisting content for inclusion in Google search returns by
first blocking it from having AdSense placed?
Is there any way to opt out of AdSense and then force Google to do a
full, unbiased, non-advertising-based, reindexing of my blog?

I am working directly with Google to push back every single article
that has been deleted from their system. I just hope it doesn’t all
disappear again without notice. Articles critical of Bush and Iraq also
seem to be disallowed by default as well as any articles that are
critical of WordPress.

I was then asked by WordPress.com support to provide examples of what I was seeing.
I did and I was told by another person in support that what I was seeing was not what they were seeing.
So, for the record, here are the screenshots I took to document the
Search Returns issue and I have since deleted Google Desktop Search
from my computer in case that was causing strange returns.

There was no
change in the Google search returns with that local engine removed from
my system. I also did these search tests on other computers with the
same results.
You may get a right return now from Google because over the past few
days I have been slowly working with them to add back — a URL at a
time — the articles that were removed to Google’s search returns.
In the first search example — “Scobleizer Found Dead in His Grave” —
Google returned the following.

Notice the strange article links to the
WordPress.com support forum and tags/category listings on
WordPress.com, but there is no longer any direct link to the article
proper on the Urban Semiotic URL:

Here’s Yahoo’s return on the same search term and the first return links directly to my article proper:

Here’s the Microsoft return and the first return links directly to my article:

Strange,
eh?
Let’s continue now with this search phrase — “Making Published Posts
Private” — and again from Google we get links to the WordPress.com
support page where the article is linked and the WordPress.com tag
system:

Here’s Yahoo:

Here’s Microsoft:

Finally,
here’s a search example for an article Winslie Gomez wrote for Urban
Semiotic from a Google search on — “If War is Human” — and here’s the
return and again the WordPress.com tag system is featured and a direct
link to the article does not appear:

Here’s Yahoo:

Finally, Microsoft:

What
do you make of these odd Google search returns?
Are you seeing the same thing on your end when try these searches or
have these missing files finally been returned to the Google search now
after I pressed them back into the Google pool?

This morning we suddenly have a slew of new comments from new readers
again — always a sign of good Google indexing — so maybe something
broken was fixed?
Perhaps something set askew was indeed set right?

3 Comments

  1. UPDATE:
    I discovered yesterday, March 29, 2008, that this article was deleted from my WordPress.com hosting. Based on the title of this piece, “Urban Semiotic Articles Deleted From Google?” makes the irony even more painful.
    I did not delete the article and I am the only one on the Urban Semiotic staff who has the backend power to remove or even change published articles.
    I have republished the article with the original date so the URL will still remain alive and linked to other sites. We will see if it stays published on WP.com or not.
    In the interest of preserving the record — and all 34 comments posted on the original publication — I was able to pull the original article out of the Google Cache! Irony sings again!
    Here is the original article — the Google Cached Version:
    http://bolesbooks.com/wp-deleted-urb.html
    Here is a PDF file of that Google cached article -– it not as pretty as the HTML version, but it is more “portable” as the file format suggests:
    http://boles.com/wp-deleted-urb.pdf

  2. Urban Semiotic Moved from WordPress to Movable Type

    Hello, and welcome to Urban Semiotic!  This blog is now running — racing, really — on Movable Type 4.1 hosted by pair Networks! This blog used to run on WordPress and then on WordPress.com and you may be wondering why…

Comments are closed.