February 03, 2011
How To

Search Marketing: How to avoid and remove Google penalties

SUMMARY: Everyone has heard of it and everyone dreads it -- a Google penalty. Honest marketers can be caught off guard when their search results disappear and traffic plummets, but there are ways to avoid penalties and ways to fix them.

Find out how to discover if you’ve been penalized, why a \"Google penalty\" might not be a penalty at all, and what to do if you have one on your hands. Includes links to additional resources and advice from a leading SEO expert.
by Adam T. Sutton, Senior Reporter

Telephone directory website Phonebooks.com worked to improve its reputation and website traffic last year, but its natural search traffic took a nosedive overnight.

Aaron Rosenthal, President, Phonebooks.com took a closer look at the numbers. Google previously brought about 76% of his natural search traffic, but it had dropped to nearly zero.

"We have had Web server problems and so forth before, so we weren't exactly sure what the problem was at first," Rosenthal said. "After running a few searches we found that the only search [in Google] that we could still be found on page one [the first Search Engine Results Page, or SERP] for -- or on any page other than four, five or six -- was for our domain name including the '.com,'" says Rosenthal.

Rosenthal was shocked. After doing some research, he suspected Google had penalized Phonebooks.com and buried its search results. Unsure of the exact problem and how to fix it, Rosenthal sought the advice of experts and worked to find a solution.

Below are eight tactics for marketers who find themselves in a similar situation -- and for marketers who want to avoid the ordeal. The tactics come from Rosenthal's experience, as well as advice from Barry Schwartz, CEO, RustyBrick. Schwartz is also the founder of Search Engine Roundtable and News Editor at Search Engine Land.

Tactic #1. Routinely check site metrics

Rosenthal first uncovered this problem by looking at Phonebooks.com's site metrics. This tactic is so vital that it should go without saying -- but we're going to say it anyway.

Always check your site metrics. Look for large deviations in traffic, conversion rates, time spent, etc. Investigate any large and sudden changes that you cannot explain. This will help uncover a range of problems, whether they stem from ad campaigns or affiliate partnerships.

In Rosenthal's case, a huge decline in natural search traffic from Google signaled the problem. Google will sometimes alert companies to penalties or other major changes, Schwartz says, but Rosenthal's team did not receive an alert.

Tactic #2. Do not immediately send a 'reconsideration request'

A common reaction is to sign into Google Webmaster Tools and submit a "reconsideration request" (see useful links below). This is a note to Google explaining the situation and why Google should remove the penalty.

Submitting this request before you've researched and fixed the problem is a mistake:

- Problem might not be a penalty

There are a range of technical issues that can prevent Google from listing a site (which we'll describe below). Google's operators cannot solve these problems -- only you can.

- Penalized issues need to be resolved

Marketers who receive penalties need to submit reconsideration requests, but only after they've identified and resolved the causes. Again, there are a range of possible reasons Google would penalize a site (which we'll describe below).

Submitting a request too soon is a waste of time. You will not gain any additional information about the cause and receiving a response can take weeks.

Tactic #3. Check if Google can access your site

Once you've noticed a sharp drop in Google traffic, it's time to investigate. First, check if Google can access and index your site. Here are two ways:

- Site-specific search

Search for all the pages Google has indexed for your site by typing "site:YOURURL.com" into the search engine. If you see a healthy list of your site's pages (MarketingSherpa has more than 33,000), then Google is likely accessing and indexing your site.

If no pages are listed, then Google is not listing your site. This is likely because Google can no longer access your site for technical reasons on your end, or Google might have de-listed your site as a penalty.

- Check for crawl errors

By signing into your Google Webmaster account, you can check if Google's search-indexing spider, Googlebot, reported any errors when attempting to crawl your site. You can access the errors page by signing into your account and navigating to:
o Webmaster Tools Homepage
o Select your site
o Under "Diagnostics," click "Crawl errors"

This tool will list the URLs Google had trouble crawling as well as the type of problem and sometimes the pages containing the errors.

Tactic #4. Investigate common technical issues

The steps above will hopefully point you in the right direction if a technical issue is your problem. Otherwise, here are some common technical issues that cause penalty-like situations:

- Duplicate content

Duplicate content -- or content that is nearly identical on two different pages of the same site -- typically does not earn a penalty. Google will choose which page to list and suppress the other(s).

Some marketers with suppressed pages have interpreted this as a penalty, but Google does not see it that way. There are several ways to signal to Google which page among duplicates you want listed -- which Google calls "canonicalization" (see useful links below).

[Note: If Google believes the content's intent is to deceive visitors or manipulate search engine results, a penalty may be administered.]

- Metadata: 'no-index' tags

Google will completely drop a page from its search results if it has a simple meta-tag (see useful links below) in its HTML code, such as:

< meta name="googlebot" content="noindex" >

- Mistakes in robots.txt file

Site administrators can use a "robots.txt" file to signal to search engines that a site or some of its pages should not be indexed. Mistakes in this document can block you from Google search results (see useful links below).

- Others technical issues

There are a range of other possible technical reasons your natural search traffic has plummeted. You may have accidentally password-protected your homepage, or your site might be blocking Google's IP addresses.

Check the useful links below for Google's list of technical guidelines.

Tactic #5. Identify the type of penalty

If Google is not having trouble accessing your site and you've noticed a huge drop in traffic -- you may have a penalty. Now you need to determine the exact nature of your penalty.

Here are three good areas to check:

- Check rankings for keywords

Search for your website in Google using your top keywords and note any large drops in your rankings. Everything might appear normal for some keywords. On others, your page-one SERP ranking might have fallen to page five or 10.

If you notice that your results have been buried for certain keywords, your team should look into all SEO work done targeting those keywords and determine if any less-than-honest tactics were used. Be sure to check the inbound links that have anchor text using those phrases.

You might notice that all of your results are buried. Rosenthal, for example, noticed that his team's results were buried for all keywords except for searches for "phonebooks.com".

- Look for missing URLs

Google might have blocked certain URLs completely from its index. Conduct a "site:" search in Google for your website and check if your high-traffic pages are being listed. If you cannot find some, they've likely been removed due to a violation of Google's guidelines. Identify the problem, fix it, and submit a reconsideration request.

- Check site security

Google will remove websites from its index that have been hacked. If your entire site is not showing in the search results, take a look at your site security and talk to your site administrators. Google may alert your company if the site is removed for this reason.

Tactic #6. Fix everything that violates quality guidelines

Once you've found the pages or keywords in question, you need to determine what is wrong with them and fix them as soon as possible. Google publishes quality guidelines in addition to its technical guidelines. Many prohibit black-hat SEO tactics, such as:
o Using hidden text
o "Cloaking" or using sneaky redirects
o Keyword stuffing (loading a page with irrelevant terms)
o Participating in link schemes, such as link purchasing or excessive reciprocal linking

Other prohibited areas include installing viruses, running phishing scams and creating sites with little or no original content (see useful links below).

Using any of these tactics is an extremely bad idea. First, they create a poor user experience and undermine customers' trust in your company. Second, such tactics earn penalties from Google.

All preaching aside, if your company is or has ever used such tactics, you should stop immediately and remove them from your website. Only when you've completely cleared out the bad apples can you submit a reconsideration request.

- Inbound links are hard to fix

If a company has earned a penalty from Google for one of the above tactics, "typically it's links," Schwartz says. Google has determined that it doesn't like either the links you're pointing to from your site, or the links pointing into your site.

Most of the black-hat tactics listed above are controlled by a website's operators, which makes them relatively easy to eliminate. Inbound links, however, are hosted on other sites and can be difficult to remove.

If Google slaps your site with a penalty for excessive unnatural inbound linking, you need to methodically identify the links and ask that they be removed.

Tactic #7. Visit Google Webmaster Help Forum

The solution to a Google penalty is not always clear and the situation can be distressing. Be sure to reach out to your industry contacts and get a few experts to review your site.

Also, if you're stumped, Schwartz suggests posting to the Google Webmaster Help Forum (see useful links below). Describe your situation and provide as much relevant information as possible.

- Be completely honest

"You really can trust the people [there]," Schwartz says. "They do provide really good advice from their perspective. But really, try to be as honest and as upfront with these people as possible. The more honesty you give them, the more honesty they will give back."

Look for comments from Google employees and from "top contributors" who have been identified as providing sound advice by Google.

Tactic #8. Find other sources of traffic

As of publication, Rosenthal's team has not been able to solve PhoneBooks.com's problem. Although the experience is a major blow to the site, Rosenthal has learned from the experience.

"If the bulk of your business comes from one traffic source, it's time to diversify," he says. "Diversify before something happens to all that traffic."

"I can't say that everything that has come out of the situation is bad," he says. "We have been trying new avenues, whether PR, other advertising or just other natural ways of getting the word out, and we have had some success."

Useful links related to this article

Subscribe to the MarketingSherpa Inbound Marketing newsletter

Sherpa Chart: SEO Tactics for the B2B Marketer

Google Caffeine: Use social media and quality content to get a jolt for your site

Google Webmaster Guidelines: Design, technical and quality guidelines

Google Webmaster Help Forum

Google Webmaster: Requesting reconsideration

Google Guidelines: Duplicate content and "canonicalization"

Google Guidelines: Little or no original content

Google Guidelines: Link schemes

Google Blog: Demystifying "duplicate content penalty"

Google Webmaster Blog: Dealing with low-quality backlinks

Using meta tags to block access to your site

About robots.txt



Search Engine Roundtable

Search Engine Land

Improve Your Marketing

Join our thousands of weekly case study readers.

Enter your email below to receive MarketingSherpa news, updates, and promotions:

Note: Already a subscriber? Want to add a subscription?
Click Here to Manage Subscriptions