A solid keyword list is the foundation of a successful search-marketing strategy. Marketers must continually work to find relevant search terms and understand the volume of traffic they provide -- as well as the number of competitors using the same phrases -- to shape their SEO and pay-per-click strategies.

A variety of online tools are available to uncover new keywords, supplement existing lists with related terms, and gain insight into competitive factors. Each can influence the effectiveness of certain phrases in a PPC or SEO campaign. And each one approaches the task differently through a range of features, analytical capabilities, data sources and pricing options.

We spoke to several search marketers and search agency specialists to assemble this guide to nine of the most popular online keyword research tools. The list breaks down each tool’s research functionality, analytical capabilities and pricing, plus tasks for which each tool is best suited.

Before we get started, here are five takeaways on how to incorporate keyword-research tools into your list-building and bidding strategy:

-> Takeaway #1. Tools can’t replace your own research

You can’t build a search campaign solely on the terms generated by a keyword research tool. Your competitors are probably using the same tools as you are using, and can find the same terms.

Instead, you need to use those tools to augment the proprietary keyword research you’re performing for your company or clients:

- Look carefully at your own product names, popular terms used in your website copy and page titles, and internal site search queries terms.
- Study existing marketing materials, press releases and other content for important industry terms.
- Conduct your own online research on user forums, blogs and social media sites for terms and topics that users are talking about.
- Perform your own competitive analysis of how other companies are positioning their sites and search ads.

After completing your research, use a keyword research tool for the following tasks in generating your overall list:

- Starting a keyword list for a new product or new ad campaign.
- Seeing additional, related terms to supplement your internally generated lists when you’ve exhausted your own creativity.
- Checking your existing list for estimates of search volume and competition for PPC and organic placement.
- Gaining insight into unusual search syntax based on historical queries, which can help you generate new search phrases.

-> Takeaway #2. Data isn’t 100% accurate

The reliability of data is the most common caveat we heard from marketers who use online keyword-research tools.

The data these tools generate is typically only a snapshot of overall search activity, which depends on:
o Sample size
o Source of the data (one search engine vs. several; ISP data vs. proprietary Web scraping tools, etc.)
o Frequency of update of data

“Volume estimates should always be taken with a grain of salt,” says Greg Hayes, Online Marketing Specialist, G.A.P Adventures. “They could be low or high depending on the day they were collected.”

Data accuracy can be a bigger issue as your search criteria become more granular. For example:

- Niche terms and long-tail keywords that have lower search volume may be less well represented in keyword databases. That was an issue for Bill Shaughnessy, Director, Online Marketing, EDGAR Online, as he conducted keyword research for long-tail financial search queries.

“The challenge I faced was with some of those less popular, less searched terms that an equity analyst might use, such as ‘derivative’ or ‘derivative models,’” says Shaughnessy. “I thought that [the tools] were lacking, because they kind of led you back to broader-based terms that I already had.”

- City-specific or regional data for geotargeting might not be well represented in keyword databases due to the lower volume of that search activity captured by the tools.

“There’s not enough data to make it statistically relevant,” says Melissa Mackey, Online Marketing Manager, Fluency Media.

-> Takeaway #3. Search advertising cost estimates are directional at best

Several tools will give you estimates for the average cost per click companies are paying on top keywords. But that data can’t tell you exactly how much you can expect to spend on a particular ad campaign.

For starters, many cost-estimating features only provide a cost index, showing whether the term in question typically commands a high, medium or low cost per click. Tools that provide actual price estimates typically display results in a range -- for example, $0.50-$4.50 -- that can lead to a wide variation in your potential budgeting needs.

Don’t look to tools for hard and fast CPC spending guidelines. Use them to create estimated costs for new keyword lists or develop an hypothesis on how keyword changes might impact your budget.

-> Takeaway #4. Use multiple tools for best results

Each tool uses different databases and analytics. So, their results will differ. But you can use those differences to your advantage by compiling data from several tools.

“I don’t think there is any one source that can do the job for you,” says Stuart Meyler, Principal, Stinson Partners. “We look at three or four of them to come up with a baseline.”

Paul Elliot, Partner, Acquisition Marketing, Brulant Inc., had his team create special algorithms that reconcile data from multiple research tools.

The more you experiment with different tools, the more you’ll find features and data sets that are most relevant to your business. You may trust one tool’s volume scores, but find better-related keyword suggestions elsewhere. Another might be particularly good at researching competitors’ search marketing activities. Use them all to enhance your strategy.

-> Takeaway #5. Always test the keywords in your own campaigns

Even the most accurate aggregate data is no match for live test results. Whenever you use an external tool to create a new keyword list, conduct a test of those terms as part of your research strategy.

“We always test a keyword list for 30 days to see if we have a realistic baseline or benchmark for where we should be moving forward,” says Kent Lewis, President, Anvil Media.

Here’s a look at nine online keyword research tools, their size and pricing. You may notice that Yahoo! is missing from this list. Yahoo! used to offer a free Keyword Selector Tool, but warned users in January that the tool’s responsiveness would be “diminished.” Yahoo!’s new keyword research tool can be accessed only by users with a Panama advertising account, accessed through this link: http://sem.smallbusiness.yahoo.com/searchenginemarketing

Compete.com Search Analytics
http://searchanalytics.compete.com/site_referrals/
Database size: 19 million terms, 3 million sites.
Update frequency: Daily
Research tools:
- Site-referral tool that shows most popular terms driving traffic to a given site or an industry category.
- Keyword-destination tool that shows which sites receive the most traffic from a specific search term.
- Site comparison to show which of two sites has the traffic and engagement advantage for specific keywords.
Analytics:
• Top referring keywords for specific sites
• Site’s share of total referrals for keyword
• Keyword engagement (average amount of time people spent on the site after searching on the keyword)
• Keyword effectiveness (combination of total number of people and time spent on site)
• Site ranking for key terms
• Average monthly search referrals
Price: Free registration, but viewing complete reports cost one or two site credits. Credit packages range from $20 - $500.
Best use: Competitive research.

Google AdWords Keyword Tool
https://adwords.google.com/select/KeywordToolExternal
Database size: Unavailable.
Update frequency: Unavailable.
Research tools:
- Keyword generator that displays list of relevant and related keywords to word or phrase entered.
- Keyword list generation based on website content for URL added.
- Keyword cost estimates for PPC campaigns.
Analytics:
• Advertiser competition for keywords
• Previous month search volume
• Average search volume
• Search volume trends
• Month in which highest search volume occurs
• Estimated ad position
• Average CPC
• Estimated traffic volume for list
• Sort by match type (broad, phrase, exact, negative)
Price: Free
Best use: New keyword generation; keyword list augmentation; pricing guidance for AdWords campaigns; integrating keyword lists into AdWords campaigns.

Keyword Discovery
http://www.keyworddiscovery.com/index.html
Database size: 36 billion searches from 200 search engines worldwide.
Update frequency: Monthly.
Research tools:
- Top phrases that include search term.
- Related terms.
- Common misspellings.
- Top searches for particular industry.
- Domain researcher that identifies available domain names that contain keywords and phrases.
Analytics:
• Search volume
• Occurrence of term on sites
• Keyword Effectiveness Index
• Predicted daily volume
• Keyword density on your site or competitors’ sites
• Seasonal trends from previous 12 months
Price: Standard subscription, $69.95 monthly/$599.40 annually. Enterprise subscription, $495 monthly/$4455 annually.
Best use: Keyword list building; keyword list augmentation; global search campaign planning; competitive research.

KeywordSpy
http://www.keywordspy.com
Database size: 1 billion keywords.
Update frequency: Every two weeks.
Research tools:
- Keyword search shows top sites ranked on a given term for paid search and organic results.
- Domain search shows top keywords bid on by that domain, along with ad variations and landing page URLs being used.
- Keyword and domain search by specific market (United States, UK, Canada, Australia, Austria, Brazil, Switzerland, Germany, Spain, France, Greece, Ireland, India, Italy, Mexico, Singapore, South Africa).
Analytics:
• Total keywords by site listed
• Ad position for keywords
• Approximate price per click on keyword
• Average clicks per day on a keyword
• Estimated ROI for keywords
• Number of competitors bidding on keywords
• Geographic breakdown of competition and keyword usage
• Historical trends for keyword usage
• Position of a given domain in organic search results
• Number of competitors using a keyword in SEO
Price: Free level limited to 20 search results per query; complete results for $89.95 monthly.
Best use: Competitive research; international campaign planning; directional data for keyword advertising selection.

Microsoft adCenter Add-in for Excel 2007
http://advertising.microsoft.com/advertising/adcenter_addin
Database size: Unavailable.
Update frequency: Unavailable.
Research tools:
- Keyword suggestion shows terms related to a submitted keyword list.
- Keyword extraction shows popular terms contained in a submitted URL.
Analytics:
• Relevancy scores for suggested keywords
• Relevant terms contained in past search queries
• Campaign association (terms that have been bid on in adCenter)
• Top frequent keywords
• Top “spiky” keywords demonstrating recent and sudden popularity
• Suggested keyword categories
• Monthly or daily traffic forecasts
• Geographic search data
• Demographic search data (gender and age group)
• Monetization Key Performance Indicators (clicks, impressions, position, CTR, match type)
• Monetization metrics within industry verticals
Price: Free download for users with Excel 2007 and an AdCenter account.
Best use: Keyword list development and pricing guidance for MSN campaigns. Data can be extrapolated to estimate search traffic on Google and Yahoo.
Price: Free for adCenter account holders (must have Excel 2007). Still in beta.

SEO Digger
http://seodigger.com/
Database size: 60 Million English keywords in Google; 44 million English keywords in MSN, 11 million Russian keywords in Google.
Update frequency: Approximately every two weeks.
Research tools:
- Domain search shows all the key phrases for which a given site achieves a top-20 Google ranking.
Analytics:
• Site position on search engine results page
• Keyword popularity from Wordtracker
• Keyword popularity from Overture
• Keyword popularity from Google
Price: Free. Unregistered users limited to 5 queries per hour, and can only see results for the specific link entered. Registered users can check entire domains (including internal pages), and can conduct 1,000 queries per hour, and 10 complete domain searches per hour.
Best use: Analysis of own site’s high-ranking terms; competitive research; ideas for search phrase semantics.

SpyFu
http://www.spyfu.com
Database size: 13 million domains, 2.5 million keywords.
Update frequency: Monthly.
Research tools:
- Keyword search shows which sites and advertisers rank highly for a given term.
- Domain search shows statistics on a site’s advertising budget, top organic positions, top ad placement, and top competitors.
- Pre-assembled lists illustrate searching and advertising trends, including Top 500 Advertisers, Top 500 Organically Ranked Domains, Top 500 Most Clicked Terms, Top 1000 Most Expensive Keywords.
-SpyFu UK, which offers domain and keyword data for the UK market.
Analytics:
• Keyword cost per click
• Keyword clicks per day
• Keyword cost per day
• Number of advertisers bidding on a keyword
• Total keyword search results
• Historical trends in keyword average cost per click and clicks per day
• Top ads for a keyword
• Top search results for a keyword
• Domain advertising budget
• Domain total clicks per day
• Domain average cost per click
• Domain average clicks per day
• Domain average ad position
• Domain average number of ad competitors
• Domain average ad percentile
Price: Free registration for basic research functions. Subscription packages for full additional features -- expanded lists of domain top organic positions and ad placements -- range from $6.75 for a three-day trial; $38.50 monthly (auto renewal); $45 monthly (manual renewal); $296 annually (auto-renewal); $308 annually (manual renewal).
Best use: Competitive research, directional data for keyword advertising selection.

Wordtracker
http://www.Wordtracker.com
Database: 330 million search terms.
Update Frequency: Daily.
Research tools:
- Keyword Universe displays popular related terms to a keyword or phrase, and how those terms are used in longer search phrases.
- Keyword Researcher allows you to import a list of up to 500 existing terms to find how those terms are used in exact searches, which terms are most popular, and how they can be re-ordered for additional coverage.
Analytics:
• Count (number of times the term appears in searches)
• Prediction of number of times the search term will appear in next 24 hours
• Keyword Effectiveness Index
• Estimated competition for keywords
Price: Free tool generates list of up to 100 related keywords. Full functionality with subscription for $59 monthly, $329 annually.
Best use: Keyword list building; keyword list augmentation.

Wordze
http://www.wordze.com
Database size: 552 million keywords, 68 million unique search terms.
Update frequency: Monthly.
Research tools:
- Related keyword search for terms entered.
- Analysis of competing sites for a specific term.
- Download lists of top weekly or monthly search phrases.
Analytics:
• Total number of keywords found
• Estimated search volume
• Keyword Effectiveness Index
• Historical keyword data
• Keyword difficulty (Top domains for each keyword, with average links per month, total links, length at that position and other features to analyze how long it would take to rank in top 10 for a given term)
• Searcher demographics for keywords
Price: $7.95 one-day trial, $35 monthly.
Best use: Keyword list building; keyword list augmentation; SEO keyword strategies.

Useful links related to this article

Sign up now for the SherpaSearch newsletter:
http://www.marketingsherpa.com/page/Search-Marketing-Ne
sletter-2008

Sherpa’s Ultimate Search Marketing Glossary:
http://www.marketingsherpa.com/article.php?ident=30549

7 No-Cost SEO Tools to Help You Increase Your Website Traffic:
http://www.marketingsherpa.com/article.php?ident=30614

How to Use Niche Keywords to Boost Search Traffic: 5 Steps to a 20% Lift:
http://www.marketingsherpa.com/article.php?ident=30359

4 Steps to Automate Keywords, Cut Costs & Lift ROI:
http://www.marketingsherpa.com/article.php?ident=30330

SEO Tactics to Quadruple Lead Gen - Link-Building, Keyword Targeting & Landing Page Tests:
http://www.marketingsherpa.com/article.php?ident=30204

Google AdWords blog - with updates and tips:
http://adwords.blogspot.com/

Keyword Workshop - blog dedicated to reviewing keyword research tools:
http://www.keywordworkshop.com/

Seer Interactive
http://www.thinkseer.com/

SEOmoz - blog about keyword research:
http://www.seomoz.org/blog/category/16