Posted on February 10, 2010 by Guru Bob

In a previous blog post titled ‘The Key Is The Keywords‘ I discussed why keywords and key phrases are so important on the Internet and even more important where Search Engines (like Google and Yahoo) are concerned. It is critical to realise that if you want to receive traffic from the search engines for a specific keyword or key phrase then you must publish relevant, unique and optimised content on your site or blog that is targetted to that keyword. Additionally, if you want to give yourself a realistic chance to rank high in the search engines for a given keyword, you must filter your candidate keywords (that you could optimize for) based on the amount of SEO competition (phrase match competition or SEOC) that exists for those words.

I am going to make the assumption in this post that you haven’t choosen your main target keyword yet and are still trying to make head or tail or which keyword to optimise your website for relative to the market/niche/microniche you wish to enter. In the context of this post, I refer to SEO competition (SEOC) as the number of competing pages on the internet that contain the specific key phrase you are analyzing. SEO competition can also be thought of in terms of the quality and authority of a website that contains a specific keyword but I want to discuss that aspect of SEO competition in another post.

At the beginning of the Thirty Day Challenge (30DC) we teach the importance of choosing a keyword or key phrase that is being searched for at least 80 to 100 times a day and where Google is aware of less than 30,000 pages on the Internet that contain the specific phrase. You can find the specific phrase match competition easily in Google by entering the phrase in quotes. In the example search (click on the image to the left) I have searched for “Robert Somerville” and Google was aware of 67,800 pages on the Internet that contained the specific phrase at the time I did this search and captured the image shown. Competition changes daily (sometimes hourly) so don’t be surprised if the number varies when you search.

Now why was it that I choose 30,000 as the cutoff competition variable for phrase match competition and why don’t I use intitle (number of times the keyword is found in the title of a webpage) or inurl (number of times the keyword is found within a URL) or whether the keyword is contained within at least 1 link pointing back to the site? Those are the questions I mainly want to address in this post.

When I began preparing the basic research that lead to the development of the SEO strategy that was taught in the 2007 30DC, I was mainly interested in determining simple filter variables that, when applied, significanly increased the probability that the keywords that remained after the application of the filter, would have a sufficient traffic reward and were not too competitve that it wouldn’t be posible to rank on page one of Google for that keyword or key phrase within 2 – 4 weeks.

I realised after many tests, that the more competing pages that exist for a given keyword or key phrase, the greater probability that there would be too many authoratative web pages (that contained the specific phrase already) meaning I would have trouble out ranking them to get onto page one in Google. The fewer the number of competing pages the less chance that there would be too many authoritative sites that happen to contain or were specifically optimised for the keyword or keyphrase I was interested in.

Remember, at the stage that you are applying the traffic and competition filter you are trying to filter out a mass of keywords that are semantically related to your initial area of enquiry to identify a much smaller subset of keywords that meet the criteria. We are trying to drill down from a Market level of enquiry (where keywords have 100′s of thousands or millions of web pages that contain those words) to a micro-niche level of enquiry. And we want to do this to give us a reasonable chance to rank for keywords with a given level of competition so we can test the commercial value of the traffic associated with those keywords before we jump in a develop an extensive website and incur a large time and/or monetary cost.

I’m often heard to say that it takes as much time and money to develop a website for a keyword that has no value at all (no traffic, too much compeition or no interest within the traffic to buy or take advantage of a commercial offering) than it does to target a commercially valuable keyword or set of keywords. This being the case, surely it is in your interest to reduce the probability of failure and that is what the traffic and competition filters are designed to achieve when you commence an initial level of enquiry.

One often mis-understood aspect of the strategy we teach in the 30DC, is that we assess the traffic based on broad match searches (meaning we assess the maximum traffic reward possible for a keyword/key phrase) but we assess the competition based on phrase match. Many people question this approach and I will provide an answer to this seeming paradox based on my practical understanding of how Google works.

If a web page contains the words of your target key phrase but not the phrase itself, then the web page only has a weak probability of ranking for the specific key phrase in the Google search engine results pages (SERPS). Additonally, Google realises that when people search on a key phrase (more than one word) they are really interested to find pages that contain the key phrase not just pages that contain the words in the phrase but not the phrase itself. Fortunately, when a web page has a high ranking for a specific keyphrase when searched for in quotes (phrase match search) Google preferentially ranks that webpage in the SERP’s for the broad match search as well (key phrase without quotes) because most people don’t realise they will get more relevant returns if they conduct phrase match searches. In effect, most people aren’t using Google in the most effective way to get the most relevant results.  Google knows this and tries to compensate to giver .

So by filtering and optimizing based on the phrase match competition, the resulting keyword optimized content should receive preferential ranking for the broad match search (for that keyword or key phrase), but this only occurs when your web page has sufficient authority and relevance (mainly created by your keyword optimized off and on-page link network) to rank high in Google for the phrase match search.

If we are going to get preferential broad match ranking at a certain point (when we gain sufficient authority) then it is in our best interest to establish the traffic reward that comes from having a good broad match ranking which is why we initially assess traffic based on broad match rather than phrase match. Even though the broad match traffic is less relevant to our phrase match focus, it is still more relevant than completely generic traffic and we would want to expose our website to the maximum amount of traffic over time.

To round out this post, I want to point out another key factor that I established from my initial testing and research that also prompted me to land on the number of 30,000 for the SEOC competition variable. As I was creating websites and targetting keywords I began to realise that some of my websites were being sandboxed by Google (they were indexed but weren’t appearing in the SERP’s for the keywords I was targetting) and some weren’t. It turned out after a number of tests for keywords with a range of phrase match competition that Google was less likely to sandbox my site when the keyword it was relevant for had lower phrase match competition. This is very important early on when you are trying to acquire sufficient authority in the eyes of Google (mostly link building) and that sometimes Google observes the link building as too aggressive and sandboxes the site until everything settles down and the relative authority can be established. But when the phrase match competition is low, Google is very reluctant to sandbox a site becuase it reduces the overall relevance of it’s index for that keyword. When there is a large number of equally relevant pages, however, then Google doesn’t mind sandboxing a site for a while (days to months) because there are many equally relevant pages that can appear in the SERP’s so the quality of the index is maintained.  It turned out, back when I was doing this testing, that around 30,000 phrase match competition for a keyword or key phrase that Google rearely sandboxed a site if it was highly relevant for that keyword or key phrase even if my link building activites were a little too aggressive.

I also believe that Google is morely likely to index and rank a new site or web page and place it into the SERP’s for a keyword if the keyword has very low phrase match competition although I have less actual evidence to back this up. But it makes sense really, if we produce content for a keyword where there is a low number of competing pages then it makes sense that Google would want to rank that new content efficiently and high in the SERP’s particularly if the on and off-page factors are well optimized for that particular keyword. We are in essence giving Google exactly what it wants and increasing the quality of the Google index for that particular keyword or key phrase.

So there you have it, these are some of the reasonings that sit behind the SEOC variable of 30,000. Let me remind you that the application of this variable (and the traffic filter) is merely to quickly filter a list of keywords within a particular subject area so you can quickly establish whether there is a sufficient traffic reward and competition profile for those keywords.  You still need to review the quality of the websites that are currently ranking for your target keyword/s and preferably seek confirmation that there is a commercial focus for those keywords online before you decide to go on a create a website to test those keyword/s. I see investigating the Inurl and Intitle variables for a keyword as part of the quality analysis for a keyword which I only do after a keyword or keywords meet the traffic and competition criteria. More of that in a future post.

The more we can do to reduce the probability of failure, it stands to reason we are increasing the probability of success for those keywords that pass our filtering criteria. But at a certain point analysis can lead to paralysis of action which we must constantly guard against. The 30DC strategy can be implemented quickly when you have reached a stage 4 level of competence. So if your analysis leaves you in doubt but you can see a reason to go forward (based on traffic, volume of competition, quality of compeition and evidence of commerciality) then it may be easier to throw up a quick blog and actual test the keyword for real. As long as you take heed of the early results and not let the test drag on too long if it turns out to be a failure, then you should be OK.

This subject can be challenging and can be difficult to comprehend when you are beginning to learn about Search Engine Optimization (SEO). Feel free to ask any questions in the comments or send me an email to emailgurubob@gmail.com and I’ll do my best to respond as soon as I can.

It is…as it is.

GuruBob

Tips And Tricks

Technorati Tags: google competition, guru bob, GuruBob, search engine optimization tips, seo competition, seo guide, seo keyword, seo keywords, seo strategies, SEO Strategy, seo tutorial