In a previous blog post titled ‘The Key Is The Keywords‘ I discussed why keywords and key phrases are so important on the Internet and even more important where Search Engines (like Google and Yahoo) are concerned. It is critical to realise that if you want to receive traffic from the search engines for a specific keyword or key phrase then you must publish relevant, unique and optimised content on your site or blog that is targetted to that keyword. Additionally, if you want to give yourself a realistic chance to rank high in the search engines for a given keyword, you must filter your candidate keywords (that you could optimize for) based on the amount of SEO competition (phrase match competition or SEOC) that exists for those words.

I am going to make the assumption in this post that you haven’t choosen your main target keyword yet and are still trying to make head or tail or which keyword to optimise your website for relative to the market/niche/microniche you wish to enter. In the context of this post, I refer to SEO competition (SEOC) as the number of competing pages on the internet that contain the specific key phrase you are analyzing. SEO competition can also be thought of in terms of the quality and authority of a website that contains a specific keyword but I want to discuss that aspect of SEO competition in another post.

At the beginning of the Thirty Day Challenge (30DC) we teach the importance of choosing a keyword or key phrase that is being searched for at least 80 to 100 times a day and where Google is aware of less than 30,000 pages on the Internet that contain the specific phrase. You can find the specific phrase match competition easily in Google by entering the phrase in quotes. In the example search (click on the image to the left) I have searched for “Robert Somerville” and Google was aware of 67,800 pages on the Internet that contained the specific phrase at the time I did this search and captured the image shown. Competition changes daily (sometimes hourly) so don’t be surprised if the number varies when you search.

Now why was it that I choose 30,000 as the cutoff competition variable for phrase match competition and why don’t I use intitle (number of times the keyword is found in the title of a webpage) or inurl (number of times the keyword is found within a URL) or whether the keyword is contained within at least 1 link pointing back to the site? Those are the questions I mainly want to address in this post.

When I began preparing the basic research that lead to the development of the SEO strategy that was taught in the 2007 30DC, I was mainly interested in determining simple filter variables that, when applied, significanly increased the probability that the keywords that remained after the application of the filter, would have a sufficient traffic reward and were not too competitve that it wouldn’t be posible to rank on page one of Google for that keyword or key phrase within 2 – 4 weeks.

I realised after many tests, that the more competing pages that exist for a given keyword or key phrase, the greater probability that there would be too many authoratative web pages (that contained the specific phrase already) meaning I would have trouble out ranking them to get onto page one in Google. The fewer the number of competing pages the less chance that there would be too many authoritative sites that happen to contain or were specifically optimised for the keyword or keyphrase I was interested in.

Remember, at the stage that you are applying the traffic and competition filter you are trying to filter out a mass of keywords that are semantically related to your initial area of enquiry to identify a much smaller subset of keywords that meet the criteria. We are trying to drill down from a Market level of enquiry (where keywords have 100′s of thousands or millions of web pages that contain those words) to a micro-niche level of enquiry. And we want to do this to give us a reasonable chance to rank for keywords with a given level of competition so we can test the commercial value of the traffic associated with those keywords before we jump in a develop an extensive website and incur a large time and/or monetary cost.

I’m often heard to say that it takes as much time and money to develop a website for a keyword that has no value at all (no traffic, too much compeition or no interest within the traffic to buy or take advantage of a commercial offering) than it does to target a commercially valuable keyword or set of keywords. This being the case, surely it is in your interest to reduce the probability of failure and that is what the traffic and competition filters are designed to achieve when you commence an initial level of enquiry.

One often mis-understood aspect of the strategy we teach in the 30DC, is that we assess the traffic based on broad match searches (meaning we assess the maximum traffic reward possible for a keyword/key phrase) but we assess the competition based on phrase match. Many people question this approach and I will provide an answer to this seeming paradox based on my practical understanding of how Google works.

If a web page contains the words of your target key phrase but not the phrase itself, then the web page only has a weak probability of ranking for the specific key phrase in the Google search engine results pages (SERPS). Additonally, Google realises that when people search on a key phrase (more than one word) they are really interested to find pages that contain the key phrase not just pages that contain the words in the phrase but not the phrase itself. Fortunately, when a web page has a high ranking for a specific keyphrase when searched for in quotes (phrase match search) Google preferentially ranks that webpage in the SERP’s for the broad match search as well (key phrase without quotes) because most people don’t realise they will get more relevant returns if they conduct phrase match searches. In effect, most people aren’t using Google in the most effective way to get the most relevant results.  Google knows this and tries to compensate to giver .

So by filtering and optimizing based on the phrase match competition, the resulting keyword optimized content should receive preferential ranking for the broad match search (for that keyword or key phrase), but this only occurs when your web page has sufficient authority and relevance (mainly created by your keyword optimized off and on-page link network) to rank high in Google for the phrase match search.

If we are going to get preferential broad match ranking at a certain point (when we gain sufficient authority) then it is in our best interest to establish the traffic reward that comes from having a good broad match ranking which is why we initially assess traffic based on broad match rather than phrase match. Even though the broad match traffic is less relevant to our phrase match focus, it is still more relevant than completely generic traffic and we would want to expose our website to the maximum amount of traffic over time.

To round out this post, I want to point out another key factor that I established from my initial testing and research that also prompted me to land on the number of 30,000 for the SEOC competition variable. As I was creating websites and targetting keywords I began to realise that some of my websites were being sandboxed by Google (they were indexed but weren’t appearing in the SERP’s for the keywords I was targetting) and some weren’t. It turned out after a number of tests for keywords with a range of phrase match competition that Google was less likely to sandbox my site when the keyword it was relevant for had lower phrase match competition. This is very important early on when you are trying to acquire sufficient authority in the eyes of Google (mostly link building) and that sometimes Google observes the link building as too aggressive and sandboxes the site until everything settles down and the relative authority can be established. But when the phrase match competition is low, Google is very reluctant to sandbox a site becuase it reduces the overall relevance of it’s index for that keyword. When there is a large number of equally relevant pages, however, then Google doesn’t mind sandboxing a site for a while (days to months) because there are many equally relevant pages that can appear in the SERP’s so the quality of the index is maintained.  It turned out, back when I was doing this testing, that around 30,000 phrase match competition for a keyword or key phrase that Google rearely sandboxed a site if it was highly relevant for that keyword or key phrase even if my link building activites were a little too aggressive.

I also believe that Google is morely likely to index and rank a new site or web page and place it into the SERP’s for a keyword if the keyword has very low phrase match competition although I have less actual evidence to back this up. But it makes sense really, if we produce content for a keyword where there is a low number of competing pages then it makes sense that Google would want to rank that new content efficiently and high in the SERP’s particularly if the on and off-page factors are well optimized for that particular keyword. We are in essence giving Google exactly what it wants and increasing the quality of the Google index for that particular keyword or key phrase.

So there you have it, these are some of the reasonings that sit behind the SEOC variable of 30,000. Let me remind you that the application of this variable (and the traffic filter) is merely to quickly filter a list of keywords within a particular subject area so you can quickly establish whether there is a sufficient traffic reward and competition profile for those keywords.  You still need to review the quality of the websites that are currently ranking for your target keyword/s and preferably seek confirmation that there is a commercial focus for those keywords online before you decide to go on a create a website to test those keyword/s. I see investigating the Inurl and Intitle variables for a keyword as part of the quality analysis for a keyword which I only do after a keyword or keywords meet the traffic and competition criteria. More of that in a future post.

The more we can do to reduce the probability of failure, it stands to reason we are increasing the probability of success for those keywords that pass our filtering criteria. But at a certain point analysis can lead to paralysis of action which we must constantly guard against. The 30DC strategy can be implemented quickly when you have reached a stage 4 level of competence. So if your analysis leaves you in doubt but you can see a reason to go forward (based on traffic, volume of competition, quality of compeition and evidence of commerciality) then it may be easier to throw up a quick blog and actual test the keyword for real. As long as you take heed of the early results and not let the test drag on too long if it turns out to be a failure, then you should be OK.

This subject can be challenging and can be difficult to comprehend when you are beginning to learn about Search Engine Optimization (SEO). Feel free to ask any questions in the comments or send me an email to emailgurubob@gmail.com and I’ll do my best to respond as soon as I can.

It is…as it is.

GuruBob

Technorati Tags: google competition, guru bob, GuruBob, search engine optimization tips, seo competition, seo guide, seo keyword, seo keywords, seo strategies, SEO Strategy, seo tutorial

“Handle them carefully, for words have more power than atom bombs.” - Pearl Strachan Hurd

Over the next few months, I plan to write about various aspects of the Thirty Day Challenge (30DC) and search engine optimisation (SEO) and so I couldn’t start without going back to what I believe is the fundamental principle behind traffic on the internet and the key to search engine optimization.

And that is…the keyword (or key phrase).

Before I got involved in Internet Marketing, I was associated with a wealth creation company that got most of it’s traffic from a standard offline marketing strategy – newspaper advertising. I couldn’t believe the expense and wastage associated with this strategy and when I looked into radio and tv advertising it was even more expensive – with no increased guarantee of success. I quickly came to realise that if you couldn’t pay the opportunity cost for these traffic sources or didn’t have an offer that converted at a rate sufficient to at least pay for the cost of the traffic, then you would fail…and fail quickly.  This is the main problem for small business…how to get traffic efficiently and at an acceptable cost that leaves you with a profit.

In the offline world, it’s about capturing eyeballs and the more eyeballs a particular advertising channel commands, the more expensive it is to advertise through that channel. But advertising through those channels doesn’t mean you get more targetted traffic, so you have to make sure your offer converts. Of course you often don’t know whether your offer will convert until you try (a chicken and the egg argument that can be very costly). It’s a numbers game and as a former mentor said to me, most large scale advertising strategies are really throwing mud against the wall and hoping some of it sticks.

Then I came onto the internet and realised that whether you are paying for clicks via PPC channels (like Google Adwords) or receiving organic traffic from the search engines you are esesentially receiving targetted traffic from people that are actually searching for the keyword you are bidding on or optimizing for.

I thought to myself, how good is this. Finally, you have the potential to allocate time and money to traffic that might actually want what you are marketing or at least have some interest in it.  So just by focussing on the Internet for traffic you significantly decrease one of the key risks of offline advertising…that you have to pay to advertise to people that don’t want what you are marketing. And the cost of advertising to those dis-interested people very often renders your advertising campaign unprofitable.

I also remember a time on the Internet, before the current search engines existed, when it was very hard to find things even when you knew what you were looking existed.  Google changed all that of course and with that, the science and business of search was created. If you have any interest in the whole concept of search (and the history of Google) I highly recommend the book “The Search” by John Batelle.

Have you ever noticed how simple Google’s home page is…have you ever considered why this is?

It is very elegant really, Google is waiting for you to tell it what you want (or need, or are looking for). And when you do so, every aspect of Google’s technical expertise is bought to bear to ensure you receive back content that is as relevant to your search as it can possibly be. But you start the process by entering…a keyword or key phrase.

Everything starts with the keyword. And as a potential marketer and receiver of traffic (related to a keyword) you must be aware of:

• The number of people using the keyword in their searches per day
• The number of websites that also use that keyword in their content
• The probability that when people use that keyword to search they might be interested in buying something
• The quality and authority or websites that already rank in the search engines for that keyword.
• The competitiveness and cost of buying traffic for this keyword via PPC

So much to discover and it’s only in the last 2 – 3 years that we have been able to quantify some of the variables discussed above. Of course Market Samurai and other keyword research tools have simplified the research process immensely but more of that in a future post.

While I did want to focus on the quantative aspects of keywords primarily (traffic, competition etc), I did want to draw your attention to some of the more subtle aspects of keywords which are harder to grasp but equally as important.

The most important of these is semantic relevance. There are many definitions of semantic relevance but to me a word is semantically relevant to another word, if a human being is likely to use that word in association with the keyword being focussed on. The reason why you need to give consideration to semantic relevance is that over time, Google will rank your content in the search engines not only for the main keyword you are optimizing for but also for the semantically related keywords particularly if you mention them in association with the main keyword. The more authority (defined as the number and quality and links that point to your content from other sites) your content has, the higher you will rank for your target keyword…and also for the semantically related keywords.

And this is where tagging comes in.  You can signifcantly increase the potential that Google and other search related sites understands the relationship between your content and the keywords it relates to by effectively tagging your content with semantically related keywords or key phrases.  So few people do this and fewer still do it well, but it is incredibly important and it’s value increases proportionally over time as more and more websites send you traffic not only for the main keyword you are targetting but also for the semantically related keywords you tag for.

There is so much to cover in a discussion regarding keywords and it is hard to do the subject justice in one post. I do hope from this post that you get a sense of the importance of keywords on the Internet and also the importance of the semantic relationships between keywords. And if you create content online, video, articles, audio, blog posts whatever…make sure you tag the content with the most relevant keywords.

If you have understood anything from this post, the keywords you use to optimize content and tag won’t be just any keywords but keywords that have associated traffic and acceptable levels of competition.  There is very little point optimizing and tagging for keywords that have no traffic or extreme levels of competition. Either way, you guarantee that you won’t get any traffic either.

If you have any questions regarding some of the subjects covered in this post please comment and I will respond either in the comments or through future posts.

It is…as it is.

GuruBob

ps I spoke about this subject at length at the Beechworth conference. If you subscribe to receive email updates at the bottom of the page you will receive access to a video of my Beechworth presentation.

Technorati Tags: best keyword, best keywords, find keyword, GuruBob, internet keyword, keyword, keyword analyzer, keyword generator, keyword management, keyword search tool, keyword selection, keyword selector tool, keywords, keywords tool, local keyword, semantic relevance, tag, tagging