By definition, SEO refers to the process of optimizing a website with the goal of having major search engines (primarily Google, Yahoo! and MSN Live Search) return pages from that website in highly-ranked search engine results. SEO is almost always employed as a form of marketing, but it is a very specific form of marketing that takes place within the search engines.

Lately, some industry blogs have suggested that SEO has grown beyond its primary parameters, suggesting that successful SEO includes expanding into other realms of marketing. I don’t think that makes any sense. By definition, SEO - search engine optimization - does not include any form of marketing that goes outside of search engines. The use of the term SEO in such a way is disingenuous and confuses many webmasters and owners of ecommerce sites.

For more about this controversial topic, visit the full article on Practical eCommerce here.

Google’s announcement that they would automatically opt-in many of their ads to appear on the mobile platform seems somewhat controversial, since one supposes that a number of advertisers might not notice the change and might be irritated that their ads may now be running on an unanticipated platform. After all, many online retailers who advertise expect that some percentage of users clicking through would be placing online purchases, and mobile devices don’t effectively support transactions yet. Google’s patent for Gpay might pave the way for this, but it hasn’t launched yet.

On one level, Google’s act to opt everyone into this program might not seem all that bad—after all, advertisers are paying for exposure and referrals, too. Greg Sterling reports that Google’s opt-out policy during this trial period is similar to policies Yahoo and Microsoft have used in the past for new/underutilized programs. Since it can be reasonably supposed that CTR on mobile ads is likely relatively low, the cost to advertisers is probably low-to-nil, and Google’s motive is likely just a desire to get research data on mobile ads while also introducing many advertisers to the medium.

A New York Times article quotes Dilip Venkatachari, Product Management Director for AdSense, as saying that “the ads would provide a new source of revenue for publishers and could encourage more online sites to create mobile-focused Web sites. Like most other Google advertising systems, ad prices will be set through an auction and advertisers will pay when a user clicks on its ad.”

So, this should encourage more sites to create mobile friendly versions of their sites? While it’s an idealistic desire, the lack of standardization through the wireless devices pose significant challenges to large companies who are already challenged by making things simultaneously attractive, usable, cross-platform compatible, and optimal for search engines. Not to mention that devices such as the iPhone and Palm Treo for Windows are fast making it possible to view “non-mobilized” sites just fine, reducing any urgency that companies might feel about getting their content to work well for the new environment.

Interactive advertising on mobile is still very new, and there’s little research as to what’s effective and what works for consumers, much less developers. What many analysts are very sure of is that mobile search is likely to be particularly effective for locally-oriented businesses. As Paul J. Bruemmer reports, “While local search marketing is well established, local-mobile search marketing is barely getting started.”

Paul further voices a concern that many have expressed about the advent of ads on phones, a “General intolerance of advertising messages on a personal device.” At least thus far, I’m not hearing a lot of consumer complaints about this yet, perhaps because the ads are not all that obtrusive so far, and perhaps because internet users have already become so accustomed to seeing contextual ads.

For instance, check out these ads I found in Google when doing a search for “florists in Boston” on my Treo running Windows and Internet Explorer:

The ads are located down at the bottom of the search results, and they’re clearly labeled “Ad”, so I don’t find them all that annoying. I have seen one ad appear at the top of Google Mobile SERPs before, but it seems to be a very infrequent position for now.

Notice the ad with the URL that appears to be Google-hosted? I wonder if that’s one of the mobile landing pages that Google states that they may “adapt” for users who click through? This seems particularly odd to me, and it looks like the sort of thing where Google ought to spell out under what circumstances they’ll choose to automatically generate a landing page to “enhance” the user-experience. This seems very controversial to me, because Google is likely charging advertisers when they click through to those generated landing pages, and advertisers may not realize this, causing them to misinterpret effectiveness of apparent conversion rates.

Now, Yahoo provides advertisers with the ability to purchase similar mobile-friendly pages which I believe are intended for this very purpose, but I don’t think they automatically generate a page in the way that Google does—it’s voluntary. For instance, check out these ads for “shoes” from Yahoo Mobile:

When you click on a mobile ad in Yahoo, they provide an optimized mobile landing page. Here’s one that came up for me when I clicked on a Teleflora ad:

In some cases, Google is choosing to dump users onto the landing pages of the advertiser. For instance, here are ads for a “shoes” search in Google Mobile:

And when you click on the ad for Zappos, you get dumped onto their homepage which is not intended for mobile users:

I’m not sure why Google would land a user onto the graphic-intensive Zappos page in this case, instead of using their optimized mobile landing page. Perhaps Google Mobile didn’t think I needed it, since I’m running on a flavor of Internet Explorer? More likely, I’m supposing if the advertiser’s existing landing page includes a phone number, as every page on the Zappos site does, Google may conditionally choose not to automatically generate a more pared-down mobile landing page.

If you’re interested in seeing how your landing page or webpage might be automatically adapted by Google to appear for mobile users, check it out through this interface: http://google.com/gwt/n

Just from observation, Yahoo’s Mobile Ad Service appears broader and more robustly envisioned, allowing a variety of ad types for mobile, including Display Ads, Search Ads, and Video Ads. Yahoo’s promo copy reads “…click through to a promotional site, enable the consumer to find or call a store directly, offer a coupon, send an SMS message—the options keep expanding. If you don’t have a promotional site to link to, Yahoo can create one to your specifications.”

Since Mobile Search and Local Mobile Search are still relatively virgin marketing territories, Google can be forgiven some early program gaffes. But, online marketers may want to go and opt-out some of their current campaigns from running automatically in the mobile end since it could obscure some of their conversion data, particularly if the ads are intended to primarily drive online transactions. It would be better to set up separate campaigns with separate reporting and check to see if an optimal mobile landing page can be associated with the ad.

Also, for mobile ads, be aware that there’s theoretically greater potential for click-fraud. As I earlier outlined in “A Thorny Issue: Detecting Mobile Search Click-Fraud“, invalid clicks may be a lot harder to identify for mobile ads.

The problem is you are staring at the tip of the iceberg, oblivious to the iceberg below. Something gigantic lies beneath. And you need more sophisticated tools and methods to understand and diagnose the opportunity in order to take appropriate action. In fact, chances are high that your natural search channel is seriously underperforming as a result.

As NSO best practices become commonplace amongst your competition, taking this evolutionary step is required for merchants to gain advantage. Fortunately, by approaching the natural search channel with an understanding of long tail dynamics, new key performance indicators (KPIs), and yield management techniques, savvy merchants can move beyond the traditional project-based mentality of NSO, to a more sophisticated, yet DM-friendly approach – one that leverages website size, brand power, and direct-marketing principles to capture the enormous potential that lies beneath the surface.

The Long Tail of Natural Search

To understand the KPIs and management techniques, let’s examine what lies beneath the surface. In a word, it is not an iceberg, but the “long tail” - a colloquial term referring to a common statistical distribution featuring a “tail-shaped” curve. Thanks to Chris Andersen’s book, the long tail concept has received much press recently, and carries many significant Internet business-based implications. But there is also a long tail distribution that contains the promise of natural search for large, dynamic e-commerce merchants.

Source Netconcepts, 2006

In fact, to appreciate the enormity of this potential, contrasting it against a known quantity is helpful. According to research published recently by Netconcepts (“Chasing the Long Tail of Natural Search”, 2006) for every search that occurs for the average merchant’s brand name, nearly 40 relevant searches occur for more generic, brand-neutral keywords. For instance, if there are 100,000 searches for “LL Bean” this month, there are an estimated 4,000,000 searches for hundreds of thousands of “unbranded” keyword markets LL Bean pages could compete within, ranging from “furniture slipcovers” to “women’s flannel pajamas” to “men’s reindeer sweater,” and more.

For these coveted and targeted keywords, LL Bean is likely to have matching category or product pages somewhere on the website, that ideally could position the LL Bean brand high enough in the search results to win these millions of click decisions (and sales) on the cheap. And even if these pages do not convert 100% of these searches into clicks, if well ranked, these pages still serve the purpose of building brand by association with these millions of corresponding “unbranded” searches, and by extension, the searchers that used them.

This is the long tail of natural search – the universe of diverse, unbranded keyword markets that, while perhaps not as frequently searched on an individual basis, cumulatively, add up to orders-of-magnitude greater (40x, in our research) search potential than the brand search traffic that most merchants naturally receive. This is the prize every savvy merchant seeks.

Since brand popularity is relative, a more objective way to think of this long tail potential is as a function of your website size. In particular, our research suggests that for every unique page that exists on a merchant’s website, nearly 100 unbranded searches are conducted in an average month. If you have a 20,000 page website, your long tail potential would be calculated as 2,000,000 unbranded searches, and so on. Tracking your unique pages and their yield therefore is a critical measurement to understanding the dynamics of your natural search channel performance.

Capturing the Elusive Tail

The long tail of unbranded keywords is a new entity for most marketers to grapple with. What drives it? How do you quantify it? What is good performance? What is bad? How do you capture it?

We set out to answer these questions in the research alluded to earlier. To this end, we developed a theory we refer to as “Page Yield Theory.” Page Yield Theory aims to breakdown, quantify, and model the components of a merchant’s long tail into manageable units, and was informed via natural search data gathered from a few dozen top online merchant (clients utilizing our GravityStream™ natural search optimization proxy technology). In the process, we developed a set of metrics to understand the dynamics of the long tail. We have since applied these same ratios as key performance indicators (KPIs) explained below, for growing natural search channel programs under management.

KPI #1 – Brand to Non-Brand Mix

What percentage of your natural search comes from brand keywords versus non-brand keywords? What does it mean if your keyword curve is dominated by brand terms? If you discover that most of your traffic is coming from searches for your brand, we view this as symptomatic of a larger problem lying just beneath the surface - that very few of your pages are actually yielding traffic.

Many retailers find 95% of their search traffic comes for brand terms with, not coincidentally, a very small percentage of their site powering that brand traffic. Whereas what you may find, when you have done some search optimization, is a very different distribution curve. You may find that 40% of your pages are yielding traffic, but that 60% of that traffic is unbranded keyword traffic. This in fact, is the core hypothesis of Page Yield Theory – unbranded keyword traffic volume grows as the number of pages yielding natural search traffic grows.

KPI #2 – Unique Pages

Just how big is your website? How many pages are there? This is a critical metric for establishing a yield management foundation. You may be tempted to approximate this number using a product count. This will most likely significantly understate your actual pages. Or you might want to use a search engine’s reported index. This too will return mixed results, as each engine indexes dramatically different amounts of pages for each website.

However, search engines are in the business of discovering pages. So we recommend using the number of pages that bots (such as googlebot) are able to crawl over a 30 – 60 day period as the most useful proxy for how many unique pages are available on your site. (This assumes your URLs are in a condition that permits bots to crawl deeply. If they are not, you may need to resort to an approximation.)

As a comparison, the average merchant in our research had roughly 73,000 uniquely crawled pages.

Note: Most analytical packages do not track page crawl metrics because they rely on Javascript to track user actions. Since search engine bots do not have Javascript enabled, their crawl activity goes unregistered – leaving marketers blind to the dynamics of their “search conversion funnel.

KPI #3 – Pages Yielding Traffic

What percentage of those pages yield search traffic? Is it 10%? 90%? This ratio essentially dictates the length (the X axis) of your unbranded keyword long tail, and suggests remaining potential.

With this KPI in hand, you understand your missed opportunity cost, and monitor your progress throughout your NSO engagement.

If you do not know your amount of unique pages or yielding pages, the above matrix can be consulted for an estimate. It was developed using data from the average merchant in our research and illustrates the inverse relationship between Page Yield and Brand / Non-Brand Traffic. Simply seek your current level of brand and non-brand traffic to estimate how many pages are accounting for that traffic.

The average merchant in our study had 14% of pages yielding traffic.

KPI #4 – Keywords per Page Yield

Now that you have an idea of your Page Yield rate, how many keywords does each of those producing pages yield over the course of a month – 2 keywords? 10? This is the fourth KPI: keyword yield. This KPI is responsible for creating scale. That is, the more keywords each yielding page attracts or targets, the longer your tail.

The average merchant in our study found 2.4 keywords produced per yielding page.

KPI #5 – Visitor per Keyword Yield

This fifth KPI – Visitor per Keyword Yield – is basically determined by how highly a page ranks for a keyword, and tells you how much traffic each keyword drives. This metric determine the height or thickness of your long tail.

The average merchant in our study experienced 1.9 visitors per keyword.

KPI #6 – Index-to-Crawl Ratios

The first step in the search conversion funnel is converting a crawled page to an indexed page. While we do not put significant stock in the reported engine indexes, this KPI can illustrate engine differences or potential trouble-spots.

For instance, the average merchant in our study saw a 3:1 index-to-crawl ratio in Google. That is, for every page that was crawled, there were 3 in the index, as strange as it sounds. If a merchant finds the index shrinking to say, only 50% of the crawled pages, that may be a signal of crawl pattern changes, or perhaps pages are being shifted into the supplemental index.

KPI #7 – Engine Yield

A final KPI to monitor is engine yield. Each engine has different audience sizes. How do you fairly compare the referral traffic you get from each? Simply calculate how much referral traffic each sends for every page it crawls or consumes. Compare engine by engine. What we have found is that MSN and Yahoo! tend to crawl a lot more pages, but the yield per crawled page from Google is typically significantly higher.

Reconstructing the Tail

Once known, these KPIs can be reassembled to create a vivid picture of your performance and empower smarter decisions. For instance, brand and non-brand traffic composition (KPI #1) provide a baseline on the effectiveness of your channel yield. This can be contrasted against your total available pages (KPI #2) and actual yielding pages (KPI #3) to understand the gap between current and potential performance. The rate at which these pages yield keywords (KPI #4), times the visitor driven by each of those keywords (KPI #5) form the length, and height, of your tail, and can each be optimized individually. Meanwhile your index-to-crawl ratio (KPI #6) and engine yield (KPI #7) help quantify your site’s effectiveness at converting web pages into units of “search produce.”

For instance, if the number of pages yielding traffic is low (under 10%), it may be time for some basic, scalable, URL rewriting to maximize the flow of link juice through your internal link text. On the other hand, if 80% of pages are yielding traffic at a rate of one keyword per page and one visitor per keyword, it may be time to conduct some auto-generated title tag tests to scalably lift rankings of thousands of pages, by incorporating more relevant keywords per page. And perhaps a pinch of organic link building via RSS may be in order to power-up the link juice that flows through the inbound link text. Each situation is unique and requires a custom approach to optimization.

The secret to capturing the long tail of natural search is this: Fully maximize the page yield of your website. That is, develop a metrics-driven optimization process that empowers each page to be viewed as the authority on the subject – be it for “furniture slipcovers,” “men’s reindeer sweater,” “18 volt Makita power drill” or other. This is not without challenge; empowering thousands of pages to yield natural search traffic may seem daunting, but it can be achieved by focusing on these new KPIs, managing your NSO channel effort against these yield metrics, and committing to an iterative NSO testing program.

Increasingly, the natural search game revolves around getting your brand associated with unbranded terms. While these new KPIs do not change the basics of NSO, they can provide the context and management discipline that enable marketers to make more informed decisions how to best spend time and budget optimizing your long tail.

Today’s focus is on improving the site’s visibility, accessibility, popularity, usability and measurability. Using the power of the Internet to ascertain market intelligence that is widely available online for free and other paid tools such as Hitwise and Nielsen NetRatings helps to make decisions for each of these areas.

What underpins all online marketing activity is “search” marketing as search engines are responsible for delivering the majority of traffic to your website including traffic generated by offline media such as TV, radio, press, billboards and magazines. Ultimately all marketing activities, whether it is online or offline, influence what people search for on the Internet.

As a brand owner, it is important to recognise the role in which search marketing plays in the wider marketing mix and understand that it is more than sprinkling a handful of keywords on pages. Natural search marketing or search engine optimization is more than just keywords.

The first step of online marketing is to ensure that you have a search friendly, accessible and user friendly website.

Don’t assume that every programmer or web designer will know how to build a website that is balanced for the target market and major search engines alike. Many a website designer and developer are still stuck in the 90’s using their old-school web design methods because that is what they are familiar with. Unlike your GP, they do not necessarily have to upgrade their skills every year. On the surface their websites are glossy and picture perfect; however under the bonnet it could be a different story.

Good websites allow search engines to easily access the content within it so the way it is built deeply impacts on whether “bots” can enter the door. When Google comes knocking, you definitely want to say “come in, make yourself at home and by the way do you want milk with your coffee”?

A common misconception, especially in the advertising agency world, is that keeping “on brand” is only about the look and feel of the site. Many of these types of websites force all typography and copy into images thus preventing search engines from accessing the good words that are contained within.

Don’t get me wrong, I love flash animated sites, but creating a whole site out of flash is not doing the client and their target market any favours. It’s all mouth and no trouser!

Even though you can use flash or image replacement technology to help the site along this is not an ideal situation. I dare say that Google has no intention in the near future to index flash pages because this technology is too complex for search engines.

Obviously the more pages that Google, MSN and Yahoo! are aware of within your site, the higher the probability of appearing in search results. A search engine friendly site that has 1,000 pages compared to a site that has 30 pages will have higher brand exposure through natural search listings than the latter.

Simply by typing into Google a “site:www.yoursite.co.nz” query command will enable you to know how many pages are being indexed in their database. A comparison between Noel Leeming (www.noelleeming.co.nz), Dick Smith Electronics (www.dse.co.nz) and an unknown brand Computer Lounge (www.computerlounge.co.nz) in New Zealand demonstrates that they have 4,930, 920 and 2,300 pages indexed respectively at time of writing.

Why is it that a small unknown quantity such as Computer Lounge has more pages indexed by Google than Dick Smith Electronics? Surely Dick Smith has more than 920 products on their site?

This is one indicator that the way the Dick Smith website is built is hindering search engines from accessing their product pages. Without pages being indexed, their visitor numbers, brand exposure and related sales will be much lower than what they should be.

Another indicator is that their pages have no PageRank score, a 0-10 measurement of how “important” these pages are in the eyes of Google.

The advertising they have invested in including press and unaddressed mail influence people to search for those related products online. Yes when people search for the term “dick smith” they will be ranked number one in all major search engines mostly due to other websites linking to them using this term in the link anchor text, however with their current site they will find it extremely difficult to achieve good traffic for product related terms such as “epson printer” or “palm pilot”. They are still to learn the power of the “long tail” where the majority of their sales will come from.

If Dick Smith Electronics are not ranking well for product related terms then they are opening themselves up to local and international competition to take advantage of the online demand they created.

Marketers need to remember that offline marketing activity influences what people search for online. If a company spends millions of dollars on television, print, radio and outdoor advertising but forgets to technically optimize and make branded and related non-branded words available on the site to search engines, then they are opening themselves up to their competition taking advantage of the marketplace generated online by their own advertising investment.

The focus is not only allowing search engines access to your website pages, but also to be a good corporate citizen by making the site accessible to vision impaired people. People who cannot see have site reader technology that reads a website very much like the way a search engine does.

A majority of websites on the Internet are not accessible. Essentially the reader software will read all navigation, copy and html code of a website out aloud to the user. If you were blind and a machine was reading out gobble-de-gook such as “untitled document, table with one column and two rows, link logo1, nesting table, link graphic images logo2, image spacer clear, table end…” this would not make for a good user experience. And yet this is the case for majority of websites on the Internet.

Ensuring that the site has an accessibility statement, text that is resizable, all images, form fields and tables are labeled in the html of the site, CSS is used properly and “skip links” are utilized to enable vision impaired people to navigate the site easily all contribute to creating an accessible site.

To enable a vision impaired person to browse the website and read navigation and copy easily, the site needs to be built using table-less development approaches using CSS (cascade style sheets) correctly by separating the layout and the structure of the site to create clean html code. This sounds very technical to the average marketer, but it is a little tip to help you gain brownie points with search engines and vision impaired people alike.

Are unaccessible sites discriminating against people who are blind? It is quite arrogant to think that vision impaired people do not surf the web. If your website is accessible it is most likely that it will be search engine friendly too.

Darren Fittler, a blind lawyer from Sydney who spoke at this year’s Webstock conference in Wellington demonstrates online how his site reader called JAWS reads a website. This hour long video can be viewed in windows media format.

By people naturally linking to your website, search engines become aware of its level of importance and its popularity. However, it’s not about the number of the links, but more the quality of the links.

Believe it or not, the web is divided up into good and bad neighbourhoods so where the link comes from is equally as important. Links from government, education and non-for-profit organisations are more trusted by search engines than the usual .co.nz or .com site.

Since the Internet today is a competitive marketplace, raising the popularity of your site is increasingly becoming an integral component of an online marketing plan. Link building activities can include online pr, placing link bait on the site, blogging, content distribution, tagging, social bookmarking and good old fashioned advertising to raise the awareness of the site including PPC (pay-per-click) or banner advertising.

As well as getting the right words in the right places on your website, you need to get the words that you want to rank well for in search engines used in the anchor text of links going to your site. The words used in anchor text communicate to the search engine what the next page is about.

If many websites or blogs were linking to a page using the term “epson printers” in the text link, then this will help boost the ranking of that “linked to” web page for that term in search engine result pages. The way search engines think is “If everyone is saying that this page is all about epson printers, then it must be true. Therefore we will show this page in our results when people search for this term”.

Achieving good quality links naturally under a Web 2.0 framework is both a science and an art. The game of online marketing has changed and good online marketers know this.