I have had a clean record with Google for many years now, survived dances, updates, Panda attacks and (other than technical issues) avoided penalties and problems with the big G, watching traffic grow or at least remain stable for a wide variety of sites I manage. Until now. This week I was hit with a problem, i.e. the Penguin update and I discovered that I may have been the victim of negative SEO.

The Penguin update and my appeal to Google
This week the Penguin update happened, the first update that has ever had a seriously negative effect on sites I manage. And all the sites affected are sites I own. For the record, the Penguin update is meant to weed out web spam. Cool story, bro G. many of us who own affiliate sites and review sites would like to differ. I had a review site drop from 40 visits a day to 0. I had affiliate sites lose anywhere from 30 to 50 percent of their traffic. None of these sites uses spammy tactics but offers a unique user experience that pre-Penguin Google was just fine with.

My favourite site on the web, beerclassifieds.com lost about 40% of its traffic. It dropped rankings for top keywords from 1st to 4th, from 2nd to 7th and more like that. Enough to seriously affect income, not enough to be a true penalty or whack.

So Google offers a form to appeal the Penguin update (You can also sign a petition asking Google to roll it back. I did, for whatever it’s worth.) The form asks you to point out a SERP that has less than ideal results. Below is the appeal I wrote, and I think my complaint is legitimate. (let me reiterate: seriously, a Yahoo Directory page??)

Penguin feedback
This query ["beer antiques" was the one I chose but there were others] is one of several queries related to this site for which this site’s rankings have fallen. I realize that I am coming from the perspective of self-interest. And yet… The listings on google.com are all much less helpful than my site. My site offers helpful information on antique valuations, advice about where to buy, a look at historical context for many collectibles and antiques and it lists actual antiques from (yes, my affiliation with) eBay. (The listings on my site, though, are sorted and sortable and organized based on what people look for in this marketplace, and I believe I have created a very unique, worthwhile user experience with this site.)

The sites ranked above me include (among others) a Yahoo directory page (seriously?), a horrible user experience like this: [site name removed] and a mall in Denver! ([site name removed]) I update my site frequently, engage with social media and otherwise seek to make this a very lively site unlike others that suddenly outrank me now. I have seen other search results where youtube pages and other unhelpful resources also rank above sites of mine that are more relevant to the user’s query (e.g. “paintball sniper rifles.” People are not looking for videos, they are shopping, period). I believe these are all “non-ideal” search results. For my part, I will go back to work and make my sites even better than they already are, but I honestly believe this update needs tweaking. Thank you for listening.

Reconsideration request and negative SEO
I actually filed a reconsideration request last night. I have always prided myself on being strictly white hat. My site cheapfobs.com has a somewhat unfortunately spammy-sounding domain name but it offers comprehensive programming instructions for many, many makes and models of vehicles and it actually lives up to its name of offering the lowest prices online for keyless entry remotes thanks to eBay.

I think my Penguin complaints might be peeing in the wind but I think I have a legit concern here and the traffic graph for this site is crazy wrong. When I looked at the link graph for this site it looks VERY suspicious. But I did nothing to acquire many of the spammy links it has recently acquired.
Here’s a cap of some of the absolutely atrocious-looking domains that now link to my site.

So the wording of my request (for interest’s sake) goes like this:

Dear Google,
I am writing with regard to the site cheapfobs.com to request that you reconsider its rankings and its value to users compared to competing sites in niches it covers. I have checked technical issues and content issues related to this site and nothing has changed drastically in the three years I have owned this site, other than two site outages last summer for which I paid a temporary penalty last fall. After the site was reinstated into SERPs it again dropped off recently, with no technical issues to explain this.

My best guess concerning the problem with this site is that I have been the victim of negative SEO. I realize that sounds disingenuous or dubious perhaps but I have worked very hard to keep this site’s link portfolio very clean. I did buy this site several years ago, but from a publisher who is also very reputable. The link portfolio for the site was entirely from good sites where myself or the previous owner offered links to users to relevant content on cheapfobs.com. There are also article sites and some guest posts, which as far as I understand are part of the norm in today’s web marketing world.

However, in a recent check of the site’s link portfolio I came across many links that appear very spammy. Recently, the site has received a high number of links from Polish and East European websites and domains such as the following:
[bad links listed here]

I did nothing to ask for these links from these horrible sounding domain names! Many of these “links” do not work now but show up in Backlink Explorer, the tool I use to track my profile. I do not know how to remove them but to me they seem like someone has done negative SEO on my site.

I would like to know if anyone has filed a spam report on my site and if this is the reason my site has suffered a big loss in traffic in recent months. If someone has, then I think this is a clear indication of Google Bowling or negative SEO, wherein a competitor has purchased links pointing to my site with the intention of deceiving Google into thinking that I have schemed to get these links. If you think that I have bought these links or someone has reported that I have paid for such spammy links I can only say that I would never do any such thing!

I have a five year history with Analytics and Webmaster Tools. I own several sites and manage content for a number of others and I have always used white hat tactics and coached others to do the same. In fact, many of my sites have a very low number of links because I avoid link penalties out of fear of getting ‘whacked’ (I once long ago worked for a company that was penalized by Google Adwords for arbitrage so I am very paranoid!)

If my site has been manually banned in Google, please consider de-penalizing my site, cheapfobs.com.

I’m currently working with a number of newer companies who are showing varying degrees of commitment to this process. I can point them to various examples of websites where I took over the SEO and helped them grow traffic but here I’ll offer a fresh look at a case study in SEO investment. This is a broad look at a case that is, in some ways a uniquely successful case (as all successes are) but in other ways typical and even predictable.

The initial approach
This company approached me about two years ago around the middle of 2009 and told me that they were spending X amount of dollars (a mid three figures number) per month on SEO and it wasn’t doing anything for them. Unfortunately, there are many companies and people (let me emphasize unfortunately) who take money in the SEO business without delivering growth or, at least, measurable value in some respect. This company had built some backlinks from some time before but other than that had done little to continue growing traffic to the client’s website.

Every SEO has their own unique strengths
When I looked at the website’s situation, I realized that I could do a lot to grow the site’s traffic through content development. This is an admitted strength of mine (hence the name of this site) and is a focus of mine vs. the focus of many SEO’s which is linkbuilding (while there are still others who, it seems, are questionable as to whether they have any focus).

The work
I took over the site, got to work and started developing content, telling the site owners that if they invested about $1000 per month I would double their search engine traffic in 6 months time: from about 50 visitors per day to 100 for a concentrated niche that was somewhat competitive (for page one results, anyway) and also highly interested. I never quite topped out on this budget as I follow the old adage to under-promise and over-deliver.

Since the company was already investing money in SEO, though, the investment was relatively easy for them.

Here’s a graph of those first six months:

What you see in the graph is that the traffic was basically flat-lined when we commenced work on SEO for this site. (I say ‘we’ because I did work closely with the site’s developer on some critical changes.) During initial work, traffic rose only very slowly, so little that it barely registers on this graph. We could see progress in terms of pages getting indexed and getting some traffic from new keywords.

However, there was no great payoff and we had promised the client that traffic would double within six months. They kept spending the requisite high three figures per month, though. They may have been growing worried, I’m not sure, but I don’t think I was.

What happened was that there was a cumulative effect of activating the site’s development and continuing to add content as well as working with partners to create links into the site, then tweaking the site structure as we went along. All of that work cumulatively added up to an eventual spike in traffic.

And that’s the way it goes.

After this, the site owners felt that the site had grown enough and they were happy with the amount of traffic they were getting for the investment they had put into it. Of course, others like continued growth, although a doubling of traffic every six months is not likely, especially as you start to top out on terms that you can truly rank for and as you run out of content ideas (as happens with sites that are focused on a small industry or consumer niches).

The point in all this, of course, is that your search engine optimization payout is often frontloaded but in the end there is a payoff. It hurts at first but in the end you will enjoy results and from there, clients can decide to continue to grow traffic or simply enjoy the benefits of your investment for a while… as long as your competition stays asleep.

First, some history…
When I went to my first Search Engine Strategies conference in Toronto in 2005, the Keynote Speaker was Danny Sullivan. He talked about how he was one of the first people to take search seriously. He was critical to the organization of many of the first SES conferences and he was told as early as the late 1990′s that “search engine optimization probably has about three to five years.” Well, by then it was already eight or more years later, he noted, and the crowd at that conference was the biggest yet, as were crowds at all conferences. That was over five years ago. In 2007, Jakob Nielsen said, “I am pessimistic about the long-term prospects for SEO jobs…. but maybe I am wrong.”

And then there’s the old saw, “SEO is dead.” An extremely prophetic fellow from 2008 gets number one ranking for a search for the phrase; according to his post, the industry will be dead by November of this year (right after the rapture?) Recently this other prophet said it again. When Google Caffeine was announced, other (in this case, remarkably badly informed) soothsayers said that this was ‘it’ for SEO; ditto for this year’s Panda I’m sure, although I may have grown blind to seeing the phrase by now, so I don’t remember seeing it in print anywhere. Clickz published this “SEO is Dead” post in 2007 and then went on to proclaim the value of SEO in another article, three years later. The latter article (predictably perhaps) generated more comments, interest and agreement.

Every one of these sage prophets of doom (like all prophets of doom, we might add) has been dead wrong. (Of course, though, being wrong is no reason to stop saying the very same thing again.)

Is SEO here to stay?
Probably, people.

In early 2010, SEMPO announced that companies predicted an SEO spend of 14% increase over what they’d spent in 2009 (long after SEO should have been long dead).

And later in the year, Rand Fishkin released details of his 2010 survey of the industry. The reason it was a bit late was because he got more responses than he ever bargained for. The industry was growing much more than even he realized. One survey question discussed spending and survey results told that even in the US (where outsourcing and a weakened economy should have taken a big bite out of the market), over 70% of respondents indicated an increase in demand for SEO services.

All forecasts for the year 2011 show that search engine optimization and search marketing budgets rule the day for any company serious about getting found on the web. Budgets for SEO (and its close sister SEM) are growing and show no signs whatsoever of slowing down. SEMPO predicts that 2011′s increase over 2010 will be greater again – not just in money but in percentage of growth (16% this time).

In other words, the search engine marketing and search engine optimization industries are still vectoring! After all this time. So where, O death is now thy sting?

Like Rock and Roll, SEO will never die. And here’s why.
All of this reminds me of a feature article I wrote in graduate school about rock and roll music. In the essay I discussed how rock music had been declared ‘dead’ throughout its history (by Don McLean, by The Who and by a lot of other people no one should remember, including Lenny Kravitz). But like Neil Young rebutted, rock and roll will never die. I argued that the reason for this is simple. It’s because the idea is timeless: rebellion, anger, and crashing away on drums and guitars will never, ever go out of vogue permanently. It might change, it might go through some valleys but by the nature of the beast, there will always be a way to renew it. It’s not necessarily tied to a form but to an idea with staying power.

The same holds true for SEO.

Why? Simple: because search will always be done by machines, based on algorithms and there will always be a gap in the logic of the algorithm. And if we’ve learned anything from the Terminator movies, the Matrix and every other movie that explores the Frankenstein complex it is that people are more resilient and adaptable than machines. There will always be a way to game search results for profit. And of course if you didn’t know this, search is never going to go away, either, because people will always be looking for new information.

And when they do they will use a machine that is using an algorithm to search for that information and that algorithm will give privilege to certain qualities of the information and someone somewhere will be gaming that algorithm in order to be seen as the source for that information… because there will be money in it.

And I’ll bet that 20 years from now… 30, 40, 50 years from now, people will be listening to rock music as they use a search engine (I’ll even bet Google, in some form), and someone somewhere will be listening to Neil Young as they do so, and they will be singing along “…There’s more to the picture than meets the eye…”

Update:  Embarrassingly, Leo Laporte said it at PubCon 2011.

In the early days of "hypermedia" relevant content would be easily and freely linked

Recently I answered a question on a forum by pointing the questioner to a site I work for, adding an html link. You know, those things the Web is supposed to be made of? Oddly, the forum’s owners allow links to be active, no questions, no nofollow, no problem. The backlink probably helped the ranking of my client’s site a little… but what else happened was a bit unusual. The link referred scads of traffic to the client’s site. As in, noticeable amounts. In fact, nearly as much as the site currently gets from Google for generic search terms.

Pretty cool.

….But then again that’s what good backlinks are supposed to be really about. I mean, right? If a link is relevant to the rest of the content on the page and useful to the page’s visitors it ought to improve the rank of the page linked to. Right? Yet, I doubt that this is accounted for in Google’s algorithm. A link is either nofollowed or followed and that’s probably as sophisticated as Google gets. I’m speculating but with good reason.

Anyway, what’s sadly surprising is that I’m surprised.

How did we get here, I wonder? How did we get to the point where a backlink that actually directs traffic is noteworthy and out of the ordinary? Why do some sites refuse to allow links that ought to drive web traffic (yep, you again, Yahoo Answers). Is it all Google’s fault for over-determining links and not coming up with a way of better grading them algorithmically?

Maybe.*

But anyway, for now, I’m happy that my client is getting so much traffic from a legitimate link to their site.

Do you honestly think Google grades links algorithmically in terms of how effective they are at driving traffic to the target site? Incorporating things like bounce rate, time on (target) site, etc? How much weight do you think they put on any of those variables versus plain old, easy-to-understand follow vs. nofollow?

*Off topic: but some people who I don’t agree with think that links are bad. (This is a whole other argument, of course.)

People expect security
Don’t be afraid to create a login for your application or ask mobile users to login. Make this easy (say, ask only for a user name) but protect user security while they are mobile. Mobile usability consultants are sure that users like this or at least don’t mind it.

Don’t worry about iPad applications
The iPad is still running along nicely on all the hype it has had this year but you need an iPad app like you need a hole in your wallet. Seriously, if anyone tells you about their expertise in designing for the iPad, start walking away. Web browsers work fine on the iPad and as long as your website works well on a laptop it will work fine on the iPad.  (Update Oct 27, some confirmation from eMarketer: “mobile device users appear to think browsers offer the better user experience” even beyond the iPad.)

Get listed locally
This ought to be your number one SEO concern when it comes to mobile and it may be the only one you’ll ever need. Most SEO best practices for mobile are the same as they are for desktop but local (e.g. Google Places) has gotta be hu-u-uge for mobile.

If you think about it, there’s every reason to think that Google created local listings partly in anticipation of mobile’s growth. If Google’s local listings are not already generated based on your GPS signal, Google will be rolling this out soon, you can bet.

Quick tip about Google Places: If your company or chain has multiple offices or stores, be sure to list the office or outlet closest to the city centre. There are a number of SEO people who advise that local listings are ranked to some degree based on proximity to city center. I’m not saying this will help your ranking in Google Places but I’m 100% sure it won’t hurt.

One more thing: I’ve had to tell this to a number of people lately: Google Places is free. Free, no cost, I’m 100% sure of that. Oh, and you should also list locally with Bing and any of those other guys, too.

This is a very young demographic
That doesn’t mean you only focus on trendy 2.0 graphics and all that. This means that you understand that this group behaves fundamentally differently in many ways.

They think differently. Here’s one example: younger users are much less likely to want to use a call center and if they do they are going to be ashamed to have to do it. Make help available and make it easy for them to help themselves and figure out how to do it on their own.

Note that a significant portion of this demographic is using their parents’ money. They may need their parents’ approval to buy certain things or do certain things. Figure out how to use this in your mobile site or application without it being intrusive or embarrassing.

Ensure usability if you’re serious
The best way to be sure that your site or application is mobile ready is to hire an experienced mobile usability company

That ideal and others (e.g. the Star Trek scenario of being able to yell out at a ‘Computer!’ that waits for us to address it, inside the ceiling) are all still a ways away. Still, it’s an exciting step forward.

Matt Cutts, meanwhile, was quick to add that SEOs will still have jobs (presumably assuring himself simultaneously that there will always be a need for a Google spam cop).

Update: there’s a dev site where you can try Google Instant.

Update (Sept 11, 2010) :It has not yet rolled out to Google.ca but Google.com is Instant.

My opinion? It is freaking cool – and a useful, completely worthwhile improvement.

In any case, it’s fun to see that that page still ranks well and to see that canoe.ca or anyone else has not even thought of trying to come up with their own list. I get an email about once a month, on average. In some cases these are quite interesting. One of the most recent ones I got was from Sammy Kohn, drummer for the Watchmen, who wrote to lament that they were not included originally, which I admitted required correction. I just got another email from someone in Tampa Bay, in fact.

It’s a case in point that creating a unique web page and doing it properly can have rewards over many years. When you appear at or near the top of Google results, people believe you are authoritative… apparently whether you are or not. All this from simply throwing up a page that was simply optimized properly, even if it was highly self indulgent (and remains so).

Anyway, for anyone who wants to comment on the list (which I just updated) in the future, please comment below.

Cheers!

I should admit that not everything I work on has been a great success. In almost all cases where things have not worked out as well or have had middling results, there is a shortage of commitment from companies.

I can do some SEO that will keep your company afloat but you need to help me out. You need to update the design of your site, for example, especially as more pages are added and your site’s design falls farther behind the times and changes to search engine algorithms.

That being said, I am still pleased to be well ahead of the curve in some cases and doing stuff that still works, in other cases.

I should blog more – but I am just overbooked.

Any change to Google’s algorithm is of course big news, spawning reams of Internet chatter, a whole lot of speculation and probably some fear-mongering, along with some brand new websites (e.g. Compare Caffeine!)

What exactly Google Caffeine will do, though, is largely unknown even though some have reported that it will be a change to Google’s indexing methods and Matt Cutts (and webmasters who tried the beta version) note that there will be changes to rankings. One thing that will not change will be the look and feel of Google.

A big change noted by many (of us) who previewed Google Caffeine was the speed of results – it’s about twice as fast, if you can believe that.

Matt Cutts summarizes on his blog:

The Caffeine update isn’t about making some UI changes here or there. Currently, even power users won’t notice much of a difference at all. This update is primarily under the hood: we’re rewriting the foundation of some of our infrastructure. But some of the search results do change.

While Google has very kindly (mercifully, for those who will inevitably move down) held off until after the Christmas season, that is largely because of flack they have received in the past. It does not seem to be because Caffeine will have as profound an effect on search results as have major updates of the past.

Lilengine.com provides a good overview of changes it found in a comparison of Caffeine to “Old Google” (i.e. the one we’re using today). Some changes they identified are as follows:

* Google Sitelinks disappearing from some results.
* Rich Snippets disappearing from some results.
* Caffeine has difficulties handling 301’s. ( This is a bug )
* Page 1 results mostly contain the same sites, however positioning is quiet different
* Page 3 and onwards seem generate completely different results.
* The index page on root domain now has more weight.
* Pages with heavy keyword stuffing are been penalised.
* Pagerank Sculpting is no longer effective. ( further confirming the rel=nofollow topic )
* Brand name / Domain trust further effecting SERP. ( a step further in the Google Vince Update? )
* Number of inbound links from external domains
* External links using targeted keywords as anchors carry more weight.

The last four points are nothing new by the way. It seems to me, then, that Caffeine will simply entail that SEOs continue to explore the long tail of relevant content, staying away from suicidal blackhat practices like keyword stuffing and over optimization and continuing to acquire legitimate organic links. So once again, Caffeine seems like no big news for SEO, although it might be relatively big news in Search.

What Caffeine actually looks like and how it will actually affect all us webmasters will, of course, have to wait to be seen. I suspect it will affect legitimate sites very little and it will only give webmasters and SEOs more work to do. In any case, we will all find out for sure, come 2010.

To learn more, see:
http://www.readwriteweb.com
http://blog.360i.com/
http://www.computerweekly.com

Google Caffeine
The buzz is that this is the biggest change to Google’s algorithm since late 2003 (aka the Florida update/dance). It has been in beta for some time and is going live right now. I didn’t like the beta version (now retired) because it was only available for google.com.

Social Search
Google rolled out social search, which integrates your results with listings from friends in your various social networks. It’s been met with some criticism and it even gives some people the creeps. But expect Google to keep tweaking things. Maybe it’ll be a good Google product in the end; maybe it’ll be useless to most of us.

There is smaller news of course, such as the way Google’s local business results have really taken off in the past few months, squeezing publishers of directories and other local marketing and inciting SEOs to get pissed off about Google hogging search results. Twitter’s real time search was old news from late in 2008 but Twitter has continued to improve its search and overall functionality. Most importantly, Twitter is an undeniable Force-to-be-Reckoned-With when anyone discusses Internet marketing (if not SEO) today. Finally, also, Google dropped PageRank in Webmaster Tools, possibly in favor of Trustrank, one might wonder.

For all that, there is very little news in mainstream search engine optimization (SEO), as should perhaps be expected at this point in the game. A list of top 50 SEO tips from this week’s pubcon confirms it: not much new in the world of actual day-to-day SEO. That may bode poorly for professional SEOs, only because people may think that all they need to do is pick up an SEO starter guide from Google or an SEO primer from a third party SEM company or any of the other hundreds of sources of basic SEO information and think that that is all they need. I say ‘I don’t really think so’ because not one of these tools will match the value of having a professional SEO in charge. From Google’s to every other guide I’ve seen I see mainly dodgy tips, teasers only and flat-out red herrings (people who even mention ‘meta tag keywords,’ I’m looking at you.)

Search engine optimization may get rocked by Google Caffeine but SEOs will still be the best qualified to handle this and any other news in Search. Bits of news and occasional flurries of news are best understood in the context of the last five to ten years of changes at Google and other search engines.