Category: Search engines

On search engines & Yellow Pages

Yaron GalaiLeave a comment

Yellow_pages Interesting (though far from surprising) news yesterday from Yahoo! on evolving Yahoo Local into a de-facto Yellow Pages platform. Original announcement here, coverage by TechCrunch here, and by John Battelle here.

This reminded me of a an email I sent to a colleague last year. The background was a growing number of requests from Yellow Pages (YP) companies to use our services to expose their specific YP listings on the major search engines[1]. My email below (slightly edited to maintain  confidentiality, etc) was part of the discussion around this direction. You can substitute the name I chose (YP-Co.) for mostly any traditional yellow pages player out there as anecdotal evidence suggests that mostly all of them are either considering or already feeding their listings into the major search engines:

From a strategic point of view, I think YP-Co is going to do a big mistake by feeding their listings into Google. The way I see it, they’re digging their own grave, for the sake of short term benefits. Here’s why:

Google is quickly taking over YP-Co’s market, and YP-Co is lending them a generous hand. By feeding their listings to Google, Google gets the local coverage they badly need, while building their own advertiser base in parallel. For Google this just primes their real future business and makes it easier for them to conquer the local space directly; for YP-Co this is a good way to get a temporary spike in traffic and then slowly decline as Google builds more and more direct relationships. I can’t see any long term strategic value for YP-Co coming from this deal. I also can’t see them sustaining a long term business where they’re an expensive middle-man that essentially provides a dumbed-down version of what Google offers.

In addition, YP-Co are essentially starting to flood the search engines with advertisers competing with YP-Co on the same keywords and inflating YP-Co’s own marketing expenses. At the end of the day, a click going to a YP-Co plumber page is 10x more valuable to them than a click going directly to a random plumber. The YP-Co click has residual value and over time gets people to understand there’s value in searching for services within YP-Co. With every click going directly to a random plumber, they’re enforcing the perception that Google is actually THE go-to place for service providers, and further obscuring YP-Co. So not only are they hurting themselves by obscuring their brand value, in the process of doing that they’re inflating their marketing budgets on very important channels (push even as little as 2-3 advertisers on a category, and the YP-Co ads are either going away, or will cost them a shitload of money… )

eBay is a good example for getting this – They spend tons of money on SEM (probably the biggest spender in the world), but always promote only their category pages and never ever ever promote a specific seller page. They seem to be extremely strict about this policy, as they understand that 10x value thing between their own page and a random seller’s page. YP-Co’s case is even more extreme, as the ‘random seller’s’  page on eBay is at least under the eBay brand and within their framework… with YP-Co the links just go off to an external random site.

In short, it seems to me like YP-Co is in a lame dog chase after Google, while trying to convince themselves that they’re making smart moves ahead of the curve.

This is of course a classic example of companies caught in the eye of the Aggregation Paradox. Unfortunately, most companies blinded by the chase after next quarter’s numbers fall right into the Aggregation Paradox trap. And that’s what makes the companies that get it (Google, Yahoo, etc) such wonderful businesses.

[1] Disclosures, disclaimers, clarifications and other vegetables:
A distinction should be made between two cases of promoting yellow pages (YP) content on search engines:

  1. Promoting of the YP category pages on the search engines. Example here.
  2. Promoting of specific listings (or – businesses) that are listed within the YP database.

While I sound skeptical about the 2nd type of SEM, I applaud YP companies for being smart and practicing the 1st type.

Beyond the obvious fact that this attracts highly qualified traffic, the 1st type of SEM actually increases the YP’s brand and the importance of their site. A user reaching a category page on a YP site must still engage with the site (drill down, search, browse, whatever) in order to see the business listings. Next time s/he is looking for a business there are higher chances of him/her going directly to the YP site and skipping the (now) redundant search engines step. That is great for the YP and helps maintain a sustainable and even growing business.

It’s the other type of promotion, that of specific business listings within the YP, that I am referring to in my mail and which I think is a slippery slope in the long term.

And to the disclosure – My company is involved in different ways in both flavors of YP promotion on search engines.

 

Share this post!

The Aggregation Paradox, Part 2

Yaron GalaiLeave a comment

In a previous post I outlined the Aggregation Paradox and its ramifications on a variety of online businesses.

A curious fact is that one category of web companies has been able to completely avoid being aggregated. It’s really quite fascinating, because this category includes the biggest aggregators of them all – the search engines.

Theoretically, a company with very little infrastructure could crawl Google’s search result pages (SERP’s) and provide an almost identical service to Google sans the costs involved with crawling and indexing the entire web. Moreover, the company could scrape and aggregate results from multiple search engines and provide a superior user experience with a fraction of the cost.

I know – this all sounds like the good old meta-search engines which have been around since, well, just about since search engines have been around. But meta-search never really took off, while aggregation in so many other categories has in a big way. Why is that? True – there are some technological barriers that make search trickier to aggregate than some of other categories (size of the index, dynamic blurbs, etc). But those are all solvable. So what really makes search aggregation different?

Simple – The search engines, being the mothers of all aggregators, know better than anyone else the Aggregation Paradox and the long term risks it imposes and those who get aggregated.

The engines have all jumped through hoops over the years to prevent anyone from aggregating them in a serious way. They’ve placed legal, economic and technical barriers to prevent aggregation. Here, for example, is a piece from Google’s search T&C’s:

No Automated Querying
You may not send automated queries of any sort to Google’s system without express permission[1] in advance from Google. Note that "sending automated queries" includes, among other things:

  • using any software which sends queries to Google to determine how a website or webpage "ranks" on Google for various queries;
  • "meta-searching" Google; and
  • performing "offline" searches on Google.

Please do not write to Google to request permission to "meta-search" Google for a research project, as such requests will not be granted.

Kind of an interesting policy for a company that’s built entirely on NOT applying ANY of these restrictions to the sites that it crawls and aggregates. BTW – I can personally testify that this is probably one of Google’s most enforced terms. Try to aggregate a few search result pages, and Google’s legal team will come after you within minutes.

How do the search engines get away with this? Why isn’t anyone calling them out?! The answer lies of course in the Aggregation Paradox, and in who gets it (Google, Yahoo, etc) and who doesn’t (mostly everyone else that’s addicted to the short term traffic…).

More on the Aggregation Paradox, and possible solutions to it, soon.

[1] "Express permission" = paying Google so much per query that a meta-search will find it very hard to earn anything…

Share this post!

Danny Sullivan totally misses tagging

Yaron Galai1 Comment

In a recent post, Danny Sullivan goes ballistic about social tagging (such as del.icio.us) and its usefulness for improving search engines:

…we’ve had tagging on the web for going on 10 years, and the experience on the search side is that it can’t be trusted.

The meta keywords tag has been around for nearly a decade. The idea behind it in part was that people could use the tag to classify what their pages are about, as well as provide copy that search engines could index if the relevant text for some reason couldn’t fit on the page itself.

…In addition, none — NONE! — of these search engines now or ever has made use of the tag in a way to let you perhaps see all the pages "tagged" to be on a particular subject. Why not? The data is largely useless.

Mark me extremely dubious that tagging will make major inroads in improving search. And if I’m wrong, I’ll happily mea cupla. But after 10 years of tagging, the experience so far gives me good reason to be dubious.

Danny misses the whole point. Meta-tags and ‘bookmark tags’[1] like del.icio.us are totally different beasts. The difference is not, as Danny points out, the fact that meta-tags are self-provided tags and del.icio.us tags are not – that hardly matters. The huge difference is the basic underlying value proposition that each of these tagging types offers:

Meta-tags were designed primarily for exposing certain keywords to a search engine as a means for improving ranking on those keywords. That is their sole purpose and sole value. So I, as a creator of meta-tags for my site, am not ‘awarded’ for creating good meta-tags, or useful or relevant ones. I am awarded only by getting improved search rankings and increased traffic on those keywords I choose. There’s no personal value gained by writing good meta-tags… on the contrary – if everyone else is doing keyword stuffing, I may drop in ranking if I write proper meta-tags.

This underlying design of meta-tags almost incentivizes misuse and is therefore fundamentally flawed. No wonder none of the search engines use a flawed system.

Bookmark tagging solutions like del.icio.us on the other hand provide a fundamentally different value proposition. As a del.icio.us tagger, I am organizing interesting pages for myself so that I can go back to them in the future and easily find them. I am doing it for pure egoistic reasons, and I have all the reasons in the world to do a good job of it. Sure – whatever tags I put on del.icio.us may have some grander value for someone else (meme trackers, search engines, etc). But frankly I don’t really give a shit about those other uses when I’m tagging something. Therefore the underlying nature of social tagging apps (or ‘bookmark tagging’) incentivizes proper tagging[2].

Of course there could be attempts to abuse bookmark tags like del.icio.us if a significant financial incentive gets tied to their tags (in the form of using tags for search engine rankings).  But:

  1. The more people use it for their own selfish reasons, the more difficult it is to abuse it. Think of this as sort of a Wikipedia… sure – some folks may try to abuse it once in a while and change entries, but the more people you have using it legitimately, the more difficult it is for abusers to misuse the system.
  2. Abuse attempts are 100x easier to identify on a bookmark tagging service than they are on meta-tags or links because the usage patterns of a non-abuser (big variety of URL’s, variety of tags, etc) are so different than an abusing pattern (stuffing of specific tags on specific sites). In the meta-tag world there is hardly any such differentiation, making it extremely difficult to sort the valid from the abusive.

Saying del.icio.us tags are necessarily crap because meta-tags are, is like saying all self-published sites (aka ‘blogs’) are crap just because self-published sites on Geocities were crap 10 years ago. What’s the connection? Only the word ‘tag’ I guess… not much else.

I applaud Yahoo for foreseeing this and making one of the smartest moves in search in recent years (and I believe this was primarily a search move).

[1] The proper name for del.icio.us class applications is social bookmarking. However, for the purpose of this discussion, I think that bookmark tagging is probably a better name for these apps. While they do have a significant social aspect, the social piece has very little to do with the motivation people have in tagging on del.icio.us. The primary motivation is much more of a ‘bookmark’ motivation – ‘how can I save something so that I can easily find it later?’.

[2] I’ve quoted Joshua Porter in the past, but this applies here as well:

From now on I’m going to call this idea the “Del.icio.us Lesson”. This is the lesson that personal value precedes network value: that selfish use comes before shared use. We’re seeing it more and more everyday in services like Del.icio.us, Flickr, and is an interesting aspect of networked applications. Even though we’re definitely benefitting from the value of networked software, we’re still not doing so unless the software is valuable to us on a personal level first.

Share this post!