The Yield Optimization Dilemma

Pickpocket_1
With Yahoo recently announcing their intention to join the rest of the civilized world in optimizing their text ads for yield (rather than for bids, which can be fairly meaningless in a PPC world[1]), and with Wall St shaking up a little, it’s a good time to look into an interesting, and generally overlooked, aspect of the yield optimization algorithms used by Google, Yahoo and MSN.

But before jumping into the Yield Optimization Dilemma, a little bit of background:

First came Overture and ranked results by cost-per-click bid. The highest bidder always got 1st place, etc.

Then came Google, and by combination of: a) having to bypass Overture’s patent on CPC-based ranking, and b) figuring out that CPC-ranking can, again, be quite meaningless[1], they introduced a yield-based algorithm for ranking ads.

Yield-based ranking basically means that ads are ranked according to the following formula:

Yield = CPC x CTR

Or in English – The yield of each ad is determined by what the advertiser is bidding (CPC) times the # of clicks users are actually committing on that ad.

The beauty of yield optimization is that it inherently improves the relevancy of the ads shown over time, and therefore is good for the publisher (more $$ money[2] for its screen real estate), good for the advertiser (ads shown where users "vote" them to be relevant), and obviously good for the ad network (again – more money).

On its surface, the yield optimization formula (Yield=CPCxCTR) has the feeling of being ‘scientifically true’, and can therefore always be applied to auction-based ad networks as-is.

But there’s a devious little detail in these formulas that is completely overlooked these days, but could be a major issue in times of recession and diminishing advertising budgets. This factor, which is baked into all the yield optimization algorithms out there, can be summarized as:

The Yield Optimization Dilemma – When optimizing display of ads for potential yield, should it be the publisher’s yield be optimized, or the ad network’s yield?

I know – it’s almost petty to mention this issue these days. Google is flush with ad dollars[2] and with multiple advertisers competing for every conceivable word mutation. With about $1B estimated of unspent ad budget by Google advertisers[3], Google can almost always show the best ad (=highest yielding) on all of its page views, both on owned properties (Google.com, Gmail, etc) and on 3rd parties via the AdSense network.

But if the advertising history has taught us one thing, it’s that the ad market is cyclical and very sensitive to recessions. After every advertising bull run, it is almost guaranteed that a recession will kick in. And when that happens, the first budgets to be slashed are usually the advertising budgets.

When that day comes, and I bet it does[4], this is basically what will happen within the black boxes of Google, Yahoo and MSN millions of times per day:
"We have 1 ad with a remaining budget of $X which is the best yielding ad for keyword Y. That keyword has just been submitted by an AdSense/YPN/MSNwhatever partner, but we predict this keyword will be submitted to our own search engine (Google.com/Yahoo.com/Live.com) 100 times during the remainder of the day. Should we serve Great Ad to the partner site, or keep it for later for our property?"

Remember that every such decision is amplified about 4x by the revenue-share factor (=Google/Yahoo/MSN do not share revenue on clicks on their owned properties and therefore they make about 3-4x more on each click generated on those properties versus clicks on sites within their networks).

In numbers this is how this decision might look on 2 ads, one of which is yielding a $1 eCPM, and the other yielding $1.5 (assuming a rev-share of 70-30 with the publisher[5]):

  1. Do-No-Evil algorithm (or – Good Ad served on partner site) – publisher makes an effective CPM of $1.05, while Google makes an effective CPM of $1.45
  2. Do-Evil-As-Long-As-Nobody-Notices algorithm (or – Good Ad kept for Google properties) – Publisher will drop to a $0.70 eCPM while Google’s eCPM jumps to $1.80

That’s a 25% difference in revenues for the ad networks operating their own properties, all with a simple flick of an algorithm that only they control, and probably only they truly understand.

In times of advertising recession, when this Yield Optimization Dilemma will pop up on servers many many times a day, the decisions made may amount to tens or even hundreds of millions of dollars annually going (or more likely – NOT going) into publishers’ pockets.

Publishers should be aware of this, as it’s part of the cost of doing business when handing ad management over to companies that are big publishers themselves and have a huge financial interest in monetizing their own content before they monetize a partner’s site.

 

[1] I say that CPC-based ranking can be meaningless, because an advertiser can bid $50 per click yet have an ad so irrelevant that no one ever clicks on it, making its overall yield for Yahoo and the publisher a nice round $0 for all the impressions it was shown on.

That $50 ad may be "pushing out" an ad with a 50c bid that’s extremely relevant to the keyword and can get tons of clicks, making it a very high yielding ad.

[2] This one’s for you, Tami…. 😉

[3] As estimated by Goldman Sachs analyst Anthony Noto during Google’s recent analyst day.

[4] Of course Google is much more resilient to such economic downturns, thanks both to it’s large advertiser base (~350K?), and the fact that it is heavily geared towards direct marketers who are much less likely to slash ad spending than brand advertisers. But if I have to place my chips, I will bet that a recession will eventually hit, and even mighty Google will find itself with more content than it can supply ads and budgets for.

[5] This is obviously an extremely simplified example. There are many other parameters in the real world, but the genera idea is basically the same.

Share this post!

Yahoo & MySpace – the real story

JenSense had a great post (as usual) about Yahoo pulling its YPN ads out of MySpace:

MySpace has long been a marketing tool and a great way to drive traffic to other sites or advertising. But if you happen to be monetizing with MySpace and using Yahoo Publisher Network, you may get your account suspended, even if you have non-MySpace sites on your account as well.

Apparently YPN is shutting down the publisher accounts that are placing ads on MySpace.

So what’s really going on here? Why is YPN giving up on so many potential page views that it probably badly wants? Their termination email to the publishers mentions the reason being "poor traffic quality", but the real reason is probably more like poor targeting combined with too much traffic. Here’s the real story[1]:

While contextual advertising (such as Google’s AdSense and Yahoo’s YPN) is an amazing way to monetize content pages, there are fairly big pockets of content that are terrible for contextual matching. The main examples that come to mind are:

  • Finance pages – A reader of a financial site reading an article about Cisco is not really interested in buying a Cisco router.
  • Dating sites and social networks – The content of a dating page or social network page (about me, my favorite pet, hobbies, etc) has little or no connection to what the user is looking for while on those pages.
  • Home pages – Notorious for being difficult for contextual algorithms because they usually contain lots of very dynamic bits of data on a wide variety of topics.

But that hasn’t stopped Yahoo (nor Google) from placing ads on any of these pages in the past, so why is MySpace different?

Yahoo’s dirty little secret is that when the pages don’t contextualize well, Yahoo ignores relevancy and simply picks ads with high bids such as mortgage ads, credit card ads, etc. So even though click-through-rates on these pages are fairly low, Yahoo makes up for it because the bids on these categories are high (usually in the $2-5 range per click).

Of course the advertisers on those categories never intended to appear on pages that aren’t relevant for, well, credit cards, mortgages, etc. They’re bidding $5 to appear on a SERP or content page related to what they’re selling. They’re expecting a qualified lead for their buck. So how can Yahoo pull this trick without pissing off its advertisers?

The secret is all in the proportion that Yahoo (or Google) can maintain between their highly targeted search placements and those irrelevant placements described above above. As long as the mortgage advertiser is seeing a big % of the traffic coming from Yahoo’s search, the non-relevant clicks get diluted in the mix and fly under the radar.

But as soon as this proportion is disrupted by adding the massive MySpace exposure, those bad clicks stopped flying under the advertisers’ radars. I bet Yahoo got very angry calls from advertisers surprised to see they were being placed on pages they never intended to be on and that have no relevance to the keywords they were bidding on.

In ways, this reminds me of the old scheme for robbing a bank by shaving the rounding errors of each transaction. This works great as long as you keep shaving fractions of a penny from each transaction, but scales pretty poorly if you start taking dollars from each transaction.

The quality of traffic on MySpace is not inherently bad. The trouble is not ‘poor quality traffic’ as Yahoo is trying to claim, but rather intentional goofy ad targeting that simply doesn’t scale very well…

[UPDATE] As I mention in the footnote below, this post is based purely on educated speculation. Yahoo posted some more clarifications on this issue on their YPN Blog, so go there for the real ‘real story’.

[1] Well, scientifically speaking this is more speculation than ‘the real story’, but being that I work for Quigo (<–that was an embedded disclosure…), I think I have a fairly good insight into the dynamics of this story. Speaking of which, I think we did a fairly good job at solving the problems described in this post. But that’s for another post maybe.

Share this post!

The Portal Dilemma

Yahoo just announced its entry into the tech publishing world with the launch of Yahoo Tech. Coverage by Mike Arrington, NY Times and John Battelle.

As I mentioned in this post, the big portals (Google, Yahoo and soon MSN) are powering the ad platforms of many media companies while at the same time creating their own competing content.

Here is what’s really going on:

Yahoo and Google get 100% of the revenues on each click happening on their pages, but only about 20-30% of that same click taking place on a publisher page (via Google AdSense or Yahoo’s YPN). So a user on a Yahoo/Google page is worth to them about 4-5x what the same user does on a partner publisher page.

The dilemma therefore is not really much of a dilemma at all – A portal will always strive in the long run to create its own content and attract as many users directly to it, and away from its partner publishers.

By powering the ad sales on other media companies via AdSense/YPN, the portals are leeching on these assets to grow their own advertiser base to be used on their content pages, like Yahoo Tech.

At the end of the game (as I described in this post), the portal stays with the traffic and the advertisers, while the publisher stays with declining traffic and no advertisers.

So when choosing a partner for powering a content-targeted ad network, the first question a publisher should ask is: Am I being leeched by a portal that wants to take away my business?

If the answer is ‘yes’ (or even if it’s ‘no'[1]), you should be looking at independent non-competing companies like Quigo (disclosure: I’m founder and employee of Quigo), IndustryBrains, etc.

[1] If Google or Yahoo ain’t competing with your content or service today, take my word on it – they will! The 5x dilemma just makes it such a non-brainer for them over time to compete on content and services and attract the users over to their pages, even if they spend all their time denying this.

Share this post!