Posted by
Spotlight: On-demand video
AI Explainer: Foundation models and the next era of AI
Explore how the transformer architecture, larger models and more data, and in-context learning have helped advance AI from perception to creation.
Certainly the use of any of these tactics probably irritates web users. Just as certainly, anybody with even a modest acquaintance with the web probably can cite at least one such ad-induced headache. The typical response might turn an old cliché on its head: “I don’t know much about annoying ads, but I know them when I see them.”
That, though, isn’t sufficient for Dan Goldstein and Siddharth Suri of Microsoft Research New York City. They want to know exactly what people mean when complaining about ad annoyances—and what cost web publishers incur when displaying such ads.
That’s the impetus behind the paper Preston McAfee of Google, investigates the annoying-ad phenomenon, noting that web display ads, though a major part of the Internet economy, can, on occasion, draw the ire of innocent web users. And they’re not the only ones.
Web publishers are paid for the number of ad impressions they can deliver. That leads to an interesting shortcoming: Publishers know how much money they make from annoying ads, but they don’t know the price they pay when frustrated users tire of the intrusions and abandon their websites entirely.
The paper’s authors conducted a two-experiment investigation to:
“We drew inspiration from a young researcher from the University of Washington, Michael Toomim,” says Suri, whose research focuses on networks, including algorithms for large-graph structure analysis and web-based behavioral experiments. “He was using a technique to study the value of user interfaces. We were wrestling with the question of how to evaluate the impact of these annoying display ads and realized that his method would help.”
The research involved conducting two separate but related experiments on Amazon’s Mechanical Turk online labor market. The first let people rate and comment on a large number of actual web display ads, thereby establishing sets of annoying and innocuous ads for the second experiment, which had users categorize emails for a per-message wage. Participants, who were free to quit at any time, were assigned to one of three different pay rates and were asked to categorize emails as “spam” or “not spam” in the presence of no ads, annoying ads, or innocuous ads.
The experimental design enabled the researchers to determine how much more a person would have to be paid to generate the same number of impressions in the presence of annoying ads, compared with the absence of ads or the presence only of innocuous ones. In the experiment, the authors found that they needed to pay people more than a dollar per thousand impressions to make up for the lost traffic caused by annoying ads, compared with showing innocuous ads or no ads at all.
But why do annoying display ads even exist? Surely substandard ads turn off users, who might shun the sites where such ads are presented, which hypothetically would mean fewer ad impressions, which would seem to lead to money-hungry sites losing ground in a cutthroat web version of survival of the fittest.
Not so fast, says Goldstein, a behavioral economics researcher. The problem, he says, is that the cost of annoying ads has been difficult to quantify—until now.
“Part of the problem may have to do with incentives inside the publisher,” Goldstein says. “Many salespeople have an incentive to sell as many display ads as possible in the short term. At the same time, those steering the direction of the company may be more focused on growing the reader base and building long-term loyalty.
“Historically, the costs of these ads have been difficult to measure, so it was unclear which camp had a stronger argument. Our experiments allow this type of measurement and give a first indication that the costs can be rather high. For example, if we had been paid a market rate to run these bad ads during our experiment, we would have lost money compared with showing no ads at all.”
Suri, Goldstein, and McAfee conclude their paper by proposing a theoretical model that relates ad quality to publisher market share. Their findings could affect the economics of the Internet ad market.
How so?
“In a number of ways,” Suri says. “First, a small group of users could rate a given ad, and then the publisher could charge the advertiser based on how annoying users rate this ad. Second, one could train a machine-learning algorithm with examples of good and bad ads, and then a publisher could run a new ad through the classifier and price the ad based on its predicted negative impact.
“Third, one could allow users to close ads—or swap in another ad. Those ads that get closed more often should be priced higher.”