Top search ad placement is the coin of today’s Internet services realm. An entire industry of search engine marketing companies have emerged to help advertisers optimize their ad campaigns to deliver high returns on investment, peddling a plethora of advertising strategies. Yet, very little is publicly known about the effectiveness of online search advertising, especially when trying to compare the various campaign strategies used by advertisers. This paper presents the first large-scale measurement of the effectiveness—measured in terms of incremental conversion gains—of online search ads. We develop a simple metric called net acquisition benefit (NAB) that admits comparisons between the efficacy of different ad campaign strategies without access to advertisers’ private financial information. We study three common campaign strategies used by advertisers on a large search ad network: cannibalization, poaching, and ad extensions. Considering data from a month in the last two years, we employ NAB to identify cases where these campaign strategies are justified. Advertisers and ad agencies can replicate our methodology to apply it to other strategies of interest.