We present an app automation tool called Brahmastra for helping app stores and security researchers to test thirdparty components in mobile apps at runtime. The main challenge is that call sites that invoke third-party code may be deeply embedded in the app, beyond the reach of traditional GUI testing tools. Our approach uses static analysis to construct a page transition graph and discover execution paths to invoke third-party code. We then perform binary rewriting to “jump start” the third-party code by following the execution path, efficiently pruning out undesired executions. Compared with the state-of-theart GUI testing tools, Brahmastra is able to successfully analyse third-party code in 2.7x more apps and decrease test duration by a factor of 7. We use Brahmastra to uncover interesting results for two use cases: 175 out of 220 children’s apps we tested display ads that point to web pages that attempt to collect personal information, which is a potential violation of the Children’s Online Privacy Protection Act (COPPA); and 13 of the 200 apps with the Facebook SDK that we tested are vulnerable to a known access token attack.