Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

Microsoft Research Team Creates Seattle Gigapixel ArtZoom, an Interactive City Panorama

January 27, 2014 | By Microsoft blog editor

ZOOM OUT. A vibrant city on the water, surrounded by mountains, teeming with life.

ZOOM IN. Do I spy, with a 600mm lens, a “Thriller” flash mob under the Space Needle? Wait, and what’s over there? Do I spy an opera singer serenading Elliott Bay from Pier 69? And over there, near the Broad Street and Aurora ramp, do I spy a protest?

ZOOM OUT. On a sunny fall day, from the 25th-floor rooftop of a Belltown condo building, Seattle is a stunning city. From several vantage points on the roof, Microsoft principal research development lead Matt Uyttendaele and principal researcher Michael Cohen take it all in from the tallest building in the neighborhood.

Their team is taking more than 2,000 rapid-fire shots on this particular day, from just before noon through the afternoon. They’re using cameras and lenses more common to photographers covering a football game, and they’re going to use one of the many technologies they have created over their years working in Microsoft Research—the Image Composite Editor (ICE)—to stitch together a seamless, 20-gigapixel (20,000 megapixels), super-high-resolution, 360-degree panorama image from a set of overlapping shots taken from that rooftop. (Most cameras are lucky if they can attain 20 megapixels, so the kind of resolution in these images will … Blow. Your. Mind.)

The Seattle Gigapixel Art Zoom project pushes the boundaries of the ICE technology to deliver an image that enables anyone to pan and zoom through the city of Seattle at an amazing level of detail. In a fun twist, visitors to the site can discover more than 70 artists and performers in public spaces and in private buildings, near important Seattle landmarks and on random street corners, acting alone or interacting with onlookers. The team captured the city’s vibrancy by filling the panoramic image with dancers and actors, painters and poets, acrobats and burlesque queens. This interactive image, along with information, videos, and stories about the artists, can be found at Gigapixel ArtZoom.

ZOOM IN. That means that when you zoom in—and you will—you can drill down, find, and magnify these local artists all over Seattle doing all kinds of eye-catching activities. It’s a novel way to explore a city and discover surprises—including a marriage proposal—where you might least expect them, as hidden treasures are embedded throughout the scenery, as though it were all happening at one time.

ZOOM OUT. The team faced a laundry list of obstacles while trying to pull together this project, including 50-mph winds that shook their lenses and Seattle’s temperamental weather. And, there was also an unexpected number of wires throughout the city, so they had to move their subjects so that the lines wouldn’t obstruct their faces. But overall, they were able to find the sweet spot that brought the big picture into focus—to show the potential of the ICE technology and to immerse people in a place in a whole new way.

“We started looking at Seattle as our canvas,” adds Cohen, who not only spearheaded the project, but also did much of the location scouting and secured the condo building that was ground zero for the project. “And once the artists starting signing up, we knew we’d be able to create something special.”

ZOOM IN. With at least 12 Pacific Reporter law books under each foot, Eric Olson and Tessa Hulls were easy to find. They stood out at the Olympic Sculpture Park, near the fountain, emulating the pose of the stony figures. Tapping lots of core strength and some commendable concentration, Hulls balanced four books on her head, while Olson struggled with a stack of cardboard boxes.

But they pulled it off for the few seconds it took for the photographer to capture their performance.

Hulls, an artist who usually paints murals, is a voracious reader and was able to take advantage of a law firm’s conversion to digital media, grabbing boxes of books for this particular piece of performance art (apropos given Seattle’s rep as the second most literate city in the U.S.).

“It’s a fun project,” Hulls says. “I’m really glad they’re using technology to showcase local artists that reflect Seattle’s vibrant art community.”

ZOOM OUT. For 16 years, Elise Ballard has done a lot of work with Microsoft with various products, managing programs and projects that include Visual Studio, SQL Server, and Windows Server. But she’s never done anything like ArtZoom, which she was invited to coordinate through John Boylan, a site manager with MSDN, who has deep roots in Seattle’s sprawling—but also tight-knit—art community. The team also worked with King County’s 4Culture agency and Seattle’s Office of Arts & Culture to coordinate with a variety of local artists.

“Microsoft Research is out in front, on the bleeding edge of new technology, and the team’s vision of delivering a creative viewpoint of Seattle incorporating its art culture reflects the personal creativity of the researchers within the team,” Ballard says. “We’d worked to plan their vision, and the day of the shoot was absolutely exhilarating. It was crazy. I was literally running up and down streets of Seattle, with my walkie-talkie and a backpack, running from artist to artist, coordinating with people. It was a blast.”

The Gigapixel ArtZoom website includes the final Seattle Gigapixel image, information about the project, and individual artist pages to learn more about those featured throughout the image.

“The technology is spectacular. It’s beautiful,” Ballard says. “The way it can be used to express the feeling of a place is really amazing. The fact that you can place yourself on one rooftop and create a panorama that allows you to drill into street-level detail all-around the city with a virtually seamless image is astounding.”

ZOOM IN. You can create your own gigapixel panoramas, using your own camera and ICE. After downloading the program, it will do all the work of stitching together a series of photos (about 25 photos taken rotating incrementally 360 degrees is the equivalent of a panorama) into a zoomable panorama. (ICE imports the photos in the blink of an eye and figures out how to align all the shots taken from a single location.) And once you’ve created the perfect panorama, you’ll want to share it. Again, technology that originated in Microsoft Research can help you do that—Photosynth.

You can share the panorama you created with friends or upload it to the Photosynth website to view it in 3-D. ICE supports saving these images in many formats, including JPEG and TIFF. And if you want to create your own Photosynth photo on the fly, you can do it using the Photosynth mobile application, available for Windows Phone devices and iPhones.

ZOOM OUT. For Uyttendaele and Cohen, the project also yielded unscripted moments of serendipity.

“Sometimes, I never saw the artists through the viewfinder,” Uyttendaele says. “I had to trust that I got them.” And then, he’d see the pleasant surprise afterward—like the belly dancers on Alki or the acrobats who went all over the city.

They captured the look of astonishment from another passerby in a wheelchair (and his dog) watching a handstand sidewalk performance and diehard Seahawks fans turning up the volume at the stadium during a game. On the flip side, there were also folks who have been in Seattle long enough to not get surprised by much.

And now anyone can see that canvas, on their own time, and explore Seattle and its artists, who are more than ready for their close-up.

Up Next

Bei Liu and Jianlong Fu of Microsoft research

Computer vision, Graphics and multimedia

The poet in the machine: Auto-generation of poetry directly from images through multi-adversarial training – and a little inspiration

As the means of expressing thoughts and feelings too sublime and elusive to be conveyed in everyday language, poetry across all cultures occupies a most sophisticated and mysterious of realms, just beyond the outskirts of creativity. Along the language-based avenues of expression available to people, poetry represents a departure from casual speech; few possess the […]

Microsoft blog editor

Artificial intelligence, Computer vision, Graphics and multimedia

Teaching computers to see with Dr. Gang Hua

Episode 28, June 13, 2018 - Dr. Hua talks about how the latest advances in AI and machine learning are making big improvements on image recognition, video understanding and even the arts. He also explains the distributed ensemble approach to active learning, where humans and machines work together in the lab to get computer vision systems ready to see and interpret the open world.

Microsoft blog editor

Microsoft Pix before and after panoramic photo of Miners Landing

Artificial intelligence, Graphics and multimedia

New Microsoft Pix features let you take bigger, wider pictures and turns your videos into comics

Microsoft has released two new features with today’s update to Microsoft Pix for iOS, an app powered by a suite of intelligent algorithms developed by Microsoft researchers to take the guesswork out of getting beautiful photos and videos. The first of these features, Photosynth, helps create photos that take in more of the perspective or […]

Nicky Budd-Thanos