Judging and Then Some …

Published

Posted by Bill Buxton

Spotlight: On-demand video

AI Explainer: Foundation models ​and the next era of AI

Explore how the transformer architecture, larger models and more data, and in-context learning have helped advance AI from perception to creation.

Imagine Cup 2013 logo

Editor’s Note: Bill Buxton, a principal researcher for Microsoft Research and a relentless advocate for innovation and effective design, is sharing his experience as a judge for the Imagine Cup Worldwide Finals. See Part One and Part Two.

Tuesday, July 9

Tuesday and Wednesday were intense and busy days for students and judges alike. These were the days of judging. The projects in the competition fell into three main categories: Games, World Citizenship, and Innovation, each of which had its own international panel of judges, typically four per panel. Furthermore, there were additional prizes, each of which also had its own set of judges.

 I was a judge for the World Citizenship competition, a category sufficiently large to be split across two panels of judges. The two days of judging were quite different. Day One consisted of each student group presenting its project to a judging panel. The structure consisted of 10 minutes of setup, an uninterrupted 10-minute presentation, followed by 10 minutes of Q&A with the judges. Enforcement of the timing was rigid—but the students were so well prepared that, in virtually all cases, they nailed it almost to the second. While sessions were open to spectators, few students were to be seen among them. Those not presenting were busy refining their own presentations for either this or the following day, an activity that seemingly went on all hours of day and night!

Other than a break for lunch, the Day One sessions lasted from 8:30 a.m. to 5 p.m. Normally, this would be considered an unbearably long time to be confined to a single room. That was not the case here. The day went by in a flash. This was completely because of the strength and energies of the presentations. Judges had 10 minutes between presentations to make handwritten notes, as well as enter interim scores into a spreadsheet. The latter was a well-designed godsend. It enabled us to enter interim scores efficiently for each presentation, in a standard format, according to a number of well-chosen criteria. It also provided a mechanism for us to give feedback to the students that might help them prepare for the following day. I appreciated the efficiency and structure of this process. It helped ensure that I didn’t forget to consider any key aspect of a presentation and meant that I could capture my interim evaluation while the details were still fresh in my mind. During this phase, there was little talk among the judges. We each formed our opinions on our own, which was—I believe—a good thing. This was partially because of design and partially dictated by circumstances: We were all preoccupied with getting our notes on each project caught up.

During the initial judges’ briefing, we were cautioned not to fall behind in our notes, because by the end of the day—given the number of projects—it would be hard for us to keep them separate in our mind. This was advice that I was really grateful for and paid attention to. The last thing the students deserved was having their hard work blurred by that of others!

Wednesday, July 10

Day One introduced us to the teams and their projects. In so doing, it was a perfect setup for Day Two, which took a different form. Rather than consisting of presentations, judges met one-on-one with the teams at the booth assigned to each project. Here, we had a chance to get a hands-on view of the project and engage in conversations that—for me—were the best part of the whole experience.

Rick Rashid, who founded Microsoft Research, always said that his vision for Microsoft Research was to be the best computer-science department in the world (where we get to make our own definition of what constitutes computer science). Day Two fulfilled one part of that reality—for me, at least—by enabling me to interact with some of the best, most passionate students in the world. For an academic, it doesn’t get any better. The competition, the fatigue, the crowds, and the noise all faded away, to be replaced by intense give-and-take conversations of the best kind. This was no examination or interrogation. Sure, the prizes were certainly in the back of all of our minds, and I had to be mindful to perform my duties. But in reality, not everyone was going to win a prize. Hence, for me, these conversations were a way to do one’s best to add to what else of value they (and, selfishly, I) might leave with, independent of any prize.

Bill Buxton onstage during the Imagine Cup awards ceremonyThe Day Two judging sessions made the entire trip worthwhile for me. My biggest hope is that some of what I both felt and received was shared by the students.

But the judging finally did come to an end. And—to the students’ relief, I strongly suspect—earlier than on the previous day. With the die now cast, the students had no pressing concerns other than curiosity as to how the awards would play out. For the rest of the day, they got to play out themselves. And what a way to play—at the Peterhof, the summer residence of Peter the Great. With its gardens, fountains, buildings, and structure, their playground was as opulent as it was stunning. This was Baroque in the extreme!

Meanwhile, we remained behind to lock down our individual final evaluations. This was the first time that our panel discussed or shared where we had landed. Therefore, it was as much a relief as it was reassuring, that, independently, we had individually included the same projects in our top-five list, and—while not in the same order—the top three. If this had occurred after two days of constant discussion, I would have been wary of having succumbed to groupthink. But that was decidedly not the case, something even more reassuring given the difference in backgrounds, specialty, and culture of the panel.

Having submitted our individual final marks—each retaining our individual rankings of the top five—we joined the students at the unforgettable Peterhof.

Thursday, July 11

This day was essentially divided into four official parts, with a coda at the end for the tireless few.

For me, it began at 8 a.m., when I was due for a technical rehearsal. This was because the first part of the day was dedicated to several one-hour information sessions. Mine was in the 9-10 slot, and there was another one-hour set of talks starting at 10. During that two-hour period, there also was an Ask the Experts room, where—when (or if not) speaking—a number of us made ourselves available to answer, one-on-one, whatever questions were thrown at us.

While heading in to set up for my talk, I bumped into Clint Rutkas. (If you don’t know Clint, check out some of his work, such as his infamous T-shirt cannons or robot bartender at Monkey See, Monkey Build.) I asked what he was going to talk about, and he said, “rapid prototyping—how to get projects going fast and cheap.” My response was, “Hey, we’re giving the same talk!” At start time, Clint popped his head into my room and expressed his concern that both of our rooms were rather sparsely populated and that perhaps more than a few students might be exhausted from the previous few days and sleeping in. As things were about to start, I suggested that we just join forces and do a talk together. Clint being Clint, he was all over the idea, and so it was.

This was actually a lot of fun because, despite having met, knowing each other’s work somewhat, and having talked a bit, we actually didn’t know each other well and had never worked together. This was off-the-cuff, and based on all of three minutes of discussion.

But, boy, was it the right thing to do!

Our fears of no attendees turned out to be ill-founded—standing room only, and all the more interesting for all concerned because nobody knew what was coming next, but, hopefully, never being disappointed. I, at least, wasn’t, and it only occurred to me later in the evening that there was perhaps no better way that we could have made one of our most important points, that of collaboration, improvisation, and trust. Form and function were one, without it occurring to us at the time, perhaps simply because it is our way of life, just as natural and automatic as breathing. We didn’t even notice at the time. It was another highlight of the week for me. I love presentations where I learn as much as the attendees do.

Phase 2 of the day was when the students had a chance to shine. All of their projects were still set up in the science-fair-type arrangement where we had done the Day Two evaluations, but instead of nervously showing them to judges, they had the chance to show them off proudly to the accumulated media representatives who attended the event.

Meanwhile, I skipped off on a different caper. On the previous Sunday, my wife, Liz, a painter and former gallery owner/operator, spent almost the entire day at a large but somewhat out-of-the-way new gallery dedicated to contemporary Russian art, the Erarta. I don’t know how to say this in any other way: It was perhaps the best, most exciting contemporary-art gallery I have ever seen. The work, all by people neither of us had ever heard of, was surprising and of a consistently high standard. But it was more than that. The building, the lighting, the design, the curation, the presentation, the architectural details—all were channeled to respect and enhance the art. It was stunning. I don’t tell you this, though, to get you to visit the gallery (although if you are an art lover, you definitely should), but rather, to set up my story.

Game OverWhile there, I was struck by a large-scale piece called Game Over by a young Moscow artist, Sergey Lakotko. It consists of a prostrate male mannequin being crushed by an enormous Tetris piece. This worked for me on a number of levels—especially its imagination and humor. Hence, I took a photograph of it, something Erarta encourages, in its desire to engage the public in contemporary art!

Anyway, what should happen? In the judges’ room, who should I bump into on Tuesday but Alexey Pajitnov, Russian resident of Redmond and the inventor of Tetris! This presented an opportunity too much fun to resist—one in which I saw an opportunity to help Erarta and the Imagine Cup gain a bit of publicity, as well as to have some fun with Alexey. I called the gallery, and, with the help of Microsoft Russia’s PR team, we set up a visit to the gallery with Alexey so that he could see the piece, as well as meet with the gallery team and some reporters.

Alexey is a gentle bear of a man, soft-spoken, thoughtful, and with a twinkle in his eye. An extrovert he is not. He was a good sport, albeit a somewhat dubious one, in playing along with me on this. But I have to say that seeing his expression on encountering the gallery, much less the Tetris piece, was more than worth the effort. I will never forget the smile he had on his face when, at Liz’s urging, he got down on his haunches to take the mannequin’s pulse for the photographer. It was a great way to spend a few hours and a perfect way to make new friends and relationships, one of many such examples that took place in numerous forms and corners of the Imagine Cup. Such things are the hidden agenda of the whole thing: You can’t plan them, but you can help them along by creating the circumstances. Most won’t get documented, but believe me, they are there, and as with my little example, they are the sweetest, longest-lasting icing that you could ask for on this particular cake.

But there were awards to be won, and hence, we move to Phase 3 of the day: the awards ceremony. This event was spectacular even before anyone arrived, being held in the most beautiful theatre I have ever been in, the Alexandrinsky Theatre, the oldest national theatre in Russia, located in downtown St. Petersburg. I was presenting some awards that evening, so I had to arrive early to rehearse. This gave me a chance to see the theatre from the stage at a time when I didn’t have to worry about tripping, remembering my lines, or dropping an award on a student’s toe. As the English say, I was gobsmacked! Me treading the boards on the very stage where Chekhov’s The Seagull premiered? Speak about “imagining”! Topping it off was the opportunity to meet our host for the evening, Doctor Who, (Matt Smith), on that stage.

The awards ceremony was worthy of both the students’ work and the theatre. This is the first awards event I have sat through in which—far from suffering sore-butt syndrome—I was surprised at how quickly it seemed to end. I’m not going to go into detail, because you can see it for yourself on the Imagine Cup website. I just want to say that while there are inevitably some disappointments in such things, I was especially comfortable with the format and the wide distribution of awards, something to which our co-sponsors made a huge contribution that I really appreciated and did my best to thank them in person.

Enter Phase 4. With the awards ceremony over, there was nothing left to do but to go to the party. In my infinite wisdom, I coaxed a few people to avoid the crowds heading to the exits and to take advantage of the tranquility of the stage door. Leading the way, I discovered what a naïve idiot I am. Exiting with a self-congratulating smirk on my face, I was confronted by the largest gaggle of young girls that I have ever seen (other than in old Beatles movies). What I had not anticipated was that word had gotten out to seemingly every female in St. Petersburg between the ages of 12 and 22 about who was at the theatre. As I made my way through the gantlet—with my smirking followers in tow—I was only too aware of disappointment in the fans’ faces that it was me and not Matt. Thus was my movie-star moment.

Onto the buses we went, and off to a perfect closing to the week. There is something magical about worrying about sunburn while partying outside at 10 p.m. at a beautiful riverside location. In this northern part of Russia, they refer to it as White Nights—a perfect name and a perfect metaphor for the event. Just as the summer sun does not set on the northern lands, neither does it, nor will it, set on what happened this week, literally or figuratively. The 71 countries represented by the student participants span the world. But even while the physical sun is shining on one group rather than another, those under stars will still be there, carrying with them their experiences of the week and working on the next stage of their current project or launching a new one.

To paraphrase Margaret Mead: Never doubt that a group of determined people can change the world. In fact, it is the only thing that ever has.

After this week, I am more sure of this than ever. Hence, I thank every person involved in making the Imagine Cup happen. You did an incredible job and should take pride in what you accomplished. You deserve it. Thank you.