BILL GATES: Well, good morning. I want to thank you all for coming and taking a lot of time to hear from us about what's gone on in these last 10 years, and also hear about where we see the next 10 years going.

The message you'll get is one of incredible optimism. We feel that the investments we've paid in research have paid off not only at the very high level we expected but even beyond those expectations.

We also see that in the next 10 years the opportunities for the kind of work that's going on in Microsoft Research today to actually get into our products and to find whole new vistas where the personal computer will be used in ways that people just don't expect, that there's a lot more of that that's going to happen these 10 years than even in the last 20.

Well, Microsoft has always been an optimistic company, believing that software could create tools that would really have no bounds in terms of what was possible. And so when I sat down with Nathan Myhrvold and talked about building this research group, we talked about speech recognition, advanced 3D graphics, where if you wanted to see what a city was like, you could just zoom in and travel around in a realistic representation, we talked about having cameras that would see people, recognize who they are, respond to them according to their interests and preferences, we talked about machines that could reason, could look into data and find patterns there that would help companies decide how to make business decisions, and so we said, "These are the kinds of things that Microsoft needs to do." And Microsoft, because it's really the only large company committed to software and committed to research, that it was incumbent upon us to pull together these groups and make this happen.

Now, of course, one thing that you'll hear again and again is that these great people at Microsoft Research work in collaboration with the leading universities. The impact we get by doing collaborative research with the universities around the world is really quite phenomenal. And if there's any field that proves that the commercial world and universities can help each other out, and that's where a lot of the great ideas are born, the world of software is probably the best example of that. Biotechnology is not a bad example either, but I'd claim that we've shown even more than that field what happens when these two worlds come together.

Within Microsoft Research itself of course there's an unbelievable number of PhDs, but I think you'd be surprised at the diverse kind of talents we've brought together to explore these advanced scenarios.

When you want to understand about how user interface should work, should it be anthropomorphic, what is that like, how will people use these new communications tools, what are the kinds of privacy issues or ways that people will take these things, you've got to have psychologists, sociologists. We've got some very pure mathematicians on this staff, people who when we hired them said, "Well, I don't know if I'm going to contribute to your commercial success, but I'd love to come and work on the theoretical stuff I'm doing," and they've been an important part of the group as well. So it's quite a mix, a lot of computer scientists.

Now, at Microsoft, the Microsoft Research Group, of course, complements the overall product development activity. And so when you think of our R&D budget, that includes the product development, as well as Microsoft Research. Those groups together are where we're approaching about $5 billion of spending. And that $5 billion, it's a very stunning number. You know, I never thought I'd get up and be proud of spending $5 billion. You know, can't we write the software for $1 billion? Geez. But no we can't; it turns out it's very challenging. The problems that we're tackling are not problems that lend themselves to overnight success.

The Natural Language Group, which you'll hear a lot about how their things have gotten to our products, many of the ambitious goals that we started out on will probably take most of the next decade for them to solve those problems. Recently, they've been making a lot of advances in machine translation, but we can't stand here and predict exactly when that machine translation will be good enough that it will be the rough equivalent of a human translator.

Part of the reason that research is fun is that the problems are hard. Part of the reason it's fun is that they're not so hard that you don't make progress. Sometimes the areas you make progress in are kind of surprising, and that's why having this group be fairly unstructured -- we don't sit down and talk about, well, how many inventions did you make last month, could you make a few more; that's just not quite enough -- and so it takes some real faith on the part of management that you're bringing in the right people and planting these seeds that take many, many, many years to pay off.

One of the things we're clearly saying is that we think corporations worldwide should invest more in research. The benefits not only to society as a whole but to these companies in terms of understanding where their products should go and bringing out products at lower cost that do more things, the really big advances come from long-term research investments.

It's very interesting at this time that a lot of people are questioning research, thinking about cutting back, and so it's particularly fitting for us to stand here today and say that research has more than paid off for us and that we're continuing to increase that investment even as the economy goes through its ups and downs here in the next several years.

The Microsoft Research Group has different labs around the world and you'll get a clear understanding of the different flavor of these labs as you hear from their leaders. It's one overall effort. They're able to collaborate electronically and they visit each other a lot. We actually have a thing called the "Research Fair" where the researchers show their latest work, not only to each other, but the product group people came in and we had that as an open event and we were stunned that virtually all the developers in the company came over -- it was actually in this building about six months ago -- came and were just fascinated to see the work that's going on.

One problem you get with research that we've avoided is the idea of the product groups and the research groups not working together. Here at Microsoft we have a lot of innovative things to make sure that that boundary is a very soft boundary and that the researchers really want to get their work into the product and that it's very central work, work that we care about, work about software performance, operating system design, user interface, you know, things that are fundamental to all the scenarios that I'll be talking about.

We've got many great examples of where the research work has gotten into the products. I won't go through any of them in particular, but just cite a couple. You know, take the area of cryptography. That is a very fast-moving field and it's a field where you really have to have the world's best people, people who are as good or better than the people who are challenging cryptography, helping you design how those systems work and making them be replaceable systems that can be upgraded over time.

Things like our ClearType technology, there's actually a lot of very deep theory from signal processing that is important there that when we first had that idea, it was our signal processing people who said, "Oh yes, absolutely this can work," and made it dramatically better than the initial ideas were.

Things like spell checking, grammar checking, that's part of an evolution that we've had, where our natural language technology is taking on tougher and tougher tasks. For example, when we first put in the ability to summarize a document, it was pretty straightforward technology. You know, about five percent of our users used it. As we made that better and better with very deep parsing technology, it became something that a wide range of users found that extremely valuable.

So the range of contributions we get is very broad and has certainly changed all of the different Microsoft products.

Well, let's look forward now and talk about where these advances will be creating new products and how people will use them.

This is a time of great change in the computer industry. For the first time, we're saying that we're going to use this incredible connectivity that we've gotten through the Internet and through wireless to allow these intelligent devices to work together. This is very different than what's happened in the past. In the past, in any situation, one of the devices would be intelligent and the other devices would simply be presenting the information from that device.

So, for example, when you do Web browsing, the Web server is very intelligent. There's a lot of rich things going on there, but the place that you run the browser, there's very little going on, that you can't combine information, see information in different ways. It's only exactly what comes off of that Web server.

And there are many examples of that. The old mainframe architecture had the terminal, the 3270 also being very simple and straightforward. Or the original PC file sharing, the actual server did very little; it just stored the bits and the PC was the intelligent device.

Well, in order to take advantage of the kind of Pocket PC devices we're having, the mobile Tablet PCs and these incredible servers, which will have all this centralized information, we need a new approach. And it's an approach that's been talked about in research circles really for many, many decades. It's the idea of distributing computing where you have programs that can run, different parts of them run on different systems.

Applications like electronic commerce absolutely require this approach. You have to have intelligent machines in every one of the companies that are considering doing business and they have to have protection from each other because the developers have never met each other, and yet you have to have an agreement on what those data formats look like. It's not as simple as taking the tabular data that have characterized relational databases and exchanging those things, because the world of information is much richer than we can capture in tabular form.

And so there are advances around how programs communicate, how you make that secure, how you represent data in a self-describing way that are leading to essentially a new platform, taking the Internet and saying this is how we do distributed computing.

Now, it sounds very exciting and it is, but there are many problems that are tough research problems that have to be solved as part of getting this to critical mass. How do you debug programs across these different machines? You know, what if one of the machines is designed by somebody who's malicious? Can that lead to situations where the data is compromised or the service is not available? How do you make it simple to see what's going on when you have programs working this way? So there are a lot of advances that really need to come together for the cutting edge scenarios.

This is happening not just at Microsoft but in many aspects as an industry initiative. The standards around XML are sort of the underpinnings that will make this possible, and then we'll have new tools and really operating systems, office tools, databases, all the different elements of software will be changed by this approach. And so this is a time where the kind of advanced ideas that research has been working on, thinking about how this could come together will really get into the products even faster than ever before.

The key to what Microsoft does is taking these technical advances, the hardware advances that are done outside Microsoft and the software advances done here and at the leading universities and thinking through how do you package that up for scenarios that people will really care about. You know, many of the times where we do this, at first people say, "Well, why would that be interesting?" Graphics interface that we all take for granted today had to be incubated. The work at Xerox PARC, the work at Apple, the work here; there were many years where people questioned whether that would come into the mainstream.

And underneath the graphics interface, the idea of message-based programming, the font technologies, the graphics efficiencies, the dedicated graphics chips, many hardware and software advances were necessary before we could say, "Well, this is common sense; of course, all computers work in exactly this way."

That same thing will happen with things like talking to your computer or the computer understanding sentences that you provide and asking it to use that as the trigger for a search rather than the primitive techniques that we use today.

So we look ahead and say that the hardware pieces are going to provide us with incredible capabilities. You know about Moore's Law, which is the doubling of the power of the microprocessor. That's predicted to continue during the next 10 years. Perhaps they'll be able to extend it, but the kind of fundamental approaches they're using will give us another factor of 30 in terms of performance. And that's pretty phenomenal. It's both necessary for the kinds of algorithms we'll talk about here and it actually makes it so that the power is just there waiting for this software to come and take advantage of it.

The screens we're getting, the kind of wireless connections through so-called 802.11, the size of the storage, the speed of the optical backbone network; all of these things really give us the possibility to have software come in and do new things with it.

The one area that I'd be cautionary about is high-speed connections out to lots and lots of homes. That's the one area where things aren't moving as fast as we'd like. And that's partly because things like digging ditches are not subject to Moore's law. You know, the cost of running that last mile of wire out to homes, there's no miracle advance that's making that very inexpensive, and then you've got regulatory issues that come into that.

And so although we can say with great confidence that things like Tablet PCs and wireless connections will be pervasive within three or four years, the percentage of homes even in the United States that will have broadband, that's at about 10 percent today, it's hard to say in five years will we have gotten to 50 percent penetration. Unless there is some incredible advance or reduction in price or something that comes along, I doubt that that will be pervasive.

Now, that is limiting, but if we're clever about it, if we use the kind of storage we have on these devices and so we cache information so you don't need that high-speed connection, it's surprising what we can do even in the households that don't have the broadband connection.

So now I want to go through some of the ways that we'll use computers differently than we do right now. Part of this is in response to the common statement people make that, "Well, doesn't it already doing everything I need it to do?" Every three or four years people get to the point where they say, "Well, you know, doesn't it do everything I want it to do?"

I remember when we brought out Excel to compete with [Lotus] 1-2-3, people said to us, "Gosh, you know, sure Excel has more features, but, hey, doesn't 1-2-3 do everything that I want to do?" Well, it took about five years before we convinced all users that Excel was a better way to go, but it was basically sitting down, looking at what those users wanted to do and applying advanced technology to that.

So in the productivity area, which is probably the most developed scenario we have right now, things like business modeling, being able to do forecasting, right now the tools are fairly primitive. In fact, if you have to go and get information from your back-end ERP system, try to get it in the right format, bring it down to the spreadsheet, massage it so you can do forecasts, keep those forecasts up to date, that's a very manual process. The visibility you have to thinks like profitability or customer satisfaction are far too manual in a world where you're moving the data around and the data itself is not in this rich structured form.

So what I call XML business intelligence reaches into these productivity packages in a very deep way. The way that you think about a business process, who's going to do what work, how do things come together, the way you think about business investments or forecasts; all of that can be very explicit with these new advances.

Even simple things in terms of documents like document control, if you send out a document and you only want four or five people to see that, is it easy to do that, is it easy to say that this information when it gets out of date should disappear and it should be replaced by more up to date information.

Also today we still have a boundary between the various productivity tools, and so the idea of a single canvas where you have the capabilities of PowerPoint, the capabilities of Word altogether and you don't have to learn quite as many commands to get at that, that's still a horizon to tackle.

Some of you will remember historically there were packages like Symphony or Enable, which tried to bring everything into one package, but they didn't have the right approach to the user interface and the memory size, the disk size was very overloaded by this so-called all in one approach.

Well, the fundamental idea of bringing those things together is alive and well. It just awaits some advances that will allow it to fit in a way that makes it simpler for the user without giving up the power.

The next area is the one I think we'll see the biggest changes, and that's this whole area of communications, both real time communication and asynchronous communication. Today it's a mess. You get junk mail you don't want. You get mail when you don't want it. It interrupts you. If you want to go out and get information about changes that have taken place, you have to do that manually, go and find things. So the user's time, which is the most valuable resource, ends up getting misused. In fact, if you're not careful, if you have a pattern of using e-mail fairly carelessly, the amount of time you give up can come very close to the benefits you gain from having electronic mail. If you have long threads of conversation where people are copying more and more people and disagreeing, you know, it's a tool but it's a tool where the user right now doesn't have as much control as they'd like.

You know, they have an intuitive sense of which messages are important to them, which messages they'd like to get at different times, and yet, you know, you sit in meetings and people's cell phones ring or their Pocket PCs vibrate and they look in there and there's something of a very trivial nature.

And so the user, helping the user to specify things that they'd like to know about and the things that they wouldn't want to know about when those things happen, that's a very tough problem. It's kind of an information agent problem.

You'll probably remember that in some of these futuristic predictions, going back many decades, there's often a type of agent, the human like figure that you're sort of talking to and it has a sense of what you care about and it's working on your behalf.

Well, where is that in software today? The early implementations of that, the so-called "paperclip" that was in Microsoft Office was actually a fairly controversial feature, and one of the most exciting things we did in Office XP was to turn it off by default. For people who like Clippy and all that, it's there to be turned back on.

You might be surprised to hear that that idea of an agent is something that we believe in, but the expectation that people have for how intelligent that agent would be before they'd want to spend time engaging in a dialogue with it is much higher than current technology allows.

And so some of the work in Microsoft Research around this idea of prioritizing your communications and helping you when you're confused with software has to do with bringing that new level of intelligence to the machine so you'll want to work with what we call a social user interface.

And we also have to get to the point where you don't have multiple phone numbers, multiple e-mail addresses, multiple mailboxes. All those things should come together under the control of our preferences about what we want to see. In some ways, instant messaging is just another add-on to this that in some ways it's more work because you have your buddy lists that are separate, and if you want to archive that stuff, you do it in a separate way, and so the opportunity to pull these things together, whether it's fax or voice mail or e-mail, instant messaging, all of these things, there's a huge opportunity.

And it's across multiple devices. It's not just the single device. If you organize a trip on your PC, you'd like to get notified of certain changes that take place if they're urgent when you're away from your PC on your pocket device, and yet your pocket device, it doesn't have enough of an interface that you'd ever want to go through and specify that. You want that to happen automatically because of the work you did at the PC.

And so this idea of thinking of the user holistically across the different devices, that's one of the key tenets of what we call the .NET strategy, building services that work on behalf of the user and notice which device they're using in bringing information to them.

Now, the Pocket PC is one of the form factors we think will be very important in this communication world, the fact that you'll carry a tablet sometimes, but sometimes it will be the smaller device.

Well, one of the people at Microsoft Research has played around with how we could do some different things with that Pocket PC, so let me ask Ken Hinckley to come out and talk about his work on what he calls the sensing Pocket PC. Welcome.

KEN HINCKLEY: All right, so a lot of people are thinking about what is going to be the future of multiple devices, and I would be one of those people. And one of the questions that people ask is, you know, well maybe if we had the ultimate smart computer, that would be the way to go. And so I've been thinking about that problem.

And just sort of as a tongue in cheek way to sort of make my point, I'd draw a comparison between mobile devices today and my cat. Mr. Cleo is his name. So your current device is completely unaware of the environment around it, so if you tip it upside down it doesn't know about that. If I tip my cat upside down, it's going to complain, probably scratch me or something. My computer is completely oblivious to my presence. It doesn't know if I'm in the room or not. My cat might come over and see me and I'll pet it.

And so overall you can end up with this experience where devices can sometimes be selfish and inconsiderate, kind of like Bill mentioned your cell phone might ring in a meeting, because it doesn't know you're talking to somebody.

However, the comparison only goes so far, because my cat has all this awareness and it's still selfish and inconsiderate, because it's a cat.

So in terms of specifically what we've tried to do is that we actually built a little device that has some sensors on it. These sensors are not made by Microsoft. They're sort of out there in the industry. Other companies are building these things. And we think sort of regardless of whether Microsoft takes an interest in this or not, they're going to be out there. They're going to be free. They're essentially free, you know, very inexpensive. They're going to be everywhere. And we think these enable some interesting new things.

So we've built a device that has three particular sensors on it. There's a tilt sensor, a proximity sensor and a touch sensor. And we think these in quantity are maybe about three dollars in parts.

So if we can switch over to the demo now, I'll show you some of these.

Okay, so I have my mobile device here. The first thing I want to show you is that right now it's just sitting there; it's off. If I just pick it up and look at it, the device actually recognizes that and it turns itself on.

So just to show you the tilt sensor, I can sort of tip my device, and if I tip it back and forth, you can see there's just a little ball that's rolling around on the screen, so that's what the tilt sensor is detecting, so you can imagine a game based on this and that kind of thing.

The proximity sensor, I'll show you a demo of that. If I just sort of bring my hand closer to the device, you can just see the little font on the screen gets bigger. Again, this is not an end application; I'm just showing you what the sensor does.

It turns out there are some more interesting things you can do. One we have is that I have a little mockup of Excel here. So if I take my device and I just tip it on the side, I can view it in a landscape format. And so I don't have to know any special thing to do this; I just tip the device on its side.

So sort of one of the points I'd like to make here is that having your device understand being picked up or being tilted on its side, it's just as much a command as typing in ctrl-alt-delete to log out your computer or something; it's just a more natural way to do it. And in the past, computers haven't been able to recognize this kind of thing.

Another example is that I can actually use the same signal to scroll around on the display. So if I just touch this area on the side here, that starts a scrolling mode and I can tip the device forward and back and that will scroll, or I can go left and right, so it's just a nice natural one-handed way to sort of move around on the screen, just another example of something you can do.

A final one I'd like to show you is a lot of these devices have a capability to record something like a voice memo or you can even imagine it's a cell phone, you know, if it rings you might want to answer it. So another one we've done is that you can actually record messages on your device and all you have to do is talk into it. So if I just take my device -- and for this one just watch me -- I can say, "Set up a meeting with Bill to talk about what the demo will be." So all I have to do is just talk into the device. And so what it's recognizing there is the combination of I'm holding it up to my head, I'm tipping it and my hand is in contact with the device, so it's using the information from all these sensors to make a nice natural user experience.

And so we've been playing with other ideas as well. I showed you sort of the ball rolling around. That was sort of the 10-minute game that our intern did, but it could add up to something more interesting like a labyrinth game or something with little enemies you have to dodge.

We could also do things like detecting various types of activity. So, for example, if you're walking around with the device, the tilt sensor can also detect that. So if you're moving around with your cell phone, it might know, "Oh, well you're walking so maybe it's not a good time to interrupt you, or vice versa." So we're looking at all kinds of things like that.

There are also some silly things, like you can shake your device and it understands that, so something like an old Etch-A-Sketch, the shake to erase gesture, that's fully possible now.

So that's the sort of things we're exploring, so thanks a lot.

(Applause.)

BILL GATES: Well, another Holy Grail of PC usage is the idea of reading on the PC screen. How can we make reading off of that PC screen superior to reading off of paper? In a limited way we've already achieved this. If you look at the encyclopedia, we've gone from a world where print encyclopedias are 100 percent of what goes on now to where over 90 percent of the units of encyclopedias are sold on CDs or DVDs. And that's a particular acute case, because the cost advantage of the digital screen, the up to date nature, the rich navigation, the fact that you can put in music and timelines and just an unbelievable wealth of things that use the intelligence of the device have meant that literally within two years of encyclopedias like Encarta coming out, we reached crossover; that is, where the electronic was more popular than the print.

If we think about short e-mail messages, of course, again that's right there on the screen. It's easy to access. People read off of the screen. But if you get much beyond that to say a magazine of any length or a memo of any length, more typical behavior, even if it arrives digitally, is to print that out.

And so Microsoft Research has really studied what are the factors that make you do that. You know, there's no sort of conscious notion of what it is. If you ask people, they'll say, "Well, you know, maybe it's the resolution or maybe it's something to do with eye strain." In fact, those two factors are fairly central. Also a very central part is the fact that the display is in a fixed position, and so staring at a fixed position is actually quite fatiguing over a period of time and it defeats the idea of so-called immersive reading, where you're not even thinking about the book or the mechanism; you're just thinking about the text and appreciating what's there.

So what would it take to reach crossover? Well, of course, one thing we want to do is provide benefits where the digital reading is better, not just the timeliness of the information but simple things like if you want to scribble a little note on it and share it with someone else, that that becomes very possible or if you want to point to some piece of the text and talk or type something in there, that that just comes with everything that you look at on your screen; whether it's an Office document or something you browse, that should be possible.

We also would like to build a device that you can create documents. As long as you're going to have this nice screen, putting a digitizer on it and letting you use a pen is reasonable inexpensive, and, of course, we're pioneering that with the Tablet PC coming out next year.

There are a lot of refinements to the fonts, to the white space, to getting less clutter up on the screen when you're doing these reading things. One of the recent understandings has to do with when you want to use columnar text. When you compose text, you don't really want it to be columnar, because the whole thing about when you delete here how the columns snake around, it's just confusing, but actually when you want to read things, columnar presentations are advantageous. And so making it simple when you say just compose a piece of e-mail to get that very high resolution ClearType approach and having it be columnar, and yet if you still want to edit it, having that work, that's one of the things that the Office group and Windows group are working on with research that we think we've recently solved.

And so this Holy Grail is a very big deal, because the benefit, if your note-taking and your document searching is done digitally, is that you have a complete history of it. You can go back and look at things you've looked at before. You can get things more quickly. And I believe that this is like graphical user interface, that the day will come -- and who knows when the exact boundary is -- where people take this for granted the same way that they take graphics interface for granted today.

Another area where we think things are quite inefficient, and software with the right hardware can make a difference, is in the whole area of meetings. You know, think about planning a meeting, how you decide who can come, what the topics will be. Well, today you use e-mail and you're sort of enclosing things and people are editing those things and you'll get just streams of e-mail around that. If you have advanced reading for the meeting, you're not really sure whether people have read it or not, you're not sure whether their view is that you ought to go ahead with the thing. Actually, during the meeting itself, particularly if you have people who are not in the same room, who are today say on an audio conference or something like that, there's certainly a lot more that can be done. It should be possible to edit documents together, browse those documents together very easily just like when you make a phone call. When you make a phone call, your screen should be connected up whether it's two people or four people and you can work together.

Follow-up for the meeting is also fairly complex, the idea that somebody who wasn't in the meeting may have an interest in seeing part of it, and yet they want to skip right to that part. And it should be possible for them to watch the presentation faster than in real time, that is you can speed up presentations amazingly by almost a factor of two without people losing the ability to understand what was presented. So the people who don't attend get a real advantage; they only have to see a subset of the meeting and they see it at twice the speed as long as they didn't want to participate in the discussion.

Now, the tools to digitally record that meeting and store that information, even a camera that Microsoft Research has put together that's got a 360 degree viewing angle, that is clearly inexpensive technology. It will be easy to put out into meeting rooms everywhere. In fact, it will be as cheap as just the audio sharing stuff that people have today. But it's the software that can make a difference, and so we have to have software that thinks about this scenario.

Another scenario, of course, is the whole e-commerce area, how businesses today from small businesses to large businesses spend a lot of time setting up software and then getting their software to work with other companies, and how can we use XML hosting software on the Web to work in a complementary way with whatever they want to run inside their business, to make it so that someone like your accountant can come in and look at your books without actually having to be there, and yet the right security things happen, the right browsing happens, and they're not just looking at the back-end data, they can also call up screens and talk with you about, "Well, why did you do this this way; don't you think you ought to make this change," all of that being done at a distance.

With all of the business information, customer feedback, for example, being captured in this rich way, the field of data mining, which has been a very low volume, high price business historically, will become, like so many productivity things, something that businesses of all sizes will want to be able to get into, and instead of spending millions of dollars to mine their data, they ought to spend literally hundreds of dollars, for example, to take geographic information and look at where their customers are coming from and decide things about marketing promotions or locating new branches. Those applications just ought to fall out from the data that's there.

Moving to a couple more consumer-oriented scenarios, you know, I think we can all say with great conviction that the day will come that music is not something that you think about moving around on physical media, that is that your rights to listen to certain music travels with you. As you get in your car, it's there. If you visit a friend's house and you want to share a new tune that you like with them, you're on a trip, the music, that's your music and anytime there's a device you ought to have access to it.

The ability to organize those things or take something and actually share a snippet with a friend and say, "Hey, maybe you ought to buy a copy of this, it's really very cool;" those things can be easily facilitated. If you're listening to the radio and you say, "Yeah, I'd like to buy that song or I'd like to know more about it," of course, that information is easy to capture either by using music recognition or simply having the timeline and going up and looking at what the play list looked like coming out of that source.

There is some real controversy here in terms of the rights management, making it easy to access the material and yet having these rights be respected. Those issues I believe the technology will be there, the right balance will be struck and the user will end up dealing in music in a far better way than they do today, being able to get more that they care about, find out about more artists.

Likewise, for their own material, photos, you know, people take lots of photos. A lot of them basically never get organized in a way that you can take advantage of, and so if you said something simple like, you know, one of the photos we took of my daughter when she was three-years old, that ought to be something where immediately you can come up, you can see what you were doing in terms of trips at those times, you can see other things that were going on and just have access without having to go to extreme efforts to do the organization.

The quality of the digital photography is improving quite dramatically. The traditional companies and startup companies clearly are going to be able to provide the digital cameras at reasonable price points that the quality of the image is equivalent to what's on film.

And so the real bottleneck to making digital photography work is making it more enticing than traditional photography, making it so that you don't feel like you lose your photos if you rely on them being on your hard disk.

So, Microsoft Research has a group that's looked at what new things could you do with photos once they're in this form. And so I would like to ask David Vronay to come out and talk to us about his work, which he calls Photo Story.

Welcome.

MR. VRONAY: I'm Dave Vronay and I work in a group in Microsoft Research called Social Computing, and unlike a lot of the other research groups, we focus not on technology but on the social and emotional aspects of your computing experience. And one thing that we have been looking at is digital photos, and why it is that sharing digital photos on the Internet, or email, or whatever, is less compelling than sitting down on the couch with someone with a stack of photos and sharing them that way.

And what we found is that photo sharing is really about personal stories. It's not about the bits themselves. And so, as I give you some holiday photos here as an example, and we look at how this might look on the Internet today, what I might see is some -- I might get the photos, I might get a title, and maybe I have a little text description of it. And, technically I can kind of get the photo out there, but it's not very interesting, and my story of my holiday doesn't really come across.

We call this the story problem. The thing is, stories are really an emotional experience. In order to hear a story, you'd have to be moved to listen to a story. And in order to tell a story, you also have to be in the mood to tell the story. And so the big challenge here is, how do we overcome people's resistance to being emotional in front of their computer.

Okay, so in terms of receiving the story, it's actually pretty solvable because people are already very used to getting stories from boxes, it's called television. And the key is really to make this experience be cinematic or television-like instead of computer-like. And so there are various rules you can follow for these things, eliminating flow and keeping things moving and so on.

And so, if I take a look here, let's look at how those same exact photos might look if we were watching them on television. Okay, here we are, this is our Christmas tree, the kids are being restrained before they open their presents. Okay, and now, here they go, ripping everything down.

Okay, what you notice here is that things are kind of moving around, I have the narration, you can hear my story telling, and basically the story comes across that way. So the next challenge that we face is, okay, this is great, it works really well. But how do you create these.

And this is really the problem that we worked on, because this is not a complicated thing to make there are tools out there that are relatively easy to use so that you make these now. The problem is that these tools are very hard to use while you're trying to be emotional. The editing and what-not interferes with your being in the storytelling mood. You're focusing on the act, not on your story.

So what we did is, we worked on an off screen tool that gets rid of everything that we found interferes with emotion, and focused only on the storytelling, and also on pointing, we found that pointing, moving your hands, is very important in storytelling.

And so let's look at how you might make that presentation that we just saw. This is an application called PhotoStory. And we've started by loading some pictures. And it shows me the pictures at the bottom, and then I start the story. What the interface does, it kind of dims the lights here, kind of makes everything black, and all I do to use this app is, I talk about my pictures. I say here's Nina and Thomas, here's their tree, and when I'm done talking about a picture, I just click, and it goes to the next picture. And while I'm talking I also point. And what the system does is, it watches me pointing, and it uses that to automatically compute those pans and zooms, and do the synchronization for you.

And so by the time I'm done talking about these photos, that's it. The entire movie is done being made, and then I can save it and send it over to my friends or family.

So, to kind of conclude here, this is all about telling stories. It's not about the images themselves, it's not about technology. And the broader point here and that you're going to see more and more in the future is that emotional content of your data is just as important, if not even more important, than the binary data.

For example, technical flaws in photos, like red eye are just much less noticeable or objectionable in PhotoStory than in still images. And in the future, we're going to be getting emotional data the same respect that we already give the bits, just as we find it unacceptable for a picture to come into a program and show upside down or red or something, we're going to in the future find it unacceptable that if I have a picture I care a lot about I don't have that same warm feeling about it on the computer.

Thanks.

(Applause.)

BILL GATES: Well, another consumer scenario to think about is TVs and gaming, and how is that going to change. There's a constant theme here, which is about putting the user in control. So, here the idea of finding the shows you might be interested in, whether they're airing right now or airing in the past, there's a theme of communication. If you're watching a sports show, you want to be able to talk with a friend or chat with a friend who is in another location while you're watching that show, that should be very easy to do. And, in fact, the continuum between TV watching and playing a rich video game, a lot of that space will get filled in. The ads will allow you interact, and indicate if you have more input. If you're watching a sports show, you'll be able to get background information that corresponds to your particular interest. A game show will bring you into the interaction, and see if you can get to the answers that the contestants are involved in.

So that TV experience will be different. In fact, this interactive TV area is one that started research on right at the very beginning of Microsoft Research. It's been over 10 years. This is an area where nothing has really caught on. The kind of set-top box that we have today is fairly limited. There is some degree of PC online gaming, where communities have come together, but most of the video games are not connected in this kind of rich way.

And yet, with the cost of the graphics chip, the processing chips coming down, and the home network that delivers the video, the cable type networks now getting to a digital capability, here is no doubt that this will happen. Some of the advances that are necessary here involve making it easy to create these computer games. Making it so an artist can come in and create the game. Traditionally, the people who wrote the games were people who could write great game code, because simply getting the performance out of the device, sort of making it look like it had more speed or more resolution than it did, that was the heart and soul of what had to happen.

But the kind of improvements, having the hard disk, more memory, having higher level tools now, we're really letting artistry take control of what goes on. Microsoft Research graphics work, although a lot of it is focused on very serious business-type uses and it has been very useful there, we've been able to take really rich advantage of it with things like Xbox games. You know, things like water, grass or rain, that are actually very hard to represent because we have a certain expectation about the reflectivity and the density, and how it occludes things in different ways, the ability to actually take that and make that in these Xbox games incredibly realistic is dependent on very advanced techniques that Microsoft Research pulled together.

And this is simply one venue, one place that people will be working. It's another venue where you want access to your information, so that multi-device approach absolutely comes in and makes sense.

The last scenario that I wanted to mention actually in some ways is not a scenario at all, but I'm sure that as I presented these neat new things, some of you sitting there are saying to yourself, well, that would be nice, but even the things I do today, the computer is often confusing. It even crashes from time to time. And if he's going to layer on all these neat new things, is that just going to make the thing that much more intimidating in terms of the number of commands, or in terms of the software that doesn't seem to do what I expect it to do, and that's a very valid point.

One of the things that we have to do is use all this software and hardware power to make the experience one that people can think of as something that always works. The way to do that is actually pretty straight forward. What we have to have is constant monitoring of any of these devices that are connected up to the Internet, and logging any time the user's expectations aren't met. You could even think of that as being in two categories, the extreme case where something didn't work at all or it crashed, let's call that sort of the red button case, and the sort of more mild case, where well that was a little slow, or that was sure hard to find.

That feedback can be logged on the system even if the system is not connected, it can be logged so that later when you get connected that information comes in and is used in order to say, well, here's exactly where we need to get this driver fixed, here's exactly where these two applications aren't coming together. And then to complete the loop the software has to be updated on a regular basis by using the Internet, and a facility like Windows updates. So with this complete cycle, the ability to make sure that the entire ecosystem, that the quality is extremely high, all the right things are being done, you get a perfect ability to understand that.

A trivial example of that is MSN when they were working to get rid of busy signals, they never knew which of our providers were overloading their modems, or were turning off their modem pools and what was going on. But, as soon as we actually put into the MSN client software complete logs of every dial attempt, and whether there was a busy signal, or what the delay to get connected was, as soon as we had that data then we had a perfect understanding of exactly who was fulfilling the quality commitments, where capacity was needed. And it became a dynamic process where literally within hours of any type of problems going up you had changes that would make sure that didn't happen again.

Well, for the entire software stack that kind of complete feedback loop actually is very important. Part of always works also is taking the state on the machine and being able to back it up very easily, so that you don't feel like if you lose that machine, or upgrade that machine, or switch to a new machine, you yourself are like a scavenger trying to go around and find the different files, or preference settings, or just give up when you're on the new machine, and reconstruct all of those different things.

Another element of this is using the connectivity so that if you have somebody who's got more expertise than you do you can connect up to them, share your screen with them, have them guide you in how to use the software. At the server side, this is technologies like the fault tolerant approach that we're using, where you have many different servers, and new servers can even be called in if the loads are very heavy, and if the server fails, the person who's connected up to that application doesn't even know it because through the use of transactions the work is shifted to the other machine.

So there's an incredible number of things in management tools, Windows Update, the logging type capability and how we take that information that will allow us to take the scenarios we have today, and these new scenarios and deliver them in a way that the performance and the experience is quite a bit better than anything people have experienced so far.

A good example of this is some work in research called Loader .NET. And I'd like to ask Hoi Vo to come up and talk to us about how this is going to make getting new software very different than it is today.

MR. VO: Hi, my name is Hoi Vo, and I joined Microsoft Research in '93, and it's so much fun I never thought I left school. Today I'm here to talk about a technology that we developed inside of the Programmer Productivity Research Center (PPRC), and it's a group within Microsoft Research. This technology basically allows you to blur the lines between applications, so you no longer have a single application running on your client system, but you could have multiple things running together, and you no longer have an idea of a single application running on your system.

On the client side you will see actually what's happening on the client when you actually run the client, and this will be an illustration on the server side to show you actually what gets downloaded to the client. So to improve the user experience, basically, when you bring up Excel, for example, and when it starts running you can see red stuff gets downloaded, and then gradually turning yellow, the yellow stuff persist on the client. Before that Excel doesn't exist on the client.

So to run through a scenario, just like you go through -- you take a shopping cart to buy mixings to build your cake, and you just take what you need, you no longer need to buy the whole thing, and go home and try to build part of it. Now, as we go through you can see that as much as you need to run through the scenario, which is most of the things we need right now for what I need to do in my job, you don't have to cache everything on the client. You can basically throw it away, the next time you run it you bring up the new stuff. So if you have an update or anything like that it's done automatically. And also patching is also for -- if you have a problem on Excel you don't need to download the whole thing, if it's not on your system you don't have it downloaded yet, so you don't have to fix it.

Another one I would like to show is Access, when you see when Access was running everything was packed nicely, nicely up on top. When Access is running it's kind of scattered all over the place, because we don't build a scenario that you are running. So I bring up a database, which we had no profile before. So you can see it's kind of random all over the place. But, really on the client, since we download just what we need, everything is packed together the way it's supposed to be, just like Excel. So as far as optimization is concerned for the performance wise, you don't suffer from changing your scenario around, or we have to know exactly what the code is actually doing to do the right thing.

That's the current work we're doing within PPRC. I'd like to show one more thing, which is basically working with online SQL for games. I don't think that I need any introduction for Quake. Everybody pretty much knows what Quake is, so Quake is the first time that we ever got data to run as well as code. Code is down below and data is up on top. Every game has a lot of data it has to go through. You can see that now the code is downloaded, you can start running it, and right now we're overlapping execution and download at the same time. The data is still downloaded right now as it goes through the scenario, and goes through the demo right now, what it's doing.

If I bring up a single player mode, you can see these download a lot more as you need it, so now I'm going through the play the game myself, you can see it starts going through and doing whatever it does. Since I'm not a great player I'm dying right away. So anyway, so you can tell that we really work and we don't play games around here. So anyway that's what we tried to do, and if you have any questions we have a demo showcase later on, and you can ask any questions if needed.

Thank you very much.

BILL GATES: So the basic idea is that even if we're downloading software on your behalf, instead of your thinking about it as I go and do the download, and then I run the application, we're actually doing the download in the background and bringing down the things of immediate interest, so the system will be responsive, and yet it's anticipating future things you'll use and doing that in the background.

I hope you've gotten a sense that we feel that we're just at the beginning in terms of what software can do, particularly at this time it's important to understand that long term research is where the new scenarios are going to come from. There's a lot of hard problems to make all of these things come together. There are many parties that have to come together before you get the breakthroughs. You get the collaboration we have with the hardware industry partners, you get the collaboration inside Microsoft between the research group and the product group, the collaboration between Microsoft Research and the universities working in all these different areas, as well as whatever commercial partners we have.

So in many cases like reading, say, you could talk about literally hundreds of companies or thousands of people that are building the pieces to make that possible. And yet, once it's done it's kind of like common sense, of course this was definitely going to come together. It's for reasons like these that we think that the number of hours you spend with your PC and the value you get out of it, we can easily more than double both of those things in the years ahead, and that's for knowledge workers, that's for students, that's for people at home. In some ways some of this stuff you can think of it like science fiction. And of course, when you make it real it's no longer science fiction. The kind of work we're talking about here is what makes it fun for all of us to work at Microsoft, because we get to drive the future forward, and it's based on the commitment we've made, and the great luck we've had in bringing wonderful people together.

So today you'll get to hear from a lot of them. And again, thanks for coming, and listening to this story.

(Applause)