- Evaluation area: The evaluation area will include two rooms and the small hallway surrounding them. The total evaluation area is approximately 2500 square feet. More information about the evaluation area could and should be collected by the teams during the setup day.
The evaluation area will contain furniture. There will also be people present in the rooms during evaluation. In addition, the furniture placement will change between the setup and evaluation days.
- Setup Day (Sunday, April 13th): all teams will be given a 7-hour window (9am-4pm) to setup their systems. This is the time to deploy your custom hardware (if any), profile the space, and calibrate your systems in the best possible way. You won’t be allowed to make any changes to your systems after the setup day. Early in the morning of Sunday, we will share with you, and also clearly mark in the evaluation area, a specific point that will serve as the origin point in the coordinate system that will be used for the evaluation. All locations reported by the teams in the competition should be with respect to this origin point. For instance, if a person is standing 0.5m along the X axis from the origin point, and 3 meters along the Y axis, then your system is expected to report the following location: (0.5m,3m).
Important notes:
- In order to localize anchor nodes or points used during fingerprinting, each team will have to measure the distances along the X and Y axis from the indicated origin point. It is up to the team to decide how to do this (i.e., measurement tape, laser range finder etc.). Since we expect a large number of participants, we recommend each team to be equipped with a laser range finder to simplify the process of distance measurements from the origin point.
- Note that each team can choose its own way of showing the estimated location (i.e., webpage, text file, phone, laptop etc.) as long as a member of the team can clearly point the estimated location out to the evaluator in real-time. It is not enough for the evaluator to be told what the estimated location is. The evaluator will also have to see that location somehow on the system under test.
- Each team should report the most accurate location possible. For instance, if the system under test can report mm level position, it should report location (0.515m,3.001m) instead of just (0.5m,3m).
- Evaluation Day (Monday, April 14th): After the teams have completed their setup on Sunday, the organizers will mark on the floor a number of points and manually measure the ground truth location for every point in the reference coordinate system. The contestants will not be aware of these points until Monday morning where the actual evaluation will take place. During evaluation, the organizers will carry the device indicated by each team to all of the test points, and record the location reported by the device at these points. The final score for the team will be the average localization error for all test points. For every test point, we will compute the Euclidean distance between the inferred location provided by the system under test and the ground truth location of the test point. At the end, we will compute the average of the distance between the measured and ground truth locations across all test points (average localization error). The team that achieves the lowest average localization error wins.
Note that the system under test could be continuously computing its location. The evaluator will only record the location at the pre-specified test points. The evaluator carrying the device will stand at each of these points for a few seconds, and will record the reported location by the system under test.
Important: The system under test cannot assume that when powered up it is at a known location that is either predefined or manually entered. Any systems that do this will be disqualified.
- WiFi Access points: A lot of the submissions rely on generic WiFi access points. To avoid the deployment of 10 WiFi access points from each participating team, we will be providing the WiFi access points. We will deploy 10 WiFi routers at different places within the evaluation area. The access points will be called MSLocalizationX (X=1,…,10). Participants that rely on generic WiFi access points should only use these 10 WiFi routers. The hotel’s access points in the area are only 3 and can only be used for connectivity and NOT localization purposes. If your team relies on generic WiFi access points, there is no need to bring and deploy your own access points. Only those teams that require specialized WiFi access points (i..e special software/hardware) will be allowed to deploy their own WiFi access points. If you want to deploy your own access points, you first need to request permission from the organizers (dlymper@microsoft.com (opens in new tab)).
- Hardware Deployment: the hotel is happy to support our hardware deployment requirements. We will be able to deploy hardware on the floor, on the walls, on top of metallic/plastic tripods, and if really needed on the ceiling (this is a really complicated option, please contact me if your team really needs this!). Please make sure you bring with you the necessary equipment for deploying your custom hardware (i.e., tripods etc.). The hotel should be able to accommodate some of our needs, but given the number of teams participating in the competition, it might be hard to accommodate every single request. Please try to be as self-sufficient as possible, and remember that the competition is taking place at a hotel, where there are always restrictions on what can and how it can be deployed.
- Poster Session: Every participating team will have the chance to present its approach during the poster session on Tuesday, April 15th. The preferred poster size is the following: DIN A1 posters in portrait orientation (594 mm x 841 mm, resp. 23.39 in x 33.11 in – http://en.wikipedia.org/wiki/Paper_size (opens in new tab) ).
- Indoor Localization Panel: On Wednesday, April 16th, a panel on Indoor Localization will be held from 4pm to 5:15pm (http://ipsn.acm.org/2014/program.html (opens in new tab) ). All teams are encouraged to participate. The panel will consist of people from academia and industry. The panel’s main goal will be to discuss the current state-of-the-art in indoor localization, future directions, and of course the results and experiences of the competition.