Accurate indoor localization has the potential to transform the way people navigate indoors in a similar way that GPS transformed the way people navigate outdoors. Over the last 15 years, several indoor localization technologies have been proposed and experimented by both academia and industry, but we have yet to see large scale deployments. This competition aims to bring together real-time or near real-time indoor location technologies and compare their performance in the same space.
The team from LINZ CENTER OF MECHATRONICS GMBH put together a video showing some of the action during the competition:
04/15/2016: The official results of the 2016 competition have been announced. An overview of the results can be found HERE. The detailed results for all teams can be found HERE. Congratulations to all the teams that participated!
04/04/2016: The competition’s detailed schedule has been announced. All registered teams have been assigned a 15min evaluation time slot on Monday April 11th, 2016.
03/17/2016: Details about registration, evaluation area, and various logistics can be found HERE.
03/16/2016: The competition will take place on Sunday April 10th, and Monday April 11th. The results will be officially announced on Tuesday April 12th (currently working to move the announcement from Wednesday April 13th as listed on the official program to Tuesday April 12th).
02/19/2016: people that want to attend the competition as observers and not contestants can do it by registering as competition observers: http://www.cpsweek.org/2016/reg.html
02/19/2016: the registration for this year’s competition is currently open: http://www.cpsweek.org/2016/reg.html. All contestants planning to attend the competition must officially register for the competition! Registration for the IPSN conference is NOT required.
02/11/2016: the teams participating in this year’s competition along with the competition categories have been announced!
02/11/2016: We have received 49 submissions this year! The competition is on!
Registration deadline: January 20th 2016
10/20/2015: The 2016 Microsoft Indoor Localization Competition has been officially announced!
Check the 2015 and 2014 competition websites above to get a better idea of the evaluation setup, the type of systems expected, and pictures of the different teams in action!
The team from JRC that performed the ground truth mapping of the evaluation area has made the 3D scanning data available. For more information on the collected 3D scanning dataset you can contact Simone Ceriani and Pierluigi Taddei.
Important information about the results and the final ranking of the teams
- Marinis et al. in the 2D category was not considered for the awards. The system required a significantly different evaluation process compared to the official one. The team required to go over specific evaluation points multiple times, and to also record the exact evaluation point they were at, at any given time. Given that no other teams followed this process, this team was not included in the awards.
- Elias et al. competed with two versions of the same system: one that requires explicit initialization to a known ground truth location before the evaluation, and one where the initial location was automatically inferred by the system based on WiFi and magnetic fingerprints. Both versions of the system complied to the rules of the competition, and the version of the system with initialization clearly achieved the lowest error. However, given that the system from Su et al. that achieved very similar accuracy to that of Elias et al. did not use explicit initialization, we believe that a fair comparison can only be made among the Elias et al. without initialization and the rest of the teams. Given this, Su et al. got the first place, and Elias et al. the second place. Given the technology leveraged by both teams, the achieved accuracy by these teams is practically identical.
- Morin et al. in the 3D category was an exhibition team from the competition’s sponsor Leica. They used a backpack equipped with 5 high-end camera modules, 2 LIDARs and a commercial-grade GPS receiver. This system was not a real-time system as they had to post-process image data and manually select the evaluation points in these images. As a result, it was not considered for the awards.
- Flores et al. in the 3D category: during the evaluation process, the organizers wrote down the wrong x-coordinate for evaluation point 15. Even though the team reported (13.38, 0.44, 3.65), the organizers wrote down (30.38, 0.44, 3.65). With this correction, the average location error for this team is 1.74m and not 2.86m.
- Kulm et al. in the 3D category reported coordinates without adjusting for their 1.5m tall tripod they used to attach their tag to. Unfortunately, they informed the organizers about this after the competition was completed. When adjusting for the 1.5m fixed offset of the tripod, the location error of this team comes down to 0.43m from the 1.43m shown in the graphs above.
We are happy to announce we have received 49 submissions this year! All 49 submissions have been classified into two categories: 3D and 2D localization. All 27 teams in the 3D localization category require to deploy custom hardware, and will have to provide 3D locations during evaluation. All 22 teams in the 2D localization category rely on already available signals such as WiFi and FM, and will have to report 2D locations during evaluation.
Teams in green fonts have at least one team member officially registered for the competition, and will be allowed to deploy their systems during the competition. Teams in red fonts do not have any of their team members officially registered, and therefore they will NOT be allowed to deploy their systems during the competition. (based on registration data as of Monday March 21st).
The detailed schedule for both days of the competition:
|Sunday:04/10/2016 (Setup Day)||9:15 – 10:00||Welcome, space walk-through, rules overview and team check-in|
|10:00 – 17:00||Setup time: all teams deploy, calibrate and test their systems|
|Monday:04/11/2016 (Evaluation Day)||8:30 – 17:30||Evaluation of all teams in 15min pre-assigned time slots. Teams should be present at least 1.5 hours before their assigned evaluation time slot. The time slots assigned to each team can be found in the two tables below.|
|Tuesday:04/12/2016 Official Results Announcement||14:30 – 15:30||The official results are announced. Winning teams give 5min talks describing their systems. This session will be held in the same room where the IPSN 2016 conference will take place here.|
|Tuesday:04/12/2016 Poster Session||16:00 – 18:00||Poster session co-located with the official 2016 IPSN poster session|
3D Localization Category
|Authors||Affiliation||Title||Evaluation Time Slot|
|Lazik et al||USA – Carnegie Mellon University||ALPS: The Acoustic Location Processing System||N/A|
|Hoppe et al.||Germany – Freiburg im Breisgau, Telocate GmbH||Telocate ASSIST||11:45|
|Hammer et al.||Austria – Linz Center of Mechatronics GmbH (LCM)||Sound Based Localization System for Safety Applications in Underground Mining||12:00|
|Wang et al.||China – Zhejiang University||AIDLOC: An Accurate Acoustic Indoor Localization System||12:15|
|Al-darowbi et al.||Saudi Arabia – KFUPM||3D Indoor Localization in the Presence of a Single Beacon||N/A|
|Flores et al.||Germany – Friedrich-Alexander-Universitat Erlangen-Nurnberg (FAU)||A Range-Bearing based Ultrasonic Positioning System (RB-UPS)||12:30|
|Perez-Cruz et al.||USA – Nokia Bell Labs||TDoA-based Localization Using Enhanced Multilateration||N/A|
|Dobrev et al.||Germany – Friedrich-Alexander-Universitat Erlangen-Nurnberg (FAU), Symeo GmbH, Neubiberg/Munich||A Wireless Localization Network for 6D Pose Estimation||12:45|
|Baszun et al.||Poland – Biocontrol||Real Time Livestock Tracking System Based on UWB Technology||13:00|
|He et al.||China – University of Science and Technology||System for 3D Positioning Based on UWB||13:15|
|Cabuz et al.||Japan – Ritsumeiken University|
Germany – Jacobs University Bremen, ZIGPOS GmbH
|A Distributed Indoor Localization Framework||13:30|
|Dardari et al.||Italy – University of Bologna, Uniset s.r.l., DMM s.r.l.||SEQUITUR: An Ultra-wideband Flexible 3D Locating Platform||13:45|
|Loreto et al.||Italy – NexTechs Srl||NexTechs Indoor Positioning System: An UWB Approach||N/A|
|Krikke et al.||Netherlands – OpenRTLS||OpenRTLS UWB||14:00|
|Trybula et al.||Poland – Elokon Polska Sp. z o.o.||UWB Indoor Localization System based on Two-way Ranging Measurement for Industrial Applications||N/A|
|Tabarovsky et al.||Russia – RTLS Research & Development LLC||High Accuracy Indoor Positioning System with Decawave Transceivers and Auto-calibration||14:15|
|Lou et al.||China – Hangzhou Sunsend Info Tech Co. Ltd.||UWB ToA and TDoA Hybrid Localization System||14:30|
|Terragrid||India/Bulgaria – TerraGrid||TerraGrid: Taking you where GPS cannot||N/A|
|Kulm et al.||Estonia – Eliko||Precise UWB Indoor Positioning System||14:45|
|Kempke et al.||USA – University of Michigan||PolyPoint: High-Precision Indoor Localization with UWB||15:00|
|Kämäräinen et al.||Finland – VTT Technical Research Center||UWB Localization Complemented by Impulse Radar||15:15|
|Gunes et al.||Germany – Quantitec GmbH||IntraNav – Low-cost Indoor Localization System and IoT Platform||15:30|
|Pichler et al.||Austria – Linz Center of Mechatronics GmbH (LCM)||UWB Indoor Localization in Time-of-Arrival Configuration||15:45|
|Morin et al.||Switzerland – Leica Geosystems||Leica Geosystems Pegasus Backpack||16:00|
|Vadeny et al.||USA – University of Albany||VSLAM and VLC Based Localization||16:15|
|Zhang et al.||USA – Real Earth, Inc||Compact, Real-time Localization without Reliance on Infrastructure||16:30|
2D Localization Category
|Authors||Affiliation||Title||Evaluation Time Slot|
|Huang et al.||China – Ubirouting Technology Corp.||A Magnetic Field Based Lightweight Indoor Positioning System for Mobile Devices||8:30|
|Su et al.||China – Tongji University||Precise Indoor Localization Platform Based on Wifi-GeoMagnetic Fingerprinting and Aided IMU||8:45|
|Wu et al.||China – Glodon Software Co., Ltd||Particle Filter and Support Vector Machine based Indoor Localization System||9:00|
|Smirnov et al.||Russia – Navigine||Navigine Navigation Technology for Handheld Mobile Devices||N/A|
|Zheng et al.||China – Xiamen University||Precise Hand-held Indoor Location||9:15|
|Wang et al.||China – Huawei Technologies Co., Ltd., Chinese Academy of Sciences, Beijing University of Posts and Telecommunications||MagLoc: Indoor Localization using Geomagnetic, WiFi, and INS||N/A|
|Peng et al.||China – Xiamen University||Precise Indoor Positioning System via Multiple Sensor Fusion||9:30|
|Narzullaev et al.||Malaysia – IQALAM DASTUR SDN. BHD.||CATCH Indoor Positioning System||N/A|
|Han et al.||Republic of Korea – Korea Advanced Institute of Science and Technology||KAILOS: KAIST Indoor Locating System||9:45|
|Ben-Moshe et al.||Israel – Ariel University||GoIn – An Accurate InDoor Navigation Framework for Mobile Devices||N/A|
|Herrera et al.||Mexico – Navix Indoor Navigation||Navix: Smartphone Multi-sensor, Radio and Probability Map Based Indoor Positioning||N/A|
|Elias et al.||Portugal – Fraunhofer Portugal Research Center for Assistive Information and Communication Solutions||A Motion Tracking Solution for Indoor Location Using Smartphones and Initialization||10:00|
|Elias et al.||Portugal – Fraunhofer Portugal Research Center for Assistive Information and Communication Solutions||A Motion Tracking Solution for Indoor Location Using Smartphones||10:15|
|Danaci et al.||Turkey – Ege University||Indoor Location Positioning System with WiFi Technology||N/A|
|Shu et al.||China – Zhejiang University||R-Loc: A Robust Wifi-based Indoor Localization System||11:30|
|Mosharafa et al.||Egypt – Nile University||MazeIn: An Indoor Localization Engine||10:30|
|Yoshizawa et al.||Japan – Ricoh Company, Ltd.||Indoor Positioning by Fusing Pedestrian Dead Reckoning and Proximity Beacon Technologies||10:45|
|Liao et al.||Taiwan – Aquaways CO. Ltd.||Indoor Localization Using Broadcasting FM Signals TDOA Triangulation||N/A|
|Barbadillo et al.||Philippines – University of the Philippines||Auto-Calibrated 2D Localization System Using RSSI||N/A|
|Rea et al.||Spain – IMDEA Networks Institute, University Carlos IIISwitzerland – Armasuisse||Robust WiFI Time-of-Flight Positioning System||11:00|
|Rosen et al.||Israel – Infuse||Infuse: Indoor Localization Using Sensor Fusion||N/A|
|Schmitz et al.||Real-time Indoor Localization with TDOA and Distributed Software Defined Radio||Real-time Indoor Localization with TDOA and Distributed Software Defined Radio||11:15|
|Marinis et al.||Italy – DUNE s.r.l.||Italy – DUNE s.r.l.||11:30|
Call for Contesters
Both academia and industry submissions are encouraged. All location techniques, such as ranging, fingerprinting, infrastructure, or device free, are welcome, except those that require end users’ manual measurements. Contesters can deploy their own infrastructure of up to 5 devices. Normal RF interference is expected, but no jammers from other deployments are allowed. The results must be shown on a portable device, such as a phone or a tablet/laptop that a person can easily carry around.
Demo submissions that do not meet one or more of the guidelines above will be included in the poster session and will be evaluated as a regular submission, but they will not be considered for prizes.
The competition will take place if at least 5 teams respond to this preliminary call for competition.
Depending on the nature and number of submissions multiple categories might be defined based on the accuracy (i.e., point-based vs. area based), the size, the cost, or the type (i.e., software vs. hardware) of the proposed solution.
Given our past experience, this year we expect to have the following categories (note that these categories have not been finalized. The final categories will be announced shortly after the registration deadline and they will depend on the number and type of submissions received):
- Commercial off-the-shelf (COTS) Technologies: Submissions in this category should be able to work with unmodified commercial off-the-shelf devices such as laptops, phones, and tablets. In this category, the unmodified COTS device is localized. Teams in this category will not be allowed to interface any custom hardware to the COTS devices (i.e., UWB or ultrasound hardware). Only changes to the software of the devices will be allowed. Submissions in this category could be further classified to infrastructure-based and infrastructure-free depending on their requirement to deploy custom hardware (i.e., BLE beacons) in the evaluation area.
- Commercial off-the-shelf (COTS) Technologies with Initialization: Submissions in this category should meet all the criteria of the COTS Technologies category, with the additional requirement to initialize the location of the COTS device being localized to a ground truth location before the evaluation.
- Modified Commercial off-the-shelf (COTS) Technologies: Submissions in this category could interface custom hardware to the COTS devices to be localized (i.e., UWB or ultrasound hardware). Submissions in this category could be further classified to infrastructure-based and infrastructure-free depending on their requirement to deploy custom hardware (i.e., BLE beacons) in the evaluation area. Most, if not all, submissions in this category are expected to be infrastructure-based.
- WiFi fingerprinting approaches that also leverage inertial sensors will be classified as infrastructure-free COTS Technologies as they do not require to interface any custom hardware to the COTS device being localized.
- WiFi fingerprinting approaches that also deploy custom BLE beacons in the evaluation area will be classified as infrastructure-based COTS Technologies as they don’t modify the COTS device being localized, but they require the deployment of custom hardware.
- Approaches based on inertial sensing that require to be initialized to a ground truth location before being evaluated will be classified as COTS Technologies with Initialization.
- UWB approaches will be classified as infrastructure-based modified COTS Technologies as they require modifications to the COTS device being localized, and they need to deploy custom hardware in the evaluation area.
In the past, teams were asked to report 2D locations (X,Y). In this year’s competition, teams will be required to report 3D locations (X,Y,Z). The evaluation area we have reserved, even though it does not contain multiple floors, it includes locations with different elevation characteristics. On top of that, we plan to place the devices to be localized at different heights during the competition to evaluate the 3D localization accuracy of each team.
Not all teams will have to report 3D locations. We understand that some technologies are not suited for 3D localization (i.e., WiFi/geo-magnetic fingerprinting, dead reckoning, BLE-based etc.), but they are still excellent candidates for commercial indoor location systems. With this in mind, the organizers will determine shortly after the registration deadline which teams will be classified as 3D localization teams. It is not up to the individual teams to decide if they will report 2D or 3D locations. The organizers will ensure that all the teams in a given category will either report 2D or 3D locations. There will be no categories containing both 2D and 3D localization teams.
Evaluation and Prize
Results are judged based on accuracy, and an award will be given to the top 3 teams in each category. When accuracy ties, infrastructure requirements will be used for tie breaking. The winning teams in each category will be invited to present their approach at the conference, and receive a cash award. The exact accuracy metrics that will be used during evaluation will be announced shortly before the competition takes place.
A poster session dedicated to all competition participants will be organized during the conference. Participants will have the opportunity to explain their system to conference attendees.
Contesters must submit an abstract describing their approach and deployment requirements by the contest registration deadline. Submissions are treated as confidential until the competition. Submissions must be at most two (2) single-spaced 8.5″ x 11″ pages, including figures, tables, and references. Submission should follow the exact same format as regular, full IPSN 2016 papers. Abstracts should include the names and affiliations of all authors. Templates can be found here.