Conclusion

7. Conclusions

In this thesis, we investigated several aspects in the field of geometric registration for augmented reality (AR).

Geometric registration as a generic term encompasses a multitude of methods with the goal that the displayed virtual content appears in the right position and perspectively correct from the user’s view onto reality.

Geometric registration for AR is a dynamic motion tracking problem in the first place, requiring continuous estimations of the camera (or user) movements in real time. Therefore, a major focus of past research has been laid on the exploration of various SLAM techniques that allow to capture simultaneously the environment and the relative motion path of the camera. Tremendous progress has been made, and as a result, highly performant and robust solutions emerged.

The fact that SLAM systems are only targeted towardsrelative motion tracking means that they have no geometric connection to an absolute reference. Pure SLAM-based methods work in a self-selected coordinate system to represent the estimated camera path and the reconstructed real-world model. However, in order to visualize pre-existing virtual content in a coherent manner with regard to reality, the spatial relation between the reconstructed real-world model and this content must also be known or defined. An obvious possibility is to integrate the virtual models interactively into the SLAM model by manual editing until the overlay looks plausible (SLAM-centric authoring, see Sec. 5.2). However, due to the tight coupling of both models, reusing the created content in a different setup or exchanging the tracking model (or method) then becomes difficult.

Rather than defining the virtual content relative to a randomly chosen and non-controllable coordinate system, it seems more appropriate to spatially register the SLAM model itself to the virtual or predefined target coordinate system.

Another problem is that SLAM alone cannot provide a consistent or even metric reconstruction that meets the accuracy requirements in many industrial applications, even if the results are globally optimized via bundle adjustment (BA). This is because BA is a poorly-conditioned problem due to the large number of parameters to be estimated, in which small errors in the measurements can lead to large changes in the estimated values (see Sec.

3.1). As a result, reconstructions of the environment and camera paths calculated by the BA are often deformed.

In practical AR applications, this manifests itself in a perceivable, systematic offset of augmentations, when the user changes his location and observes the scene from a different viewpoint. It is likely that SLAM systems will continue to be further improved in future, but it also seems reasonable to assume that the necessary effort to achieve higher accuracy could be very large. Even a highly integrated system equipped with a variety of cameras, depth sensors, and inertial units, such as today’s Hololens, cannot sufficiently solve this problem. Therefore, instead of relying only on the SLAM as the authoritative source of truth for geometric registration, available scene knowledge should be fed back and integrated into the computations in order to correct its inaccuracies.

Spatial registration can be solved via CAD-model-based tracking that automatically recognizes elements of the virtual scene in the image feed of a camera. However, this requires the availability of corresponding CAD models that match reality sufficiently well, and appropriate algorithms must also be applicable in this context (Sec. 1.2). Moreover, a fully automatic ad-hoc registration on untextured models is currently not possible, so that still a certain amount of aid from the end-user is required at runtime of the application. Using additional assumptions or sensors (GPS or Mahhattan-World assumption), some automatic registration methods for urban outdoor environments exist (Sec5.2), but their general applicability is restricted, as they are bound by these extra

conditions. Automation and general applicability are often conflicting goals, asking to what extent a user can be supported with easy-to-use tools and algorithms that allow highly reproducible results in a wide application context.

In addition to the geometric alignment of SLAM, another registration problem relates to the way the virtual information is visually presented to the user. When using optical see-through (OST) devices, virtual objects are typically displayed via a semi-transparent reflector as combiner. In this setup, the user observes the surrounding reality from his own perspective and not from the viewpoint of the tracking camera (as in video see-through AR). While it is difficult, if not impossible to directly access the user’s personal perception, it is nonetheless important to have an abstract model that emulates this situation as close as possible. It is necessary to know, how the rendering can be controlled accordingly so that the virtual content most likely appears seamlessly integrated into his view on the real world. This demands for calibration methods to retrieve the parameters of such a model in a quantitative manner.

In the present work, we have addressed the above-mentioned problems with the aim of fusing virtual worlds and reality in a geometrical sense. As stated at the beginning of this work (see Sec. 1.1), this requires equiva-lence relations that define what belongs together in each of the respective domains, and we posed the following questions:

• What is the source and how is this information provided (data association)?

• In which way can it be exploited algorithmically to the best extent possible for spatial registration of a SLAM system (algorithmic exploitation)?

• How do these equivalence relations actually appear in optical see-through devices and how can they be measured and used for display calibration (modeling and measuring geometric perception in OST-devices)?

The contributions of this thesis regarding these questions can be summarized as follows.

Data association: In this work, we proposed methods that allow a user to contribute with his context knowledge by providing correspondences between the real environment and the virtual world for spatial registration of SLAM systems. With a human operator in the loop who understands a scene and can make informed decisions about it, it becomes possible to cover situations where the automatic association is difficult and to retrieve implicit user knowledge that may be hard to formalize. We allow that the information might be provided in sparse and incomplete form, which serves to map the user knowledge in a more targeted manner and to simplify the process of information transfer from the user.

In a first variant, we have proposed a preparative process for setting up and registering a tracking system based onnatural features(Chap.5). Based on a pre-recorded and reconstructed image sequence, a simple user interface helps to establish links between the coordinate systems of the virtual scene and the SLAM. The user can browse through the sequence, select reconstructed feature points, and assign these to corresponding 3D points of a virtual model. In addition, this preparative phase is also used to retrieve additional information in an automated fashion.

A feature classifier for tracking initialization and a feature management system for visibility handling are trained using the input sequence. The output is a registered and ready-to-use tracking model for a broad range of AR applications. The system was used in several industrial and academic AR projects, and its accuracy and general applicability was proved at various public benchmarks with high success.

As a second variant, we present a spatial registration approach for SLAM systems, in which the user spec-ifies the connections to the virtual coordinate system by placing somereference markersin the scene (Chap.

4). The important part is that the position and orientation for each of the reference markers only needs to be defined partially, i.e. they can be positioned freely on edges or surfaces, which notably simplifies the setup.

The registration information is encoded in the partial references to the virtual coordinate system of each marker,

and the geometric arrangement of these distributed connections over the entire marker setup. During runtime, spatial registration is accomplished automatically once enough reference markers have been reconstructed by the SLAM. The approach was implemented as a pure marker-based method, but it can be easily combined with a feature-based reconstruction. It was developed for an industrial quality inspection AR-application (Sec.1.5). In our evaluations, we could show that the proposed system attained an absolute accuracy of 1mm or below.

Algorithmic exploitation: A central part of this thesis is the optimal exploitation of the sparse and distributed partial information for spatial registration provided by the user. This information is internalized intoclosed-form as well asiterative, nonlinear minimization formulations.

We presented aclosed-form algorithm(Chap.2) to determine the parameters of a Euclidean or similarity trans-formation based on different types of correspondences, namely point-to-point, point-to-line, and point-to-plane.

It comprises several advantageous properties: it is applicable to minimal and over-constrained configurations, it has linear complexity, and it returns all minima at once in case of ambiguities. Our algorithm represents a gener-alization of several, most recent algorithms for absolute pose problems, including the well-known perspective-n-point (PnP) problem. Apart from being more general, it is also faster than existing algorithms. Due to its general applicability, the algorithm also optimally solves other problems in computer vision, such as the perspective-n-line (PnL) problem and generalized variants of PnP and PnL for multi-camera systems (GPnP and GPnL). In the context of this work, the algorithm was used for spatial registration between the virtual and the real word domain.

In particular, it was indispensable for achieving spatial registration of the marker-based SLAM of Chap.4with reference markers positioned on surfaces (point-to-plane) or edges (point-to-line) that in this way only provided partially indeterminate links to the virtual model.

In Chap. 3we proposed a variant of theiterative bundle adjustment, in which the links to the virtual target coordinate system are imposed as constraints. Instead of using the traditional Lagrangian method as a generic tool for constrained problems, we resort to optimization-on-manifold techniques, with the benefit of preserving the least squares character of the problem and reducing rather than increasing the total number of optimization variables. Our constrained BA minimization comprises three steps. After a transformation of the parameters into the coordinate system of the constraints (1), the parameters are projected onto their respective constraint manifolds (2). For the iterative minimization of the objective function (3), the constraint manifolds are modeled via appropriate derivative functions and parameter update rules along geodesic paths. For this last (third) step, we also analyzed to what extent existing sparse minimization frameworks can be used as a substitute for our own BA implementation. Our proposed constrained BA variant is capable of compensating the low-frequency defor-mations caused by the ill-conditionedness of this minimization problem to a large extent. This was demonstrated in each of the use-cases of Chap.4and Chap.5.

Modeling and measuring geometric perception in OST-devices: For the calibration of optical see-through (OST) displays, we present a parametric display model, in which user-related and hardware-related parameters are decoupled from one another. This is particularly useful, as user related data, such as the head position in head-up displays (HUD) or the exact eye centers in head-mounted displays (HMD), is subject to frequent variations, whereas the hardware parameters generally remain constant throughout the lifetime of the device. Our calibration model includes a parametric perspective camera model and a generic multivariate polynomial distortion model for pre-compensation of view-dependent aberrations by the optics and the combiner.

In the calibration procedure presented in this thesis, a moving camera is used to emulate the users perception from various viewpoints and to see how virtual content displayed on the screen appears with regard to reality. As the parameters need to be determined directly in the reference coordinate system, it is necessary that the camera itself is also registered with this reference system. Thus, our display calibration can be considered as a further

use-case of the above-described methods for the spatial registration of the real model and the camera path to a desired target coordinate system.

A. Publications

The majority of the work described in this thesis has been peer-reviewed and presented at conferences and in journals. This is a list of the publications derived from this work:

1. WIENTAPPER F., SCHMITTM., FRAISSINET-TACHETM., KUIJPERA.: A universal, closed-form ap-proach for absolute pose problems. Computer Vision and Image Understanding (CVIU)(2018) — [WS-FTK18]

2. WIENTAPPER F., KUIJPERA.: Unifying algebraic solvers for scaled Euclidean registration from point, line and plane constraints. InProc. Asian Conf. on Computer Vision (ACCV)(2017), pp. 52–66 — [WK17]

3. WIENTAPPER F., WUESTH., ROJTBERGP., FELLNERD.: A camera-based calibration for automotive augmented reality head-up-displays. InIEEE Proc. Int’l Symp. on Mixed and Augmented Reality (ISMAR) (Oct 2013), pp. 189–197 — [WWRF13]

4. WIENTAPPER F., ENGELKE T., KEIL J., WUEST H., MENSIK J.: [demo] user friedly calibration and tracking for optical stereo see-through augmented reality. InIEEE Proc. Int’l Symp. on Mixed and Aug-mented Reality (ISMAR)(Sept 2014), pp. 385–386 — [WEK^∗14]

5. WIENTAPPERF., WUEST H., KUIJPERA.: Composing the feature map retrieval process for robust and ready-to-use monocular tracking.Computers & Graphics 35, 4 (2011), 778 – 788 — [WWK11a]

6. WIENTAPPER F., WUESTH., KUIJPERA.: Reconstruction and accurate alignment of feature maps for augmented reality. InIEEE Proc. of Int’l Conf. on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT)(2011), pp. 140–147 — [WWK11b]

The following patent has been published based on the work on HUD-calibration:

7. GIEGERICHP., WIENTAPPERF., WUESTH.: Method and apparatus for controlling an image generating device of a head-up display. Patent. WO/2015/044280, 02 04, 2015 — [GWW15]

The following publications describe applications that have been realized using the presented methods and algo-rithms:

8. ENGELKET., KEILJ., ROJTBERGP., WIENTAPPERF., SCHMITTM., BOCKHOLT U.: Content first: A concept for industrial augmented reality maintenance applications using mobile devices. InACM Proc. of the Multimedia Systems Conference (MMSys)(New York, NY, USA, 2015), MMSys ’15, ACM, pp. 105–

111 — [EKR^∗15]

9. KEILJ., ZOELLNERM., ENGELKE T., WIENTAPPER F., SCHMITTM.: Controlling and filtering infor-mation density with spatial interaction techniques via handheld augmented reality. InVirtual Augmented and Mixed Reality. Designing and Developing Augmented and Virtual Environments(Berlin, Heidelberg, 2013), Shumaker R., (Ed.), Springer Berlin Heidelberg, pp. 49–57 — [KZE^∗13]

10. ENGELKE T., KEILJ., ROJTBERG P., WIENTAPPER F., WEBEL S., BOCKHOLT U.: Content first - a concept for industrial augmented reality maintenance applications using mobile devices. InIEEE Proc.

Int’l Symp. on Mixed and Augmented Reality (ISMAR)(Oct 2013), pp. 251–252 — [EKR^∗13]

11. KEILJ., ZÖLLNERM., BECKERM., WIENTAPPERF., ENGELKET., WUESTH.: The house of olbrich — an augmented reality tour through architectural history. InIEEE Proc. Int’l Symp. on Mixed and Augmented Reality - Arts, Media, and Humanities (ISMAR-AMH)(Oct 2011), pp. 15–18 — [KZB^∗11]

Publications in the related field of model-based tracking have also been co-authored. They are only succinctly discussed and not core to this work:

12. WUEST H., ENGEKLE T., WIENTAPPER F., SCHMITTF., KEILJ.: From CAD to 3D tracking — en-hancing & scaling model-based tracking for industrial appliances. InIEEE Proc. Int’l Symp. on Mixed and Augmented Reality (ISMAR-Adjunct)(Sept 2016), pp. 346–347 — [WEW^∗16]

13. WUESTH., WIENTAPPERF., STRICKERD.: Adaptable model-based tracking using analysis-by-synthesis techniques. InProc. Int’l Conf. on Computer Analysis of Images and Patterns (CAIP)(2007), vol. 4673, pp. 20–27 — [WWS07]

14. WUESTH., WIENTAPPERF., STRICKERD.: Acquisition of high quality planar patch features. InProc.

Int’l Symp. on Advances in Visual Computing (ISVC)(2008), pp. 530–539 — [WWS08]

Not related to the present thesis are the following publications:

15. WIENTAPPER F., AHRENS K., WUEST H., BOCKHOLT U.: Linear-projection-based classification of human postures in time-of-flight data. InIEEE Proc. Int’l Conf. on Systems, Man and Cybernetics (SMC) (Oct 2009), pp. 559–564 — [WAWB09]

16. AMORETTIM., COPELLIS., WIENTAPPERF., FURFARIF., LENZIS., CHESSAS.: Sensor data fusion for activity monitoring in the PERSONA ambient assisted living project. Journal of Ambient Intelligence and Humanized Computing 4, 1 (Feb 2013), 67–84 — [ACW^∗13]

17. AMORETTIM., WIENTAPPERF., FURFARIF., LENZI S., CHESSAS.: Sensor data fusion for activity monitoring in ambient assisted living environments. InSensor Systems and Software(Berlin, Heidelberg, 2010), Hailes S., Sicari S., Roussos G., (Eds.), Springer Berlin Heidelberg, pp. 206–221 — [AWF^∗10]

B. Supervising Activities

The following list summarizes the student bachelor, diploma, and master thesis supervised by the author.

1. AHRENSK.:Feature-basiertes Tracking mittels Bag of Visual Words. Master’s thesis, Hochschule Darm-stadt — Universitry of Applied Sciences, Schöfferstraße 8b, 64295 DarmDarm-stadt, Germany, Oct 2011. Re-ceived thePreis des Fachbereichs Informatik der Hochschule Darmstadtaward. — [Ahr11]

2. AHRENSK.: Erkennung der menschlichen Körperhaltung mit Hilfe einer Tiefenbildkamera. Bachelor’s thesis, Hochschule Darmstadt — Universitry of Applied Sciences, Schöfferstraße 8b, 64295 Darmstadt, Germany, May 2009. — [Ahr09]

In the years from 2008 to 2017, the author also contributed to the lectureVirtual and Augmented Realityat the TU Darmstadt and recited there on the subjectFeature Extraction, Matching, and Pose Estimation.

Bibliography

[AAL^∗09] ANDERSENM., ANDERSENR., LARSENC., MOESLUNDT. B., MADSENO.: Interactive assembly guide using augmented reality. InProc. Int’l Symp. on Advances in Visual Comput-ing (IJVC), Bebis G., Boyle R., Parvin B., Koracin D., Kuno Y., Wang J., Wang J.-X., Wang J., Pajarola R., Lindstrom P., Hinkenjann A., Encarnação M. L., Silva C. T., Coming D., (Eds.).

Springer Berlin Heidelberg, Berlin, Heidelberg, 2009, pp. 999–1008.4,86

[ACW^∗13] AMORETTIM., COPELLIS., WIENTAPPERF., FURFARIF., LENZIS., CHESSAS.: Sensor data fusion for activity monitoring in the PERSONA ambient assisted living project.Journal of Ambient Intelligence and Humanized Computing 4, 1 (Feb 2013), 67–84. 134

[AD03] ANSAR A., DANIILIDISK.: Linear pose estimation from points or lines. IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI) 25, 5 (May 2003), 578–589. 16

[Adv] ADVANCED REALTIME TRACKING GMBH:. http://www.ar-tracking.com/

technology/markers/. Accessed: 2013-04-13.114

[AFS^∗11] AGARWAL S., FURUKAWA Y., SNAVELY N., SIMON I., CURLESS B., SEITZ S. M., SZELISKIR.: Building rome in a day. Commun. ACM 54, 10 (Oct. 2011), 105–112. 37, 39,62,82

[AHB87] ARUN K. S., HUANGT. S., BLOSTEIN S. D.: Least-squares fitting of two 3-D point sets.

IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI) PAMI-9, 5 (Sept 1987), 698–700.13,15,30,93

[Ahr09] AHRENSK.: Erkennung der menschlichen Körperhaltung mit Hilfe einer Tiefenbildkamera.

Bachelor’s thesis, Hochschule Darmstadt — Universitry of Applied Sciences, Schöfferstraße 8b, 64295 Darmstadt, Germany, May 2009.135

[Ahr11] AHRENS K.: Feature-basiertes Tracking mittels Bag of Visual Words. Master’s thesis, Hochschule Darmstadt — Universitry of Applied Sciences, Schöfferstraße 8b, 64295 Darm-stadt, Germany, Oct 2011. Received thePreis des Fachbereichs Informatik der Hochschule Darmstadtaward.135

[AHRL17a] ARMAGAN A., HIRZERM., ROTHP., LEPETITV.: Accurate camera registration in urban environments using high-level feature matching. InProc. of the British Machine Vision Conf.

(BMVC)(September 2017), BMVA Press.88

[AHRL17b] ARMAGAN A., HIRZER M., ROTH P. M., LEPETIT V.: Learning to align semantic seg-mentation and 2.5D maps for geolocalization. InIEEE Proc. Conf. on Computer Vision and Pattern Recognition (CVPR)(July 2017), pp. 4590–4597.88

[AKr12] ASKE., KUANGY., ÅSTRÖMK.: Exploiting p-fold symmetries for faster polynomial equa-tion solving. InIEEE Proc. Int’l Conf. on Pattern Recognition (ICPR)(Nov 2012), pp. 3232–

3235.26

[AMO] AGARWAL S., MIERLE K., OTHERS: Ceres solver. http://ceres-solver.

org(accessed 18 March 2017 ). 39,57,61,62

[AMS07] ABSILP.-A., MAHONYR., SEPULCHRER.: Optimization Algorithms on Matrix Manifolds.

Princeton University Press, 2007.46,54,118

[App17] APPLE INC.: ARKit. https://developer.apple.com/arkit/, 2017. Accessed:

2017-09-14.2,84,85

[APV^∗15] ARTHC., PIRCHHEIM C., VENTURAJ., SCHMALSTIEGD., LEPETITV.: Instant outdoor localization and SLAM initialization from 2.5D maps. IEEE Transactions on Visualization and Computer Graphics (TVCG) 21, 11 (Nov 2015), 1309–1318.88

[ARTC12] AGRAWALA., RAMALINGAMS., TAGUCHIY., CHARIV.: A theory of multi-layer flat re-fractive geometry. InIEEE Proc. Conf. on Computer Vision and Pattern Recognition (CVPR) (June 2012), pp. 3346–3353.15

[ASS^∗09] AGARWALS., SNAVELYN., SIMONI., SEITZS. M., SZELISKIR.: Building rome in a day.

InIEEE Proc. Int’l Conf. on Computer Vision (ICCV)(Sept 2009), pp. 72–79. 43

[ASSS10] AGARWALS., SNAVELYN., SEITZS. M., SZELISKIR.: Bundle adjustment in the large. In Proc. European Conf. on Computer Vision (ECCV)(Berlin, Heidelberg, 2010), Daniilidis K., Maragos P., Paragios N., (Eds.), Springer Berlin Heidelberg, pp. 29–42.39,43,46,62,82 [ATR10] AGRAWAL A., TAGUCHI Y., RAMALINGAM S.: Analytical forward projection for axial

non-central dioptric and catadioptric cameras. InProc. European Conf. on Computer Vision (ECCV), Daniilidis K., Maragos P., Paragios N., (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 2010, pp. 129–143.15

[AWF^∗10] AMORETTI M., WIENTAPPER F., FURFARI F., LENZI S., CHESSA S.: Sensor data fu-sion for activity monitoring in ambient assisted living environments. InSensor Systems and Software(Berlin, Heidelberg, 2010), Hailes S., Sicari S., Roussos G., (Eds.), Springer Berlin Heidelberg, pp. 206–221.134

[AWK^∗09] ARTH C., WAGNERD., KLOPSCHITZ M., IRSCHARAA., SCHMALSTIEGD.: Wide area localization on mobile phones. InIEEE Proc. Int’l Symp. on Mixed and Augmented Reality (ISMAR)(2009), pp. 73–82. 95

[Azu97] AZUMAR. T.: A survey of augmented reality.Presence: Teleoperators and Virtual Environ-ments 6, 4 (1997), 355–385.1

[Azu16] AZUMAR. T.: The most important challenge facing augmented reality. Presence: Teleoper-ators and Virtual Environments 25, 3 (2016), 1–30.3

[BÅ09] BYRÖD M., ÅSTRÖM K.: Bundle adjustment using conjugate gradients with multiscale preconditioning. InProc. of the British Machine Vision Conf. (BMVC)(2009), BMVA Press, pp. 36.1–36.10. doi:10.5244/C.23.36.43

[BÅ10] BYRÖDM., ÅSTRÖMK.: Conjugate gradient bundle adjustment. InProc. European Conf.

on Computer Vision (ECCV)(Berlin, Heidelberg, 2010), Daniilidis K., Maragos P., Paragios N., (Eds.), Springer Berlin Heidelberg, pp. 114–127.43

[BAC^∗16] BERGAMASCOF., ALBARELLIA., COSMOL., RODOLÀE., TORSELLOA.: An accurate and robust artificial marker based on cyclic codes. IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI) 38, 12 (Dec 2016), 2359–2373. 68

[BB02] BURCH D., BRAASCHM.: Enhanced head-up display for general aviation aircraft. InIEEE Proc. Digital Avionics Systems Conf.(2002), vol. 2, pp. 11C6–1–11C6–11. 111

[BD14] BANSAL M., DANIILIDISK.: Geometric urban geo-localization. InIEEE Proc. Conf. on Computer Vision and Pattern Recognition (CVPR)(June 2014), pp. 3978–3985.88

Bibliography

[BEK17] BERNARDF., ENGELKE T., KUIJPERA.: User friendly calibration for tracking of optical stereo see-through head worn displays for augmented reality. InIEEE Proc. Int’l Conf. on Cyberworlds (CW)(Sept 2017), pp. 33–40.126

[Bel03] BELL B.: CppAD: A package for differentiation of C++ algorithms. http://www.

coin-or.org/CppAD, 2003. Accessed: 2017-06-23.57

[Ber99] BERTSEKASD. P.: Nonlinear programming, second ed. Athena scientific Belmont, 1999.

38,44

[BH93] BURGESSM., HAYESR.: Synthetic vision-a view in the fog.IEEE Aerospace and Electronic Systems Magazine 8, 3 (1993), 6–13.111

[BJM07] BHASKERE., JUANGR., MAJUMDERA.: Registration techniques for using imperfect and partially calibrated devices in planar multi-projector displays. IEEE Trans. on Visualization and Computer Graphics (TVCG) 13, 6 (2007), 1368–1375. 113

[BL95] BASUA., LICARDIES.: Alternative models for fish-eye lenses. Pattern Recognition Letters 16, 4 (1995), 433–441. 113

[BLE03] BÖRLINN., LINDSTRÖMP., ERIKSSONJ.: A globally convergent gauss-newton algorithm for the bundle adjustment problem with functional constraints. InOptical 3-D measurement techniques : applications in GIS, mapping, manifactoring, quality control, robotics, naviga-tion, mobile mapping, medical imaging, VR generation and animanaviga-tion, vol. 2 ofOptical 3-D Measurement Techniques. Wichmann-Verlag, 2003, pp. 269–276.45

[BM04] BAKER S., MATTHEWS I.: Lucas-kanade 20 years on: A unifying framework. Int’l J. of Computer Vision (IJCV) 56, 3 (2004), 221–255. 37,116

[BM07] BHASKERE., MAJUMDERA.: Geometric modeling and calibration of planar multi-projector displays using rational bezier patches. InIEEE Proc. Conf. on Computer Vision and Pattern Recognition (CVPR)(2007), pp. 1–8. 113,121

[BNR02] BARATOFFG., NEUBECKA., REGENBRECHTH.: Interactive multi-marker calibration for augmented reality applications. InIEEE/ACM Proc. Int’l Symp. on Mixed and Augmented Re-ality (ISMAR)(Washington, DC, USA, 2002), ISMAR ’02, IEEE Computer Society, pp. 107–.

68,69

[Bou] BOUGUET J.-Y.: Camera calibration toolbox for matlab. http://www.vision.

caltech.edu/bouguetj/calib_doc/. Accessed: 2019-01-04. 65

[BPR15] BAYDINA. G., PEARLMUTTERB. A., RADULA. A.: Automatic differentiation in machine learning: a survey.CoRR abs/1502.05767(2015).56

[Bro71] BROWND. C.: Close-range camera calibration. Photogrametric Engineering 37, 8 (1971), 855–866.64,113,121

[BS96] BENDTSENC., STAUNING O.: FADBAD, a flexible C++ package for automatic differen-tiation. Tech. Rep. IMM-REP-1996-17, Department of Mathematical Modelling, Technical University of Denmark, 1996.57

[BS09] BLESER G., STRICKER D.: Advanced tracking through efficient image processing and visual-inertial sensor fusion.Computer & Graphics (C&G) 33, 1 (2009), 59–72.96,97 [BSKP12] BAATZG., SAURERO., KÖSERK., POLLEFEYSM.: Large scale visual geo-localization of

images in mountainous terrain. InProc. European Conf. on Computer Vision (ECCV)(Berlin, Heidelberg, 2012), Fitzgibbon A., Lazebnik S., Perona P., Sato Y., Schmid C., (Eds.), Springer Berlin Heidelberg, pp. 517–530.87

[BSS06] BAZARAA M. S., SHERALIH. D., SHETTYC. M.: Nonlinear programming: theory and algorithms, 3rd ed. John Wiley & Sons, 2006. 38,44

[BvES11] BABOUD L., ˇCADÍKM., EISEMANNE., SEIDELH. P.: Automatic photo-to-terrain align-ment for the annotation of mountain pictures. InIEEE Proc. Conf. on Computer Vision and Pattern Recognition (CVPR)(June 2011), pp. 41–48.88

[BWS06] BLESERG., WUESTH., STRICKERD.: Online camera pose estimation in partially known and dynamic scenes. InIEEE/ACM Proc. Int’l Symp. on Mixed and Augmented Reality (IS-MAR)(2006), pp. 56–65.81,87,90

[CB14] COLLINST., BARTOLIA.: Infinitesimal plane-based pose estimation. Int’l J. of Computer Vision (IJCV) 109, 3 (2014), 252–286. 14

[CC04] CHENC.-S., CHANGW.-Y.: On pose recovery for generalized visual sensors. IEEE Trans.

on Pattern Analysis and Machine Intelligence (PAMI) 26, 7 (July 2004), 848–861.16 [CC12] CHOIC., CHRISTENSENH. I.: 3D textureless object detection and tracking: An edge-based

approach. InIEEE/RSJ Proc. Int’l Conf. on Intelligent Robots and Systems (IROS)(Oct 2012), pp. 3877–3884.4,86

[CCC^∗16] CADENAC., CARLONEL., CARRILLOH., LATIFY., SCARAMUZZAD., NEIRAJ., REID

I., LEONARD J. J.: Past, present, and future of simultaneous localization and mapping:

Toward the robust-perception age. IEEE Trans. on Robotics 32, 6 (Dec 2016), 1309–1332.

37,83

[CDH08] CHEN Y., DAVIS T. A., HAGER W. W.: Algorithm 887: Cholmod, supernodal sparse cholesky factorization and update/downdate.ACM Trans. on Mathematical Software (TOMS) (2008), 1–14.43,61,62,63

[CGC14] CHU H., GALLAGHER A., CHEN T.: GPS refinement and camera orientation estimation from a single image and a 2D map. InIEEE Proc. Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW)(June 2014), pp. 171–178. 88

[Che90] CHENH. H.: Pose determination from line-to-plane correspondences: existence condition and closed-form solutions. InIEEE Proc. Int’l Conf. on Computer Vision (ICCV)(Dec 1990), pp. 374–378.15,16,23

[CLO07] COXD. A., LITTLEJ., O’SHEAD.: Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, third edition (Undergraduate Texts in Mathematics). Springer-Verlag, Berlin, Heidelberg, 2007.17

[CNSD93] CRUZ-NEIRAC., SANDIND. J., DEFANTIT. A.: Surround-screen projection-based virtual reality: the design and implementation of the CAVE. InACM Proc. Conf. on Comp. Graphics and Interactive Techniques (SIGGRAPH)(1993), pp. 135–142.113

[CSP16] CAMPOSECOF., SATTLERT., POLLEFEYSM.: Minimal solvers for generalized pose and scale estimation from two rays and one point. InProc. European Conf. on Computer Vision (ECCV), Leibe B., Matas J., Sebe N., Welling M., (Eds.). Springer International Publishing, Cham, 2016, pp. 202–218.15,16

[Dav03] DAVISONA. J.: Real-time simultaneous localisation and mapping with a single camera. In IEEE Proc. Int’l Conf. on Computer Vision (ICCV)(Oct 2003), pp. 1403–1410 vol.2.37,82 [Dav06a] DAVIS T.: Direct Methods for Sparse Linear Systems. Society for Industrial and Applied

Mathematics, 2006.43

Bibliography

[Dav06b] DAVIST.: SuiteSparse.http://faculty.cse.tamu.edu/davis/suitesparse.

html, 2006. Accessed: 2017-11-08. 43

[DC00] DRUMMOND T., CIPOLLA R.: Application of lie algebras to visual servoing. Int’l J. of Computer Vision (IJCV) 37, 1 (2000), 21–41. 47

[DC02] DRUMMONDT., CIPOLLAR.: Real-time visual tracking of complex structures.IEEE Trans.

on Pattern Analysis and Machine Intelligence (PAMI) 24, 7 (Jul 2002), 932–946. 4,47,86 [DD95] DEMENTHON D. F., DAVISL. S.: Model-based object pose in 25 lines of code. Int’l J. of

Computer Vision (IJCV) 15, 1 (Jun 1995), 123–141.17

[Dee92] DEERINGM.: High resolution virtual reality. InACM Proc. Conf. on Comp. Graphics and Interactive Techniques (SIGGRAPH)(1992), pp. 195–202.113,121

[Del12] DELLAERTF.: Factor graphs and GTSAM: A hands-on introduction. Tech. Rep. GT-RIM-CP&R-2012-002, Georgia Institute of Technology, 2012.39,61,62

[Der94] DERMANIS A.: The photogrammetric inner constraints. ISPRS Journal of Photogrammetry and Remote Sensing 49, 1 (1994), 25 – 39.50

[DF01] DEVERNAYF., FAUGERASO.: Straight lines have to be straight.Machine Vision and Appli-cations 13, 1 (2001), 14–24. 113,121

[DGLN04] DAVIST. A., GILBERTJ. R., LARIMORES. I., NGE. G.: A column approximate minimum degree ordering algorithm. ACM Trans. Mathematical Software (TOMS) 30, 3 (Sept. 2004), 353–376.43

[DHD] DAVIS T., HAGER W., DUFF I.: Suitesparse. http://faculty.cse.tamu.edu/

davis/suitesparse.html. Accessed: 2017-07-21.63

[DK06] DELLAERT F., KAESS M.: Square root SAM: Simultaneous localization and mapping via square root information smoothing. The International Journal of Robotics Research 25, 12 (2006), 1181–1203.37,43,69

[DRLR89] DHOME M., RICHETIN M., LAPRESTE J. T., RIVESG.: Determination of the attitude of 3D objects from a single perspective view. IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI) 11, 12 (Dec 1989), 1265–1278.16,23

[Ead] EADEE.:. http://ethaneade.com/lie.pdf. Accessed: 2017-06-20.56

[ED06] EADEE., DRUMMONDT.: Scalable monocular SLAM. InIEEE Proc. Conf. on Computer Vision and Pattern Recognition (CVPR)(June 2006), vol. 1, pp. 469–476.37,82

[EE11] ELQURSHA., ELGAMMALA.: Line-based relative pose estimation. InIEEE Proc. Conf. on Computer Vision and Pattern Recognition (CVPR)(June 2011), pp. 3049–3056.15

[EKR^∗13] ENGELKET., KEILJ., ROJTBERGP., WIENTAPPER F., WEBELS., BOCKHOLTU.: Con-tent first - a concept for industrial augmented reality maintenance applications using mobile devices. InIEEE Proc. Int’l Symp. on Mixed and Augmented Reality (ISMAR)(Oct 2013), pp. 251–252.9,133

[EKR^∗15] ENGELKET., KEILJ., ROJTBERGP., WIENTAPPERF., SCHMITTM., BOCKHOLTU.: Con-tent first: A concept for industrial augmented reality maintenance applications using mobile devices. InACM Proc. of the Multimedia Systems Conference (MMSys)(New York, NY, USA, 2015), MMSys ’15, ACM, pp. 105–111.9,133

[ENvG07] ESS A., NEUBECK A., VAN GOOL L.: Generalised linear pose estimation. In Proc. of the British Machine Vision Conf. (BMVC)(2007), BMVA Press, pp. 22.1–22.10.

doi:10.5244/C.21.22.14,17

[ESC14] ENGELJ., SCHÖPST., CREMERSD.: LSD-SLAM: Large-scale direct monocular SLAM. In Proc. European Conf. on Computer Vision (ECCV), Fleet D., Pajdla T., Schiele B., Tuytelaars T., (Eds.). Springer International Publishing, Cham, 2014, pp. 834–849.6,47,83

[ESN06] ENGELS C., STEWÉNIUS H., NISTÉR D.: Bundle adjustment rules. InPhotogrammetric Computer Vision (PCV)(2006).40,46

[EvdH09] ERIKSSON A., VAN DEN HENGEL A.: Optimization on the manifold of multiple homo-graphies. InIEEE Proc. Int’l Conf. on Computer Vision Workshops (ICCVW)(Sept 2009), pp. 242–249.46

[FALS92] FOYLE D., AHUMADA A., LARIMERJ., SWEET B.: Enhanced synthetic vision systems:

Human factors research and implications for future systems. SAE T.: Journal of Aerospace 101(1992), 1734–1741.111

[FAR] FARO TECHNOLOGIES INC.:. http://www.faro.com/products/metrology/

faroarm/overview. Accessed: 2013-04-13.114

[FB81] FISCHLERM. A., BOLLESR. C.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM 24, 6 (June 1981), 381–395. 16

[Fia10] FIALAM.: Designing highly reliable fiducial markers. IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI) 32, 7 (July 2010), 1317–1324.68

[Fio01] FIOREP. D.: Efficient linear solution of exterior orientation.IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI) 23, 2 (Feb 2001), 140–148.16

[FLWW98] FYLSTRAD., LASDONL., WATSON J., WARENA.: Design and use of the microsoft excel solver.Interfaces 28, 5 (1998), 29–55.45

[FN03] FOXLINE., NAIMARK L.: VIS-Tracker: A wearable vision-inertial self-tracker. InIEEE Proc. Virtual Reality (VR)(2003), pp. 199–206. 96

[Fua99] FUAP.: Using model-driven bundle-adjustment to model heads from raw video sequences.

InIEEE Proc. Int’l Conf. on Computer Vision (ICCV)(1999), vol. 1, pp. 46–53 vol.1.44,46 [Gal05] GALLAGHER A. C.: Using vanishing points to correct camera rotation in images. InIEEE

Proc. Canadian Conf. on Computer and Robot Vision (CRV)(May 2005), pp. 460–467.88 [Gan84] GANAPATHYS.: Decomposition of transformation matrices for robot vision. Pattern

Recog-nition Letters 2, 6 (1984), 401 – 412.16,19

[GFG08] GILSONS. J., FITZGIBBONA. W., GLENNERSTERA.: Spatial calibration of an optical see-through head-mounted display. Journal of Neuroscience Methods 173, 1 (2008), 140–146.

113

[GFK14] GABBARDJ. L., FITCHG. M., KIMH.: Behind the glass: Driver challenges and opportuni-ties for AR automotive applications.Proc. of the IEEE 102, 2 (Feb 2014), 124–136. 7 [GHA03] GAO C., HUA H., AHUJAN.: Easy calibration of a head-mounted projective display for

augmented reality systems. InIEEE Proc. Virtual Reality (VR)(2003), pp. 53–60. 113 [GHT11] GAUGLITZS., HÖLLERERT., TURKM.: Evaluation of interest point detectors and feature

descriptors for visual tracking.Int’l J. of Computer Vision (IJCV) 94, 3 (2011), 335.4,37 [GHTC03] GAO X.-S., HOUX.-R., TANGJ., CHENG H.-F.: Complete solution classification for the

perspective-three-point problem. IEEE Trans. on Pattern Analysis and Machine Intelligence

Bibliography

(PAMI) 25, 8 (Aug 2003), 930–943.16

[GIMS18] GRUBERTJ., ITOHY., MOSERK., SWANJ. E.: A survey of calibration methods for optical see-through head-mounted displays. IEEE Trans. on Visualization and Computer Graphics (TVCG) 24, 9 (Sept 2018), 2649–2662. 114

[GJMSMCMJ14] GARRIDO-JURADOS., MUÑOZ-SALINASR., MADRID-CUEVASF., MARÍN-JIMÉNEZM.:

Automatic generation and detection of highly reliable fiducial markers under occlusion. Pat-tern Recognition 47, 6 (2014), 2280 – 2292. 70

[GJU96] GRIEWANK A., JUEDES D., UTKEJ.: Algorithm 755: ADOL-C: A package for the auto-matic differentiation of algorithms written in C/C++. ACM Trans. on Mathematical Software 22, 2 (June 1996), 131–167. 57

[GKS^∗10] GRISETTI G., KÜMMERLE R., STACHNISS C., FRESE U., HERTZBERG C.: Hierarchical optimization on manifolds for online 2D and 3D mapping. InIEEE Proc. Int’l Conf. on Robotics and Automation (ICRA)(May 2010), pp. 273–278.39

[GLTH12] GAUGLITZS., LEEC., TURKM., HÖLLERERT.: Integrating the physical environment into mobile remote collaboration. InACM Proc. Int’l Conf. on Human-computer Interaction with Mobile Devices and Services (MobileHCI)(New York, NY, USA, 2012), MobileHCI, ACM, pp. 241–250.85

[GNTH14] GAUGLITZS., NUERNBERGERB., TURKM., HÖLLERERT.: World-stabilized annotations and virtual scene navigation for remote collaboration. InACM Proc. Symp. on User Interface Software and Technology (UIST)(New York, NY, USA, 2014), UIST ’14, ACM, pp. 449–459.

85

[Goo17] GOOGLEINC.: ARCore.https://developers.google.com/ar/, 2017. Accessed:

2017-09-14.2,85

[GSB^∗07] GEORGELP., SCHROEDERP., BENHIMANES., HINTERSTOISSERS., APPELM., NAVAB

N.: An industrial augmented reality solution for discrepancy check. InIEEE/ACM Proc. Int’l Symp. on Mixed and Augmented Reality (ISMAR)(Nov 2007), pp. 111–115.3

[GSC^∗07] GOESELEM., SNAVELYN., CURLESSB., HOPPE H., SEITZS. M.: Multi-view stereo for community photo collections. InIEEE Proc. Int’l Conf. on Computer Vision (ICCV)(Oct 2007), pp. 1–8.37

[GSV^∗12] GAUGLITZS., SWEENEY C., VENTURAJ., TURKM., HÖLLERER T.: Live tracking and mapping from both general and rotation-only camera motion. InIEEE Proc. Int’l Symp. on Mixed and Augmented Reality (ISMAR)(Nov 2012), pp. 13–22. 85

[GTFC12] GEORGEP., THOUVENINI., FREMONTV., CHERFAOUIV.: DAARIA: Driver assistance by augmented reality for intelligent automobile. InIEEE Proc. Intelligent Vehicles Symp.(2012), pp. 1043–1048.111

[GTKN01] GENC Y., TUCERYANM., KHAMENEA., NAVABN.: Optical see-through calibration with vision-based trackers: propagation of projection matrices. InIEEE/ACM Proc. Int’l Symp. on Augmented Reality (ISAR)(2001), pp. 147–156.113,126

[GTN02] GENC Y., TUCERYAN M., NAVAB N.: Practical solutions for calibration of optical see-through devices. InIEEE/ACM Proc. Int’l Symp. on Mixed and Augmented Reality (ISMAR) (2002), pp. 169–175.126

[Gue07] GUENTER B.: Efficient symbolic differentiation for graphics applications. ACM Trans. on Graphics 26, 3 (July 2007).57

Im Dokument Optimal Spatial Registration of SLAM for Augmented Reality (Seite 148-177)