A comprehensive survey of augmented reality assembly research

Abstract

In the past two decades, augmented reality (AR) has received a growing amount of attention by researchers in the manufacturing technology community, because AR can be applied to address a wide range of problems throughout the assembly phase in the lifecycle of a product, e.g., planning, design, ergonomics assessment, operation guidance and training. However, to the best of authors’ knowledge, there has not been any comprehensive review of AR-based assembly systems. This paper aims to provide a concise overview of the technical features, characteristics and broad range of applications of AR-based assembly systems published between 1990 and 2015. Among these selected articles, two thirds of them were published between 2005 and 2015, and they are considered as recent pertinent works which will be discussed in detail. In addition, the current limitation factors and future trends in the development will also be discussed.

Keywords

Augmented reality (AR) / Assembly process / Assembly guidance / Assembly systems

References

[1] Whitney DE. Mechanical assemblies: their design, manufacture, and role in product development, 2004, Oxford: Oxford University Press

[2] Zhou F, Duh H-L, Billinghurst M (2008) Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR. Proceedings of the 7th IEEE international symposium on mixed and augmented reality (ISMAR 2008), Cambridge, pp 193–202

[3] Sielhorst T, Feuerstein M, Navab N. Advanced medical displays: a literature review of augmented reality. J Disp Technol, 2008, 4(4): 451-467.

[4] Henderson SJ, Feiner SK. Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Trans Vis Comput Graph, 2011, 17(10): 1355-1368.

[5] Ridel B, Reuter P, Laviole J, et al. The revealing flashlight: interactive spatial augmented reality for detail exploration of cultural heritage artifacts. J Comput Cult Herit, 2014, 7(2): 1-18.

[6] Bower M, Howe C, McCredie N, et al. Augmented reality in education-cases, places and potentials. Educ Media Int, 2014, 51(1): 1-15.

[7] Google Glass. http://www.google.com/glass/start/. Accessed 10 July 2015)

[8] Nee AYC, Ong SK, Chryssolouris G, et al. Augmented reality applications in design and manufacturing. CIRP Annal Manuf Technol, 2012, 61(2): 657-679.

[9] Ong SK, Yuan ML, Nee AYC. Augmented reality applications in manufacturing: a survey. Int J Prod Res, 2008, 46(10): 2707-2742.

[10] Fite-Georgel P (2011) Is there a reality in industrial augmented reality? In: Proceedings of the 10th IEEE international symposium on mixed and augmented reality (ISMAR 2011), Basel, pp 201–210

[11] Leu MC, ElMaraghy HA, Nee AYC, et al. CAD model based virtual assembly simulation, planning and training. CIRP Annal Manuf Technol, 2013, 62(2): 799-822.

[12] Boothroyd G, Dewhurst P, Knight WA. Product design for manufacture and assembly, 2011, Boca Raton: CRC Press

[13] Seth A, Vance JM, Oliver JH. Virtual reality for assembly methods prototyping: a review. Virtual Reality, 2011, 15(1): 5-20.

[14] Cruz-Neira C, Sandin DJ, DeFanti TA, et al. The CAVE: audio visual experience automatic virtual environment. Commun ACM, 1992, 35(6): 64-72.

[15] Cruz-Neira C, Sandin DJ, DeFanti TA (1993) Surround-screen projection-based virtual reality: the design and implementation of the CAVE. Proceedings of the 20th annual conference on computer graphics and interactive techniques, Anaheim, California, pp 135–142

[16] Kuehne R, Oliver J (1995) A virtual environment for interactive assembly planning and evaluation. Proceedings of ASME design automation conference

[17] Pere E, Langrana N, Gomez D et al (1996) Virtual mechanical assembly on a PC-based system. Proceedings of ASME design engineering technical conferences and computers and information in engineering conference, Irvine, pp 18–22

[18] Jayaram S, Connacher HI, Lyons KW. Virtual assembly using virtual reality techniques. Computer Aided Design, 1997, 29(8): 575-584.

[19] Jayaram S, Jayaram U, Wang Y, et al. VADE: a virtual assembly design environment. Comput Graph Appl, 1999, 19(6): 44-50.

[20] Jayaram S, Jayaram U, Wang Y et al (2000) CORBA-based collaboration in a virtual assembly design environment. Proceedings of ASME design engineering technical conferences and computers and information in engineering conference

[21] Jayaram U, Tirumali H, Jayaram S (2000) A tool/part/human interaction model for assembly in virtual environments. Proceedings of ASME design engineering technical conferences

[22] Taylor F, Jayaram S, Jayaram U (2000) Functionality to facilitate assembly of heavy machines in a virtual environment. Proceedings of ASME design engineering technical conferences

[23] Coutee AS, McDermott SD, Bras B. A haptic assembly and disassembly simulation environment and associated computational load optimization techniques. J Comput Inform Sci Eng, 2001, 1(2): 113-122.

[24] Coutee AS, Bras B (2002) Collision detection for virtual objects in a haptic assembly and disassembly simulation environment. ASME design engineering technical conferences and computers and information in engineering conference, Montreal, pp 11–20

[25] Seth A, Su HJ, Vance JM (2005) A desktop networked haptic VR interface for mechanical assembly. ASME 2005 International mechanical engineering congress & exposition, Orlando, pp 173–180

[26] Seth A, Su HJ, Vance JM (2006) SHARP: a system for haptic assembly & realistic prototyping. ASME 2006 international design engineering technical conferences and computers and information in engineering conference in Philadelphia, Pennsylvania, pp 905–912

[27] Graphics Processing Unit (GPU). http://www.nvidia.com. Accessed 10 July 2015

[28] Caudell TP, Mizell DW (1992) Augmented reality: an application of heads-up display technology to manual manufacturing processes. Proceedings of the 25th Hawaii international conference on system sciences, Hawaii, pp 659–669

[29] Sims D. New realities in aircraft design and manufacture. Comput Graph Appl, 1994, 14(2): 91.

[30] Mizell D. Barfield W, Caudell T. Boeing’s wire bundle assembly project. Fundamentals of wearable computers and augmented reality, 2001, Mahwah: CRC Press 447-467.

[31] Feiner S, Macintyre B, Seligmann D. Knowledge-based augmented reality. Commun ACM, 1993, 36(7): 53-62.

[32] Webster A, Feiner S, Maclntyre B et al (1996) Augmented reality in architectural construction, inspection and renovation. In: Proceedings of ASCE 3rd congress on computing in civil engineering, Anaheim, pp 913–919

[33] Curtis D, Mizell D, Gruenbaum P et al (1998) Several devils in the details: making an AR application work in the airplane factory. In: Proceedings of the International Workshop Augmented Reality in San Francisco, California, pp 47–60

[34] Schwald B, Figue J, Chauvineau E, et al. Stanford-Smith B, Chiozza E, et al. STARMATE: using augmented reality technology for computer guided maintenance of complex mechanical elements. E-work and e-commerce-novel solutions and practices for a global networked economy, 2001, Amsterdam: IOS Press 196-202.

[35] Syberfeldt A, Danielsson O, Holm M et al (2015) Visual assembling guidance using augmented reality. In: International manufacturing research conference 2015, Charlotte, pp 1–12

[36] Molineros J, Sharma R (2001) Computer vision for guiding manual assembly. In: Proceedings of the 2001 IEEE international symposium on assembly and task planning (ISATP 2001), Fukuoka, pp 362–368

[37] Schwald B, Laval BD (2003) An augmented reality system for training and assistance to maintenance in the industrial context. In: International conference in central Europe on computer graphics, visualization and computer vision (WSCG), Plzen, pp 425–432

[38] Regenbrecht H, Baratoff G, Wilke W. Augmented reality projects in the automotive and aerospace industries. IEEE Comput Graph Appl, 2005, 25(6): 48-56.

[39] Saaski J, Salonen T, Hakkarainen M, et al. Ratchev S, Koelemeijer S, et al. Integration of design and assembly using augmented reality. Micro-assembly technologies and applications, 2008, London: Springer 395-404.

[40] Salonen T, Saaski J. Yan XT, Jiang C, Eynard B. Dynamic and visual assembly instruction for configurable products using augmented reality techniques. Advanced design and manufacture to gain a competitive edge, 2008, London: Springer 23-32.

[41] Salonen T, Saaski J, Hakkarainen M et al (2007) Demonstration of assembly work using augmented reality. In: Proceedings of the 6th ACM international conference on image and video retrieval (CIVR 2007), Amsterdam, pp 120–123

[42] Salonen T, Saaski J, Woodward C (2009) Data pipeline from CAD to AR based assembly instructions. In: Proceedings of the ASME-AFM 2009 world conference of innovative virtual reality (WINVR09), Mediapole, pp 165–168

[43] Tumkor S, Aziz ES, Esche SK et al (2013) Integration of augmented reality into the CAD process. In: Proceedings of the ASEE annual conference and exposition, Atlanta

[44] Makris S, Pintzos G, Rentzos L, et al. Assembly support using AR technology based on automatic sequence generation. CIRP Annal Manuf Technol, 2013, 62(1): 9-12.

[45] Sanna A, Manuri F, Lamberti F et al (2015) Using handheld devices to support augmented reality based maintenance and assembly tasks. In: 2015 IEEE international conference on consumer electronics (ICCE), Berlin, pp 178–179

[46] Kollatsch C, Schumann M, Klimant P, et al. Mobile augmented reality based monitoring of assembly lines. Procedia CIRP, 2014, 23: 246-251.

[47] Yuan ML, Ong SK, Nee AYC. Augmented reality for assembly guidance using a virtual interactive tool. Int J Prod Res, 2008, 46(7): 1745-1767.

[48] Sukan M, Elvezio C, Oda O et al (2014) ParaFrustum: visualization techniques for guiding a user to a constrained set of viewing positions and orientations. In: Proceedings of the 27th annual ACM symposium on user interface software and technology, Honolulu, pp 331–340

[49] Posada J, Toro C, Barandiaran I, et al. Visual computing as a key enabling technology for industry 4.0 and industrial internet. IEEE Comput Graph Appl, 2015, 35(2): 26-40.

[50] Zhang J, Ong SK, Nee AYC. RFID-assisted assembly guidance system in an augmented reality environment. Int J Prod Res, 2011, 49(13): 3919-3938.

[51] Henderson SJ, Feiner SK (2011) Augmented reality in the psychomotor phase of a procedural task. In: Proceedings of the 10th IEEE international symposium on mixed and augmented reality (ISMAR 2011), Basel, pp 191–200

[52] Neumann U, Majoros A (1998) Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance. In: Proceedings of IEEE virtual reality (VR 1998), San Francisco, pp 4–11

[53] Andersen M, Andersen R, Larsen C et al (2009) Interactive assembly guide using augmented reality. In: Proceedings of the 5th international symposium on advances in visual computing: Part I, Las Vegas, pp 999–1008

[54] Khuong BM, Kiyokawa K, Miller A et al (2014) The effectiveness of an AR-based context-aware assembly support system in object assembly. In: Proceedings of IEEE virtual reality, Minneapolis, pp 57–62

[55] Friedrich W (2002) ARVIKA augmented reality for development, production, and service. In: Proceedings of the 1st IEEE international symposium on mixed and augmented reality (ISMAR 2002), Darmstadt, pp 3–4

[56] Rentzos L, Papanastasiou S, Papakostas N et al (2013) Augmented reality for human-based assembly: using product and process semantics. In: Analysis, design and evaluation of human-machine systems, Las Vegas, pp 98–101

[57] Zhu Z, Branzoi V, Wolverton M et al (2014) AR-Mentor: augmented reality based mentoring system. In: Proceedings of the 13rd IEEE international symposium on mixed and augmented reality (ISMAR 2014), Munich, pp 17–22

[58] Radkowski R, Oliver J. Shumaker R. Natural feature tracking augmented reality for on-site assembly assistance systems. Virtual, augmented and mixed reality, systems and applications, 2013, Berlin: Springer 281-290.

[59] Lowe DG (1999) Object recognition from local scale-invariant features. In: Proceedings of the 1999 international conference on computer vision, Kerkyra, pp 1150–1157

[60] Chen CJ, Hong J, Wang SF. Automated positioning of 3D virtual scene in AR-based assembly and disassembly guiding system. Int J Adv Manuf Technol, 2015, 76(5–8): 753-764.

[61] Damen D, Gee A, Mayol-Cuevas W et al (2012) Egocentric real-time workspace monitoring using a rgb-d camera. In: 2012 IEEE/RSJ international conference on intelligent robots and systems (IROS), Vilamoura, pp 1029–1036

[62] Vignais N, Miezal M, Bleser G, et al. Innovative system for real-time ergonomic feedback in industrial manufacturing. Appl Ergon, 2013, 44(4): 566-574.

[63] Haringer M, Regenbrecht H (2002) A pragmatic approach to augmented reality authoring. In: Proceedings of IEEE 1st international symposium on mixed and augmented reality (ISMAR 2002), Darmstadt, pp 237–245

[64] Zauner J, Haller M, Brandl A et al (2003) Authoring of a mixed reality assembly instructor for hierarchical structures. In: Proceedings of the 2nd IEEE international symposium on mixed and augmented reality (ISMAR 2003), Tokyo, pp 237–246

[65] Servan J, Mas F, Menendez JL et al (2012) Using augmented reality in AIRBUS A400M shop floor assembly work instructions. In: The 4th manufacturing engineering society international conference, Cadiz, pp 633–640

[66] Petersen N, Stricker D (2012) Learning task structure from video examples for workflow tracking and authoring. In: Proceedings of the 11th IEEE international symposium on mixed and augmented reality (ISMAR 2012), Atlanta, pp 237–246

[67] Mura K, Petersen N, Huff M, et al. IBES: a tool for creating instructions based on event segmentation. Front Psychol, 2013, 4: 1-14.

[68] Petersen N, Pagani A, Stricker D (2013) Real-time modeling and tracking manual workflows from first-person vision. In: Proceedings of the 12th IEEE international symposium on mixed and augmented reality (ISMAR 2013), Adelaide, pp 117–124

[69] Bhattacharva B, Winer E (2015) A method for real-time generation of augmented reality work instructions via expert movements. IS&T/SPIE Electronic Imaging: 93920G-1-93920G-13

[70] Mohr P, Kerbl B, Donoser M et al (2015) Retargeting technical documentation to augmented reality. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, Seoul, pp 3337–3346

[71] Hou L, Wang X. A study on the benefits of augmented reality in retaining working memory in assembly tasks: a focus on differences in gender. Autom Constr, 2013, 32: 38-45.

[72] Odenthal B, Mayer MP, Kabub W, et al. A comparative study of head-mounted and table-mounted augmented vision systems for assembly error detection. Hum Fact Ergon Manuf Serv Ind, 2014, 24(1): 105-123.

[73] Hou L, Wang X, Truijens M. Using augmented reality to facilitate piping assembly: an experiment-based evaluation. J Comput Civil Eng, 2013, 29(1): 05014007.

[74] Baird KM, Barfield W. Evaluating the effectiveness of augmented reality displays for a manual assembly task. Virtual Reality, 1999, 4(4): 250-259.

[75] Tang A, Owen C, Biocca F et al (2003) Comparative effectiveness of augmented reality in object assembly. In: Proceedings of the SIGCHI conference on Human factors in computing systems (CHI 03), Fort Lauderdale, pp 73–80

[76] Wiedenmaier S, Oehme O, Schmidt L, et al. Augmented reality (AR) for assembly processes design and experimental evaluation. Int J Hum Comput Interact, 2003, 16(3): 497-514.

[77] Odenthal B, Mayer MP, Kabub W et al (2011) An empirical study of disassembling using an augmented vision system. In: Proceedings of the 3rd international conference on digital human modeling (ICDHM’11), Orlando, pp 399–408

[78] Boud AC, Haniff DJ, Baber C et al (1999) Virtual reality and augmented reality as a training tool for assembly tasks. In: Proceedings of 1999 IEEE international conference on information visualization, London, pp 32–36

[79] Rios H, Hincapie M, Caponio A et al (2011) Augmented reality: an advantageous option for complex training and maintenance operations in aeronautic related processes. In: Proceedings of 2011 international conference on virtual and mixed reality—Part 1, Orlando, pp 87–96

[80] Suarez-Warden F, Cervantes-Gloria Y, Gonzalez-Mendivil E (2011) Sample size estimation for statistical comparative test of training by using augmented reality via theoretical formula and OCC graphs: aeronautical case of a component assemblage. In: Proceedings of 2011 international conference on virtual and mixed reality-systems and applications—Part II, Orlando, pp 80–89

[81] Gavish N, Gutierrez T, Webel S, et al. Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks. Interact Learn Environ, 2013

[82] Radkowski R, Herrema J, Oliver J. Augmented reality-based manual assembly support with visual features for different degrees of difficulty. Int J Hum Comput Interact, 2015, 31(5): 337-349.

[83] Henderson SJ, Feiner SK (2009) Evaluating the benefits of augmented reality for task localization in maintenance of an armored personnel carrier turret. In: Proceedings of the 8th IEEE international symposium on mixed and augmented reality (ISMAR 2009), Orlando, pp 135–144

[84] Gattullo M, Uva AE, Fiorentino M, et al. Legibility in industrial AR: text style, color coding, and illuminance. IEEE Comput Graph Appl, 2015, 35(2): 52-61.

[85] Hou L, Wang X (2011) Experimental framework for evaluating cognitive workload of using AR system for general assembly task. In: Proceedings of the 28th international symposium on automation and robotics in construction, Seoul, pp 625–630

[86] Hou L, Wang X, Bernold L, et al. Using animated augmented reality to cognitively guide assembly. J Comput Civil Eng, 2013, 27(5): 439-451.

[87] Stork S, Schubo A. Human cognition in manual assembly: theories and applications. Adv Eng Inform, 2010, 24(3): 320-328.

[88] Reiners D, Stricker D, Klinker G et al (1998) Augmented reality for construction tasks: door lock assembly. In: Proceedings of the 1st international workshop on augmented reality (IWAR98), San Francisco, pp 31–46

[89] Boulanger P (2004) Application of augmented reality to industrial teletraining. In: Proceedings of the 1st Canadian conference on computer and robot vision, Ontario, pp 320–328

[90] Horejsi P. Augmented reality system for virtual training of parts assembly. Proc Eng, 2015, 100: 699-706.

[91] Simon V, Baglee D, Garfield S et al (2014) The development of an advanced maintenance training programme utilizing augmented reality. In: Proceedings of the 27th international congress of condition monitoring and diagnostic engineering, Brisbane

[92] Peniche A, Treffetz H, Diaz C et al (2012) Combining virtual and augmented reality to improve the mechanical assembly training process in manufacturing. In: Proceedings of the 2012 American conference on applied mathematics, pp 292–297

[93] Fiorentino M, Uva AE, Gattullo M, et al. Augmented reality on large screen for interactive maintenance instructions. Comput Ind, 2014, 65(2): 270-278.

[94] Liu Y, Li SQ, Wang JF, et al. A computer vision-based assistant system for the assembly of narrow cabin products. Int J Adv Manuf Technol, 2015, 76(1–4): 281-293.

[95] Raczynski A, Gussmann P (2004) Services and training through augmented reality. In: The 1st European conference on visual media production (CVMP 2004), pp 263–271

[96] Gorecky D, Worgan SF, Meixner G (2011) COGNITO:a cognitive assistance and training system for manual tasks in industry. In: Proceedings of the 29th annual European conference on cognitive ergonomics, Rostock, pp 53–56

[97] Matsas E, Vosniakos GC. Design of a virtual reality training system for human-robot collaboration in manufacturing tasks. Int J Interact Desig Manuf, 2015

[98] Pathomaree N, Charoenseang S (2005) Augmented reality for skill transfer in assembly task. In: 2005 IEEE international workshop on robot and human interactive communication in Nashville, Tennessee, pp 500–504

[99] Kreft S, Gausemeier J, Matysczok C (2009) Towards wearable augmented reality in automotive assembly training. In: ASME international design engineering technical conferences and computers and information in engineering conference, San Diego, pp 1537–1547

[100] Charoenseang S, Panjan S. Shumaker R. 5-finger exoskeleton for assembly training in augmented reality. Virtual and mixed reality-new trends, 2011, Berlin: Springer 30-39.

[101] Re GM, Bordegoni M. Shumaker R, Lackey S. An augmented reality framework for supporting and monitoring operators during maintenance tasks. Virtual, augmented and mixed reality applications of virtual and augmented reality, 2014, Berlin: Springer 443-454.

[102] Kruger J, Nguyen TD. Automated vision-based live ergonomics analysis in assembly operations. CIRP Annal Manuf Technol, 2015, 64(1): 9-12.

[103] Webel S, Bockholt U, Engelke T, et al. An augmented reality training platform for assembly and maintenance skills. Robot Autonom Syst, 2013, 61(4): 398-403.

[104] Westerfield G, Mitrovic A, Billinghurst M, et al. Lane HC, Yacef K, Mostow J, et al. Intelligent augmented reality training for assembly tasks. Artificial intelligence in education, 2013, Berlin: Springer 542-551.

[105] Westerfield G, Mitrovic A, Billinghurst M. Intelligent augmented reality training for motherboard assembly. Int J Artificial Intell Educ, 2015, 25(1): 157-172.

[106] Chimienti V, Iliano S, Dassisti M, et al. Ratchev S, et al. Guidelines for implementing augmented reality procedures in assisting assembly operations. Precision assembly technologies and systems, 2010, Berlin: Springer 174-179.

[107] Webel S, Bockholt U, Engelke T, et al. Alem L, Huang WD, et al. Design recommendations for augmented reality based training of maintenance skills. Recent trends of mobile collaborative augmented reality systems, 2011, New York: Springer 69-82.

[108] Webel S, Bockholt U, Keil J (2011) Design criteria for AR-based training of maintenance and assembly tasks. In: Proceedings of the 2011 international conference on virtual and mixed reality: new trends - volume part 1, Orlando, pp 123–132

[109] Gavish N, Gutierrez T, Webel S et al (2011) Design guidelines for the development of virtual reality and augmented reality training systems for maintenance and assembly tasks. In: BIO web of conferences, 00029

[110] Billinghurst M, Hakkarainen M, Woodward C (2008) Augmented assembly using a mobile phone. In: Proceedings of the 7th international conference on mobile and ubiquitous multimedia, Umea, pp 84–87

[111] Wu CL, Wang H (2011) A multi-modal augmented reality based virtual assembly system. In: Proceedings of the 6th international conference on embedded and multimedia computing, Enshi, pp 65–72

[112] Marcinčin JN, Barna J, Janak M et al (2011) Utilization of open source tools in assembling process with application of elements of augmented reality. In: Proceedings of the 10th international conference on virtual reality continuum and its applications in industry, Hong Kong, pp 427–430

[113] Marcinčin JN, Barna J, Torok J. Precision assembly process with augmented reality technology support. Key Eng Mater, 2014, 581: 106-111.

[114] Woll R, Damerau T, Wrasse K et al (2011) Augmented reality in a serious game for manual assembly processes. In: IEEE international symposium on mixed and augmented reality: arts, media, and humanities (ISMAR-AMH11), Basel, pp 37–39

[115] Wang X, Kotranza A, Quarles J et al (2005) A pipeline for rapidly incorporating real objects into a mixed environment. In: Proceedings of the 4th IEEE international symposium on mixed and augmented reality (ISMAR 2005), Vienna, pp 170–173

[116] Wang YT, Shen Y, Liu D et al (2010) Key technique of assembly system in an augmented reality environment. In: The 2nd international conference on computer modeling and simulation, Sanya, pp 133–137

[117] Velaz Y, Arce JR, Gutierrez T, et al. The influence of interaction technology on the learning of assembly tasks using virtual reality. J Comput Inform Sci Eng, 2014, 14(4): 041007.

[118] Theis S, Mertens A, Wille M et al (2015) Effects of data glasses on human workload and performance during assembly and disassembly tasks. In: Proceedings of the 19th triennial congress of the IEA, Melbourne, pp 1–8

[119] Valentini PP. Interactive virtual assembling in augmented reality. Int J Interact Desig Manuf, 2009, 3(2): 109-119.

[120] Wei W, Chen G (2010) Research on interaction in virtual assembly based on augmented reality. In: The 2nd international conference on computer engineering and technology, Chengdu, pp 341–345

[121] Lee H, Billinghurst M, Woo W. Two-handed tangible interaction techniques for composing augmented blocks. Virtual Reality, 2011, 15(2–3): 133-146.

[122] He HW, Wu YM, Zheng DT, et al. Computer vision-based augmented reality system for assembly interaction. Appl Mech Mater, 2010, 37: 263-269.

[123] Boonbrahm P, Kaewrat C. Shumaker R, Lackey S. Assembly of the virtual model with real hands using augmented reality technology. Virtual, augmented and mixed reality, designing and developing virtual and augmented environments, 2014, Cham: Springer International Publishing 329-338.

[124] Arroyave-Tobon S, Osorio-Gomez G, Cardona-McCormick JF. AIR-MODELLING: a tool for gesture-based solid modeling in context during early design stages in AR environments. Comput Ind, 2015, 66: 73-81.

[125] Radkowski R, Stritzke C (2012) Interactive hand gesture-based assembly for augmented reality applications. In: The 5th international conference on advances in computer-human interactions, Valencia, pp 303–308

[126] Ong SK, Wang ZB. Augmented assembly technologies based on 3D bare-hand interaction. CIRP Annal Manuf Technol, 2011, 60(1): 1-4.

[127] Wang ZB, Shen Y, Ong SK et al (2009) Assembly design and evaluation based on bare-hand interaction in an augmented reality environment. In: 2009 international conference on cyberworlds (CW), Bradford, pp 21–28

[128] Wang ZB, Ong SK, Nee AYC. Augmented reality aided interactive assembly design. Int J Adv Manuf Technol, 2013, 69(5–8): 1311-1321.

[129] Piumsomboon T, Clark A, Billinghurst M, et al. et al. Kotze P, Marsden G, Lindgaard G, et al. et al. User-defined gestures for augmented reality. Human-computer interaction-INTERACT 2013, 2013, Berlin: Springer 282-299.

[130] Raghavan V, Molineros J, Sharma R. Interactive evaluation of assembly sequences using augmented reality. IEEE Trans Robot Autom, 1999, 15(3): 435-449.

[131] Liverani A, Amati G, Caligiana G. A CAD-augmented reality integrated environment for assembly sequence check and interactive validation. Concurr Eng, 2004, 12(1): 67-77.

[132] Pan W, Wang Y, Du P. Automatic disassembly navigation for accurate virtual assembly path planning. Assem Autom, 2014, 34(3): 244-254.

[133] Reinhart G, Patron C. Integrating augmented reality in the assembly domain fundamentals, benefits and applications. CIRP Annal Manuf Technol, 2003, 52(1): 5-8.

[134] Zhang FQ, Zhao L, Liang XH et al (2010) The augmented reality research progress in collaboration environment of CAR-CA. In: Proceedings of the 9th ACM SIGGRAPH conference on virtual reality continuum and its applications in industry, Seoul, pp 229–236

[135] Ong SK, Pang Y, Nee AYC. Augmented reality aided assembly design and planning. CIRP Annal Manuf Technol, 2007, 56(1): 49-52.

[136] Pang Y, Nee AYC, Ong SK, et al. Assembly feature design in an augmented reality environment. Assembl Autom, 2006, 26(1): 34-43.

[137] Fiorentino M, Monno G, Uva AE. Tangible digital master for product lifecycle management in augmented reality. Int J Interact Desig Manuf, 2009, 3(2): 121-129.

[138] Fiorentino M, Monno G, Uva AE. Maad S. Tangible interfaces for augmented engineering data management. Augmented reality, 2010, Taipei: Tech Open Access Publisher 113-128.

[139] Fiorentino M, Uva AE, Monno G. Augmented technical drawings: a novel technique for natural interactive visualization of computer-aided design models. J Comput Inform Sci Eng, 2012, 12(2): 024503.

[140] Fiorentino M, Radkowski R, Stritzke C, et al. Design review of CAD assemblies using bimanual natural interface. Int J Interact Desig Manuf, 2013, 7(4): 249-260.

[141] Wang ZB, Ng LX, Ong SK, et al. Assembly planning and evaluation in an augmented reality environment. Int J Prod Res, 2013, 51(23–24): 7388-7404.

[142] Ng LX, Wang ZB, Ong SK, et al. Integrated product design and assembly planning in an augmented reality environment. Assembl Autom, 2013, 33(4): 345-359.

[143] Wang X, Ong SK, Nee AYC (2014) Augmented reality interfaces for industrial assembly design and planning. In: Proceedings of the 8th international conference on interfaces and human computer interaction, Lisbon, pp 83–90

[144] Engelke T, Keil J, Rojtberg P et al (2013) Content first: a concept for industrial augmented reality maintenance applications using mobile devices. In: Proceedings of the 12th IEEE international symposium on mixed and augmented reality (ISMAR 2013), Adelaide, pp. 251–252

[145] Tan W, Liu H, Dong Z et al (2013) Robust monocular SLAM in dynamic environments. In: Proceedings of the 12th IEEE international symposium on mixed and augmented reality (ISMAR 2013), Adelaide, pp 209–218

[146] Alvarez H, Aguinaga I, Borro D (2011) Providing guidance for maintenance operations using automatic markerless augmented reality system. In: Proceedings of the 10th IEEE international symposium on mixed and augmented reality (ISMAR 2011), Basel, pp 181–190

[147] Yang X, Cheng KT (2012) LDB: an ultra-fast feature for scalable augmented reality on mobile devices. In: Proceedings of the 11th IEEE international symposium on mixed and augmented reality (ISMAR 2012), Atlanta, pp 49–57

[148] Calonder M, Lepetit V, Strecha C et al (2010) BRIEF: binary robust independent elementary features. In: Proceedings of 11th European conference of computer vision (ECCV), Crete, pp 778–792

[149] Petit A, Marchand E, Kanani K (2013) Augmenting markerless complex 3D objects by combining geometrical and color edge information. In: Proceedings of the 12th IEEE international symposium on mixed and augmented reality (ISMAR 2013), Adelaide, pp 287–288

[150] Dong S, Kamat VR (2010) Robust mobile computing framework for visualization of simulated processes in augmented reality. In: Proceedings of the 2010 winter simulation conference (WSC 2010), Baltimore, pp 3111–3122

[151] Zheng F, Schubert R, Welch G (2012) A general approach for closed-loop registration in AR. In: Proceedings of the 11th IEEE international symposium on mixed and augmented reality (ISMAR 2012), Atlanta, pp 335–336

[152] Yang J, Dai Y, Li H et al (2013) Single-shot extrinsic calibration of a generically configured RGB-D camera rig from scene constraints. In: Proceedings of the 12th IEEE international symposium on mixed and augmented reality (ISMAR 2013), Adelaide, pp 181–188

[153] Waegel K, Brooks FP (2013) Filling the gaps: hybrid vision and inertial tracking. In: Proceedings of the 12th IEEE international symposium on mixed and augmented reality (ISMAR 2013), Adelaide, pp 1–4

[154] Szalavári Z, Schmalstieg D, Fuhrmann A, et al. Studierstube: an environment for collaboration in augmented reality. Virtual Reality, 1998, 3(1): 37-48.

[155] Billinghurst M, Weghorst S, Furness T. Shared space: an augmented reality approach: for computer supported collaborative work. Virtual Real, 1998, 3(1): 25-36.

[156] Shen Y, Ong SK, Nee AYC. AR-assisted product information visualization in collaborative design. Comput Aided Des, 2008, 40(9): 963-974.

[157] Ong SK, Shen Y. A mixed reality environment for collaborative product design and development. CIRP Annal Manuf Technol, 2009, 58(1): 139-142.

[158] Oda O, Feiner S (2012) 3D referencing techniques for physical objects in shared augmented reality. In: Proceedings of the 11th IEEE international symposium on mixed and augmented reality (ISMAR 2012), Atlanta, pp 207–215

[159] Ranatunga D, Adcock M, Feng D et al (2013) Towards object based manipulation in remote guidance. In: Proceedings of the 12th international symposium on mixed and augmented reality (ISMAR 2013), Adelaide, pp 1–6

[160] Newcombe RA, Izadi S, Hilliges O et al (2011) KinectFusion: real-time dense surface mapping and tracking. In: Proceedings of the 10th IEEE international symposium on mixed and augmented reality (ISMAR 2011), Basel, pp 127–136

[161] Jancosek M, Pajdla T (2011) Multi-view reconstruction preserving weakly-supported surfaces. In: Proceedings of 2011 IEEE conference of computer vision and pattern recognition, Colorado, pp 3121–3128

[162] Dou M, Fuchs H, Frahm JM (2013) Scanning and tracking dynamic objects with commodity depth cameras. In: Proceedings of the 12th IEEE international symposium on mixed and augmented reality (ISMAR 2013), Adelaide, pp 99–106

[163] Zhong Y, Qin Y, Huang M, et al. Automatically generating assembly tolerance types with an ontology-based approach. Comput Aided Des, 2013, 45(11): 1253-1275.

[164] Gorti S, Gupta A, Kim G, et al. An object-oriented representation for product and design processes. Comput Aided Des, 1998, 30(7): 489-501.

[165] Rachuri S, Baysal M, Roy U, et al. Information models for product representation: core and assembly models. Int J Prod Dev, 2005, 2(3): 207-235.

[166] Lee JY, Rhee G. Context-aware 3D visualization and collaboration services for ubiquitous cars using augmented reality. Int J Adv Manuf Technol, 2008, 37(5–6): 431-442.

[167] Zhu J, Ong SK, Nee AYC. An authorable context-aware augmented reality system to assist the maintenance technicians. Int J Adv Manuf Technol, 2013, 66(9–12): 1699-1714.

[168] Chandrasegaran SK, Ramani K, Sriram RD, et al. The evolution, challenges, and future of knowledge representation in product design systems. Comput Aided Des, 2013, 45(2): 204-228.

[169] Hervas R, Bravo J, Fontecha J. An assistive navigation system based on augmented reality and context awareness for people with mild cognitive impairments. J Biomed Health Inform, 2014, 18(1): 368-374.

[170] Schaumloffel P, Talha M, Gorecky D et al (2011) Augmented reality applications for future manufacturing. In: International conference on manufacturing science and education (MSE-11), pp 2–5

[171] Hakkarainen M, Woodward C, Billinghurst M (2008) Augmented assembly using a mobile phone. In: Proceedings of the 7th IEEE international symposium on mixed and augmented reality (ISMAR 2008), Cambridge, pp 167–168