31 March 2019 to 5 April 2019
Academia Sinica
Asia/Taipei timezone

Deep Learning and Augmented Reality as a tool to explore the naturalistic richness of urban areas

3 Apr 2019, 14:00
30m
Conference Room 1 (Academia Sinica)

Conference Room 1

Academia Sinica

Oral Presentation Humanities, Arts, and Social Sciences (HASS) Applications Humanities, Arts, and Social Sciences Application

Speaker

Mr Giulio Bianchini (University of Perugia)

Description

In the past two decades increasing efforts have been devoted to diversify the tourism industry, such is the case with urban trekking and geotourism, which has become an important channel for promoting geological knowledge (Del Monte at al., 2013). The recent advancements in Augmented Reality technologies as well as the increasing availability of ‘born digital’ data like those gathered from social media, create the basis for the development of immersive and customized touristic experiences. Urban scientific tourism, Augmented Reality, and data mining are the key elements of the HUSH project. Its first focus is the identification of the naturalistic components in a given urban area (flora, fauna and geological features), through literature surveys and scientific research. These components come to be Points of Interest (PoIs) along touristic paths, where they are connected to the historical and artistic components of the area. Augmented Reality is the mechanism by which the user can access these contents, by means of the HUSH mobile application. In the geodatabase, each PoI is defined by a target image. This allows the users to access the augmented content by framing the target element for the component with their mobile device. The contents are delivered as videos, text, images, or interactive 3D models. The access to the PoIs can be performed in different ways. The user has the possibility to choose between predefined paths, paths suggested according to a keyword-based search and “intelligent paths” based on a Deep Neural Network (DNN). From an infrastructure point of view, the latter represents the most innovative element of the HUSH project. The users can access the application in several ways. One of these is the social login, which allows them to volunteer their social information in order to customize their touristic experience taking into account their general interests and preferences. The information acquired from their social profiles are stored in an anonymous form in a dedicated server. Due to the heterogeneous nature of the data collected, the use of NoSQL database is preferable, as it allows achieving scalable performance, strong resilience and wide availability. Once properly stored, user data are preprocessed and used to train an unsupervised neural network model. Such a type of model represents the future of deep learning, since phenomena like human behavior are largely unsupervised (LeCun et al., 2015). Once properly trained, the model helps identify and select the PoIs that could be of major interest for the user, in order to suggest a customized touristic experience. To achieve this task, the PoIs are properly classified in the database according to their characteristics (e.g., semantic and geographic). The use of the application and the associated increase of user data will improve the accuracy of the model over time. Moreover, based on the consideration that not all users may have or want to use their social profile to login to the application, information regarding the application usage (i.e., PoIs visited by the users) are also collected and used as additional training data for the model. **Acknowledgments** The project ‘HUSH – Hiking in Urban Scientific Heritage. Realtà aumentata per un nuovo approccio al patrimonio culturale e ambientale urbano di Perugia’, 2017.0103.021 Ricerca Scientifica Tecnologica, is supported by the Fondazione Cassa di Risparmio di Perugia. **References** Del Monte M., Fredi P., Pica A., Vergari F. 2013. Geosites within Rome City center (Italy): a mixture of cultural and geomorphological heritage. Geografia Fisica e Dinamica Quaternaria, 36, 241-257. LeCun Y., Bengio Y., Hinton G. 2015. Deep learning. Nature, 521, 436-444.

Primary author

Mr Giulio Bianchini (University of Perugia)

Co-authors

Dr Daniele Spiga (INFN Perugia) Prof. Diego Perugini (University of Perugia, Department of Physics and Geology) Dr Francesca Vercillo (University of Perugia, Department of Physics and Geology) Dr Laura Melelli (University of Perugia, Department of Physics and Geology) Prof. Livio Fanò (University of Perugia, Department of Physics and Geology ; INFN Perugia) Dr Luisa Liucci (University of Perugia, Department of Physics and Geology) Dr Sabrina Nazzareni (University of Perugia, Department of Physics and Geology)

Presentation materials