Images do not only carry visual information about their content but also an intent, namely the one of person who shared it. Disruption images, uploaded by citizens in urban reporting systems, constitute a fantastic lab to develop algorithms able todetect the different message users want to deliver. Citizen-based reporting systems, e.g. FixMyStreet / ZürichWieNeu, rely on private citizens to report urban incidents/damages which require an intervention by the city administration. Most of these systems give the users the possibility to upload a picture together with a textual description of the incident.

The peculiarity of the images uploaded is that they don't necessarily want to depict specific objects or situations, but they are rather aimed to the communication of an underlying problem. In this sense, these images can be considered bearing an intent. Intent-detection is still a fairly new application field for deep learning, usually however limited to textual data.

The goal of this project is to develop a deep learning algorithm to perform multi-modal intent detection, combining images and problem descriptions provided by users. This could serve as a first well-defined use case to develop and test an approach, which could be generally expanded to othervisual-rich contexts.