Throughout the first part of ‘Human-Centred Research and Design in Crisis’ course, I’ve been interested in both international covid-tracing apps (Assignment 2) and design beyond visuality (Assignment 3). I developed the Miro board further, and for the final presentation I joined both of the interests and elaborated on multi-modal Voice for Urban Mobility project. Although digital technology and apps cannot be always seen as the best solution for limited senses (e.g. firefighters’ case), but when coded right, they can give more independence to permanently visually impaired users.
Voice for Urban Mobility platform aims to tackle covid-19 pandemic (and other possible viruses) by providing relevant information about the situation to travellers and more vulnerable groups in society via mobile app and public kiosks both in Tallinn and Helsinki (Voice for Urban Mobility 2020). As the title of the platform suggests, both of the solutions should be multi-modal and voice-based, including diverse range of users, specially more marginalised groups (e.g. users without sight). However, as the public kiosks and application both try to give more information about the virus and guidelines for staying healthy and safe in urban environment, they have different level of personality and privacy.
As already briefly mentioned, the main target groups would be commuting workers between Tallinn and Helsinki, tourists, vulnerable residents (e.g. users without sight), elderly citizens and migrants. Residents with either Finnish or Estonian social security number could download the app and use more personalised solution, while the public kiosks were accessible to all users (specially tourists), not depending on the citizenship.
On local level, first and foremost both THS in Finland and Health Board in Estonia are the main sources of relevant information about the virus itself and the protective measures on both sides of Gulf of Finland. Migration services and border authorities could provide most accurate information about crossing borders and what are the requirements of entering the countries. City councils and public transport authorities have to be consulted in order to have permission to install kiosk to public places and bigger public transport stations (e.g. Rautatientori in Helsinki and Balti Jaam in Tallinn).
Information extracted from mobile network traffic data could be used in live population mapping (Khodabandelou, Gauthier, El-Yacoubi, Fiore 2016), therefore telecom companies could provide access to collected and anonymised data to map densely populated areas in towns. If telecom data might be difficult and risky to access, then in Helsinki, one option would be using data and information from the City Bike system. As user has to register their every ride, the system tracks the rides automatically and real-time information about the stations is publicly available online (Stations 2020). Although the information from the City Bike system cannot provide exact number of people, but it can outline most popular stations and thus reflect the number of people in area, which in turn can provide useful information for more vulnerable groups.
Moreover, as both Finland and Estonia are developing their national contact tracing apps, then it should be considered whether to create separate application or try to integrate Voice for Urban Mobility voice-based UI with existing solutions. Therefore, the companies behind national contact tracing apps should be considered part of stakeholders as well.
On international level, governments, specially Ministries of Foreign Affairs and Health, are coordinating international relations and responsible for the decisions made. In addition, as the data should be protected from both national and international level, EU authorities can monitor whether the proposed solution is in accordance with the legislations. As mentioned before, public kiosks would be in bigger public transport stations and on ships, so ports, airports and their administrators have to be considered part of international stakeholders as well.
Map of Ecologies
Nitin Sawhney and Anh-Ton Tran (2020) have proposed a framework for mapping and unpacking ecologies that influence debates emerging in participatory design. Issues of power are necessary to address in order to understand possibilities and limitations of proposed solutions.
Finns and Estonians have similar languages, set of values, legislations, understandings of privacy and level of digital literacy, so there’s solid basis for collaboration. However, one of the main concerns addressed throughout the course is the element of trust. Specially, as Voice for Urban Mobility aims to be cross-border solution, the question of who owns the data and has access to the information has to be answered. Although patients e-healthcare data can be accessed both in Estonia and Finland by doctors, there’s history of medical data sharing, but it has been more case-specific. Moreover, instead of contact tracing, could there be more trust towards users and let them share what they find relevant?
Although the voice-enabled solution is taking into account visually impaired users’ needs, are sighted users willing to interact by voice in public spaces? Studies in United States have shown that smartphone-based voice assistants are used mostly in private environments (e.g. in the car or at home), but in public, the percentage of users remains relatively low (Baker, 2018). As the platform would be multi-modal, all users could choose the form of interaction they feel the most comfortable with, but on the other hand, if the number of VUI users remained low, it could shift the development from voice to visual interaction and leave non-sighted users in vulnerable position yet again.
Despite the fact that Voice for Urban Mobility platform tries to cater for different needs, public kiosks and apps can have touching points, but the target groups differ, as well as the level of privacy. As kiosks are more oriented to sharing information about the virus, symptoms and rules-regulations, app or sub-platform Urban Well-being Diaries is more personalised and has similarities with contact tracing apps. Therefore, as both are ambitious projects, which one should be on the top of the priority list?In a way I feel that as many companies are developing covid-apps, kiosks are tackling the situation from a fresh angle, and in public, they are accessible to wider audience than national tracing apps.
Although the app attempts to have more holistic approach and take into account both physical and mental health and urban well-being, it could be still seen as tracing app. One of the proposed features, making it different from other similar solutions, is collective/in-family symptom monitoring. However, even such data sharing raises the question of individual privacy – as families are different, then it has to be considered that maybe not all family members are comfortable with one person having access to their medical data. Therefore, syncing app with different family members can seem more data-preserving than one user having access to other’s medical data; the latter can be more vulnerable if the device gets lost or stolen.
It can seem obvious, but public spaces tend to be noisy. Hence, it raises the question of what the right trigger would be to start the conversation, so it’s distinguishable from the background noise. On the other hand, as the kiosks probably will not be isolated, how to preserve by-passers privacy and not record random conversations from the street? There have been privacy issues with voice-enabled solutions in domestic environments, where voice assistants have been activated by mistake and they’ve recorded private conversations (Requejo, 2018). As there’s lots of noise in public spaces, how to prevent random recordings?
As there are many languages spoken in Tallinn and Helsinki, numerous languages should be included in the platform. The initial idea was to start with English and Finnish, but if Voice for Urban Mobility wants to be accessible for more vulnerable groups, then specially languages like Estonian, Russian and Arabic should be included. Moreover, even though the current lingua francais English, speakers from all over the world have different accents and dialects, which sometimes aren’t recognised or understood by the AI. Of course, the pilot voice-enabled UIs can be in Finnish and English in order to see how users are interacting with kiosks in general, and more languages can be included in visual interface.
Including visually impaired users to the development process could take their needs into account from the beginning and acknowledge that nice visuals are not enough. Drawing on what Jakob Rosin mentioned during his presentation in UX Tartu conference (2020), both application and kiosks should have voice-based interface from the start, so the experience for visually impaired user would be intuitive. In addition, audible identity with different tones and voices could create more engaging experience for all, especially non-sighted users could understand different paragraphs and sections in the text better. More attention should be paid to the code written as well and check its compatibility with screen-readers, so visually impaired users would get relevant information, not hear “button” every time they tap somewhere.
As audible information is important for doctors to diagnose, University of Cambridge is developing machine learning platform for diagnosing covid-19 by the respiratory sounds (Mascolo et al. 2020). They crowdsourced dataset, which had cough samples from both covid-19 positive and negative users, so they could see the difference between them and distinguish patterns/sounds describing the current virus. Moreover, mobile app reminded users to provide samples every couple of days, so the progression of sounds was traced. The combination of breath and cough predicted covid-19 with nearly 80% accuracy (Brown et al. 2020), however the data is not robust enough to use it singularly for screening. Despite that, stepping away from contact tracing and letting users/patients to monitor and record their own symptoms could be something worth integrating to Urban Well-being Diaries. Moreover, with more data, AI could give suggestions of what to do further based on the respiratory sounds and whether to take covid-19 test to confirm the potential diagnosis or consult with GP.
Voice for Urban Mobility project and Urban Well-being Diaries try to create more holistic and inclusive solution between two countries, stepping away from the national covid-tracing apps. Despite having numerous pressure points, the platform has benefits beyond covid-19 pandemic. As the virus has affected our behaviour, voice-based public kiosks could be ‘urban assistants’, highlighting more popular areas in town, giving information about possible restrictions in public spaces and reducing fear towards the virus. Furthermore, even when the pandemic is over, the kiosks could remain in public places and provide relevant information about two towns (e.g. traffic jams, events in town), being urban assistants as suggested.
Baker, J. (2018, November 27). Voice User Interfaces (VUI) - The Ultimate Designer’s Guide. Retrieved July 21, 2020, from https://medium.muz.li/voice-user-interfaces-vui-the-ultimate-designers-guide-8756cb2578a1
Brown, C., Chauhan, J., Grammenos, A., Han, J., Hasthanasombat, A., Spathis, D., Xia, T., Cicuta P., Mascolo, C. (2020). Exploring Automatic Diagnosis of COVID-19 from Crowdsourced Respiratory Sound Data. arXiv preprint: 2006.05919 (2020)., fromhttps://arxiv.org/pdf/2006.05919.pdf
Khodabandelou, G., Gauthier, V., El-Yacoubi, M., Fiore, M. (2016). Population estimation from mobile network traffic metadata. 2016 IEEE 17th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM). doi:10.1109/wowmom.2016.7523554
Mascolo, C., Cicuta, P., Floto, A., Brown, C., Chauhan, J., Grammenos, A., . . . Spathis, D. (2020). Covid-19 Sounds App. Retrieved July 21, 2020, from https://www.covid-19-sounds.org/en/
Sawhney, N., Tran, A. (2020). Ecologies of Contestation in Participatory Design. Proceedings of the 16th Participatory Design Conference 2020 – Participation(s) Otherwise – Volume 1. doi:10.1145/3385010.3385028
Stations. (n.d.). Retrieved July 21, 2020, from https://kaupunkipyorat.hsl.fi/en/helsinki/stations
Requejo, D. (2018, April 06). How Voice User Interface is taking over the world, and why you should care. Retrieved July 21, 2020, from https://medium.com/@goodrebels/how-voice-user-interface-is-taking-over-the-world-and-why-you-should-care-54474bd56f81
Rosin, J. (2020, May 29). How UX shapes the world of blind and visually impaired users.UX Tartu 2020. https://www.youtube.com/watch?v=6emfy_Z5Njk
Voice for Urban Mobility. (n.d.). Retrieved July 21, 2020, from https://voiceformobility.wordpress.com/