Toward a better COVID-19 tracer app user experience

A tweet by Peter Dunne has stirred up a small backlash of denial and insecurity that I have found endemic to NZ society. In the tweet, Dunne criticized, somewhat obnoxiously and with politicized overtones, the New Zealand Government's COVID-19 tracer app's design. The tweet was brief and vague, but importantly attributed the “woeful” usage rate of the app to its “clunky” design.

Whatever else the tweet may convey about the political characters involved, the fundamental premise that the design of a system determines its usage is a sound one. Let's see why.

I would like to dive into how that plays out with the COVID-19 tracer app in more specific detail. But before that, let me pontificate somewhat on the state of society:

The backlash mostly consisted of the usual unsound tropes: it-works-for-me, you-should-harden-up, it's-better-than-something-worse, and don't-criticize-without-suggesting-alternatives. The app may work for some people — especially those who are able to, and so they have, accustomed themselves to it. But it does not suggest that it works enough of the time for enough people. By extension, the notion that critics should stop complaining and just get better at using the app, is to take a victim-blaming or rotten-apple or personal-responsibility approach (as opposed to systemic analysis), which is antithetical to any progressive cause. That the app is basically better than something which is worse is a truism, and it neglects to account for the “woeful” usage rate, which remains the more important measure. Finally, the demand that any criticism of this nature should only be accompanied by an alternative suggestion is a misguided one: as any subsequent constructive discussion shows, one needs to have some level of expertise and insight to be able to produce alternatives, which is an unrealistically high bar for users to give feedback. One does not expect a doctor or nurse to retort to a patient's medical complaint with, “what is the suggestion you have for an improvement?”.

These are common, popular reactions to critiques of common, popular systems. I wonder whether it is because users of these systems develop an attachment or identification with it, or a sense that they worked hard to accustom themselves to it, so any complaint that diverges from their experience must be personally undermining their own experience. I see this all the time in my advocacy and activism for disability accessibility (“it works for me”), safer street design (“no bike lanes please, you should harden up if you want to cycle here”), ethnic or gender/sexuality minority rights (“we're not as bad as those other parts of the world”), and so on. It is a scary thing to confront a crowd — including ostensibly progressive people — to deliver a criticism or complaint about an incumbent system that makes them feel insecure about their own relationship to it. People feel like their whiteness is threatened when there are complaints from non-white folks about racism; or that their straight marriage is threatened by gay marriage equality; or that their environmental credentials are threatened when someone else rides a bike; etc. So it is when popular Government software that lives in their phones and intimately tracks their lives by shaping their daily habits gets criticized — it seems to threaten their own identity.

Nevertheless, although I disagree that there ought to be an expectation of alternatives or “constructive” feedback, it so happens that I have some degree of expertise in software and system usability, as it is my line of work. Because of this, I can shed some light on the shortcomings of the COVID-19 tracer app beyond the vague critique that the original tweet in question offered. But I also am not closely connected with the software development project for the COVID-19 tracer app, nor the Ministry of Health, so it would be impossible to give very detailed feedback based on an understanding of the project's constraints. I am also lacking meaningful user research data which would normally guide any decision-making in these matters. So I will document some of my own limited user experience, but with a generally-informed analysis that could prove useful. I hope, more than anything, I may disprove the reactionary response that, “I cannot conceive of a way to make [the app] more simple and straight forward to use”.

While I try to use the app regularly, I do often fail to scan in. Below are some of the failure modes I have encountered. I do not generally include glitches or bugs, but rather design problems and workflows that are poorly accommodated.

Failure scenarios

Access

The system requires that a QR code must be scanned by visual contact at close range, usually from an A3 or A4 sheet of paper stuck to a door, window or wall near the entrance of a venue.

This is often failure-prone because of the non-standard placement of the codes. Height, glare, and angle are often problems that slow down the scanning process. All of these properties affect the position and angle in which the phone must be held and I find it a high-friction process — if I have found a QR code poster at all.

What are some ways to reduce this friction? Redundancy is the most obvious and cheapest, low-tech response, i.e. having multiple posters, especially at varying heights and angles.

Usually when I am operating my handset, I hold it at an angle pointing somewhat downwards. While poised with the phone in hand, it is most convenient (and discreet) to keep it pointed in that downwards direction. Therefore, I would find it useful for posters to also be available nearer the ground, but angled to face up, perhaps on a plinth (nothing fancier than cardboard), rather than being placed at approximately eye level which requires the phone to be roughly parallel (that requires holding it up in front of the face, conspicuously). As a taller person, I find it also inconvenient to have to position the camera so it is parallel to the poster but below my eye level, and then simultaneously operate the user interface that is now on a screen awkwardly tilted up.

The situation is usually worse on a bus where the space is more cramped. (More on buses later.)

Sometimes the QR code simply won't scan — not just due to glare or angle but for focus distance.

While reducing the friction of scanning a single QR code helps, defence-in-depth is perhaps more useful — ensuring that posters are repeated not just at the entrance to a venue but throughout its premises. Some cafes place QR codes on tables and menus — that's good. But my local supermarket has an entrance QR code but none amongst the aisles — this is a lost opportunity. Another lost opportunity is that supermarkets and shops invariably print receipts — this could include a QR code that can be scanned at the user's convenience. A techy extension of this could be for shops to issue timestamped QR codes in receipts to be scanned later (but automatically backdated to the timestamp), if the app would support that.

A broader technological response to this situation is to just accept that whilst users may want to scan in, still for one reason or another cannot or will not do so — whether it's a technical or accessibility limitation, or even discomfort and inconvenience. The app should not judge but rather be more forgiving of such scenaries and allow retrospective logging.

One obvious first step of handling scanning failures is to record that a user had a problem accessing a QR code (and why) and to submit that information to the Ministry of Health, which should develop the capacity to enforce guidelines and standards with its QR code registrants. The app has a manual entry log, but it could also optionally collect feedback on why a manual entry was needed — was there a problem with the QR code poster? Should the Ministry follow up with a retailer or bus operator, say? Are many people having the same problem in the same places?

There are still other ways to be forgiving too, and more ideas are given below.

Oops!

One issue with the fickle camera positioning problem is that the app and device, in my case, often requires two hands free. But my hands — often while shopping — are occupied, and I cannot operate the handset to scan in. Defence in depth is one option that helps, because it allows more opportunities to scan in later, perhaps when I have freed my hands of my cargo. But what if I am going away from the premises, or through it to another one? How can the QR code of the place I had just visited (but didn't scan into) remain available to me?

The same question arises in cases where I couldn't use my phone to sign in for any other reason. Low battery, or I left it at home. Or I was doing something else with it — making a call or in the middle of typing a text message. Switching tasks is possible but it adds friction — that's what, as designers, we want to minimize, and still achieve the goal of high usage (whereas blaming the user for being lazy or something like that achieves nothing good).

It is possible that people are not focused on the task of signing in, whether because of environmental distraction, stress, or another condition like ADHD, etc. I've certainly been in such a position many times. Again, having more opportunities to scan in later is the most straightforward option I look for.

The social aspect of performatively and conspicuously signing in is another kind of disincentive or friction. If everyone appears to be signing in, it is easier to get involved. If no one else seems to be doing it, even fewer people will break ranks to start. This matters because the first instinct of a segment of the population will be to go with the flow — i.e. skip scanning in. Sometimes, I have been in that segment — I felt uncomfortable drawing attention to myself scanning where there was no space to comfortably do so. It can be worse in confined spaces where I may be blocking others. Many indignant commentators are proud of their ability to confront and chastise others who are hostile to their scanning, but many of us are also not so hardened-up. Once again, the goal of design is to accept this phenomenon and try to achieve a high usage rate anyway. How can we do that? More opportunities to scan offers more chances of a discreet process. The earlier suggestion of lower QR code posters helps too.

But there must be some way to scan into a place even long after leaving the premises. I should be able to select the location I have recently visited in the app's manual entry screen, as if the QR code for that location is still in front of me. And why not? The Ministry issues QR codes to its registrants, who generally have pre-determined, fixed locations. This database of fixed locations could be presented both as a map view in the app, and as an auto-completing field to accurately fill in a location. The map could optionally use the device's current location or a user-selected point, and offer all nearby registered QR codes (and venue names) in a list. So if I went to a supermarket earlier and forgot to scan in, I should be able to bring up a map during manual entry and drop a pin near the supermarket's location, and be presented with a list of proximate registered premises, near the top of which I should find my supermarket to select.

Currently, the manual entry process requires a lot of typing, and does not auto-complete addresses, and does not visually confirm locations in any way. Compare this to Google Maps, Yelp, Foursquare, Zomato, etc, which offer much richer location-based user experiences and workflows. The manual entry process is high-friction and can be made much easier so that it encourages accurate, retrospective checking-in that should boost usage rates.

Another slightly advanced feature would be support for partial check-in. That is, without scanning and linking to a QR code, a user should be able to use some shortcut to rapidly log a location and timestamp to the app with very little fuss (2 seconds at most). That partial entry can then be refined later through a location-based proximity search of registered premises as above. This is a substitute for continuous GPS tracking and only logs a key point when required, with some affordance for refinement. Unrefined locations lingering the diary should cause notification prompts and reminders to be shown to the user to encourage them to fill it in completely later.

An even more far-fetched feature could use the camera not only to scan in with QR codes but to take a location-tagged photo of the area, or perhaps the logo of a shop, receipt, etc. Refinement of this raw input into a registered premises could happen later through both manual entry and possibly automated help, including image recognition (such as identifying logos, or scanning receipts). The general direction of these future ideas is to lower the friction of interacting with the app, to permit capturing essential information through shortest path, or to support smoother user experience for retrospective logging.

Pocket calculator

Confession: I don't use my handset very much. Some people do, perhaps most people do. But I and some others do not. This is one reason that developing a habit of rapidly activating my device to scan in and getting the job done isn't so easy. Why I don't use the device much is none of your business. Once more, the goal of design is to accept the limitation and work with it. So what does that mean?

It means I want to retrospectively log or refine data using my computer, not only depending on the small-screen touch device that the apps are deployed on. I want to browse a map in my web browser and type input text using a larger keyboard.

Even with its decentralized data architecture, the app should be able to sync data (opt-in) securely across the web or with a locally-networked or Bluetooth-enabled computer. I should then be able to use a map or even a Google Street View experience to retrace my steps and log entries at the end of the day using a Ministry of Health web app in my large-screen browser.

Combined with location-based, partial check-in using a mobile device, and a more accessible user interface on computers, this can offer some Disabled people a better way to operate the tracer app system.

Vroom

While redundancy and location-based logging can be useful for fixed premises, it is more difficult for mobile premises such as public transport vehicles. Let's use buses in Auckland as an example.

Scanning in when using the bus has been difficult for me. Most of the time I don't do it. Of course, there is HOP card data that would help with contact tracing, but that is not always reliable (e.g. I recently rode the bus without tagging on/off because the merciful driver allowed me to get on and top up my empty balance later). The biggest issue is finding QR code posters on buses — they are very inconsistent on the routes I ride most often. Some buses have them, some don't. Some seats have them, some don't. Going out of one's way and being conspicuous to scan a QR code — perhaps encroaching upon another's personal space — is, for me, out of the question. So what to do?

There isn't much that defence-in-depth can offer here, because if a bus has no accessible poster, then there is no other place to put one. The bus stop where I alight can't offer me the QR code of the bus that just departed. Or can it? I think it can, at least in Auckland Transport's jurisdiction, with the cooperation of the COVID-19 tracer app. If bus stops also had QR codes, I could scan into the bus stop I just arrived at and declare that I got off a bus just now (with timestamp logged). Then, as a refinement of my partial check-in, the app could offer me options for which buses are known to have made the trip (or were scheduled to have made the trip) within some time range before and after my check-in, based on Auckland Transport's transit feed data (which documents just about all trips). So the app could offer a button not just for “add manual entry” but more specific scenarios like “I just got off a bus here”.

A button also for “I just got on a bus” would be useful. It would log my location and time in the first instance. And if I am online, or I decide to refine my entry, I could receive options for which bus I could be riding based on transit feed data or my input of bus route name, etc. This would take the awkwardness out of scanning in with unreliable and inconsistent QR codes on buses and allow the user to discreetly operate their device without much social interaction or a public display of scanning (friction).

Retrospective entry for a bus trip should also be possible through a map view with a public transit layer of routes and stops and times.

Currently, manual entry of bus trips is high-friction because finding and typing in the bus details is cumbersome, including the stops, vehicle number, route number, etc. This can be made much easier by integrating with already available public transit data.

We're in this together, aren't we?

This is a situation I found myself in more than once. I was in a group or with one other person and for reasons that don't matter, one of us or most of us did not scan in, while at least one other did scan in. I think there is a valid sentiment that a group should be able to scan in all together with just one QR code interaction, rather than each individual pausing and taking time. Can this be supported?

Now, the need for such a feature is perhaps reduced with redundancy of QR code posters (defence in depth), because it allows people more opportunities to scan in by themselves (e.g. when settling into a cafe table). However, let's suppose that doesn't work for some arbitrary reason and the app must cope regardless.

One way of supporting shared scan-in is to have a friend list. If people in a group are known to each other socially, they might be allowed to build a friend list of each other's devices, perhaps by Bluetooth, or even their usual contact list of phone numbers. Perhaps this can allow one person who scans in the option of forwarding their entry to all nearby friends by Bluetooth (selected by default), or also manually chosen friends from their contact list (not selected by default). This should cause a notification prompt on the receiver's device allowing them to accept the scan-in to their diary too. I can imagine this opt-in feature working most usefully for family groups etc who may regularly go out on trips together. That is, as long as the user experience is designed sufficiently non-intrusively.

Another simpler possibility is for the user who scanned-in to be able to select a diary entry and display a QR code on their screen for sharing. Their companions can then more conveniently copy that entry by scanning the QR code of the first user, rather than all queuing up at the entrance of the premises at the same time.

The neat thing about this kind of social approach is that it leverages influential members of groups to promote others scanning in. Given that such people already exist in many social groups, we can look to make their 'ask' easier. It is harder for them to nudge their friends/family to scan if the 'ask' they have to make is for someone to stop, go out of their way, or backtrack right away. However, if the app allows them to, say, share a QR code they just scanned, the 'ask' that others copy it without going anywhere is more personal, discreet and accessible.

Déjà vu

Much of the time I am scanning into places I visit regularly, like the local supermarket. Do I have to go through the ritual of its QR code poster awkwardly-positioned every time?

Why not let me save a list of most-frequently visited or favourite locations to speed up the scanning process and to allow bypassing the poster on every visit?

This would reduce the awkward scanning interaction to the first visit alone and make it more likely that I will keep scanning in during future visits.

Conclusion

This assortment of failure scenarios I listed above comes with suggestions for lowering friction and allowing more flexibility across time for logging visits to places. Why these are necessary is not important, and I would caution strongly against blaming users for not simply doing something that you may have habituated yourself to doing. What is more important is to identify all such disincentives and to remedy them through the design of the system as much as possible, to allow diverse populations to interact with the app in diverse ways, to achieve the main goal of logging data for easier and more reliable contact tracing.

I've sliced through this problem space with only a few of my personal experiences and frustrations using the app, and some of the features I found myself wishing it had. These are not all going to be comprehensive solutions but it should illustrate the kind of approach needed to continuously improve the app and increase participation in logging data for contact tracing.

I have intentionally avoided the realm of gamification to drive up participation, because of its likelihood of unintended consequences. So I have stuck to the removal of barriers, obstacles and frictions that get in the way of intrinsically incentivized usage.

Ultimately, to criticize the app and system's design and deployment is not a bad thing, and to acknowledge that usage rates may be declining because of its inherent frictions is a sound starting point to try to reverse the trend.

Postscript: the Dunne effect

If I were being charitable, I would have to accept that, to some extent, the backlash on Twitter has been motivated as a reaction to the messenger, not the message. The messenger has developed a reputation for taking dubious positions on COVID-19 policy issues in the past, and has continued that form by crafting the current tweet as a political attack. What, then, can we reasonably say as a counterpoint to Dunne's message?

For one thing, while it does recognize the causal link between system design and usage patterns, it fails to acknowledge that this has not been disputed by the Ministry or the software's developers. In practice, the project's trajectory has been that of progressive improvement to make usability easier.

That trajectory has been reportedly constrained — as is typical in software projects — so that not all desirable features could be delivered by now. There is room to debate the prioritization (especially the neglect of software accessibility) within the envelope of what is possible, but that is not Dunne's line of inquiry. Rather, Dunne simply fails to acknowledge the constraints the project is operating within.

For Dunne to tie, then, the usage of the app to the Minister (i.e. 'Hipkins' as a verb) is a misplaced association. It is unlikely that many questionable gaps in the app's development are a matter of ministerial policy, but rather largely the effect of constraints inherent to software development subject to urgency and the tactical choices made by the development team in an industrial environment that is poorly regulated for quality standards (there is a bigger issue down this path with technology regulation, which, also, is beyond the scope of Dunne's commentary).

In other words, politicizing the app as a Ministerial failure is an irresponsible mistake. It can have unintended consequences by casting it as a pet project of a Labour Member of Parliament, rather than the product of Government and public servants.

The Minister has not helped the case, however, by reinforcing the browbeating message that users should simply use the app more once again, without emphasizing the improvements that have been made and could still be made to address friction in the user experience. While the project has followed this path of development, the Minister does not appear to be giving assurances to that effect, which undermines somewhat the reciprocal trust needed between vendor and user to keep up engagement. If Dunne can make political hay out of the Minister's dissonant messaging, then the responsibility lies with the Minister.

So where Dunne's message is not wrong is the premise this post began with: that a system's design determines its usage. Taking the flatly opposite position through denial and dismissal merely to oppose the messenger is not only fruitless, but also possibly harmful to marginalized folks who also depend on the critique of incumbent systems to advance equality and justice.

A more focused rebuttal to Dunne's tweet is warranted, not the works-for-me and I-don't-see-a-problem attitude that it has provoked.