Statistically Accurate vs Individually Accurate Systems
Many tools go through data to come up with conclusions or recommendations. Like an app that suggests you go for a run now. Or one that warns you you’re spending too much money this month.
Such apps can be either individually accurate or statistically accurate. Individually accurate is a high bar: it means that the results are almost always correct. Otherwise people will lose trust in them. For example, Outlook has Meeting Insights, which is supposed to go through your emails and find mails and attachments relevant to a meeting. But when I’m interviewing a candidate, it shows me other candidates’ CVs, adding to, rather than reducing, my cognitive load. Another example is my Garmin fitness tracker that reminds me occasionally to move, but it used to buzz and wake me up when I was sleeping in the afternoon, so I turned it off. People accept other humans making mistakes, but have little tolerance for machines making mistakes. If a machine makes a couple of mistakes, we’re quick to turn it off. Individual accuracy is a high bar to meet. Even if a system is accurate for others, if it doesn’t work very well for you, you won’t use it. And you’ll share your experience with your friends. Since one bad recommendation needs multiple good recommendations to offset, they won’t use it either.
Building an individually accurate tool is hard because different people use tools differently. In the above example of my Garmin waking me up during my afternoon sleep, maybe you don’t sleep in the afternoon. Or maybe you do, but you configure your phone to be in Do Not Disturb, and you expect your activity tracker to respect that. A system that’s individually accurate needs to cater to multiple workflows.
On the other hand, a statistically accurate system is right often, not always. Or, more precisely, a statistically accurate system is allowed to be wrong more often than an individually accurate system. An example is insurance pricing: if a company deploys an algorithm that causes it to make increased profit on 80% of customers than the increased loss on 20%, it will come out ahead as compared to the status quo.
If you’re planning to build an individually accurate system, know that it’s a research project: it takes a highly skilled team of engineers working for years in a stable work environment with stable funding of millions of dollars. You can’t exhort your team to ship this month because <insert startup mantra here>.
You can’t ask your team, “When will it be ready?” They won’t know ahead of time. And it may never work out — that’s always a risk with research.
These are the constraints you should be aware of if you want to build an individually accurate system.
If you can’t afford this, it’s better to acknowledge reality now and instead pivot to building a statistically accurate system. For example, if you’re considering building a system to give people health recommendations, that’s individually accurate: you can’t be casual about giving wrong suggestions and ruining someone’s health because it works for 80% of users. You could instead pivot to building a system that prices medical insurance, or which analyses prescriptions and tells a hospital how many patients have a particular problem.