One of the big themes in the popular news on COVID research these days is studies that report on how much masks help reduce the spread of COVID. These studies generally have some serious methodological limitations; I have yet to see one that can distinguish between the hypothesis that masks limit the spread of COVID by blocking stuff from the hypothesis that people wearing masks will be more mindful of other restrictions, and thus reduce the spread of COVID due to better adherence to social distancing and hand-washing guidelines.
This is the start of a series of posts discussing some of these studies. This post will give an overview of the different types of evidence typically used to argue this point; specific studies will be addressed in future posts.
The gold standard for medical studies in the Randomized Clinical Trial. To do this for the question of whether wearing masks helps reduce the spread of COVID, you'd need to randomize experimental subjects (communities?) to either be told to wear masks (and maybe related enforcement measures) or not to be told to wear masks...and hope the non-mask wearing communities don't start wearing masks. There are ethical concerns in doing this, not the least of which is getting informed consent from, well, everybody in a community (probably won't happen). Then there are the compliance issues. How long do you want to run the study for? What if the results are negative, but that's because the wrong type of mask was used? What if, say, Iceland were to do this? Would the results hold for Brazil, due to climate differences, population density differences, demographic differences, etc.?
More restricted, we have laboratory tests showing what masks block. Short of going for high-grade masks (e.g. N95 masks) worn correctly, which is not practical for the general population, masks stop large droplets, but not finer aerosols. How do we think we know this? Lab tests, where masks are exposed to air flows to simulate breathing, with air with different types of particulates on one side of a mask, and devices to measure what gets through on the other? Which matters for spreading COVID? Based on genetically similar viruses, it appears the aerosols are more important; they are more likely to get deeper into the long and avoid the outer layers of the bodies protections (e.g. nose hairs). Is COVID the same? We don't know. Moreover, if *you* are sick, masks will reduce the amount of virus you shed, reducing the viral loads others are exposed to. So the mask might not protect you, but may protect others. And all of this ignores the question of whether the laboratory set-up is a good simulation of environmental exposure.
Looser than Randomized Clinical Trials, we have an idea that has become popular in the Economics literature of the Natural Experiment. Find two otherwise identical areas except for some external influence, and see how their fates differ due to this external influence. If needed, panels of areas can be used to synthesize a better match. This approach relies heavily on the two areas being otherwise identical. Strictly speaking, you cannot infer causation from such a study, because there might be an important difference you did not measure that is instead causing the difference you observe. However, this approach does not have the ethical concerns of a Randomized Clinical Trial; you're just doing the best you can with the observational data you have.
Looser yet is that you compare the fate of an area before and after mask control are put in place. If the spread of COVID slows after masks are required in an area, you conclude masks help. However, other changes in behavior can also be causing the slow-down; other trends are not being controlled for.
Finally, you can just compare areas that require masks with areas that don't, without any effort to match other characteristics. Any observed difference can be due to differing demographics, differing work roles, differing population density/population habits, differing environments, differing population vulnerabilities, or a host of other differences I'm not thinking of, I have trouble considering such studies as useful contribution to the discussion.
And outside of the laboratory, these methods also suffer from the challenges of comparing COVID rates across communities previously discussed. But as I've said before, perfect should not be the enemy of good, and most of these studies are worth thinking about.
This series will end with a post discussing a challenge I find fascinating - how should one go about making decisions in the face of uncertainty? Especially when we can't rigorously quantify that uncertainty, the uncertainty in our uncertainty is too large.
No comments:
Post a Comment