Ricochet is the best place on the internet to discuss the issues of the day, either through commenting on posts or writing your own for our active and dynamic community in a fully moderated environment. In addition, the Ricochet Audio Network offers over 50 original podcasts with new episodes released every day.
Help Me Understand the Testing Percent Positive Benchmark….
I mostly lurk here, but I think this might be a good place to get help with a question. Here in Arizona, one of the three benchmarks for re-opening schools, businesses, gyms, etc. is the percent of positive tests. It is often reported that a high percentage (above 5% is the benchmark in AZ) shows that there isn’t enough testing available and that the virus is not under control. I could understand the purpose of this benchmark if the state were either a) testing everyone they could get their hands on and/or b) testing a random sample in a community, not just people who self-select to test.
In Arizona, there is surely not a lack of available testing. That may have been true six weeks ago, but not now. This is widely reported (not enough people showing up for tests), but I’ll share my own anecdote.
Over the weekend I was ill, similar to a stomach virus, and since pretty much every common symptom is linked to COVID I figured, well, why not get a test? Arizona State University has developed a saliva test, which sounded much less uncomfortable to me so I went to their site and noticed that they had many open appointments. There were multiple spots open every five minutes up until closing time. I made an appointment, drove over to the site and the huge stadium parking lot set up with serpentine lanes was all but empty. Imagine a university stadium parking lot with four cars. I pulled up at 9:25, was handed the vial, did my spit thing, and drove away at 9:35. Easy-peasy. Although they estimated a 48-hour turnaround, at 8 pm that evening I received a text that my results were in and I was negative. Obviously, they are not overwhelmed. (As a side note, I highly recommend the saliva test. It is thought to be more accurate and was by far more comfortable than a nasal swab!).
We are at about a 10% positive test rate right now. Nothing can open until we are at or below 5%. I really didn’t think I had COVID, but I thought: hey, maybe I’ll help lower the percent positives! My kids really want to go to school! If testing sites are empty like this one, which was in a very busy metro area, I assume people who do not have symptoms are not getting tested. Oh ….and it’s all free, too, so that’s not an obstacle. How in the world are we supposed to ever lower that percentage by much if only symptomatic people get tested? I suppose it will go down as the virus wanes, but should this really be a benchmark? Isn’t this statistic somewhat meaningless? Even if everyone is encouraged to get tested, why would one do that willingly? Why would I get tested when I am feeling well? It tells me nothing other than the fact that at that moment in time, I do not have COVID. Tomorrow is a different day.
Am I missing something here? I just don’t get the constant harping on the percent positive of tests benchmark and why it is being used to guide all reopenings.
Published in General
I asked this question on another post, and it was never answered.
Yes, I would think that with all the Virus experts around someone would have the answer to that very important question.
@mimac?
@arizonapatriot or @rodin?
Statistics are a blight; they make us feel that our betters understand things that they don’t, and turn innocent facts into weapons of public policy.
5% is being used because A) it has a reasonable sound to it, and B) under that standard we will never fully reopen and C) we will just have to have mail-in voting. It’s OK, though. No limits on riots.
Let’s assume the metric includes a background level of testing for people that have to get tested for employment and thus there are many people that don’t test in response to having symptoms. If the number of required tests is high enough, then a testing level of 5% would probably show the achievement of herd immunity. The false positivity problem persists, so having goal value less than 5% is bad.
We’re good to go in Oregon, then. According to the Oregon Health Authority, today’s positive rate is 4.8%. We’ll see if that leads to any change in our lockdown status.
How does testing “control” the virus? It is a Wiley sneaky thing that eludes control.
If you test today, but come across the virus on your way home, what is controlled? Are you to test each day?
I think the Chinese, while wrong, did their best to control the virus by welding people into their homes.
15-days to flatten the curve in March.
Now its 5% of some test mid-August.
I call BS. Big time BS.
Thats nonsense. Lots of states have gotten their positive rate well below 5%, some hard hit early on.
As of August 4
With all due respect to @kozak, I think that @sandy is on to something. As @wanitten points out if only people who are feeling possible symptoms are tested then falling below 5% positives is really an achievement and a welcome outcome. Testing only people who present with symptoms is a form of “adverse selection”. In a perfect world you would target something like whatever your state experience was with various contagious diseases and periodically test a random sample of residents for COVID-19 positivity. (Clinical tests would continue to be performed to the extent useful in planning patient care, but would not be determinative for public policy.) But even this would not necessarily justify lockdowns even if the infection rate were higher so long as the severity of illness/death is not higher than experienced for other illnesses that we deal with without lockdowns.
The school situation is problematic because data on COVID-19 suggests that very few school aged children suffer significant illness. That does not minimize the tragedy for those persons and their families for those few who have atypical reactions to infection. So the real issue is whether we continue to quarantine children to protect their teachers and/or elderly family members. This does not seem (to me ) to be a proper public health question. It is for those teachers and their employers and for the families to make decisions that are limited to the relevant persons and not society as a whole.
The 5% rate is chosen b/c to achieve it you need to have a significant number of tests AND a low infection rate. If you only test people with symptoms you will have a high percent positive (look at some of the 3rd world data-some of the positive rates are incredible-but I presume they are only testing people in the hospital/ER). The situation you describe (no lines-lots of open appointments) doesn’t sound like a large number of tests are being done-which makes interpretation of the results difficult. Your area may be containing the virus but not testing enough-or the virus is not well controlled. The problem with relying on hospitalization, ICU capacity and deaths is that they are lagging indicators-it only tells you that 2-3 weeks ago you didn’t have the virus contained-the use of % positive is an attempt to get ahead of the virus rather than always being behind it.
I don’t think the positive rate has much value except, perhaps, to statisticians. There seem to be about as many protocols for arriving at that figure as there are states. For example, in Oregon the only people who are tested are those referred for testing by a physician because of symptoms, paid for by the state or insurance and those who take the test on their own initiative, paid for out of pocket. Those two conditions would surely yield far more positives than would random testing of the general population. (Or is that the goal?)
True, but it is a better indicator of what public (as opposed to private) actions should be done. If we got a do-over on this epidemic it should have been to have a random prevalence testing protocol to watch the spread of infection and hospitalization/death data to determine severity.
Perversely incentives have been created to overstate the severity of the illness in service of central control.
Sorry, I don’t know the answer to the question about the 5% benchmark.
Most places are open in Arizona, though, at least in Tucson. My teenage son is back at his job at Top Golf. The schools are closed, though.
This is precisely what very few of our esteemed leaders have been smart enough to realize.
Oregon is in the process of testing all residents and staff in care homes. Also, a test is required for some people prior to undergoing certain medical procedures. (My wife had to get one prior to an endoscopy.) I have no idea of the significance of those numbers compared to the testing of symptomatic people.
Expect a re-lockdown freak out within a week.
I say two weeks.
Forgive me if I’m dense, but I’m really trying to understand this.
It’s a ratio, positive tests divided by the number of tests given. So the result depends on who is tested. If you are testing only hospitalized patients, the rate will be high. If you are testing random people, the rate will be very low. I suspect that most of the tests in my state (Oregon) are given to symptomatic people, although tests are given to others (see comment #19). Other than care home staff, I don’t think employees are being tested much. (My daughter works in a hospital, and employees there are not tested unless symptomatic, even if exposed.) The contact tracing system here does not require testing unless symptoms are present. Anecdotally, testing here does not seem as easy to come by here as in, for instance, Arizona. So did somebody keep track of the reasons for the tests and put it into a formula to come up with the 5%? What is the formula? And why is the figure 5% for all states, even though the criteria and availability for testing differs among states?
This is true.
These are very legitimate questions.
To start, it’s important to remember that targets like positivity rate are calculated using models based on assumptions. And while I’m not privy to the actual decision-making processes used to determine individual state targets, from the general literature/guidelines it appears that there are two major assumptions underlying the selection of positivity rate cut-offs:
There are likely other factors that went into determining the positivity rate thresholds, but these two criteria will certainly have been the main driving forces. And they’re not that far fetched: about two months ago when states were setting their official re-opening gates, tests were indeed in short supply and demand for tests vastly outstripped their availability.
Whether or not current prevalence is a legitimate metric for re-opening is a separate conversation unto itself. But if we grant the premise that re-opening should be based on the current level of viral spread, the assumptions used to determine the positivity rates were reasonable at the time they were made.
The problem is that those assumptions are no longer true in many states: test availability has gone up, while many people don’t feel the need to get tested since they aren’t as concerned about the threat of the virus to their own personal health as they might have been a few months ago. If demand for testing no longer outstrips supply, it’s legitimate to ask whether the positivity rate thresholds are still realistic, or even whether using test positivity rate is a useful metric at all.
This would be the desirable approach in theory but is much more difficult to accomplish in practice than it sounds.
One issue is that the underlying threshold of “acceptable levels of community transmission” is so low as to be difficult to ascertain statistically. If 5% is the positivity rate cut-off among a self-selected group (that is much more likely to be infected than the population as a whole), that means the actual population-wide target infection prevalence is likely somewhere around 1-2%.
1-2% of the population is so low that detecting these levels with statistical confidence requires a large number of tests. Like thousands. Add to that the fact that the virus doesn’t spread homogeneously, but in clusters – that means even more people would need to be tested. But lots of people aren’t eager to volunteer an hour of their time to drive to a central test center and get a stick jabbed into their brain if they don’t think they’re infected. So you need to invite about twice as many people as you think you need.
And then repeat this every week.
All of this takes lots of manpower, manpower that doesn’t exist because most US states don’t keep fleets of public employees on standby in the event of emergencies like this, and don’t maintain central databases of their residents that can be used to quickly draw up lists of random participants. Most commercial labs also aren’t interested in this kind of work, so public health authorities need their own labs or good relationships with academic labs.
Finally, if the supply of tests is indeed limited, background surveillance usually takes the lowest priority for testing, behind people who are high-risk patients and testing used for isolation purposes.
TL;DR: background surveillance takes the type of long-standing government infrastructure that the US doesn’t have. The only places that have done a good job on this front so far are small, rich European social democracies like Switzerland or Austria.
That does not fit my actual real life clinical experience.
Most of the people being tested are not symptomatic. They are being tested because they were exposed, or work is requiring they have a negative test to work, or school, or they are traveling and need a test.
Well – in both NH and MA the “rates” are below the magic 5% and neither state has relaxed much if any restrictions. In fact – Baker has imposed more restrictions in the form of super special magic quarantine travel demands. Hotels in the Whites are still serving food on to go boxes with plastic utensils. So I think the testing activity is drama and the resulting “data” – even ignoring the credible reports of flawed testing and flawed reporting – is not used by administrations for anything except filler in press releases. The key data – regarding the cause and distribution of deaths – is ignored. I’ve come to think the whole script is something like me demanding – and getting – all the road signage in NH changed to accommodate my severe far sighted astigmatism so I can read everything without wearing glasses when I drive. I would be dangerous to myself and others if I didn’t wear glasses when I drive, but it would be so much more convenient to me if I didn’t have to take that precaution.
As usual, @mendel ‘s and @kozak ‘s contributions to this discussion are invaluable. What we are finding out is that knowledge of prevalence is difficult to come by and may not be that useful in informing public policy as a result, but politics makes waiting for death data before responding unacceptable. Hospitalizations are likely the best indicator with the following caveats: (1) only hospitalizations to treat symptoms should be counted for policy purposes while (2) testing of other patients should be done to determine clinical isolation needs, and (3) hospitalizations to treat persons with symptoms of COVID-19 directly imported from outside, e.g. Mexico, Brazil, etc. should be excluded from local data for policy setting purposes. All of this assumes that you are adopting policies to control the lives and conduct of healthy persons as opposed to policies only affecting the sick.
The % Positive (%+) is useful in a relative sense only. Lower is better than higher. Decreasing over time is better than increasing. And, with the caveat that all areas are using roughly similar protocols for who gets tested, one could use the %+ to rank areas. Your 15%+ is worse than my 6%+; but mine is not as good as their 1.5%+. But that’s as far as I’d be comfortable using it. Designating any particular number as a go/no-go indicator seems to me to be putting too much weight on too weak a reed. But there is an old quip from business – “That which gets measured gets managed.” And having a particular number to point to sure sounds better than “We’ll re-open when I say so!“. Even if that’s exactly what they mean.
I have never been tested. What part of the percentage am I? This is part of then lefties wanting us under their control. We know who is vulnerable and who isn’t. I don’t see any figures for the flu.