Your post is likely to be severely misinterpreted. I've already seen people sharing it with the takeaway comment that, "The real world data has shown that the death rate among the vaccinated, if infected with COVID, can be 3 to 5.7 times higher than the death rate of the unvaccinated."
A majority of people will read that as saying that the vaccine makes you 3 to 5.7 times more likely to die of COVID. Some of those who read more carefully will notice the "if infected" and interpret it to mean that if you catch COVID-19 after being vaccinated you are 3 to 5.7 times more likely to die than if you had caught COVID-19 without being vaccinated. As you point out in the "More graphs" section, and as discussed in the comment by Sandra and your response, THAT'S NOT WHAT IT MEANS.
The 5.7 is a completely meaningless number found by dividing the fatality rate in a group roughly half of whose members are over 50, by the fatality rate in a group only 2% of whose members are over 50. It is dividing apples by oranges. It says nothing about how getting vaccinated affects the chances of an individual dying if they get COVID-19. If the only confounding variable were whether someone were over 50, then the data in that table would imply that if you are over 50 and you get a breakthrough infection you have a 1 - (220/13,427)/(131/2,337) = 71% LOWER chance of dying than if you had gotten infected without being vaccinated, and if you are under 50 it is 1 - (4/15,346)/(34/119,063) = 9% lower, with wide confidence intervals on both. (This is a nice example of Simpson's paradox -- the trend in each group is different from the apparent overall trend.) There are many other confounding characteristics, such as finer breakdown of age, comorbidities, immune strength, etc, that are likely to correlate both with the chance of a breakthrough infection and with the chance of dying if infected, so after correcting for them it is likely that vaccination decreases the chance of death if infected even more. Of course you are correct that there's no reason to believe the decrease is as much as 95%, and certainly the data that is being presented doesn't show it, but none-the-less, it is very likely that there is some benefit, and probably a substantial one for the most vulnerable people.
You've done an excellent job of explaining why claims of vaccine benefit for those with breakthrough infections has been overstated, but most people will not read it all, and will instead come away with a faulty understanding in the opposite direction. Rather than interpreting it to mean that people need to be careful even after being vaccinated, it will be interpreted by many people to mean that they should not get vaccinated at all. I'm sure that is not your intent.
Could you edit your post to replace the assertion about 3 to 5.7 higher death rate with something that is less likely to be misinterpreted?
For now I am going to leave it as is. Given this is responding to people who are deliberately overstating the protection given, the initial shock of a true counterintuitive result is an important tool to help correct it.
I do not make any statements that indicate the vaccine is causing increased deaths - so to the extent people may be misusing the results to make that claim it's hard for me to completely stop.
If some does use to mislead, and the person they misled reads the article and sees why the numbers are high, they will at least be aware now - unlike before.
I'm not saying your concern is incorrect, but I have seen a lot of shares and discussion and at the moment it is not my assessment that people are walking away with the wrong interpretation. I'm sure some are, but it's not most, as far as I can tell.
What I mostly see are people who respond to claims that "even if you get covid you're are a lot less likely to go to the hospital/die" with this article. Which is mostly the misinformation I am trying to help correct.
What to make of the information coming out of Florida? “The virus has a new target: the unvaccinated and younger people,” says Mary C. Mayhew, President and CEO, Florida Hospital Association in a recent release. “Previously healthy people from their teens to their 40s are now finding themselves in the hospital and on a ventilator. Regardless of your age, get vaccinated, if eligible! What you heard last year and last spring about this virus mostly targeting seniors and those with pre-existing conditions is not true today.”
The media is echoing this. But the data in this article suggests the opposite — if I’m understanding correctly?
Looking at the definition of "vaccinated" in the Israel chart, it only includes people post 14 days after their second round. It isn't stated explicitly, but it appears that "unvaccinated" includes everyone else: those never vaccinated, vaccinated with one dose, or vaccinated with two doses but within 14 days of the second.
Isn't this a much bigger problem? How can we know what the relative risks are for vaccination without knowing the risks during the "actually vaccinated but still counted as unvaccinated" phase? Shouldn't the data have at least 3 categories? Ideally, each category would have separate counts.
Not likely that infection was verified by lab culture or some other means than a single PCR positive. Also, govt guidance on testing changed for the vaccinated. That is, unless showing symptoms, the vaccinated were excluded from PCR testing. Did this apply to the trials as well? Also guidance changed when testing the vaccinated so that the Cycle Threshold was reduced markedly from the rather high Cycle Threshold used routinely prior to the trials. Also, these vax trials at some point eliminated the control group by vaccinating. What is of concern is that the data gathering process has been severely botched and this may produce significant error or weak conclusions that can not actually guide policy-making much less the decision-making of individuals based on their circumstances. Hence the broad push for all to get vaccinated in a hurry. However, the rollout to the general population has overlapped the trials which are ongoing. Very messy all around.
Thank you for your effort. I hope I understood your article correctly. However, I am still puzzled by this:
">90% less likely to get infected — true"
... so there is a benefit for getting a vaccine after all? Just in the event If I get infected after I am fully vaccinated the chances are such as described in article? Please correct me if I am wrong.
Before setting up the trials, what was the substantive baseline risk of infection? Stratified by age or other criteria? The infection fatality rate appeared very low before the trials began.
Do I understand correctly? The initial trials done by the pharma companies showed a reduction in disease but it was a complete mystery why. They did not measure the rate of infection. Combined with the fact that some vaccinated people still got infected, many suspected that it just lessened symptoms but not infection (like IPV polio vaccine). But the results of this study suggest that the trial results are indeed explained by reduction in infection and it has almost no effect on symptoms.
To my knowledge there was no particular mystery. Yes the original trials did not test for infection the same way the later Israeli population study did. The original trials waited until symptoms appeared before they considered it an infection.
"Confirmed Covid-19 was defined according to the Food and Drug Administration (FDA) criteria as the presence of at least one of the following symptoms: fever, new or increased cough, new or increased shortness of breath, chills, new or increased muscle pain, new loss of taste or smell, sore throat, diarrhea, or vomiting, combined with a respiratory specimen obtained during the symptomatic period or within 4 days before or after it that was positive for SARS-CoV-2 by nucleic acid amplification–based testing"
The Israeli study includes earlier assessments involving positive tests, whether symptomatic or not.
So the original study wasn't particularly looking to see if it increased or decreased asymptomatic COVID to any degree. Just if those vaccinated became symptomatic and tested positive and at what rate compared to the placebo.
Looking at the first reference, I fail to see how "the death rate among the vaccinated, if infected with COVID, can be 3 to 5.7 times higher1 than the death rate of the unvaccinated". For example, Table 5 - for <50 group, the death rates are near identical. For >=50 they are 1.638% (220/134.27) for double-vaccinated vs 5.61% (131/23.37) for unvaccinated.
WHAT AM I READING WRONG HERE, I WANT TO UNDERSTAND
There are a lot of things delved into in the article at large. The 5.7 number is the infection death rate of the whole population. "those vaccinated with two doses died at a rate of 0.0078 per infection and those unvaccinated died at a rate of 0.0014 per infection"
I do explain how and why the infection death rate, among the whole population, increases.
That specific phenomenon does disappear when you split the whole population into age brackets.
When you do so, which I go through in the article, most groups do not show a difference in infection death rates. It is only as you get to the older population that any reduction in infection death rate can be seen.
BUT. When you vaccinate everyone, you wind up reducing infections in the people who were never going to die. The young. That means the IFR in the whole population of vaccinated people is higher.
I do not claim this is evidence of vaccine harm - I am alerting people to this to prevent conclusions that may not be true when the IFR rises after vaccination.
Interesting. The virus was associated with Severe Acute Respiratory Syndrome but what proportion of hospitalizations and deaths showed the hard symptoms of SARS? Mild symptoms are common for other identifiable maladies. Where the subjects tested for other maladies that could also require hospitalization and can cause death? If reduction of mild symptoms is counted in the overall 'reduction of illness' statistics, then, this would inflate the significance of the outcome.
Don't mean to be a pain, BUT: if we look at the entirety of the population, IFRs become 0.78 vs 0.13. However, since the first one is for fully vaccinated people who got covid, and if we eyeball it, vaccinations are about 80% protective, so we can assume that this brings in a reduction of factor 5. From here, IFRs become 0.156 (for all fully vaccinated, regardless of whether they caught it or not) against (can we assume flat-line) 0.13 for unvaccinated.
Assuming this logic is not full of shit, if we now extend it on the two age groups, it suddenly becomes 0.00522 vs 0.0284 for <50s, and 0.5276 vs 5.6 for >=50. The question here is whether the effectiveness goes down or holds across all age groups (the former could make the above somewhat different but I don't see how it could be significant).
I do break down the age groups for the Israeli data in the article - and show what the IFRs are for each age group provided.
16-44 and 45-64 show now evidence of additional protection. Those 65+ show some, but nowhere near 98%
But when you combine all the age groups - the IFR increases.
This is purely a mathematical effect. And if you don't notice it exists ahead of time, you could make incorrect conclusions in the future.
I do not claim that the infection death rate being higher is a problem - I claim not understanding it fully (by spreading misinformation) can lead to misreading the numbers in the future. Mistakenly concluding the virus is getting more lethal when it isn't (because you never noticed the higher infection death rate in the original data)
I also argue the false information can lead people to take more risk getting infected than the otherwise would.
The total population IFR should not be considered here, given that we know that the fatality rates increase with decades of life, reliably and consistently, without vaccines. It follows that only age group specific IFRs should be considered with this data (see https://en.m.wikipedia.org/wiki/Simpson%27s_paradox)
Your post is likely to be severely misinterpreted. I've already seen people sharing it with the takeaway comment that, "The real world data has shown that the death rate among the vaccinated, if infected with COVID, can be 3 to 5.7 times higher than the death rate of the unvaccinated."
A majority of people will read that as saying that the vaccine makes you 3 to 5.7 times more likely to die of COVID. Some of those who read more carefully will notice the "if infected" and interpret it to mean that if you catch COVID-19 after being vaccinated you are 3 to 5.7 times more likely to die than if you had caught COVID-19 without being vaccinated. As you point out in the "More graphs" section, and as discussed in the comment by Sandra and your response, THAT'S NOT WHAT IT MEANS.
The 5.7 is a completely meaningless number found by dividing the fatality rate in a group roughly half of whose members are over 50, by the fatality rate in a group only 2% of whose members are over 50. It is dividing apples by oranges. It says nothing about how getting vaccinated affects the chances of an individual dying if they get COVID-19. If the only confounding variable were whether someone were over 50, then the data in that table would imply that if you are over 50 and you get a breakthrough infection you have a 1 - (220/13,427)/(131/2,337) = 71% LOWER chance of dying than if you had gotten infected without being vaccinated, and if you are under 50 it is 1 - (4/15,346)/(34/119,063) = 9% lower, with wide confidence intervals on both. (This is a nice example of Simpson's paradox -- the trend in each group is different from the apparent overall trend.) There are many other confounding characteristics, such as finer breakdown of age, comorbidities, immune strength, etc, that are likely to correlate both with the chance of a breakthrough infection and with the chance of dying if infected, so after correcting for them it is likely that vaccination decreases the chance of death if infected even more. Of course you are correct that there's no reason to believe the decrease is as much as 95%, and certainly the data that is being presented doesn't show it, but none-the-less, it is very likely that there is some benefit, and probably a substantial one for the most vulnerable people.
You've done an excellent job of explaining why claims of vaccine benefit for those with breakthrough infections has been overstated, but most people will not read it all, and will instead come away with a faulty understanding in the opposite direction. Rather than interpreting it to mean that people need to be careful even after being vaccinated, it will be interpreted by many people to mean that they should not get vaccinated at all. I'm sure that is not your intent.
Could you edit your post to replace the assertion about 3 to 5.7 higher death rate with something that is less likely to be misinterpreted?
Thank you.
For now I am going to leave it as is. Given this is responding to people who are deliberately overstating the protection given, the initial shock of a true counterintuitive result is an important tool to help correct it.
I do not make any statements that indicate the vaccine is causing increased deaths - so to the extent people may be misusing the results to make that claim it's hard for me to completely stop.
If some does use to mislead, and the person they misled reads the article and sees why the numbers are high, they will at least be aware now - unlike before.
I'm not saying your concern is incorrect, but I have seen a lot of shares and discussion and at the moment it is not my assessment that people are walking away with the wrong interpretation. I'm sure some are, but it's not most, as far as I can tell.
What I mostly see are people who respond to claims that "even if you get covid you're are a lot less likely to go to the hospital/die" with this article. Which is mostly the misinformation I am trying to help correct.
I will keep your concern in mind though.
What to make of the information coming out of Florida? “The virus has a new target: the unvaccinated and younger people,” says Mary C. Mayhew, President and CEO, Florida Hospital Association in a recent release. “Previously healthy people from their teens to their 40s are now finding themselves in the hospital and on a ventilator. Regardless of your age, get vaccinated, if eligible! What you heard last year and last spring about this virus mostly targeting seniors and those with pre-existing conditions is not true today.”
The media is echoing this. But the data in this article suggests the opposite — if I’m understanding correctly?
If I can find good reliable data from Florida I will take a look.
There's a lot of data to sift through as you can imagine.
Looking at the definition of "vaccinated" in the Israel chart, it only includes people post 14 days after their second round. It isn't stated explicitly, but it appears that "unvaccinated" includes everyone else: those never vaccinated, vaccinated with one dose, or vaccinated with two doses but within 14 days of the second.
Isn't this a much bigger problem? How can we know what the relative risks are for vaccination without knowing the risks during the "actually vaccinated but still counted as unvaccinated" phase? Shouldn't the data have at least 3 categories? Ideally, each category would have separate counts.
According to the study "Individuals were defined as unvaccinated if they had not
received any doses of BNT162b2"
So I believe we can be confident that those marked unvaccinated are truly unvaccinated, and not partially vaccinated.
This is found on the third page of the study (page number 1821) under "statistical analysis"
Good to know, thanks. The CDC is not so transparent :-)
Not likely that infection was verified by lab culture or some other means than a single PCR positive. Also, govt guidance on testing changed for the vaccinated. That is, unless showing symptoms, the vaccinated were excluded from PCR testing. Did this apply to the trials as well? Also guidance changed when testing the vaccinated so that the Cycle Threshold was reduced markedly from the rather high Cycle Threshold used routinely prior to the trials. Also, these vax trials at some point eliminated the control group by vaccinating. What is of concern is that the data gathering process has been severely botched and this may produce significant error or weak conclusions that can not actually guide policy-making much less the decision-making of individuals based on their circumstances. Hence the broad push for all to get vaccinated in a hurry. However, the rollout to the general population has overlapped the trials which are ongoing. Very messy all around.
Hello,
Thank you for your effort. I hope I understood your article correctly. However, I am still puzzled by this:
">90% less likely to get infected — true"
... so there is a benefit for getting a vaccine after all? Just in the event If I get infected after I am fully vaccinated the chances are such as described in article? Please correct me if I am wrong.
Are there any updates on this topic? I have come across these calculations: https://www.covid-datascience.com/post/israeli-data-how-can-efficacy-vs-severe-disease-be-strong-when-60-of-hospitalized-are-vaccinated
Why do these differ so much?
Before setting up the trials, what was the substantive baseline risk of infection? Stratified by age or other criteria? The infection fatality rate appeared very low before the trials began.
You're right the baseline rate is low. That is estimated using the data from the control group.
Do I understand correctly? The initial trials done by the pharma companies showed a reduction in disease but it was a complete mystery why. They did not measure the rate of infection. Combined with the fact that some vaccinated people still got infected, many suspected that it just lessened symptoms but not infection (like IPV polio vaccine). But the results of this study suggest that the trial results are indeed explained by reduction in infection and it has almost no effect on symptoms.
To my knowledge there was no particular mystery. Yes the original trials did not test for infection the same way the later Israeli population study did. The original trials waited until symptoms appeared before they considered it an infection.
"Confirmed Covid-19 was defined according to the Food and Drug Administration (FDA) criteria as the presence of at least one of the following symptoms: fever, new or increased cough, new or increased shortness of breath, chills, new or increased muscle pain, new loss of taste or smell, sore throat, diarrhea, or vomiting, combined with a respiratory specimen obtained during the symptomatic period or within 4 days before or after it that was positive for SARS-CoV-2 by nucleic acid amplification–based testing"
The Israeli study includes earlier assessments involving positive tests, whether symptomatic or not.
So the original study wasn't particularly looking to see if it increased or decreased asymptomatic COVID to any degree. Just if those vaccinated became symptomatic and tested positive and at what rate compared to the placebo.
Looking at the first reference, I fail to see how "the death rate among the vaccinated, if infected with COVID, can be 3 to 5.7 times higher1 than the death rate of the unvaccinated". For example, Table 5 - for <50 group, the death rates are near identical. For >=50 they are 1.638% (220/134.27) for double-vaccinated vs 5.61% (131/23.37) for unvaccinated.
WHAT AM I READING WRONG HERE, I WANT TO UNDERSTAND
There are a lot of things delved into in the article at large. The 5.7 number is the infection death rate of the whole population. "those vaccinated with two doses died at a rate of 0.0078 per infection and those unvaccinated died at a rate of 0.0014 per infection"
I do explain how and why the infection death rate, among the whole population, increases.
That specific phenomenon does disappear when you split the whole population into age brackets.
When you do so, which I go through in the article, most groups do not show a difference in infection death rates. It is only as you get to the older population that any reduction in infection death rate can be seen.
BUT. When you vaccinate everyone, you wind up reducing infections in the people who were never going to die. The young. That means the IFR in the whole population of vaccinated people is higher.
I do not claim this is evidence of vaccine harm - I am alerting people to this to prevent conclusions that may not be true when the IFR rises after vaccination.
Interesting. The virus was associated with Severe Acute Respiratory Syndrome but what proportion of hospitalizations and deaths showed the hard symptoms of SARS? Mild symptoms are common for other identifiable maladies. Where the subjects tested for other maladies that could also require hospitalization and can cause death? If reduction of mild symptoms is counted in the overall 'reduction of illness' statistics, then, this would inflate the significance of the outcome.
Don't mean to be a pain, BUT: if we look at the entirety of the population, IFRs become 0.78 vs 0.13. However, since the first one is for fully vaccinated people who got covid, and if we eyeball it, vaccinations are about 80% protective, so we can assume that this brings in a reduction of factor 5. From here, IFRs become 0.156 (for all fully vaccinated, regardless of whether they caught it or not) against (can we assume flat-line) 0.13 for unvaccinated.
Assuming this logic is not full of shit, if we now extend it on the two age groups, it suddenly becomes 0.00522 vs 0.0284 for <50s, and 0.5276 vs 5.6 for >=50. The question here is whether the effectiveness goes down or holds across all age groups (the former could make the above somewhat different but I don't see how it could be significant).
I do break down the age groups for the Israeli data in the article - and show what the IFRs are for each age group provided.
16-44 and 45-64 show now evidence of additional protection. Those 65+ show some, but nowhere near 98%
But when you combine all the age groups - the IFR increases.
This is purely a mathematical effect. And if you don't notice it exists ahead of time, you could make incorrect conclusions in the future.
I do not claim that the infection death rate being higher is a problem - I claim not understanding it fully (by spreading misinformation) can lead to misreading the numbers in the future. Mistakenly concluding the virus is getting more lethal when it isn't (because you never noticed the higher infection death rate in the original data)
I also argue the false information can lead people to take more risk getting infected than the otherwise would.
The total population IFR should not be considered here, given that we know that the fatality rates increase with decades of life, reliably and consistently, without vaccines. It follows that only age group specific IFRs should be considered with this data (see https://en.m.wikipedia.org/wiki/Simpson%27s_paradox)
This is the exact thing I am outlining in this article.
Your objections are actually in agreement with the article.
The care you have taken in your explanations is highy commendable. Thanks for digging in.