Comparative analysis of stuff like foodborne illness or deaths across countries is difficult. Methodologies and protocols differ. But it’s not impossible. You can make estimates, you just need to be clear and transparent on what your sources are and any assumptions you’re making.
However, despite these difficulties there is one thing about which we can be absolutely certain.
The Reality Check piece that Morris fronted made the schoolboy error of comparing numbers measuring two completely different fucking things. For the US, he used estimates of total illnesses whilst for the UK he used lab-confirmed cases.
And estimates are always way higher than confirmed cases; 9.3:1 for Campylobacter and 4.4:1 for Salmonella in the UK.
The bit that troubles me – well, one of the many aspects of this that I can’t get my head around – is that data for lab-confirmed cases are readily available for both countries. A more rigourous comparison could have been easily made. But for some reason it wasn’t.
The most recent US numbers show 46,623 lab-confirmed cases for Salmonella and about the same for Campy, in the UK it’s 10,089 and 63,946 resepectively. If we take population into account:
Yes, comparisons between different studies are fraught with difficulties but based on those data we do have available, rates would seem to be higher in the UK – certainly for Campylobacter than the US. (Note that Salmonella is rare and serious, Campy is common and unpleasant but generally not too serious).
One could argue that methodological inconsistencies mean we need to treat the numbers with caution – but that concern didn’t stop Reality Check making the ‘stretchy’ assertions that have been parroted elsewhere ever since.
In order to settle the matter there would need to be a comparison of rates of foodborne illnesses by country, gathered and analysed using a common methodology, preferably performed by an organisation with a degree of credibility.
Oh, fuck me, it just so happens there is one. The WHO did it.
This was what was cited by US Ambassador in his defence of US food safety standards. But Chris Morris reckons the 265-page WHO report is not relevant because it gave numbers:
“…only for total foodborne diseases and not the specific pathogens (Salmonella and Campylobacter) relevant to chlorine-washed chicken”
Not true. There is a useful summary of the report in PLoS Medicine which is easier to digest than the rather lengthy UN report. Check out Table 4 and you can see rates for every single foodborne agent including Campylobacter and Salmonella.
There is a flaw; it gives estimates by region rather than country and the AMR 1 region comprises the USA, Canada and Cuba. But given the US comprises >85% of the population of AMR 1 the study has power and comparisons with Western Europe region (EUR A) has meaning. And don’t forget all the countries in EUR A have common food safety and hygiene standards.
North America (and by extension the US) is estimated to have a lower burden of:
So, the position that Campylobacter and Salmonella rates are 4 and 20 times higher in the US than the UK is based on a fatally-flawed interpretation of the available data and I can’t find any original or robust sources that even come close to supporting a similar premise.
Granted, there is no perfect comparison, but the available data are consistent with the US Ambassador’s claim that US has lower rates of foodborne illnesses than Europe.
Morris also asserted that food-related death rates are much higher in the US than the UK and that the CDC estimates 450 deaths from Salmonella every year whereas in the UK fatalities are extremely rare.
Wrong again. As before, he compared two completely different measures: estimates of total deaths in the US with UK deaths where Salmonella was lab confirmed / recorded as the cause of death on the death certificate.
An issue I frequently have with the BBC is it seems (probably due to my confirmation bias) that whenever they report on a subject within my narrow area of dubious expertise they usually get it spectacularly wrong.
But this one is different. The data are readily available, as you can see above. Yes, there’s some nuance – but there always is. But errors like comparing numbers measuring completely different things is so basic and fundamental it’s not one I can readily attribute to an honest mistake: in life there are three things that smell like fish. One of them is fish. I have strong suspicions about this one.
Am I being too critical here? Let me know in the comments.
A month after the original story was released they changed the Reality Check page removing the most egregious assertions but thanks to the Wayback Machine you can still see the original containing the shonky claims Morris embellished in his piece on Today.
(As an aside they also cherry-pick a study saying chlorine isn’t toally effective – no shit, Sherlock, as any microbiologist will tell you.
“…so they might remain capable of causing disease”
…that’s why you cook the bloody chicken, FFS….)
The newer version is a wishy-washy ‘well, you can’t really make a comparison‘ and a bunch of innuendo that kind of assumes their position – that US food safety standards are lower.
But the damage was already done and the original zombie chlorine chicken story has been reappearing ever since across social media, the press – and even Parliament.
Egregious misrepresentation of the stats to support a preexisting political bias or just crass incompetence? You tell me. I don’t think the BBC will.