The Microbiology of Brexit

I’ll get on to wider food production questions later. I want to boil the chlorinated chicken / US livestock issues down to a simple question:

Are standards of food safety in the US worse than the UK?

This seems a simple question but it’s complicated. Let’s use a surrogate outcome and hone the question down to “is there greater incidence per capita of foodborne illness in the US than the UK?” Given this is the measure used to prop up most of the headlines let’s try to bring some rigour to it.

Having done some digging a lot of this seems to go back to March 2019 and an item on BBC Radio 4’s Today programme. For the uninitiated, this programme modestly describes itself as the BBC’s ‘flagship’ news programme and prides itself on keeping the political class honest through dogged pursuit of the facts of any matter at hand.

They were investigating a claim by US Ambassador Woody Johnson that the USA has some of the “lowest food poisoning rates in the world”.

The BBC ‘Reality Check‘ correspondent, Chris Morris, reported that he had investigated the question and that the statistics were clear:

“The Center for Disease Control in the US estimated there are about 1.2m illnesses from Salmonella in the country every year and 1.3m from Campylobacter.  Now that works out about 0.4% of the population for each of them.

“And if you compare that with the UK figures for 2017, it means you are about 4 times as likely to get an illness caused by Campylobacter in the US than the UK and roughly 20 times more likely to suffer from Salmonella poisoning.”

To summarise the core of Morris’s analysis:

BBC chief Tony Hall has described the BBC’s ‘Reality Check’ unit as being a vital part of the Corporation holding people to account in the fight against fake news. And when the Reality Check team make pronouncements such as these they are never challenged by fellow BBC journalists.

But directly accusing the US Ambassador to the UK of being full of shit on air? You’d better be pretty sure of your facts…

Comparative analysis of stuff like foodborne illness or deaths across countries is difficult. Methodologies and protocols differ. But it’s not impossible. You can make estimates, you just need to be clear and transparent on what your sources are and any assumptions you’re making.

However, despite these difficulties there is one thing about which we can be absolutely certain.

The Reality Check piece that Morris fronted made the schoolboy error of comparing numbers measuring two completely different fucking things. For the US, he used estimates of total illnesses whilst for the UK he used lab-confirmed cases.

And estimates are always way higher than confirmed cases; 9.3:1 for Campylobacter and 4.4:1 for Salmonella in the UK.

The bit that troubles me – well, one of the many aspects of this that I can’t get my head around – is that data for lab-confirmed cases are readily available for both countries. A more rigourous comparison could have been easily made. But for some reason it wasn’t.

The most recent US numbers show 46,623 lab-confirmed cases for Salmonella and about the same for Campy, in the UK it’s 10,089 and 63,946 resepectively. If we take population into account:

Yes, comparisons between different studies are fraught with difficulties but based on those data we do have available, rates would seem to be higher in the UK – certainly for Campylobacter than the US. (Note that Salmonella is rare and serious, Campy is common and unpleasant but generally not too serious).

One could argue that methodological inconsistencies mean we need to treat the numbers with caution – but that concern didn’t stop Reality Check making the ‘stretchy’ assertions that have been parroted elsewhere ever since.

In order to settle the matter there would need to be a comparison of rates of foodborne illnesses by country, gathered and analysed using a common methodology, preferably performed by an organisation with a degree of credibility.

Oh, fuck me, it just so happens there is one. The WHO did it. 

This was what was cited by US Ambassador in his defence of US food safety standards. But Chris Morris reckons the 265-page WHO report is not relevant because it gave numbers:

“…only for total foodborne diseases and not the specific pathogens (Salmonella and Campylobacter) relevant to chlorine-washed chicken”

Not true. There is a useful summary of the report in PLoS Medicine which is easier to digest than the rather lengthy UN report. Check out Table 4 and you can see rates for every single foodborne agent including Campylobacter and Salmonella.

There is a flaw; it gives estimates by region rather than country and the AMR 1 region comprises the USA, Canada and Cuba. But given the US comprises >85% of the population of AMR 1 the study has power and comparisons with Western Europe region (EUR A) has meaning. And don’t forget all the countries in EUR A have common food safety and hygiene standards.

North America (and by extension the US) is estimated to have a lower burden of:

So, the position that Campylobacter and Salmonella rates are 4 and 20 times higher in the US than the UK is based on a fatally-flawed interpretation of the available data and I can’t find any original or robust sources that even come close to supporting a similar premise.

Granted, there is no perfect comparison, but the available data are consistent with the US Ambassador’s claim that US has lower rates of foodborne illnesses than Europe.

Morris also asserted that food-related death rates are much higher in the US than the UK and that the CDC estimates 450 deaths from Salmonella every year whereas in the UK fatalities are extremely rare.

Wrong again. As before, he compared two completely different measures: estimates of total deaths in the US with UK deaths where Salmonella was lab confirmed / recorded as the cause of death on the death certificate.

An issue I frequently have with the BBC is it seems (probably due to my confirmation bias) that whenever they report on a subject within my narrow area of dubious expertise they usually get it spectacularly wrong.

But this one is different. The data are readily available, as you can see above. Yes, there’s some nuance – but there always is. But errors like comparing numbers measuring completely different things is so basic and fundamental it’s not one I can readily attribute to an honest mistake: in life there are three things that smell like fish. One of them is fish. I have strong suspicions about this one.

Am I being too critical here? Let me know in the comments. 

A month after the original story was released they changed the Reality Check page removing the most egregious assertions but thanks to the Wayback Machine you can still see the original containing the shonky claims Morris embellished in his piece on Today.

(As an aside they also cherry-pick a study saying chlorine isn’t toally effective – no shit, Sherlock, as any microbiologist will tell you.

“…so they might remain capable of causing disease”

…that’s why you cook the bloody chicken, FFS….)

The newer version is a wishy-washy ‘well, you can’t really make a comparison‘ and a bunch of innuendo that kind of assumes their position – that US food safety standards are lower.

But the damage was already done and the original zombie chlorine chicken story has been reappearing ever since across social media, the press – and even Parliament.

Egregious misrepresentation of the stats to support a preexisting political bias or just crass incompetence? You tell me. I don’t think the BBC will.

Let’s be clear. I’m not making a case for or against chlorine-washed chicken or US vs European food supply chains. My only point is that this is being characterised as a food safety issue and it’s not.

Chlorine? Sounds nasty but pretty much every bagged salad you buy is washed in chlorine. Chlorine is promiscuous and dissipates rapidly. Salads don’t taste of swimming pools, nor does chicken that’s been washed in chlorine (I know, I’ve eaten it loads). In many restaurants they’ll still bung chopped leaves into a sink with a chlorine tablet – I’m not a fan of that or chlorine as a disinfectant (don’t get me started) but it’s common.

There’s also a huge cultural element. In Europe, food standards are from ‘farm to fork’ and encompass stuff like animal welfare too; that’s the EU’s principal argument to prevent cheap US poultry flooding the market. I know US food hygiene law and practice resonably well and it’s more based on ‘provided it’s safe at the point of consumption’ than the all-encompassing EU system. It’s just a different approach.

That said:

In some ways this reminds me of the horsemeat scandal; all the scaremongering headlines and hand-wringing but that was a food labelling issue, not a food safety one. Also the issues above – animal welfare, hormones and environment – are not limited to the USA. Far from it.

I can understand people latching on to food safety fears and how the mythical chlorinated chicken was an easy flag to manipulate people into rallying around. What I still can’t understand is how a food safety scare story clearly unsupported by the data could be propagated by a ‘flagship’ BBC news programme. And it’s not like they were unaware of the data, it was dismissed with a metaphorical wave of the hand by the journalist who was apparently not brought to heel by any editorial process.

As I say, crass incompetence or motivated reasoning / naked bias? You tell me. Neither is a good look…