No, not that sort of super model, the super computer model that was the basis for the lockdown which trashed the economy – that model.
Apparently the decisions in New Zealand were based on models by a group called Te Punaha Matatini, a “Centre of Research Excellence” at Auckland University. Their paper is available online and the introduction includes:
Our model is parameterised for the spread of COVID-19 through the New Zealand population (Wilson, 2020) with intervention strategies calibrated from a recent study of COVID-19 spread through the US and UK (Fergusson, 2020). Using the model we examine a range of possible interventions and their effect on the healthcare system and population fatality rate
So the Ferguson model was used to check the calibration of the New Zealand model.
And what is Mr Ferguson’s track record for model predictions?
- In 2002 he predicted 50,000 people would die of BSE. Actual number: 178 (national CJD research and survellance team)
- In 2005 he predicted 200 million people would die of avian flu H5N1. Actual number according to the WHO: 78
- In 2009 he predicted that swine flu H1N1 would kill 65,000 people. Actual number 457.
Hmmm, looks legit.
Now a computer coding expert, Sue Denim, (Psue Donym) has had a look at the Ferguson model computer code and, to put it politely, is not overly impressed.
The writer appears competent to comment.
My background. I wrote software for 30 years. I worked at Google between 2006 and 2014, where I was a senior software engineer working on Maps, Gmail and account security. I spent the last five years at a US/UK firm where I designed the company’s database product, amongst other jobs and projects.
Microsoft programmers have been attempting to clean up the model code.
The code. It isn’t the code Ferguson ran to produce his famous Report 9. What’s been released on GitHub is a heavily modified derivative of it, after having been upgraded for over a month by a team from Microsoft and others. This revised codebase is split into multiple files for legibility and written in C++, whereas the original program was “a single 15,000 line file that had been worked on for a decade” (this is considered extremely poor practice). A request for the original code was made 8 days ago but ignored, and it will probably take some kind of legal compulsion to make them release it. Clearly, Imperial are too embarrassed by the state of it ever to release it of their own free will, which is unacceptable given that it was paid for by the taxpayer and belongs to them.
The writer discusses many short-comings of the Ferguson virus computer model code, such as this one.
Imperial advised Edinburgh that the problem goes away if you run the model in single-threaded mode, like they do. This means they suggest using only a single CPU core rather than the many cores that any video game would successfully use. For a simulation of a country, using only a single CPU core is obviously a dire problem – as far from supercomputing as you can get. Nonetheless, that’s how Imperial use the code: they know it breaks when they try to run it faster. It’s clear from reading the code that in 2014 Imperial tried to make the code use multiple CPUs to speed it up, but never made it work reliably. This sort of programming is known to be difficult and usually requires senior, experienced engineers to get good results. Results that randomly change from run to run are a common consequence of thread-safety bugs. More colloquially, these are known as “Heisenbugs“.
And it goes on and on.
Imperial are trying to have their cake and eat it. Reports of random results are dismissed with responses like “that’s not a problem, just run it a lot of times and take the average”, but at the same time, they’re fixing such bugs when they find them. They know their code can’t withstand scrutiny, so they hid it until professionals had a chance to fix it, but the damage from over a decade of amateur hobby programming is so extensive that even Microsoft were unable to make it run right.
It really is not a flattering report and the conclusion is:
All papers based on this code should be retracted immediately. Imperial’s modelling efforts should be reset with a new team that isn’t under Professor Ferguson, and which has a commitment to replicable results with published code from day one.
All papers? Even the one that was used by our kind, open transparent government?
Oh dear …. could it be possible that 80,000 Kiwis were not going to die?
If you enjoyed this BFD article please consider sharing it with your friends.