What Prof Hendy Gets Wrong

Professor Shaun Hendy is another prominent adviser to the New Zealand government on covid-19 issues.  Like Professor Baker, he combines frequent commentary via popular media in support of lockdowns with papers written (with numerous co-authors) in the academic style.  However, unlike Professor Baker, he does not seem to have done any prior research in epidemiology (he is a Professor of Physics).  His epidemiological work starts with his first covid paper, which was posted to a website on 25 March 2020:

Table 2 of the paper presents predictions of the death tolls in New Zealand from a range of possible control strategies.  No control yields predicted deaths of 83,000 (1.67% of the population).  Case isolation and quarantining of members of their households reduces this to 62,500 (1.25% of the population).  Adding population-wide social distancing reduces this to 3,000 (0.06% of the population), and adding school and university closures reduces it further to 20.

On page 7, they consider a strategy they describe as “mitigation”, with a predicted death toll of 25,000 (0.508% of the population), and involving a combination of periods of low control (case isolation plus household quarantining) with periods of high control (add population-wide social distancing and school and university closures) as required to keep the number of cases within the capacity of the hospital system.  None of these strategies correspond to mitigation as defined in the 23 March published paper by Professors Baker, Wilson and Blakely (isolation of the over 60s). The most interesting features of the Hendy paper are:

1. The worst case scenario (in which no control measures are instituted) was 83,000 dead (1.67% population mortality rate, as per their Table 2).  By contrast, the worst case death toll (with no control measures) in the many papers of Professors Baker and Wilson (who were the most significant advisers to the government at this time) was 30,600 (in the Baker et al paper of 23 March).  Hendy et al do not even cite this paper, which predates theirs, let alone explain why their worst case figure is almost three times that of Baker et al.  The usual practice in academic work is to cite relevant existing work, and explain why your approach is better.  The need for this is amplified by the fact that none of the Hendy et al co-authors is an epidemiologist, while all co-authors of the Baker et al paper are.

2. The set of control strategies examined did not include lockdown (closing down all but essential businesses as well as all the restrictions described by Hendy), and yet Hendy et al concluded that deaths could be limited to 20 in the highest control state examined by them.  The only places of work that are closed down in any of the control states in Hendy’s Table 2 are schools and universities.  Since it took lockdowns on repeated occasions to achieve New Zealand’s covid death toll to date of 27, Hendy’s belief that this could be achieved without lockdowns would seem to have been far too optimistic. Interestingly, in Baker et al’s paper of 23 March, the authors do not define the restrictions involved in their high control scenario (which they call “eradication”) but the lack of specification of the restrictions at least allows for the possibility that it involved lockdowns.

3. None of the control strategies examined by Hendy et al corresponds to Level 3 or Level 4, despite these levels having been defined by the government on 21 March 2020, which was four days before the Hendy paper was released.  So, by the time the paper was released, it was already superseded by the events of 21 March.

4. The costs of adopting different control strategies are not even mentioned, let alone quantified.  Nor was there any conversion of predicted deaths to life years lost, nor valuation of this in accordance with standard methodology in the medical literature.  Again this contrasts with the Baker et al paper.

The next significant paper by Hendy et al was on 21 October 2020 and was concerned with the economic costs of the Level 3 August 2020 Auckland lockdown relative to those of an alternative Level 4 lockdown:

The paper assumes adoption of the government’s elimination strategy and is only concerned with the question of whether Level 4 restrictions would have been more or less costly (in lost GDP) than the Level 3 restrictions actually adopted in Auckland (Level 4 restrictions cost more per day than Level 3 restrictions but are likely to end sooner).   They find a modest such advantage to Level 4, because the expected time in lockdown to reach their epidemiological target is shorter in Level 4, which more than compensates for the higher costs per day.  This seems to be the first paper from Hendy et al that considers the costs of competing policies, but none of the co-authors appears to have any expertise in economics. The most interesting features of the paper are:

1. Despite considering the costs of these two options, the paper does not accord with the standard methodology in the medical literature of assessing the comparative deaths of the two options and converting this to a cost per QALY saved.

2. The data in their Tables 1 and 2 does not reconcile.  For example, Table 1 states that the cost per day in Level 3 is $57m, Table 2 gives expected days under Level 3 restrictions as 23, implying a cost of $1.3b, but Table 2 gives a cost of $1.8b instead.  The same problem applies to the Level 4 restrictions.  I raised this point with the lead author (Rachel Binny) on 17 November and received a reply from Professor Hendy but he did not address this issue over the course of several emails (in which I reminded him about the point).  I therefore presume that Table 2 is in error.

3. Page 8 of the paper says “Figure 2B shows the economic cost of the outbreak for a particular probability of elimination in the cases where the elimination was successful.”  This is not correct. The Figure is premised on exiting lockdown when cases have fallen to a level at which the probability of elimination has fallen to a particular level, and shows the economic cost for the expected lockdown period for a particular probability of elimination. Whether elimination was subsequently achieved is irrelevant to this calculation.  I also raised this point with the lead author (Rachel Binny) on 17 November and received a reply from Professor Hendy but he did not address this issue over the course of several emails (in which I reminded him about the point).

4. The authors acknowledge that their analysis does not consider the “..longer term economic costs of the measures..” (Executive Summary) and that “These factors may take the analysis to a different conclusion.” (page 10).  What then is the usefulness of the analysis?

5. Despite limiting themselves to the question examined, they note in passing that cost benefit analyses such as those performed by Heatley (2020) and Lally (2020) “…might be useful for informing a mitigation strategy but are not useful for a decision maker considering or following an elimination strategy” (pp. 4-5).  This seems to be accepting that cost-benefit analysis might be appropriate for choosing between mitigation and elimination strategies, as was the focus of Lally (2020), whilst denying its usefulness in choosing between Level 3 and 4 restrictions.  However, even if one has decided on an elimination strategy, for whatever reasons, there are competing variants of it, as Hendy et al recognises in comparing a Level 3 and Level 4 response to the Auckland outbreak, and cost-benefit analysis should also be used to choose between them, as Heatley does and Hendy et al do not.

6. Hendy et al refer again later to the cost-benefit analyses of Heatley and Lally, and state that “Combining our approach….with these more in-depth economic analyses may be useful in informing future responses.” (page 10). This seems to be accepting that cost-benefit analysis may be useful for choosing between Level 3 and 4 restrictions, thereby undercutting the contrary claim quoted in the previous point.

7. The equivocal comments by Hendy et al quoted in the last two points (“may” or “might”) suggest a lack of confidence on the part of the authors about basic economic issues that anyone offering policy advice ought to be confident about. This is understandable in view of none of the authors having any apparent expertise in economics, but it is harder to understand why they would offer policy advice about matters that they are so uncertain about.

Please share this article so that others can discover The BFD.