Feeding My Ignorance

In these days when so many are focussing on "the model" it is throwing up lots of insight into what people understand - and do not understand - about modelling and specifically models used for forecasting. Whilst much of the coverage has been OK both the media and the "experts" (not experts in modelling I should add for clarity) are often guilty of misrepresentation (to put it kindly!)

...

Nothing could illustrate the level of the lack of understanding of the subject better than this quote :

"We talked about it (coronavirus) because he had additional experience of having been much earlier. And he has developed some incredible theories, and all that information is coming over here. A lot of it's already come. We call it data. And we're going to learn a lot from what the Chinese went through."

"we call it data"!!!!! WHAT - everyone (well most people at least) call it data - data is the catchall term for what he is talking about here. It may or may not be the correct term (see the pyramid) but it is most certainly the 'accepted' terminology and, frankly, anyone who feels the need to specify what is called data in this sort of context clearly hasn't got a clue about the subject.

In a similar mis-understanding of models someone else said : (and this is a rather long statement because there are various aspects that are relevant)

I'm sure you have seen the recent report out of the U.K. about them adjusting completely their needs. This is really quite important. If you remember, that was the report that says there would be 500,000 deaths in the U.K. and 2.2 million deaths in the United States. They've adjusted that number in the U.K. to 20,000. Half a million to 20,000. We are looking at that in great detail to understand that adjustment.

I'm going to say something that is a little bit complicated but do it in a way we can understand it together. In the model, either you have to have a large group of people who a-asymptomatic, who never presented for any test to have the kind of numbers predicted. To get to 60 million people infected, you have to have a large group of a-symptomatics. We have not seen an attack rate over 1 in 1,000. So either we are measuring the iceberg and underneath it, are a large group of people. So we are working hard to get the antibody test and figure out who these people are and do they exist. Or we have the transmission completely wrong.

So these are the things we are looking at, because the predictions of the model don't match the reality on the ground in China, South Korea or Italy. We are five times the size of Italy. If we were Italy and did all those divisions, Italy should have close to 400,000 deaths. They are not close to achieving that.

Models are models. We are -- there is enough data of the real experience with the coronavirus on the ground to really make these predictions much more sound. So when people start talking about 20% of a population getting infected, it's very scary, but we don't have data that matches that based on our experience.

And the situation about ventilators. We are reassured in meeting with our colleagues in New York that there are still I.C.U. Beds remaining and still significant -- over 1,000 or 2,000 ventilators that have not been utilized.

Please for the reassurance of people around the world, to wake up this morning and look at people talking about creating DNR situations, Do Not Resuscitate situations for patients, there is no situation in the United States right now that warrants that kind of discussion. You can be thinking about it in the hospital. Certainly, hospitals talk about this on a daily basis, but to say that to the American people and make the implication that when they need a hospital bed it's not going to be there or a ventilator, it's not going to be there, we don't have evidence of that.

It's our job collectively to assure the American people, it's our job to make sure that doesn't happen. You can see the cases are concentrated in highly urban areas and there are other parts of the states that have lots of ventilators and other parts of New York state that don't have any infected. We can meet the needs by being responsive.

There is no model right now -- no reality on the ground where we can see that 60% to 70% of Americans are going to get infected in the next eight to 12 weeks. I want to be clear about that. We are adapting to the reality on the ground and looking at the models of how they can inform but learning from South Korea and Italy and from Spain and I know you will look up my numbers.

I have highlighted the phrases that I want to pull apart a little - but I have left the whole answer as it was given because context is still important.

I'm going to say something that is a little bit complicated but do it in a way we can understand it together

All that is being said is - fundamentally that the model is (as are all models) based on some assumptions and simplifications that mean that it will not fit perfectly with reality. Assuming (and I have no reason not to) that whoever is creating the model is using best practice then the forecasts from the model represent the best that can be done with the existing knowledge and information. This comes back to the "all models are wrong" statement that I referred to in my last post.  This statement is rather demeaning to those listening.

either we are measuring the iceberg and underneath it are a large group of people or we have the transmission completely wrong

or...... I would suggest that these are just two possibilities. Without knowledge of what is included in the model I cannot be sure - but I suspect that these are simply the parameters to the moel that are everyone is least confident about.

Models are models

Oh dear - playing to the crowd - yes, models are models that is a tautology - but what is implied here is that there is a "looking down" on the modellling going on - that somehow there is a belief that there is a better way. That is nonsense. At best it is an example of unfortunate phraseology - at worst it is a demonstration of a complete lack of understanding of the importance of models in any form of forecasting. Yes, we should all understand the limitations and the reasons why the model cannot give "better" answers, but there is no need to be disparaging or to look disdainfully at the modelling effort as is implied by this phrase.

There is no model right now

A second piece of absolute nonsense - because there is ALWAYS a model. This implies that "when we have better 'real' data" from South Korea, Italy, Spain or wherever then the model is no longer required. As I said nonsense -what will change is that the model can be based on less arbitrary assumptions. The models always exist, they get refined as we learn more.

OK - rant over - I could continue to pick lots of holes in the way that the "non-modellers" have characterised the models - thereby showcasing a lack of full understanding - but my experience is that it is true for everyone - to some degree - so this is enough to illustrate things.