Tests reveal major variation in flood risk models

The first ever comparison of six of the major global flood risk monitoring computer models has revealed wide discrepancies between the information they provide.

Insurers, risk analysts and many other businesses, as well as governments and NGOs use software like these models to understand which areas of the world are most at risk of flooding.

Now, an objective academic review of the six models has found a 60-70 per cent discrepancy between the areas each model said were at risk. The discovery was made following tests of the models to forecast flood hazard in Africa.

Dr Mark Trigg, a Chartered Engineer at the University of Leeds from the School of Civil Engineering led the study, which was initiated during the 2014 Global Flood Partnership conference.

The study secured permission from all six model developers to compare their results in order to better improve the service they provide.

Dr Trigg has now called for future comparisons to include commercial flood risk prediction models used by the insurance industry.

Recognising that there were differences between the models was an expected part of the development process and was not a sign of failure, Dr Trigg said. He explained it showed the developers how they could improve their models.

His research team’s results, published in the journal IOPScience, mean users and the developers themselves understand more about the variance that can exist in such tools.

Dr Trigg said: “Assessing global flood risk is hugely important for individual communities right up to governments and worldwide insurers, but we’ve found the tools used to do the job have significant variations.

“In several places the areas which these software tools predict will flood varies by many miles.”

The developers all agreed to their software’s output being reviewed by the researchers in order to improve their services – global level flood risk mapping at this stage is still a ‘young’ science.

This type of continent-wide prediction was hard to envisage even 10 years ago, but advances in algorithms for the simulation of flood flow physics means it is now possible to model risk in sufficient detail to be worthwhile.

Even now though, the researchers say, while there is some skill in the models in predicting floods at a large river level, the models have had ‘limited validation against observed flood flows and extents’.

The models were better at predicting flooding in Europe, the researchers say, because more reliable hydrological and terrain data is available.

The researchers explain how the outputs of these models and others like them are ‘vital for consistent qualification of global flood risk, and in projecting the impacts of climate change’.

But the paper adds: “However, the urgency of these tasks means that outputs are being used as soon as they are made available and before such methods have been adequately tested.”

The paper also states: “While there is encouraging agreement between the models in some areas, there are enough differences between the models in most areas that any flood risk conclusions resulting from identical analysis using different models will lead to very different implications.”

Dr Trigg says the key message for all developers, and users, is that while individual models have a level of accuracy, further development and testing is required before they can be completely relied upon.

“We can’t rush into this technology,” he says. “It is vital for areas of the world where this data is not currently available, but it is still developing and accuracy is still a concern. Many developers and users of the model results are aware of this challenge, and this research provides an important step in helping them understand where they work well, and where they don’t.”

The full open-access research paper, The credibility challenge for global fluvial flood risk analysis, has been published in IOP Science. The research was funded by the Willis Research Network and the University of Leeds.

A teaching version of the paper for children and teens has also been published by the Environmental Science Journal for Kids (and their teachers), How much do we really know about river flooding?

Further information

The above image shows a map of Africa highlighting the different flood areas described by the software models tested. 

Journalists requiring more details or interviews with Dr Trigg should contact Peter Le Riche in the University of Leeds press office, on 0113 343 2049 or email p.leriche@leeds.ac.uk

Africa was chosen as being large enough to provide a meaningful test, and the most lacking in existing flood hazard information for global planning meaning more reliance will be placed on these models in these locations.