Assessment of a large number of empirical plant species niche models by elicitation of knowledge from two national experts

Abstract

Quantitative models play an increasing role in exploring the impact of global change on biodiversity but to win credibility and trust they need validating. We show how expert knowledge can be used to evaluate a large number of empirical species niche models constructed for the British vascular plant and bryophyte flora. Key outcomes were; a) scored assessments of each modelled species and niche axis combination, b) guidance on models needing further work, c) exploration of the trade-off between presenting more complex model summaries that could lead to more thorough validation versus the longer time these take to evaluate, d) quantification of the internal consistency of expert opinion based on comparison of assessment scores made on a random subset of models evaluated by both experts. Overall, the experts judged 39% of species and niche axis combinations to be 'poor' and 61% to show a degree of reliability split between 'moderate' (30%), 'good' (25%) and 'excellent' (6%). The two experts agreed in only 43% of cases, reaching greater consensus about poorer models and disagreeing most about models rated as better by either expert. This low agreement rate suggests that a greater number of experts is required to produce reliable assessments and to better understand the reasons underlying these differences of opinion. While AUC statistics showed generally very good ability of the models to predict random hold-out samples of the data, there was no correspondence between these and the scores given by the experts. Crowd-sourcing further assessment from on-line exposure of models to expert volunteers is an obvious next step. This will allow potential users to evaluate more complex yet informative summaries of model fit even though yet these will take longer to assess.