Biodiversity Data Use

Wallace provides a fairly broad suite of evaluation metrics to use in determining which model to utilize. For our purposes, we will use AICc. Typically, the model with the lowest AICc score (or a delta AICc of 0) is considered to be the best model (balancing goodness-of-fit with simplicity). But, omission rate is also a common and effective method of evaluating binary predictions, so we will look at these as well.

a) Look at the “Full model and partition bin average evaluation statistics” table in the Results section (the top table).

b) Sort the AICc scores lowest to highest. Which model has the lowest AICc score? The name of the model tells you what the parameter settings are. RM = randomization multiplier, FC = feature class.

c) Now look at the “Individual partition bin evaluation statistics” table (the bottom results table). You’ll see that data have been evaluated using binning based on two threshold levels: the 10 percentile training (or.10p) and the minimum presence training thresholds (or.MTP). Which model has the lowest omission rate?

d) Based on this information, choose the model you think is the best fit. This will likely be a compromise—one model that outperforms the others on all evaluation metrics is quite rare. Use your best judgement, and ask for help if you’re stuck.