Exploring XAI methods

[galengorski]

These are some recent results from examining the different model runs using expected gradients. I wanted to start a new discussion because it touches on many of our previous discussions. In particular this builds off of Feature Selection (#170) and Model Comparison (#152).

One question raised in #170 was what features are responsible for the increased model performance in the model with a full set of 30 attributes (26 static, 4 dynamic) compared to the model with a reduced set of 10 attributes (7 static, 3 dynamic). The model with the full set of attributes performed better overall, but as @lekoenig showed, this was largely due to improvement at a single site (01481500 on Brandywine Creek), so that is where this analysis is focused to begin with.

For the next few plots, I have averaged the model responses and expected gradients across 5 model replicates, and in the figures "slim model" and "full model" refer to the reduced full set of attributes, respectively. All plots show results for do_mean.

Here are the model predictions at site 01481500 which show that the improvement largely come in the summer months, when do values are low and in the timing of the rising limb of the do peak in the fall.

[galengorski]

For the slim model, I plotted the time series of predictions with the time series of the dynamic attributes and their associated expected gradients. The panels on the left hand side show:

-DO observations and predictions
-Solar radiation and precipitation observations
-Solar radiation and precipitation expected gradients
-Max temp observations
-Max temp expected gradients
and on the right hand side the expected gradients of static features are shown (the two blank panels are left so the figure has the same structure as the next one for the full model)

Precipitation events produce strong EGs (both positive and negative), while solar radiation produces generally lower magnitude EGs.
Max temp also shows consistent seasonality in EGs, highest magnitude in the winter, lowest in summer.
Several of the static attributes show sharp consistent seasonal increases or decreases in EGs

[galengorski]

The same style plot for the full model shows somewhat different patterns:

Reduced EGs compared to slim model
Strong seasonal pattern for both tmax and tmin (it looks like)
Smoother seasonal patterns for static attributes (positive for Runoff and negative for topographic wetness index)

[galengorski]

Another way to visualize the difference between the models is with an area plot. Here I took the sum of the absolute value of the daily expected gradients, normalized them so that each day had a value of 1, then selected the 9 attributes with highest EGs throughout the course of the modeling time period. So we can think of this as looking at a time series of daily feature importance:

The seasonal cycles show up really prominently here, with dynamic features generally showing greater magnitude EGs during the winter months and static features during the summer month.
The full model relies much less heavily on precipitation with the full model and you can see the topographic wetness index and runoff popping out as important during the summer months.
Also interesting that in the full model, solar radiation doesn't make the the top 9 most important attributes

[galengorski]

Lastly, these figures are a little rough, I took a look at the seasonal and monthly average EG to try to pull out the seasonal signals that are pretty clear from looking at the time series:

Tmin and Tmax are most important except about May-Oct when runoff ratio and watershed area are more important.
These dynamics look pretty site specific, but there are patterns. For most of the sites, the relative important of tmin is decreased during the summer, with some static attribute becoming more important. For all the sites on Brandywine creek, the runoff ratio seems to be among the most important static attributes particularly during the summer.