LSTM-metab performance

[jsadler2]

Starting this so we have a place post updates about the LSTM-metab performance and have discussion about those.

[jsadler2]

I've been doing some trouble-shooting with the model and have found some interesting things and I have a few new things I want to try. Note that all of the plots below are for the training time period. I'm plotting this b/c I just wanted to see if the model is at least fitting that.

Baseline

Here are the baseline predictions for comparison. As a reminder, these are predictions from the model that is just predicting DO mean/min/max

1 - scaled vs. unscaled

One of the major issues with the LSTM-metab results that we first saw was that the y-data (so the metabolism and the DO data) were all being scaled and centered. Because of this, the equations to get from metabolism back to DO, were, of course not work.

predictions with scaled data:

predictions with unscaled data:

Takeaways:

• The big difference is in the DO min and DO max ranges. They are way off in the scaled and near the average in the unscaled.
• In both, the model is not fitting well at all to metabolism data. I think that this may be because we are just asking the model to do to much relative to the flexibility we are providing.
• DO mean is the only decent prediction. I'm 95% sure that that is because we let the model tune weights that are specific to that output. Because of that, the model is able to fit that output even though the rest of the outputs are garbage.

Trying with just metab

I wanted to test the "we are just asking the model to do to much relative to the flexibility we are providing" hypothesis. So I ran two different versions of the LSTM-metab. In #88, I set the loss weights for the DO outputs to 0. That way the model was only trying to minimize error in the metabolism predictions. In #87, I did the opposite: I set the metabolism errors to zero. In effect, I asked the model to only try to reduce error in the DO predictions and not to worry at all about the metabolism errors. Here are those results:

just solving for metabolism

just solving for DO (but going through our metabolism model)

Takeaways:

• The model can do a pretty good job fitting to metabolism when just looking to minimize error in the metabolism outputs
• Our "process-based" model is doing quite well. With good metabolism estimates, we can get back to pretty good DO. This is encouraging to me.
• When just trying to focus on DO, the model does terribly again. I was pretty surprised by this. I was thinking that the model would just go crazy with the metabolism outputs, but adjust them so that DO would be reasonable. Not the case.