Rethink statistical tests

[lauraporta]

          This is a nice function I like how it all come neatly together to grab the data for the test. In this instance I don't think Kruskall-Wallis is a correct test but I'm not sure we need to worry too much at this stage as is port of their analysis, and it seems common in the field (also, maybe I am misunderstanding something).

The reason I think not the right test is because Kruskal-Wallis assumes independent samples within and between groups, however in this case all data are from a single neuron (e.g. 24 x 36 matrix of stim repeat vs. combination type). A Friedman test might work better as at least that assumes repeated measures across groups, but this would still violate independence within groups. A mixed-effect model might be an option, or something like fMRI analysis in which voxel response to stimuli is modelled as a regression between the signal and predictors that indicate the timing of the different stimuli (but, there are other issues with this approach for this use case). Here are a couple of papers 1 2 I have not read but look interesting on the topic for reference, maybe these could be a book club some time. If Margrie lab are interested, maybe this could be something to talk to Gatsby about (e.g. Joaquin) who might be able to suggest a different analysis. That been said, it's possible I've completely misunderstood something and it is a completely appropriate analysis 😅

Originally posted by @JoeZiminski in #25 (comment)

[lauraporta]

One additional reason to rethink about them is that in the unit tests, when using mock data obtained with seed 101, I am obtaining one significant ROI
Check this commit

[lauraporta]

One tailed Signed Test can be useful to identify negative responses (suppression of activity) that can be relevant in layer 6