MultistepLIFNode surrogate backpropagation

[ClaasBeger]

Hello,

I am currently using MultiStepLIFNodes in my project and have recently reviewed the documentation of the surrogate gradient method on the readTheDocs page and the suggested Paper. I understand that using a surrogate such as sigmoid enables an easy gradient computation in the backwards pass during optimization, however, I am confused as to which parameters are adjusted using this information? I did not find a set of input weights, so I assume it must be the spiking threshold or the reset value, is that correct? Unfortunately I was unable to find the corresponding implementation, that is why I am asking here. Thank you very much for your help.

[fangwei123456]

I am confused as to which parameters are adjusted using this information?

The amplitude of gradients is determined by the surrogate function. For example, if you use the atan surrogate function, you can set alpha to control the gradient amplitude. Refer to: https://spikingjelly.readthedocs.io/zh_CN/latest/activation_based_en/surrogate.html

did not find a set of input weights, so I assume it must be the spiking threshold or the reset value, is that correct?

The weighted inputs, threshold and the surrogate function will influence the gradients. The reset vaule has not effect in the current time-step, but it may influence the future gradients.

[ClaasBeger]

Thank you very much for your reply. So can I deduce that both the input weights and the threshold can be adjusted by the optimizer using the result of gradient descent? I have not noticed any adjustment to the threshold in my project so far. Also, could you point me towards the implementation of the weighting of the inputs? I have not found it in the MultiStepLIFNode, or the BaseNode , so I assumed the input would be taken in unaltered for computation.

[fangwei123456]

So can I deduce that both the input weights and the threshold can be adjusted by the optimizer using the result of gradient descent?

Yes, you need to set the vthreshold as learnable parameters. Here is an example:

#371 (comment)

Also, could you point me towards the implementation of the weighting of the inputs?

Weights of inputs are weights of layers which are in front of spiking neurons. You can refer to pytorch's api doc for how to modify their weights.

[ClaasBeger]

Great, thank you for your help!