POMDP implementation?

[MHRosenberg]

Hi there, we spoke at Cosyne. Cool work! I've skimmed the repo and am not sure where to begin my hacks or whether it would be too cumbersome. Would it be possible for you to provide some instructions on how to best hack your repo to support partial observability?

Thanks!
Matthew

[ClementineDomine]

Hi there,
Good to hear back from you.

AGENT
I would advise using as an example agent the Stachenfeld SR agent. If you want to implement another RL algorithm, please follow the guideline on adding an agent here (https://github.com/ClementineDomine/NeuralPlayground/tree/main/neuralplayground/agents) ( The Stachenfeld SR model should be a good model for the structure).

ARENA
I would look at the simple 2D arena and create a class that changes the make_observation function. For now, the observation is the full state of the environment. It should be easy for you to modify it to be partially observable. Consider creating a new arena class following the guideline here (https://github.com/ClementineDomine/NeuralPlayground/tree/main/neuralplayground/arenas), basically inheriting from Simple2D class

INTERACTION
To create an interaction, follow the examples shown in the examples (https://github.com/ClementineDomine/NeuralPlayground/blob/main/examples/agent_examples/stachenfeld_2018_examples.ipynb) with your newly created arena.

I hope it helps. Thanks!
Clem and Ro