Ok thanks for the clarification, that’s a pretty standard setup for reinforcement learning type experiments but we don’t currently support everything you would need - but we should in the next month or so.
network is given 200ms
As noted above we currently hardcode the timestep to 20ms for simulator runs / transfer functions, but this is something you can easily modify in your local installation for now and something that we should support in the future (it’s noted in our backlog of tasks).
After the position of the robot is updated we want to reset the network to evaluate the new position.
The frontend does have a reset option for just the neural network, which I believe would fully reload the network from source. So theoretically you could modify the brain file in between reset events and that should let you update weights/etc in the brain file… but let’s investigate that further in the future.
We’re working on development of the “Virtual Coach” which is a command line interface that would probably be most useful in your case, but it doesn’t currently support reset (though that should be through in the next weeks). You could start/stop experiments and change your parameters and stuff, but that would add some seconds of overhead. The documentation is currently pretty non-existent since development started recently. but you can take a look at the Jupyter notebook example of a parameter search type experiment in the VirtualCoach repo (git clone ssh://@bbpcode.epfl.ch/neurorobotics/VirtualCoach) and look under the example directory.
It’s pretty much going to be ideal for your use case since you are free to subscribe to normal ROS topics and do event based things based on simulation time / etc. without having to do them manually.
What’s your long term/short term deadline for integration with the platform? We can work with you to get setup and the reset functionality for the Virtual Coach should be in place in the next couple of weeks. If you have a bit of time to delay we can probably work with you to get everything setup and running properly as needed, we should be able to run your experiment setup without too many hacks.
We’re currently working on supporting some other experiments with May/June demo/review deadlines so they have pretty hard priority, but I can ping @vonarnim who controls the planning/priorities (he’s out of the office until next week, but we can discuss then).