added toysgd benchmark

[benjamc]

Please have a look. :-)
Especially on the bounds for the coefficients and the initial x (not sure about that).

[TheEimer]

Is the history length used somewhere I don't see right now?

[benjamc]

oops forgot to delete, not used right now

[TheEimer]

I'll not say you shouldn't or can't do that, but Dict spaces are impractical without the dict wrapper. If you don't care, feel free to ignore

[benjamc]

I just took inspiration from the sgd benchmark. If that's suboptimal, feel free to remodel it :) or I could also do it tomorrow

[TheEimer]

Initial x as 0 makes sense, I think. Without haing every run it, I can't really say if the coefficient range is good or not, but it's super easy to adapt, so I think that's good.
So thanks for the benchmarks, I'll merge as soon as you adressed my comment (second one is optional).

[TheEimer]

Any updates here, @benjamc ? I found another thing to fix in the meantime, the reset doesn't seem to return a state. I missed that, but it's definitely not desired behavior ;D

[TheEimer]

Also, we may want to set an upper limit for values. The lr get huge pretty fast with actions > 1

[TheEimer]

Changes from my side:

• limited actions to stay below 1
• made reset return a state
• initialized lr with 0 (might not be smart at all, but None is worse)

Things where input would be nice:

• how can we initialize the lr without giving the agent a nan in the reset state?
• is 1 a good limit for the actions?

As soon as we settled those two, we can merge

[benjamc]

• [lr] setting the learning rate to 0 as a starter should be fine. this way the agent is definitely forced to do something :D
  and the agent does not set the learning rate itself but the log learning rate . I am not sure if this is really good or if we should set the learning rate to a really small value because the agent can never achieve a learning rate of 0 because of log.
• [actions] do not limit actions