We have made the following software available to download under
various open-source licenses:
The Pinball Domain
Pinball domain is
a fairly challenging 4-dimensional continuous and dynamic reinforcement
domain. The goal is to maneuver the blue ball into the red hole, while
avoiding (or using, since the ball is dynamic and collisions are
elastic) the obstacles. The dynamics of the ball and the presence of
obstacles result in a domain with sharp discontinuities, and the
location and shape of the obstacles can be specified so you can make
the domain as hard or as easy as you want.
The Pinball domain page contains a brief overview
and Java source code, full documentation,
RL-Glue interface, and GUI
programs for editing obstacle
configurations, viewing saved trajectories, etc.
The Pinball domain was originally described in:
The Fourier Basis
The Fourier basis is a simple,
principled basis function scheme for linear value function approximation in reinforcement learning. It has performed well over
a wide range of problems, despite (or perhaps because of) its simplicity.
Please see our Fourier Basis page
for a concise overview.
ε-greedy Sarsa(λ) Fourier Basis code
to the RL-Library project.
You can get the code, installation instructions, and a project overview at
RL Library community wiki.
The code is in Java and is fully documented. (Apache License)
We have also made available a (GPLv3 licensed) Python implementation of the Fourier basis, along with code for ε-greedy Sarsa(λ) and two
RLGym examples, here: github.com/brownirl/fourier_vfa.
The Fourier Basis is described in: