Intelligent Robot Lab
Brown University, Providence RI


We have made the following software available to download under various open-source licenses:

The Pinball Domain (GPL)

The Pinball domain is a fairly challenging 4-dimensional continuous and dynamic reinforcement learning domain. The goal is to maneuver the blue ball into the red hole, while avoiding (or using, since the ball is dynamic and collisions are elastic) the obstacles. The dynamics of the ball and the presence of obstacles result in a domain with sharp discontinuities, and the location and shape of the obstacles can be specified so you can make the domain as hard or as easy as you want.

The Pinball domain page contains a brief overview and Java source code, full documentation, an RL-Glue interface, and GUI programs for editing obstacle configurations, viewing saved trajectories, etc.

Other versions:

The Pinball domain was originally described in:

The Fourier Basis

The Fourier basis is a simple, principled basis function scheme for linear value function approximation in reinforcement learning. It has performed well over a wide range of problems, despite (or perhaps because of) its simplicity. Please see our Fourier Basis page for a concise overview.

We contributed RL-Glue-compatible ε-greedy Sarsa(λ) Fourier Basis code to the RL-Library project. You can get the code, installation instructions, and a project overview at the RL Library community wiki. The code is in Java and is fully documented. (Apache License)

We have also made available a (GPLv3 licensed) Python implementation of the Fourier basis, along with code for ε-greedy Sarsa(λ) and two RLGym examples, here:

Other versions:

The Fourier Basis is described in: