2013 Reinforcement Learning Competition and ICML Workshop
20-21 June 2013 Atlanta, USA
https://sites.google.com/site/rlcomp2013/icml_workshop

Competition goal
After a four year hiatus, the reinforcement learning competition is back. The primary aim of the competition is to test both general and domain-specific reinforcement learning algorithms, using an unbiased and transparent methodology. As a side-effect, the competition will generate a set of benchmarks domains and benchmark results in those domains, which can then be used as a basis of comparison in future work.
In the reinforcement learning competition, researchers can test their algorithms and insights in a friendly competitive way, on new and challenging domains. The reinforcement learning competition has not been organized for a while, but will take place again this year after a long hiatus. The primary aim of the competition is to test both general and domain-specific reinforcement learning algorithms, using an unbiased and transparent methodology. The domains used in the competition will form a set of benchmarks, and the results of the submitted algorithms will form a body of benchmark results, which researchers can then use as a basis of comparison in future work.
A secondary aim is to discuss methodological approaches for comparing reinforcement learning algorithms. This remains an issue in reinforcement learning in general.

Another important aim is to ensure the continuing existence of the competition, and the prevention of further hiatuses. Our aim is to ensure that the competition will organized annually again, from this year onward. To this extend we will try to build up an organization of interested researchers.
As in the previous years, the competition will be hosted at
http://www.rl-competition.org/
The format will be remain as is. Agents may compete in one or more of a set of known domains. This set will include a polyathlon event, where the agents compete in a sequence of arbitrary environments.
We envisage the actual competition to take place in June 2013.

ICML Workshop on the Reinforcement Learning Competition 2013 (WRLCOMP)
The competition is associated with an ICML Workshop in conjunction with ICML 2013. The ICML reinforcement learning competition workshop will bring together researchers, who participated in the competition, in order to present and discuss their results. We will evaluate what works, and also under what conditions established methods may not work so well. In this way we hope to broaden our insight into state-of-the-art RL algorithms, and important properties of RL problems.

The workshop will be organized along the lines of the domains. One time slot will be reserved for each competition category winner. In addition, all competition entrants will be invited to submit a short paper describing their approach.

We will reserve a special time slot for papers on methodological problems that arise when comparing reinforcement learning algorithms.
Workshop website: https://sites.google.com/site/rlcomp2013/icml_workshop
Topics
1. For entrants
All entrants are additionally invited to submit a paper describing the approach used in the competition, including findings and ideas that relate to more general research questions.
2. For everyone
You are invited to submit a paper discussing methodological issue when comparing reinforcement learning algorithms, including but not limited to the following topics:
Effects of PRNGs in large-scale comparisons
Experimental design for an unbiased evaluation
Metrics for performance evaluation
Reproducibility of results
Robustness of algorithms
Statistical testing standards for reinforcement learning

Submission

Interested authors should format their papers according to ICML formatting guidelines, available here: http://icml.cc/2013/wp-content/uploads/2012/12/icml2013stylefiles.tar.gz
Papers should not exceed 4 pages and are due by May 15, 2013. All papers must be submitted as PDF, and be made through Easychair at https://www.easychair.org/conferences/?conf=wrlcomp2013
Important Dates

Domains available: April 1
Testing starts: May 1
Competition: June 1
Paper Submission Deadline: May 15, 2013
Author Notification: June 1, 2013
Workshop: June 20-21, 2013