New libPRADA fork (Jan 2017)

Sebastian Höfer provides a fork of libPRADA with more up-to-date features.

 

libPRADA (until 2012)

libPRADA is a C++-library for model-based relational reinforcement learning in stochastic domains:

• libPRADA provides basic data-structures and methods for symbolic reasoning with relational representations. In particular, it implements probabilistic relational rules which can be used as a transition model P(s'|s,a) in a Markov decision process.
• libPRADA provides algorithms for planning with ground relational rules: PRADA (Probabilistic Relational Action-sampling in Dynamic Bayesian networks planning Algorithm), sparse sampling trees (SST) and UCT.
• libPRADA implements the algorithm for learning probabilistic relational rules by Pasula et al., Learning Symbolic Models of Stochastic Domains, JAIR (2007).

The algorithms implemented in libPRADA are described in detail in my PhD thesis and in the following paper:

Tobias Lang, Marc Toussaint
Planning with Noisy Probabilistic Relational Rules
Journal of Artificial Intelligence Research, 2010, Vol. 39, p. 1-49
PDF    Bibtex

You can test libPRADA with our simulator libARMANI.

Download

Version 1.2 (July 2012): libPRADA.tgz

You'll find a guide in the zip package. I'd be very happy to hear about your experiences, problems, complaints, desires etc. when using libPRADA!

Troubleshooting

I have maintained libPRADA until 2012. Here are some potential issues when continuing to work with it.

18 Nov. 2014 - Fixing compiler problem.
Alexandre Antunes detected and fixed a problem with the latest GNU compilers: "We recently noticed that, probably due to a GNU update, libPRADA no longer compiled. We fixed it by removing the unistd.h include from the minGW, on util.h, and adding it to the standard includes, on the same file. The error appeared on Ubuntu, with GNU 4.8 and 4.7 (we haven't tried any others). It was not loading the unistd.h file, and couldn't read the getcwd and chdir functions on util_t.cxx."

Gradient calculation for learning probabilities of rule outcomes.
In the estimation step of the probabilities of rule outcomes, you might want to replace the gradient descent calculation of the current libPRADA version with the conditional gradient descent algorithm described in the original Pasula et al. (2007) paper. For example, see Mourao (2014). libPRADA's current gradient calculation produces satisfactory results in many domains, including the contexts of my work, but you might want to check that for your own applications.