This page contains useful scripts for running experiments.

Markov Logic Network Evaluation Package

The evaluation package is to help evaluate the structure and parameters of MLN. We divide up the data into training and testing data. Use our database package to produce your training and testing data. The MLN evaluation package takes in a [foo.db] and a [bar.mln] file (the Alchemy format) and outputs the average accuracy, conditional log-likelihood, over all predicates in the dataset, and Area under Curve for binary predicates.

Installation

System Requirements

  • The system runs under Unix.
  • You need Java version 1.6 or greater
  • Compiled executables of Alchemy. For Alchemy's system requirements please see [here]. (Note: Installing Alchemy is often the hardest part.)
In order to use the package, please follow these steps.

  1. Download the package from [here].

  2. Add compiled executables (learnst, learnwts, infer) from Alchemy [here] to the Alchemy folder.
  3. Place your [foo.db] and [bar.mln] in the root directory of the package.
  4. Change to the root directory of the package and run " ./analyze.ksh foo.db bar.mln" in a Linux terminal. A folder called analysis is created in which the result for each predicate is stored in a file. The average accuracy, conditional log-likelihood, and Area under curve over all predicates in the dataset is printed in the terminal.
  5. In case there are errors you can run the package in debug mode in which a folder called output is created where you can see the log of inference over each predicate. To run in debug mode type " ./analyze.ksh *.db *.mln debug" in a Linux terminal.

Database Package

The purpose of the database package is to help create fair training and testing datasets. The inputs to the database package are 1) an SQL dump foo of a database and 2) an integer N that defines the percentage of data to be used by training as input. The package outputs the following:

  • foo_training database in your MYSQL which has N% of the number of entities of the actual SQL dump.
  • foo_testing database in your MYSQL which has the other (100-N)% of entities of the actual SQL dump.
  • foo_training.db file which corresponds to foo_training in MYSQL.
  • foo_testing.db file which corresponds to foo_testing in MYSQL.

System Requirements

  • The system runs under Unix.
  • You need Java version 1.6 or greater
  • An installation of MySQL. The command "mysql" should work from the command line.

Installation

In order to use the package, please follow these steps.

  1. Download the package from [here].
  2. Our system is designed for databases whose schema follows the Entity-Relationship model. The database is expected to consist of entity and relationship tables. The primary key of the entities should be set and the name primary key should end with "_id". The foreign keys in the relationship tables should have the same name as the primary keys and foreign key constraint should be set. The foreign keys should cascade on delete.
  3. Change directory into the directory where you extract the package. Place the sql dump foo.sql for your dataset into root/data.
  4. Modify the config.xml file to replace your connection setting to MYSQL.
  5. Run "java -jar database_package.jar [schema name][N]" on your Linux terminal where [schema name] corresponds to the name of the foo.sql file placed in root/data and N is the percentage of data that you want to use for your training set.