This supplementary material contains an implementation of the intersection and incremental infix methods.

We have also included the Brown Corpus, which we used to generate the n-grams in our experiments.

This code requires Python3, NumPy, and NLTK. These can all be downloaded using pip. 

To see a test run, you can simply run:

    python3 incremental.py
or
    python3 intersection.py

A valid run should look like the following:

    >python incremental.py 

    Number of states in P: 718
    Infix probability of ['NN']:  0.735924391092
    Infix probability of ['NN']:  0.735924391092
    Infix probability of ['NN', 'NNP']:  0.0111170294813
    Infix probability of ['NN', 'NNP']:  0.0111170294813
    Infix probability of ['NN', 'NNP', 'NN']:  0.000215855482977
    Infix probability of ['NN', 'NNP', 'NN']:  0.000215855482977
    Infix probability of ['NN', 'NNP', 'NN', 'PUNC']:  0.000106891828828
    Infix probability of ['NN', 'NNP', 'NN', 'PUNC']:  0.000106891828828
    Iterative timings [0.17996513843536377, 0.2204982042312622, 0.26677119731903076, 0.31684410572052]
    Iterative total time:  0.9840786457061768

The parameters for the runs can be changed in the intersection.py and incremental.py files. The incremental and infix methods can be used with any supplied PFA and valid infix.

For example:

>python3 

>>> import incremental, intersection
>>> P = generate_PFA() #This is some method to generate PFA
>>> w = ["a","b"]  #A word over the same alphabet as P
>>> incremental.incremental_test(P, w, trials=1)
Infix Probability of ['a']:  0.5
Infix Probability of ['a', 'b']:  0.015


We have provided code for making a PFA in common.py in the class common.WA. All that is required is to supply the correct parameter types and a PFA object will be returned.
The documentation for the WA class is provided in common.py.
