Fuzzy-Pattern Tsetlin Machine for Text Generation

Speed is the most important feature.

Fred Wilson

This repository provides an alternative Fuzzy-Pattern Tsetlin Machine implementation with zero external dependencies and blazingly fast performance. Achieves 37 million MNIST predictions per second at 98% accuracy with 4.7 GB/s throughput on a desktop CPU and demonstrates the first Tsetlin Machine–based text generation example.

• Up to 15× faster training and41× faster inference compared to the original FPTM implementation, achieved through the use of bitwise operations, SIMD instructions, and a specialized memory layout. - Binary classifier.
• Multi-class classifier.
• Single-threaded and multi-threaded training and inference.
• Specialized BitSet index over literals to improve performance on very large, sparse binary vector inputs. - Model compilation to reduce memory usage and increase inference speed.
• Save and load trained models for production deployment or continued training with modified hyperparameters.
• Automatic selection of UInt8

orUInt16

Tsetlin Automata based on the number of TA states. - Automatic switching between binary and multi-class classification depending on the dataset.

• Built-in benchmarking tool.
• Built-in explainability tools.

Talk is cheap, show me thecodesome examples.

Below is an example of character-level text generation in the style of Shakespeare.

First, install the Julia language by running the following command and following the installation instructions:

curl -fsSL https://install.julialang.org | sh

In the first terminal window, run the following command to train the model over multiple epochs:

julia -O3 -t auto examples/TEXT/train.jl

In the second terminal window, run the following command after each training epoch to observe how the quality of the generated text evolves from one epoch to the next:

julia -O3 examples/TEXT/sample.jl

After 400+ epochs, you should see output similar to the following:

ROMEO:
The father's death,
And then I shall be so;
For I have done that was a queen,
That I may be so, my lord.

JULIET:
I would have should be so, for the prince,
And then I shall be so;
For the princely father with the princess,
And then I shall be the virtue of your soul,
Which your son,--

ESCALUS:
What, what should be particular me to death.

BUCKINGHAM:
God save the queen's proclaim'd:
Come, come, the Duke of York.

KING EDWARD IV:
So do I do not know the prince,
And then I shall be so, and such a part.

KING RICHARD III:
Shall I be some confess the state,
Which way the sun the prince's dead;
And then I will be so.

Here is a quick "Hello, World!" example of a typical use case with the Tsetlin Machine.

Importing the necessary functions and the MNIST dataset:

using MLDatasets: MNIST
using .Tsetlin: TMInput, TMClassifier, train!, predict, accuracy, save, load, unzip, booleanize, compile, benchmark

x_train, y_train = unzip([MNIST(:train)...])
x_test, y_test = unzip([MNIST(:test)...])

Booleanizing input data (2 bits per pixel):

x_train = [booleanize(x, 0, 0.5) for x in x_train]
x_test = [booleanize(x, 0, 0.5) for x in x_test]

This implementation introduces some differences compared to the Vanilla Tsetlin Machine:

L

— limits the number of included literals in a clause.LF

— new hyperparameter that sets the number of literal misses allowed per clause.

CLAUSES = 20   # Number of clauses per class
T       = 20   # Voting threshold
S       = 200  # Specificity
L       = 150  # Maximum literals per clause
LF      = 75   # Allowed failed literals per clause

EPOCHS  = 1000 # Number of training epochs

Train the model over 1000 epochs and save the compiled model to disk:

tm = TMClassifier(x_train[1], y_train, CLAUSES, T, S, L, LF, states_num=256, include_limit=240)
train!(tm, x_train, y_train, x_test, y_test, EPOCHS, shuffle=true, index=false)
save(compile(tm), "/tmp/tm_last.tm")

the compiled model and evaluating accuracy:

tm = load("/tmp/tm_last.tm")
accuracy(predict(tm, x_test), y_test) |> println

Benchmarking the compiled model:

benchmark(tm, x_test, y_test, 1000 * 4, warmup=true, index=false)

37 million MNIST predictions per second on a CPU.

To explain a specific input vector or generate a heat map of the distributed literal representation on the MNIST dataset, run the following command:

julia -O3 -t auto examples/MNIST/explain.jl

This repository includes examples for MNIST, Fashion-MNIST, CIFAR-10, AmazonSales, IMDb sentiment analysis, Noisy Parity Problem, and Shakespeare character-level text generation.

Instructions on how to run the examples can be found here.

If you use the Fuzzy-Pattern Tsetlin Machine in a scientific publication, please cite the following paper: arXiv:2508.08350

@article{hnilov2025fptm,
    title={Fuzzy-Pattern Tsetlin Machine}, 
    author={Artem Hnilov},
    journal={arXiv preprint arXiv.2508.08350},
    year={2025},
    eprint={2508.08350},
    archivePrefix={arXiv},
    primaryClass={cs.LG},
    url={https://arxiv.org/abs/2508.08350},
    doi = {10.48550/arXiv.2508.08350},
}