{"slug": "fuzzy-pattern-tsetlin-machine-for-text-generation", "title": "Fuzzy-Pattern Tsetlin Machine for Text Generation", "summary": "A new implementation of the Fuzzy-Pattern Tsetlin Machine achieves 37 million MNIST predictions per second at 98% accuracy with 4.7 GB/s throughput on a desktop CPU, and demonstrates the first Tsetlin Machine–based text generation example. The implementation, developed by BooBSD, offers up to 15× faster training and 41× faster inference compared to the original FPTM implementation through bitwise operations, SIMD instructions, and a specialized memory layout.", "body_md": "Speed is the most important feature.\n\n*Fred Wilson*\n\nThis repository provides an alternative [Fuzzy-Pattern Tsetlin Machine](https://github.com/BooBSD/FuzzyPatternTM) implementation with zero external dependencies and blazingly fast performance.\nAchieves **37 million** MNIST predictions per second at 98% accuracy with **4.7 GB/s** throughput on a desktop CPU and demonstrates the first Tsetlin Machine–based **text generation** example.\n\n- Up to\n**15× faster training** and**41× faster inference** compared to the original FPTM implementation, achieved through the use of bitwise operations, SIMD instructions, and a specialized memory layout. - Binary classifier.\n- Multi-class classifier.\n- Single-threaded and multi-threaded training and inference.\n- Specialized\n**BitSet index** over literals to improve performance on very large, sparse binary vector inputs. - Model compilation to reduce memory usage and increase inference speed.\n- Save and load trained models for production deployment or continued training with modified hyperparameters.\n- Automatic selection of\n`UInt8`\n\nor`UInt16`\n\nTsetlin Automata based on the number of TA states. - Automatic switching between binary and multi-class classification depending on the dataset.\n- Built-in benchmarking tool.\n- Built-in explainability tools.\n\nTalk is cheap, show me the~~code~~some examples.\n\nBelow is an example of character-level **text generation** in the style of Shakespeare.\n\nFirst, install the [Julia language](https://julialang.org) by running the following command and following the installation instructions:\n\n```\ncurl -fsSL https://install.julialang.org | sh\n```\n\nIn the **first terminal window**, run the following command to train the model over multiple epochs:\n\n```\njulia -O3 -t auto examples/TEXT/train.jl\n```\n\nIn the **second terminal window**, run the following command after each training epoch to observe how the quality of the generated text evolves from one epoch to the next:\n\n```\njulia -O3 examples/TEXT/sample.jl\n```\n\nAfter **400+** epochs, you should see output similar to the following:\n\n```\nROMEO:\nThe father's death,\nAnd then I shall be so;\nFor I have done that was a queen,\nThat I may be so, my lord.\n\nJULIET:\nI would have should be so, for the prince,\nAnd then I shall be so;\nFor the princely father with the princess,\nAnd then I shall be the virtue of your soul,\nWhich your son,--\n\nESCALUS:\nWhat, what should be particular me to death.\n\nBUCKINGHAM:\nGod save the queen's proclaim'd:\nCome, come, the Duke of York.\n\nKING EDWARD IV:\nSo do I do not know the prince,\nAnd then I shall be so, and such a part.\n\nKING RICHARD III:\nShall I be some confess the state,\nWhich way the sun the prince's dead;\nAnd then I will be so.\n```\n\nHere is a quick *\"Hello, World!\"* example of a typical use case with the Tsetlin Machine.\n\nImporting the necessary functions and the MNIST dataset:\n\n```\nusing MLDatasets: MNIST\nusing .Tsetlin: TMInput, TMClassifier, train!, predict, accuracy, save, load, unzip, booleanize, compile, benchmark\n\nx_train, y_train = unzip([MNIST(:train)...])\nx_test, y_test = unzip([MNIST(:test)...])\n```\n\nBooleanizing input data (2 bits per pixel):\n\n```\nx_train = [booleanize(x, 0, 0.5) for x in x_train]\nx_test = [booleanize(x, 0, 0.5) for x in x_test]\n```\n\nThis implementation introduces some differences compared to the *Vanilla Tsetlin Machine*:\n\n`L`\n\n— limits the number of included literals in a clause.`LF`\n\n— new hyperparameter that sets the number of literal misses allowed per clause.\n\n```\nCLAUSES = 20   # Number of clauses per class\nT       = 20   # Voting threshold\nS       = 200  # Specificity\nL       = 150  # Maximum literals per clause\nLF      = 75   # Allowed failed literals per clause\n\nEPOCHS  = 1000 # Number of training epochs\n```\n\nTrain the model over 1000 epochs and save the compiled model to disk:\n\n```\ntm = TMClassifier(x_train[1], y_train, CLAUSES, T, S, L, LF, states_num=256, include_limit=240)\ntrain!(tm, x_train, y_train, x_test, y_test, EPOCHS, shuffle=true, index=false)\nsave(compile(tm), \"/tmp/tm_last.tm\")\n```\n\nLoading the compiled model and evaluating accuracy:\n\n```\ntm = load(\"/tmp/tm_last.tm\")\naccuracy(predict(tm, x_test), y_test) |> println\n```\n\nBenchmarking the compiled model:\n\n```\nbenchmark(tm, x_test, y_test, 1000 * 4, warmup=true, index=false)\n```\n\n37 million MNIST predictions per second on a CPU.\n\nTo explain a specific input vector or generate a heat map of the distributed literal representation on the MNIST dataset, run the following command:\n\n```\njulia -O3 -t auto examples/MNIST/explain.jl\n```\n\nThis repository includes examples for *MNIST*, *Fashion-MNIST*, *CIFAR-10*, *AmazonSales*, *IMDb* sentiment analysis, *Noisy Parity Problem*, and *Shakespeare* character-level **text generation**.\n\nInstructions on how to run the examples can be found [here](https://github.com/BooBSD/Tsetlin.jl/tree/main/examples).\n\nIf you use the Fuzzy-Pattern Tsetlin Machine in a scientific publication, please cite the following paper: [arXiv:2508.08350](https://arxiv.org/abs/2508.08350)\n\n```\n@article{hnilov2025fptm,\n    title={Fuzzy-Pattern Tsetlin Machine}, \n    author={Artem Hnilov},\n    journal={arXiv preprint arXiv.2508.08350},\n    year={2025},\n    eprint={2508.08350},\n    archivePrefix={arXiv},\n    primaryClass={cs.LG},\n    url={https://arxiv.org/abs/2508.08350},\n    doi = {10.48550/arXiv.2508.08350},\n}\n```\n\n", "url": "https://wpnews.pro/news/fuzzy-pattern-tsetlin-machine-for-text-generation", "canonical_source": "https://github.com/BooBSD/Tsetlin.jl", "published_at": "2026-07-21 09:48:58+00:00", "updated_at": "2026-07-21 10:23:06.335491+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence", "neural-networks"], "entities": ["BooBSD", "Fuzzy-Pattern Tsetlin Machine", "MNIST", "Julia"], "alternates": {"html": "https://wpnews.pro/news/fuzzy-pattern-tsetlin-machine-for-text-generation", "markdown": "https://wpnews.pro/news/fuzzy-pattern-tsetlin-machine-for-text-generation.md", "text": "https://wpnews.pro/news/fuzzy-pattern-tsetlin-machine-for-text-generation.txt", "jsonld": "https://wpnews.pro/news/fuzzy-pattern-tsetlin-machine-for-text-generation.jsonld"}}