{"slug": "multilingual-text-classification-with-scikit-llm-and-multilingual-embeddings", "title": "Multilingual Text Classification with Scikit-LLM and Multilingual Embeddings", "summary": "A tutorial published on Scikit-LLM shows developers how to build a multilingual text classification pipeline using multilingual LLM embeddings and scikit-learn without training a separate model per language. The walkthrough configures a free, local embedding pipeline with Ollama and the open-source BGE-M3 model, then trains and evaluates a logistic regression classifier on top of those embeddings using a real-world review dataset. The setup installs scikit-llm and datasets version 2.19.1, points Scikit-LLM at a local Ollama server via SKLLMConfig.set_gpt_url(\"http://localhost:11434/v1/\"), and uses a dummy API key to avoid paid services such as OpenAI.", "body_md": "In this article, you will learn how to build a multilingual text classification pipeline using multilingual large language model (LLM) embeddings and Scikit-learn, without training separate models for each language.\n\nTopics we will cover include:\n\n- What multilingual LLM embeddings are and why they eliminate the need for language-specific models.\n- How to set up a free, local embedding pipeline using Ollama, BGE-M3, and Scikit-LLM.\n- How to train and evaluate a logistic regression classifier on top of multilingual embeddings using a real-world review dataset.\n\n## Introduction\n\nBuilding **machine learning models for a global audience**, such as text classifiers based on **multilingual data**, traditionally required training a separate model for each language. Thus, the process could easily become unmanageable. Thankfully, progress in LLMs also extends to scenarios like this! Multilingual LLM embeddings are numerical representations of text produced by a model that maps text from different languages into a common vector space. With these “barrier-free” embeddings, all it takes thereafter is training a downstream, lightweight classifier on top of them. Let’s uncover how to do this step-by-step, aided by Scikit-LLM.\n\n## Initial Setup\n\nIn the sequel, we will construct a multilingual text classification pipeline aided by Scikit-LLM and scikit-learn.\n\n```\n# Installing Python dependencies\npip install scikit-llm \"datasets==2.19.1\" -q\n\n# Fix Colab's missing system dependencies first (version-dependent, use with care in other environments)\napt-get update -qq && apt-get install -y -qq zstd\n\n# Installing Ollama distribution\ncurl -fsSL https://ollama.com/install.sh | sh\n\n12345678\n\n# Installing Python dependenciespip install scikit-llm \"datasets==2.19.1\" -q # Fix Colab's missing system dependencies first (version-dependent, use with care in other environments)apt-get update -qq && apt-get install -y -qq zstd # Installing Ollama distributioncurl -fsSL https://ollama.com/install.sh | sh\n```\n\nEnsuring a 100% free and runnable solution in a variety of running environments, including notebooks, requires bypassing paid APIs like OpenAI. That’s why, instead, we have installed an Ollama distribution offering a variety of free LLMs. Accordingly, in the next steps we will configure Scikit-LLM to speak to a local Ollama server running `BGE-M3`, which is a state-of-the-art, open-source model supporting multilingual information in the embedding generation process.\n\nNext, we start the Ollama server as a background process —this is the most hassle-free way to use Ollama in a cloud-based notebook, but not mandatory if working with your own IDE and local Ollama distribution. We also pull the aforementioned multilingual model for embedding generation, **BGE-M3** (more information about this model on its [official website](https://bge-model.com/bge/bge_m3.html)).\n\n``` python\nimport subprocess\nimport time\n\n# Starting the Ollama server in the background\nsubprocess.Popen([\"ollama\", \"serve\"])\ntime.sleep(5) # Give the server a few seconds to initialize\n\n# Pulling the multilingual embedding model\nollama pull bge-m3\n\n123456789\n\nimport subprocessimport time # Starting the Ollama server in the backgroundsubprocess.Popen([\"ollama\", \"serve\"])time.sleep(5) # Give the server a few seconds to initialize # Pulling the multilingual embedding modelollama pull bge-m3\n```\n\nThe last configuration step is to use Scikit-LLM’s configuration module to point it to our Ollama instance. The configuration approach we are using does not require an actual key, but a dummy one, as shown below:\n\n``` python\nfrom skllm.config import SKLLMConfig\n\n# Point Scikit-LLM to our local Ollama instance\nSKLLMConfig.set_gpt_url(\"http://localhost:11434/v1/\")\n\n# Provide a dummy key (required by the internal client, but safely ignored by Ollama)\nSKLLMConfig.set_openai_key(\"free-friendly-dummy-key\")\n\n1234567\n\nfrom skllm.config import SKLLMConfig # Point Scikit-LLM to our local Ollama instanceSKLLMConfig.set_gpt_url(\"http://localhost:11434/v1/\") # Provide a dummy key (required by the internal client, but safely ignored by Ollama)SKLLMConfig.set_openai_key(\"free-friendly-dummy-key\")\n```\n\n## Building the Pipeline\n\nThe first major step in building our multilingual classification pipeline is, of course, getting the data. We will consider the **Amazon Multi-language Reviews** dataset, which has labeled customer reviews on a 5-star rating scale (internally encoded with labels 0 to 4). To avoid an overly time-consuming execution — especially regarding the embedding generation process later on — we will load a total of 2000 reviews in both English and Spanish. Feel free to select a larger sample if you’d like to, but try to keep it language-balanced and ensure random shuffling of your data before applying further steps like a training-test split.\n\n``` python\nfrom datasets import load_dataset\nimport pandas as pd\n\nprint(\"Loading and shuffling data to ensure class diversity...\")\n\n# 1. Loading the complete split\n# 2. Shuffling it randomly with shuffle()\n# 3. Extracting 1000 varied samples with select(range(1000))\ndata_en = (load_dataset(\"mteb/amazon_reviews_multi\", \"en\", split=\"train\", trust_remote_code=True)\n           .shuffle(seed=42)\n           .select(range(1000)))\n\ndata_es = (load_dataset(\"mteb/amazon_reviews_multi\", \"es\", split=\"train\", trust_remote_code=True)\n           .shuffle(seed=42)\n           .select(range(1000)))\n\n# Combining into a single DataFrame\ndf = pd.concat([pd.DataFrame(data_en), pd.DataFrame(data_es)], ignore_index=True)\n\n# Shuffling bilingual data\ndf = df.sample(frac=1, random_state=42).reset_index(drop=True)\n\n# Features and Labels\nX = df['text']\ny = df['label']\n\nprint(f\"Total samples: {len(X)}\")\nprint(\"\\n--- Class Verification (should have samples from 0 to 4) ---\")\nprint(y.value_counts())\n\n1234567891011121314151617181920212223242526272829\n\nfrom datasets import load_datasetimport pandas as pd print(\"Loading and shuffling data to ensure class diversity...\") # 1. Loading the complete split# 2. Shuffling it randomly with shuffle()# 3. Extracting 1000 varied samples with select(range(1000))data_en = (load_dataset(\"mteb/amazon_reviews_multi\", \"en\", split=\"train\", trust_remote_code=True)           .shuffle(seed=42)           .select(range(1000))) data_es = (load_dataset(\"mteb/amazon_reviews_multi\", \"es\", split=\"train\", trust_remote_code=True)           .shuffle(seed=42)           .select(range(1000))) # Combining into a single DataFramedf = pd.concat([pd.DataFrame(data_en), pd.DataFrame(data_es)], ignore_index=True) # Shuffling bilingual datadf = df.sample(frac=1, random_state=42).reset_index(drop=True) # Features and LabelsX = df['text']y = df['label'] print(f\"Total samples: {len(X)}\")print(\"\\n--- Class Verification (should have samples from 0 to 4) ---\")print(y.value_counts())\n```\n\nOutput:\n\n```\nLoading and shuffling data to ensure class diversity...\nTotal samples: 2000\n\n--- Class Verification (should have samples from 0 to 4) ---\nlabel\n0    444\n3    410\n2    404\n4    380\n1    362\nName: count, dtype: int64\n\n1234567891011\n\nLoading and shuffling data to ensure class diversity...Total samples: 2000 --- Class Verification (should have samples from 0 to 4) ---label0    4443    4102    4044    3801    362Name: count, dtype: int64\n```\n\nThe magic happens next. We define a scikit-learn pipeline consisting of two major stages:\n\n- Using a **GPTVectorizer** from Scikit-LLM and having it set up to utilize our previously loaded BGE-M3 model for building embeddings.\n- Feeding the embeddings to train a classifier based on a **LogisticRegression** model type.\n\n``` python\nfrom skllm.models.gpt.vectorization import GPTVectorizer\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import classification_report\n\n# Splitting into 80% training and 20% testing\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n\n# Defining the Pipeline\npipeline = Pipeline([\n    (\"vectorizer\", GPTVectorizer(model=\"bge-m3\", batch_size=32)),\n    (\"classifier\", LogisticRegression(max_iter=1000, random_state=42))\n])\n\n# Training the pipeline\nprint(\"Extracting embeddings and training classifier...\")\npipeline.fit(X_train, y_train)\n\n123456789101112131415161718\n\nfrom skllm.models.gpt.vectorization import GPTVectorizerfrom sklearn.pipeline import Pipelinefrom sklearn.linear_model import LogisticRegressionfrom sklearn.model_selection import train_test_splitfrom sklearn.metrics import classification_report # Splitting into 80% training and 20% testingX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # Defining the Pipelinepipeline = Pipeline([    (\"vectorizer\", GPTVectorizer(model=\"bge-m3\", batch_size=32)),    (\"classifier\", LogisticRegression(max_iter=1000, random_state=42))]) # Training the pipelineprint(\"Extracting embeddings and training classifier...\")pipeline.fit(X_train, y_train)\n```\n\n**Why did I say the magic takes place here? Let’s look more closely:**\n\nBGE-M3 is a multilingual embedding model that has been pre-trained on massive data spanning over 100 languages. Put another way, it is capable of internally mapping both our English and Spanish reviews into a common dimensional (embedding) space: not based on their concrete vocabulary, but based on the meaning behind it. Thus, language barriers disappear during the process of generating embeddings, with LLM outputs for *“This product is fantastic!”* and *“¡Este producto es fantástico!”* being nearly identical.\n\nAs a result, by the time the embeddings arrive at the logistic regression model for training and inference, the classifier doesn’t actually care about the language anymore. It has the information it needs to perform rating classifications on product reviews.\n\n```\nprint(\"Evaluating on the test set...\")\ny_pred = pipeline.predict(X_test)\n\nprint(\"\\n--- Classification Report ---\")\nprint(classification_report(y_test, y_pred))\n\n12345\n\nprint(\"Evaluating on the test set...\")y_pred = pipeline.predict(X_test) print(\"\\n--- Classification Report ---\")print(classification_report(y_test, y_pred))\n```\n\nResults:\n\n```\n--- Classification Report ---\n              precision    recall  f1-score   support\n\n           0       0.66      0.78      0.72        82\n           1       0.40      0.30      0.34        64\n           2       0.46      0.46      0.46        91\n           3       0.56      0.54      0.55        84\n           4       0.71      0.73      0.72        79\n\n    accuracy                           0.57       400\n   macro avg       0.56      0.56      0.56       400\nweighted avg       0.56      0.57      0.56       400\n\n123456789101112\n\n--- Classification Report ---              precision    recall  f1-score   support            0       0.66      0.78      0.72        82           1       0.40      0.30      0.34        64           2       0.46      0.46      0.46        91           3       0.56      0.54      0.55        84           4       0.71      0.73      0.72        79     accuracy                           0.57       400   macro avg       0.56      0.56      0.56       400weighted avg       0.56      0.57      0.56       400\n```\n\nThe results are just okay, but not great. There is significantly better performance in correctly predicting extreme ratings (0 for 1-star, 4 for 5-star) than for predicting intermediate ratings. Don’t panic; there are at least two reasons for this:\n\n1. The classification task at hand is inherently challenging: distinguishing between a 3-star and a 4-star review is intuitively harder than discerning, for instance, between positive, negative, and neutral reviews.\n2. More importantly, we have used just 2000 samples (80% of them for model training), but these samples are embeddings with 1024 features each. Feeding such a small amount of high-dimensional data to a classifier is most likely the perfect recipe for overfitting your model. If you have the time to run the code for longer, try using a few thousand more examples instead.\n\n## Wrapping Up\n\nIn traditional natural language processing, we were often faced with two far-from-ideal options when handling multilingual data for predictive tasks like text classification: translate all your data into a base language — a slow, expensive process with frequent loss of nuance — or train separate models: one for every language. In the pipeline we just built, the heavy burden is assumed by the multilingual embedding model (BGE-M3) leveraged through Scikit-LLM, which is capable of transparently mapping text across a variety of languages into a uniform embedding space.", "url": "https://wpnews.pro/news/multilingual-text-classification-with-scikit-llm-and-multilingual-embeddings", "canonical_source": "https://machinelearningmastery.com/multilingual-text-classification-with-scikit-llm-and-multilingual-embeddings/", "published_at": "2026-09-17 12:00:49+00:00", "updated_at": "2026-09-18 01:25:41.839391+00:00", "lang": "en", "topics": ["natural-language-processing", "large-language-models", "machine-learning", "ai-tools", "developer-tools"], "entities": ["Scikit-LLM", "scikit-learn", "Ollama", "BGE-M3", "OpenAI"], "alternates": {"html": "https://wpnews.pro/news/multilingual-text-classification-with-scikit-llm-and-multilingual-embeddings", "markdown": "https://wpnews.pro/news/multilingual-text-classification-with-scikit-llm-and-multilingual-embeddings.md", "text": "https://wpnews.pro/news/multilingual-text-classification-with-scikit-llm-and-multilingual-embeddings.txt", "jsonld": "https://wpnews.pro/news/multilingual-text-classification-with-scikit-llm-and-multilingual-embeddings.jsonld"}}