{"slug": "few-shot-text-classification-with-scikit-llm", "title": "Few-Shot Text Classification with Scikit-LLM", "summary": "Scikit-LLM, a Python library that wraps pre-trained large language models behind the scikit-learn fit() and predict() interface, enables few-shot text classification with as few as 3 to 5 labeled examples, according to a tutorial published on the project's GitHub. The walkthrough configures the library against Groq's OpenAI-compatible endpoint via SKLLMConfig.set_gpt_url and a GROQ_API_KEY secret in Google Colab, after installing the package with pip install scikit-llm. The approach targets domain-specific classification tasks where jargon, edge cases or nuanced sentiment make zero-shot prompting less reliable, without retraining the underlying model.", "body_md": "# Few-Shot Text Classification with Scikit-LLM\n\nLeveraging Scikit-LLM for one of the simplest yet most practical text classification tasks utilized in the modern AI landscape: few-shot classification.\n\n## Introduction\n\nIf you have worked with building text classification models, chances are you got familiar with the usual drill: collecting hundreds to thousands of labeled examples, cleaning the raw text, converting it to suitable numerical representations for machine learning models — like embeddings or TF-IDF — and training your model, such as a logistic regression or random forest classifier.\n\nHowever, when large language models (LLMs) came along, they changed the rules of the game — and that includes traditional machine learning pipelines to some extent. Now it is possible to classify a text by simply asking an LLM to do so. Here's the catch: shifting from classical machine learning workflows to prompt engineering can be challenging. **[Scikit-LLM](https://github.com/iryna-kondr/scikit-llm)** is a Python library that bridges the gap between both worlds. It wraps the powerful capabilities of pre-trained LLMs behind the familiar `scikit-learn` interface — namely `fit()` and `predict()`.\n\nThis article shows how to use Scikit-LLM for one of the simplest yet most practical text classification tasks in the modern AI landscape: few-shot classification, that is, classifying text based on a small number of labeled examples that guide the LLM on what to do and how.\n\n## Zero-Shot vs. Few-Shot Classification\n\nBefore getting into our Python example, it's worth clarifying the difference between few-shot classification and another form of text classification that also gained popularity with the rise of LLMs: **zero-shot classification**.\n\n- **Zero-shot classification** : The model is given a text together with a list of possible labels and asked to categorize the text under the most suitable one — all without any input-output examples. This paradigm relies entirely on the LLM's internal knowledge gained during pre-training.\n- **Few-shot classification** : A small set of labeled examples, or \"shots\" (texts with associated class labels), is passed to the model before asking it to classify a new, unlabeled text. These domain-specific examples supplement the model's internal knowledge and better guide its reasoning.\n\nFew-shot classification is typically better than its zero-shot counterpart for highly domain-specific text classification tasks where texts contain specific jargon, edge cases, or nuanced sentiments. In most cases, providing just 3 to 5 labeled examples may suffice to \"teach\" a pre-trained LLM the extra rules it needs to perform a good classification — all without retraining the whole model.\n\nThe good news: Scikit-LLM makes this process remarkably easy. Here's how it works.\n\n## Running Few-Shot Classification with Scikit-LLM, Step by Step\n\nThis example can be run entirely in a Google Colab notebook. It uses **[Groq](https://console.groq.com/keys)**'s LLM API, which provides access to free models. As a prerequisite, register on the Groq website and generate your own API key. Once generated, carefully copy your API key and paste it as a new \"Secret\" variable in Google Colab — found at the key icon on the left-hand side of Colab. Name the secret variable `GROQ_API_KEY`.\n\nAfter that, install Scikit-LLM as follows:\n\n```\n!pip install scikit-llm -q\n```\n\nWhile Scikit-LLM is built to operate with OpenAI-compatible LLM endpoints, Groq's API is among them. The following code retrieves your Groq API key and applies the necessary configuration to set up access to the endpoint.\n\n``` python\nfrom google.colab import userdata\nfrom skllm.config import SKLLMConfig\n\n# Retrieve the API key securely from Google Colab Secrets\ngroq_key = userdata.get('GROQ_API_KEY')\n\n# Point scikit-LLM to Groq's OpenAI-compatible endpoint\nSKLLMConfig.set_gpt_url(\"https://api.groq.com/openai/v1\")\nSKLLMConfig.set_openai_key(groq_key)\n```\n\nNext, we prepare the dataset. To use few-shot classification, we don't need a large labeled dataset as we would for training a new model from scratch. Instead, we need a small, representative set of labeled \"few-shot\" examples and their associated labels. Let's assume we want to classify IT support tickets into three categories. Following the classical machine learning convention, we name the few-shot examples and their labels `X_train` and `y_train` respectively. We also define `X_test`, which contains three unlabeled texts we want to classify:\n\n```\n# Our \"Few-Shot\" examples to teach the model the nuances\nX_train = [\n    \"My laptop screen is flickering heavily.\",\n    \"I need a new mouse, the scroll wheel is broken.\",\n    \"Excel keeps crashing every time I open a macro.\",\n    \"How do I update to the latest version of Photoshop?\",\n    \"I forgot my VPN password.\",\n    \"Can you grant me admin rights to the shared drive?\"\n]\n\ny_train = [\n    \"Hardware\",\n    \"Hardware\",\n    \"Software\",\n    \"Software\",\n    \"Access\",\n    \"Access\"\n]\n\n# The unseen data we want to actually predict\nX_test = [\n    \"The keyboard types double letters randomly.\",\n    \"I am locked out of my email account.\",\n    \"Zoom freezes when I share my screen.\"\n]\n```\n\nTo use Scikit-LLM, we import the key class for our target task: `FewShotGPTClassifier`. This class wraps a compatible model of our choice — such as Groq's `openai/gpt-oss-20b` — inside a task-specific layer that lets us use it just like a classical machine learning model trained with `scikit-learn`.\n\n``` python\nfrom skllm.models.gpt.classification.few_shot import FewShotGPTClassifier\n\n# Use a confirmed 2026 free-tier model on Groq, ensuring the custom_url:: prefix is present\nclf = FewShotGPTClassifier(model=\"custom_url::openai/gpt-oss-20b\")\n\n# \"Train\" the model (constructs the few-shot prompt memory)\nclf.fit(X_train, y_train)\n```\n\nImportantly, `fit()` does not perform actual training (weight updating) on the LLM. It simply builds a few-shot prompt memory for the model, containing the labeled examples it will rely on for reasoning and generating its responses.\n\nAll it takes now is calling `predict()` to obtain predictions on our unlabeled text examples:\n\n```\n# Generate predictions\npredictions = clf.predict(X_test)\n\n# Display the results\nfor text, label in zip(X_test, predictions):\n    print(f\"Ticket: '{text}'\\nPredicted Category: {label}\\n\")\n```\n\nThe classification labels the model assigns to these three support tickets are exactly what we'd expect — all three are correct:\n\n```\nTicket: 'The keyboard types double letters randomly.'\nPredicted Category: Hardware\n\nTicket: 'I am locked out of my email account.'\nPredicted Category: Access\n\nTicket: 'Zoom freezes when I share my screen.'\nPredicted Category: Software\n```\n\n## Wrapping Up\n\nThis article has shown how Scikit-LLM's elegant design lets you turn pre-trained, open LLMs into machine learning engines capable of performing domain-specific tasks like few-shot classification. No heavy model retraining is required, and no large paid models are needed — just a handful of labeled examples is enough for the model to classify new texts accurately.\n\n \n\n \n\n[**\\[Iván Palomares Carrascosa\\](https://www.linkedin.com/in/ivanpc/)**](https://www.linkedin.com/in/ivanpc/) is a leader, writer, speaker, and adviser in AI, machine learning, deep learning & LLMs. He trains and guides others in harnessing AI in the real world.", "url": "https://wpnews.pro/news/few-shot-text-classification-with-scikit-llm", "canonical_source": "https://www.kdnuggets.com/few-shot-text-classification-with-scikit-llm", "published_at": "2026-10-14 14:00:00+00:00", "updated_at": "2026-10-14 23:51:03.655728+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "natural-language-processing", "developer-tools", "machine-learning"], "entities": ["Scikit-LLM", "scikit-learn", "Groq", "Google Colab", "Python", "OpenAI", "iryna-kondr"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/few-shot-text-classification-with-scikit-llm", "markdown": "https://wpnews.pro/news/few-shot-text-classification-with-scikit-llm.md", "text": "https://wpnews.pro/news/few-shot-text-classification-with-scikit-llm.txt", "jsonld": "https://wpnews.pro/news/few-shot-text-classification-with-scikit-llm.jsonld"}}