{"slug": "auditing-model-bias-with-balanced-datasets-with-mimesis", "title": "Auditing Model Bias with Balanced Datasets with Mimesis", "summary": "Mimesis, an open-source data generation library, can be used to create balanced, counterfactual datasets for auditing bias in machine learning models. A hands-on guide demonstrates training a loan approval classifier on intentionally biased data, then using Mimesis to generate test subjects with identical financial profiles but different genders to reveal discriminatory outcomes. This approach allows developers to detect whether a model unfairly discriminates against protected groups without exposing sensitive real-world data.", "body_md": "# Auditing Model Bias with Balanced Datasets with Mimesis\n\nLearn how to use Mimesis library to generate a balanced, counterfactual dataset that helps analyze potential bias in your models.\n\n## # Introduction\n\nWhether they are well-established classifiers or state-of-the-art massive models like large language models (LLMs), building machine learning solutions often entails a risk: algorithms might silently adopt prejudices inherent in the historical training dataset they were trained on. But in a high-stakes scenario or one where data is sensitive, how can we **audit whether a model is biased** without compromising real-world information?\n\nThis hands-on article guides you in training a simple classification model for \"loan approval\" on biased data. Based on this, we will use ** Mimesis**, an open-source library that can help generate a perfectly balanced,\n\n*counterfactual*dataset. You'll be able to test \"fake\" users with identical financial backgrounds but different demographic characteristics, thereby determining whether the model discriminates against certain groups or not.\n\n## # Step-by-Step Guide\n\nStart by installing the Mimesis library if you are new to using it, or you are working on a cloud notebook environment like Colab:\n\n```\npip install mimesis\n```\n\nBefore auditing a model, we actually need to get one! In this example, we will synthetically generate a dataset of 1,000 bank customers, with just two features: gender and income. These features are categorical and numerical, respectively. The data creation will be intentionally manipulated so that the gender attribute unfairly influences the binary outcome: loan approval. Specifically, for labeling the dataset, we will consider a scenario in which men are generally approved, whereas women are only approved when they have remarkably high income.\n\nThe process to create this clearly biased dataset and train a decision tree classifier on it is shown below:\n\n``` python\nimport pandas as pd\nimport numpy as np\nfrom sklearn.tree import DecisionTreeClassifier\n\n# 1. Simulating biased historical data (1000 instances)\nnp.random.seed(42)\nn_train = 1000\ngenders = np.random.choice(['Male', 'Female'], n_train)\nincomes = np.random.randint(30000, 120000, n_train)\n\napprovals = []\nfor gender, income in zip(genders, incomes):\n    if gender == 'Male':\n        # Historically, males are approved\n        approvals.append(1)\n    else:\n        # Only females with high income are approved\n        approvals.append(1 if income > 80000 else 0)\n\ntrain_df = pd.DataFrame({'Gender': genders, 'Income': incomes, 'Approved': approvals})\n\n# Converting categories to numbers for the machine learning model\ntrain_df['Gender_Code'] = train_df['Gender'].map({'Male': 1, 'Female': 0})\n\n# 2. Training a Decision Tree classifier\nmodel = DecisionTreeClassifier(max_depth=3)\nmodel.fit(train_df[['Gender_Code', 'Income']], train_df['Approved'])\n```\n\nThe next step shows Mimesis in action. We will use this library to generate a small set of test subjects using the `Generic`\n\nclass. This will be done by defining three base financial profiles that contain random UUIDs (universally unique identifiers) and a moderate income ranging between 40K and 70K. Notice that these profiles will not have gender information incorporated yet:\n\n``` python\nfrom mimesis import Generic\n\ngeneric = Generic('en')\n\n# Generating 3 base financial profiles\nbase_profiles = []\nfor _ in range(3):\n    profile = {\n        'Applicant_ID': generic.cryptographic.uuid(),\n        'Income': generic.random.randint(40000, 70000) # Moderate income\n    }\n    base_profiles.append(profile)\n```\n\nFor example, the three newly created profiles may look something like:\n\n```\n[{'Applicant_ID': '1f1721e1-19af-4bd1-8488-6abf01404ef9', 'Income': 44815},\n {'Applicant_ID': '5c862597-7f55-43f4-9d6e-ac9cc0b9083e', 'Income': 47436},\n {'Applicant_ID': '3479d4cf-0d9b-4f06-9c43-1c3b7e787830', 'Income': 58194}]\n```\n\nLet's finish building our counterfactual set of examples, which constitutes the core of our auditing process! For each of the three base profiles, we will create two cloned counterfactual instances: one being male and the other being female. For each pair of test customers, their application ID and income will be totally identical, so the only difference will be the gender: any difference in how our trained decision tree model treats them will undoubtedly be proof of gender bias.\n\n```\ncounterfactual_data = []\n\nfor profile in base_profiles:\n    # Version A: Male Counterfactual\n    counterfactual_data.append({\n        'Applicant_ID': profile['Applicant_ID'], \n        'Gender': 'Male', \n        'Gender_Code': 1, \n        'Income': profile['Income']\n    })\n    \n    # Version B: Female Counterfactual\n    counterfactual_data.append({\n        'Applicant_ID': profile['Applicant_ID'], \n        'Gender': 'Female', \n        'Gender_Code': 0, \n        'Income': profile['Income']\n    })\n\naudit_df = pd.DataFrame(counterfactual_data)\n```\n\nThis is what the three pairs of customers may look like:\n\n```\n1f1721e1-19af-4bd1-8488-6abf01404ef9\tMale\t1\t44815\n1\t1f1721e1-19af-4bd1-8488-6abf01404ef9\tFemale\t0\t44815\n2\t5c862597-7f55-43f4-9d6e-ac9cc0b9083e\tMale\t1\t47436\n3\t5c862597-7f55-43f4-9d6e-ac9cc0b9083e\tFemale\t0\t47436\n4\t3479d4cf-0d9b-4f06-9c43-1c3b7e787830\tMale\t1\t58194\n5\t3479d4cf-0d9b-4f06-9c43-1c3b7e787830\tFemale\t0\t58194\n```\n\n**A key point to insist on here:** we have just used Mimesis to instantly build perfectly matched \"clones\" of loan applicants with identical income but different genders. This underlines the library's value in providing total statistical control, isolating a protected attribute.\n\nNow it's time to probe the model and see what it reveals.\n\n```\n# Asking the model to predict approval for our counterfactuals\naudit_df['Predicted_Approval'] = model.predict(audit_df[['Gender_Code', 'Income']])\n\n# Formatting the output for readability (1 = Approved, 0 = Denied)\naudit_df['Predicted_Approval'] = audit_df['Predicted_Approval'].map({1: 'Approved', 0: 'Denied'})\n\nprint(\"\\n--- Model Audit Results ---\")\nprint(audit_df[['Applicant_ID', 'Gender', 'Income', 'Predicted_Approval']].sort_values('Applicant_ID'))\n```\n\nThe decision-making results yielded by our model could not be clearer:\n\n```\n--- Model Audit Results ---\n                           Applicant_ID  Gender  Income Predicted_Approval\n0  1f1721e1-19af-4bd1-8488-6abf01404ef9    Male   44815           Approved\n1  1f1721e1-19af-4bd1-8488-6abf01404ef9  Female   44815             Denied\n4  3479d4cf-0d9b-4f06-9c43-1c3b7e787830    Male   58194           Approved\n5  3479d4cf-0d9b-4f06-9c43-1c3b7e787830  Female   58194             Denied\n2  5c862597-7f55-43f4-9d6e-ac9cc0b9083e    Male   47436           Approved\n3  5c862597-7f55-43f4-9d6e-ac9cc0b9083e  Female   47436             Denied\n```\n\nNotice that for the exact same `Applicant_ID`\n\nand `Income`\n\n, male clones are approved for the loan. Meanwhile, female clones with such moderate income are generally denied. The Mimesis functionalities we used based on profiles helped us hold all other variables constant, thereby successfully isolating and exposing the model's discriminatory decision-making.\n\n## # Wrapping Up\n\nThroughout this hands-on article, we have shown how Mimesis can be used to generate balanced, counterfactual data examples — without privacy or sensitive data constraints — that can help audit a model's behavior and identify whether the model is behaving in a biased manner or not. Next steps to take if your model is biased may include:\n\n- Augmenting your training data with more balanced profiles to correct historical skewness or bias.\n- Depending on the model type, using model re-weighting strategies.\n- Utilizing open-source toolkits for fairness — for instance,\n— which are helpful for bias mitigation in machine learning pipelines.[AI Fairness 360](https://ai-fairness-360.org/)\n\nis a leader, writer, speaker, and adviser in AI, machine learning, deep learning & LLMs. He trains and guides others in harnessing AI in the real world.\n\n[Iván Palomares Carrascosa](https://www.linkedin.com/in/ivanpc/)", "url": "https://wpnews.pro/news/auditing-model-bias-with-balanced-datasets-with-mimesis", "canonical_source": "https://www.kdnuggets.com/auditing-model-bias-with-balanced-datasets-with-mimesis", "published_at": "2026-05-25 14:00:46+00:00", "updated_at": "2026-05-26 13:46:46.709329+00:00", "lang": "en", "topics": ["machine-learning", "ai-ethics", "artificial-intelligence", "ai-tools", "ai-safety"], "entities": ["Mimesis"], "alternates": {"html": "https://wpnews.pro/news/auditing-model-bias-with-balanced-datasets-with-mimesis", "markdown": "https://wpnews.pro/news/auditing-model-bias-with-balanced-datasets-with-mimesis.md", "text": "https://wpnews.pro/news/auditing-model-bias-with-balanced-datasets-with-mimesis.txt", "jsonld": "https://wpnews.pro/news/auditing-model-bias-with-balanced-datasets-with-mimesis.jsonld"}}