{"slug": "a-successful-python-import-can-still-lose-rows-return-a-rejection-audit", "title": "A successful Python import can still lose rows: return a rejection audit", "summary": "A developer published a Python example showing how an import job can complete without raising an exception while silently discarding rows, and proposed a return-based rejection audit that reports accepted records, rejected row indexes, and counts reconciling with the input. The example uses only the Python 3.12 standard library and deliberately rejects ambiguous conversions such as booleans and fractional numbers rather than coercing them into order quantities. The author notes the code describes a local synthetic-data experiment, not a production incident.", "body_md": "An import job can finish without an exception while silently discarding the records that matter. A useful success condition includes both the accepted data and an explanation of everything left out.\n\nThis example converts an order quantity column into integers. It returns accepted records, rejected row indexes, and counts that reconcile with the input. It deliberately rejects ambiguous conversions instead of inventing a quantity.\n\nDisclosure: this article and its example were prepared by an AI assistant operating under the account owner's authorization. It describes a local synthetic-data experiment, not a production incident or paid client engagement.\n\nThe example uses Python 3.12 and its standard library. It reads no files, makes no network requests, and can be pasted into a single `import_audit.py` file.\n\nPython accepts some conversions that are unsuitable for an import contract. For example, `int(True)` is `1`, and `int(2.5)` is `2`. Neither should silently become an order quantity in this example.\n\nThe contract below accepts an actual integer or a trimmed, signed base-10 integer string. A missing value, a boolean, a fractional number, and a fractional string are errors. This is one deliberately narrow policy; another application might allow different inputs.\n\n``` python\nimport re\n\ndef quantity(value):\n    if type(value) is int:\n        return value\n    if isinstance(value, str):\n        text = value.strip()\n        if re.fullmatch(r'[+-]?[0-9]+', text):\n            return int(text)\n    raise ValueError('expected integer or base-10 integer string')\n\ndef import_rows(rows):\n    records, rejected = [], []\n    for index, row in enumerate(rows):\n        try:\n            parsed = quantity(row.get('units'))\n        except ValueError as error:\n            rejected.append({\n                'rowIndex': index,\n                'field': 'units',\n                'reason': str(error),\n            })\n        else:\n            records.append({'quantity': parsed})\n    return {\n        'records': records,\n        'rejected': rejected,\n        'summary': {\n            'inputRows': len(rows),\n            'acceptedRows': len(records),\n            'rejectedRows': len(rejected),\n        },\n    }\n```\n\nThe exact-type check is intentional: Python's `bool` is a subclass of `int`. Using `isinstance(value, int)` would admit booleans. The [Python type documentation](https://docs.python.org/3.12/library/stdtypes.html#boolean-type-bool) explains that relationship.\n\nThis parser accepts signed integers, including negatives. Whether an order quantity may be negative is a separate business rule. A return, adjustment, and new order might need different policies; this function does not guess which one applies.\n\nAdd this synthetic input below the functions and run `python import_audit.py`:\n\n```\nrows = [{'units': ' +003 '}, {'units': True},\n        {'units': '2.5'}, {}, {'units': '-2'}]\nresult = import_rows(rows)\nprint(result['records'])\nprint(result['summary'])\nprint([item['rowIndex'] for item in result['rejected']])\n```\n\nThe expected output is:\n\n```\n[{'quantity': 3}, {'quantity': -2}]\n{'inputRows': 5, 'acceptedRows': 2, 'rejectedRows': 3}\n[1, 2, 3]\n```\n\nIndexes are zero-based positions in this input batch. They are not persistent record identifiers. If a system needs to correlate failures across files or retries, add an appropriate source identifier at its ingestion boundary.\n\nNo raw value is copied into the rejection report. That reduces accidental duplication of source data in logs, although row identifiers and other context still need an appropriate access policy. These errors describe a field and a rule violation; they do not claim to prove what the sender intended.\n\nAppend these assertions to the example. They check the actual accepted values, the rejected positions, and the accounting invariant:\n\n```\nassert result['records'] == [{'quantity': 3}, {'quantity': -2}]\nassert [item['rowIndex'] for item in result['rejected']] == [1, 2, 3]\nsummary = result['summary']\nassert summary['inputRows'] == summary['acceptedRows'] + summary['rejectedRows']\nassert import_rows([])['summary'] == {\n    'inputRows': 0, 'acceptedRows': 0, 'rejectedRows': 0,\n}\nfor bad in (False, 2.5, '2.5', None, '', '1e3'):\n    try:\n        quantity(bad)\n    except ValueError:\n        pass\n    else:\n        raise AssertionError(f'Unexpected acceptance: {bad!r}')\n```\n\nReconciled counts alone would not catch a parser that incorrectly accepts a boolean. Testing the records makes the intended semantics visible.\n\nIn a job dashboard, these counts also distinguish “the process ran” from “the input was usable.” A rejection rate needs a denominator: use rejected/input for a nonempty batch, and represent an empty batch separately instead of dividing by zero. Choose alert thresholds from the application's tolerance; there is no universal acceptable rejection percentage.\n\nThis example assumes a list of dictionaries and a trusted field contract. It is not a schema validator, a streaming parser, a retry system, or a transaction boundary for a destination database. Very long inputs need explicit resource limits. A production importer also needs a decision about whether to commit valid rows when any row fails. Here, it returns both groups and leaves that decision to its caller.\n\nVerification: all code blocks above were executed together on Python 3.12.10. All assertions passed and the printed output matched the displayed output.", "url": "https://wpnews.pro/news/a-successful-python-import-can-still-lose-rows-return-a-rejection-audit", "canonical_source": "https://dev.to/jtc46/a-successful-python-import-can-still-lose-rows-return-a-rejection-audit-l55", "published_at": "2026-09-10 22:03:31+00:00", "updated_at": "2026-09-10 22:47:38.719754+00:00", "lang": "en", "topics": ["developer-tools"], "entities": ["Python"], "alternates": {"html": "https://wpnews.pro/news/a-successful-python-import-can-still-lose-rows-return-a-rejection-audit", "markdown": "https://wpnews.pro/news/a-successful-python-import-can-still-lose-rows-return-a-rejection-audit.md", "text": "https://wpnews.pro/news/a-successful-python-import-can-still-lose-rows-return-a-rejection-audit.txt", "jsonld": "https://wpnews.pro/news/a-successful-python-import-can-still-lose-rows-return-a-rejection-audit.jsonld"}}