{"slug": "using-oxcaml-to-implement-type-safe-reference-counting-between-ocaml-and-python", "title": "Using OxCaml to implement type-safe reference counting between OCaml and Python", "summary": "Jane Street developed PyOCaml, a bridge between OCaml and Python, but faced memory management issues where Python objects represented in OCaml were deallocated late. Using OxCaml language extensions, they implemented type-safe reference counting that statically guarantees prompt deallocation, preventing memory leaks in Python programs using OCaml libraries.", "body_md": "Jane Street is known for being an OCaml shop, but for years now Python has been our second major programming language, acting as the primary tool for data analysis and (especially importantly these days) machine learning. Most of our traders and researchers think and write in Python, even as the majority of our infrastructure is written in OCaml.\n\nSo it’s been important to support a bridge between these two languages. For that we developed PyOCaml, which lets authors expose a Python interface to their OCaml library. The trouble is that sometimes things fall off the bridge: in particular, when you represent a Python object as an OCaml value, the interactions between the different languages’ memory management systems can lead to object deallocations getting materially delayed. For simple, small-scale data types this is no big deal, but for programs working with huge data frames or scarce resources like GPU memory, it’s a real problem.\n\nWe ended up developing a solution that relied on some nifty features of\n[OxCaml](https://oxcaml.org/), a set of language extensions for OCaml intended to support\nhigh-performance programs and data-race free parallelism. These features have allowed us\nto encode prompt deallocation in a typesafe way. When PyOCaml library authors use these\nnew features, the compiler can actually statically guarantee that Python programs written\nagainst them won’t have those promptness problems. This is a big win: in the old world, it\nwas theoretically possible to write Python that avoided losing track of objects, but it\nrequired an impractical level of care and expertise. Now such offending programs are\nimpossible to write by construction.\n\nTo understand how that works, it helps first to know how Python objects are allocated and GC’d; how they’re represented in OCaml; and how we can borrow the idea of “borrowing” to implement explicit and typesafe reference counting between the two.\n\n## A primer on Python objects and their lifecycle\n\nIn Python, every object is allocated in memory using a structure with some type-specific layout. The first fields in the structure are shared across all structures, and include information like the type of the object and a reference count field.\n\nUnlike OCaml, where the garbage collector is [scanning and\nmoving](https://ocaml.org/docs/garbage-collector), Python objects are reference counted. A\nfreshly created object has reference count 1; when a new reference to the object is\ncreated (e.g., the object gets stored in a list), the reference count needs to be\nincremented; when an object goes out of scope, the count is decremented. Once the count\nreaches 0, the object is deallocated:\n\n```\nmy_list = [] # Refcount of my_list is 1\nmy_dict = {}\nmy_dict[\"my_list\"] = my_list # Refcount of my_list is 2\ndel my_list # Refcount of my_list is 1\ndel my_dict[\"my_list\"] # Refcount of my_list went to 0, deallocated\n```\n\n### Borrowing and stealing\n\nIn Python, when you pass an arg into a function, there are two ways to ensure that its reference count is managed correctly. One is called “borrowing.” When a function borrows a reference to the object, it doesn’t increment its reference count.\n\nCode can “borrow” a reference to an object. As an example, when calling a function with some argument, the argument object can be borrowed from the caller during the function call, as long as the object doesn’t outlive the call:\n\n``` python\ndef g():\n    obj = object()        # We just made a new object.\n                          # Exactly one name (`obj`) points at it. Refcount = 1.\n\n    res = f(obj)          # We call f, passing the object in.\n\ndef f(arg):\n    # arg is just another name for the same object `obj`.\n    # Counter is still 1, NOT 2.\n    # That's \"borrowing\": passing into a function\n    # does not increase the count.\n\n    res = [arg]\n    # Now we put it inside a list.\n    # The list is a new, persistent container that points at the object.\n    # That DOES bump the count: 1 -> 2.\n\n    return res\n```\n\nNote that the above code merely demonstrates the concept, but isn’t actually true, in the\nsense that the actual interpreter does things slightly differently (and even depends on\nthe exact version). The above applies only to functions that are *not* implemented in pure\nPython (but are rather exposed to Python by some extension module written in, say, C).\n\nSuppose `g`\n\ncontinued:\n\n``` python\ndef g():\n    obj = object()\n    res = f(obj)          # res is the list. The list still points at obj.\n                          # Counter on obj is 2: one from `obj`, one from the list.\n\n    del obj               # Drop the name `obj`. Counter: 2 -> 1.\n                          # The list still has it.\n\n    del res               # Drop the list. The list is freed,\n                          # which releases its reference to obj.\n                          # Counter: 1 -> 0. obj is freed.\n```\n\nThis is safe: we know inside `f`\n\nthat the reference count of arg will not go to 0, because\nthe caller `g`\n\nstill holds a reference that’s valid at least until `f`\n\nreturns.\n\nMeanwhile, some APIs “steal” a reference from a caller: they take some argument that will outlive the function call, but instead of incrementing the reference count, they “steal” it and the caller no longer owns the reference it had on the object when invoking the function. (The reference count might even have gone to 0 before the callee returns!) Stealing is not something you can see in the Python code, but rather is implemented at the C-extension level.\n\nIn borrowing-style code:\n\n```\nPyObject *obj = make_something();   // counter = 1\nlist_append(my_list, obj);          // list increfs internally: counter = 2\nPy_DECREF(obj);                     // I drop my reference: counter = 1\n```\n\nStealing-style code:\n\n```\nPyObject *obj = make_something();   // counter = 1, I own it\nlist_append_steal(my_list, obj);    // I do NOT incref.\n                                    // The list now considers itself\n                                    // the owner of that single reference.\n                                    // I'm no longer allowed to use obj.\n```\n\nWhen writing C code to implement a Python extension, a developer must manage the ownership of object references manually, since the compiler won’t catch bugs:\n\n- A missing\n`Py_DECREF`\n\ncould lead to a memory leak - A missing\n`Py_INCREF`\n\non a borrowed object could lead to an invalid pointer later on - A missing\n`Py_INCREF`\n\ncould lead to an invalid pointer when an object passed to a stealing function is later reused - A missing\n`Py_INCREF`\n\ncould lead to problems when a borrowed object is passed to a stealing function - Receiving a borrowed reference from a function could lead to an invalid pointer when using free-threaded Python\n\netc.\n\n## Representing Python objects as OCaml values can leave refcounts hanging\n\nIn PyOCaml, Python objects are represented as OCaml values of type `Py.Object.t`\n\n. These\nare “custom block” values that hold an 8-byte pointer regardless of the size of the\nunderlying Python object. This `Py.Object.t`\n\nis allocated onto the global heap and will be\nlive until garbage collection. We want to make sure that the Python object is GC’d on the\nPython side, which means ensuring that the OCaml side doesn’t hold any reference longer\nthan necessary.\n\nTrouble is, it’s easy for this to go awry. When we wrap a PyObject reference into an OCaml\nvalue, we increment its reference count; but it will only be decremented when the custom\nblock’s finalizer is called. This only happens when the OCaml garbage collector frees up\nthe `Py.Object.t`\n\n. In effect we’ve coupled the freeing of the underlying Python object\n(which could be a many-GB data frame) to the eventual GC’ing of the OCaml value. This\nmeans that when calling a PyOCaml function the reference count of an argument may remain\nincremented even after the function returns, and it’ll only be decremented when the OCaml\ngarbage collector frees up the `Py.Object.t`\n\n.\n\nConcretely, let’s say you have some OCaml code defining a PyOCaml function:\n\n``` js\nlet pyocaml_function (arg : Py.Object.t) : Py.Object.t =\n  let i = Py.Int.to_int arg in\n  let res = i + 1 in\n  Py.Int.of_int res\n```\n\nThe way this is implemented in C glue code (heavily simplified for the purposes of this post) is as follows:\n\n```\nPyObject *call_pyocaml_function(PyObject *arg) {\n    /* The C function borrows, but the OCaml value is globally allocated\n       and could outlive this function. As an example, `pyocaml_function` could\n       store it in some global map. */\n    Py_INCREF(arg);\n    value ocaml_arg = wrap_pyobject_into_py_object_t(arg);\n\n    value ocaml_result = call_pyocaml_function(ocaml_arg);\n    PyObject *result = unwrap_pyobject_from_py_object_t(ocaml_result);\n\n    /* The returned ocaml_result is a `Py.Object.t` that's globally allocated,\n       and could outlive the call to `pyocaml_function` */\n    Py_INCREF(result);\n    return result;\n}\n```\n\nBecause the `Py_INCREF`\n\nisn’t paired with a decref until the `Py.Object.t`\n\ngets GC’d by\nthe OCaml runtime, the borrowing semantics don’t work out the way they did in pure\nPython. In particular:\n\n```\narg = 1024 # Reference count is 1\nres = pyocaml_function(arg)\n# Reference count of arg is 2, reference count of res is 2,\n# unlike the pure-Python examples above\n\ndel arg # Reference count of arg is 1\ndel res # Reference count of res is 1\n\n# As long as the OCaml GC doesn't run (i.e., at least as long as we don't call any PyOCaml code),\n# reference count of arg and res remains 1 and neither are deallocated.\n```\n\nIt’s only once the OCaml GC observes that `arg`\n\nand the returned `Py.Object.t`\n\nwere no\nlonger referenced, and invoked the custom block finalizers, that the reference count of\n`arg`\n\nand `res`\n\ngo to 0 and both get deallocated.\n\nThis problem is exacerbated by the fact that the OCaml garbage collector has no knowledge\nof the amount of memory these `Py.Object.t`\n\ns are keeping alive. From OCaml’s point of view\nit’s just an 8-byte object. So the garbage collector may not work as hard to clean them\nup.\n\n## Using OxCaml modes to do typesafe borrowing and stealing\n\nWe can improve this situation thanks to\n[functionality](https://oxcaml.org/documentation/uniqueness/intro/)\n[introduced](https://blog.janestreet.com/introducing-oxcaml/)\n[in](https://blog.janestreet.com/fun-with-algebraic-effects-hardcaml/)\n[OxCaml](https://oxcaml.org/documentation/modes/intro/). Instead of always globally\nallocating `Py.Object.t`\n\nvalues with their owned reference and relying on the OCaml\ngarbage collector to clean up these references, we can use two new annotations:\n\n`@ local`\n\nmeans “this value won’t escape this function call.” If a Python API function takes its argument as`@ local`\n\n, OCaml can prove the caller still holds the object for the whole call, so we can skip the incref.`@ unique`\n\nmeans “this is the only handle to the value.” If an OCaml function returns a Python object as`@ unique`\n\n, we know that this value holds a unique reference to the object, so we can likewise hand the handle straight back to Python without the incref.\n\nWe can employ these modes in a few common cases to ensure most of our code is deallocating promptly.\n\n### Borrowing arguments\n\nMost Python API functions exposed by PyOcaml only borrow their arguments. As a result, we\ncan mark them as `@ local`\n\n. So\n\n``` php\nval Py.Int.to_int : Py.Object.t -> int\n```\n\nbecomes\n\n``` php\nval Py.Int.to_int : Py.Object.t @ local -> int\n```\n\nSimilarly, we change how arguments are passed to PyOCaml function implementations: instead\nof giving a `Py.Object.t`\n\n, the argument becomes `Py.Object.t @ local`\n\n. Our C glue code can\nthen safely construct a `Py.Object.t`\n\nvalue (in this case with no finalizer set!) without\nincreasing the reference count: the C function itself borrows the argument object, and\njust passes it on to the OCaml function, borrowed. When the OCaml function returns, we’re\nguaranteed the `Py.Object.t`\n\nvalue we called it with is no longer referenced by any OCaml\ncode / data. For safety sake, we set the pointer stored in the value to NULL.\n\n### Stealing results\n\nWhen invoked, PyOCaml functions construct and return `Py.Object.t`\n\nvalues. We can change\nthe APIs to construct Python callables from OCaml functions and require the OCaml\nfunctions to return `Py.Object.t @ unique`\n\nvalues. As a result, after our C glue code\ninvokes the OCaml function, we know the result value is no longer referenced in OCaml code\nor data, and we can steal the `PyObject *`\n\nfrom that value and return it to the Python\nruntime as-is, without increasing the reference.\n\nWe do, however, need to set the pointer stored in the `Py.Object.t`\n\nvalue to NULL, and\nmake sure the finalizer can handle this properly. If not, the finalizer would decrease the\nreference count of an object to which it no longer owns a reference (the object may even\nhave been deallocated).\n\n### Retaining uniqueness through borrowing\n\nThere’s a problem once you start using `@ unique`\n\nreturns. Suppose you build up a fresh\nPython list (`@ unique`\n\n) and you want to call `.append`\n\non it before returning it. The\ncode you’d like to write is:\n\n``` js\nlet i = Py.Int.of_int 1 in\nlet lst = Py.List.create 0 in        (* unique *)\nlet _ = Py.Object.call_method lst \"append\" [: i :] in\nlst : Py.Object.t @ unique           (* doesn't typecheck! *)\n```\n\nEven though `Py.List.create`\n\nreturns a `Py.Object.t @ unique`\n\n, once we pass it to:\n\n``` php\nval Py.Object.call_method \n   : Py.Object.t @ local \n  -> string @ local \n  -> Py.Object.t iarray @ local read \n  -> Py.Object.t @ unique\n```\n\nthe uniqueness of the value is consumed.\n\nLuckily, borrowing can save us here. The `borrow_`\n\nkeyword means “just loan this to the\nfunction, don’t actually give it up.” As long as `call_method`\n\ntakes its argument locally,\nwe know the argument value can’t outlive the function call, and hence if the argument\nvalue was unique before the function call, it’ll still be unique once the function\nreturns.\n\nThe OxCaml compiler got basic support for borrowing recently, so we can write the code above as:\n\n``` js\nlet i = Py.Int.of_int 1 in\nlet lst = Py.List.create 0 in        \n- let _ = Py.Object.call_method lst \"append\" [: i :] in\n+ let _ = Py.Object.call_method (borrow_ lst) \"append\" [: i :] in\nlst : Py.Object.t @ unique           (* typechecks! *)\n```\n\n### Safe explicit reference counting when you really need it\n\nThe mode system covers the common cases, but sometimes you really do want to manage things manually, for instance when you want to store a Python object in a long-lived OCaml global, or eagerly free a giant data frame before doing something risky. These modes allow you to write code with explicit, manual reference counting management when this would make sense. We provide the following functions:\n\n``` php\nval Py.newref : Py.Object.t @ local -> Py.Object.t @ unique\nval Py.decref : Py.Object.t @ local unique -> unit\n```\n\n`newref` allows you to construct a new global reference when given a local one. `decref`\n\nin turn consumes the uniqueness of a given value, so it can’t be used afterwards, and\nreleases the reference.\n\nMost code wouldn’t use `decref`\n\nexplicitly, since it clutters code and isn’t required (the\nGC finalizer performs a decref when the `Py.Object.t`\n\ngets garbage-collected). But if\nprofiling shows Python objects are retained longer than desired in some code, an explicit\n`decref`\n\ncould solve this problem. An example would be code which constructs some large\ndataframe and then runs some other function which might raise, and wants to proactively\nrelease the dataframe in the error path.\n\nExplicit `decref`\n\ncalls get quite tedious when you have many objects to manage,\nespecially in light of exceptions. We’ve written `ppx_release`\n\nto make all of this\nmore ergonomic. For example, the following code is safe, and doesn’t keep the Python\nobjects constructed by `create_dataframe`\n\nand `create_series`\n\nalive for longer than\nstrictly necessary, even when `do_some_calculation`\n\nraises an exception:\n\n```\nlet%release df = create_dataframe ()\nand series = create_series () in\ndo_some_calculation df series\n```\n\nThis approach allows for type-safe, compiler-checked, code-directed reference count handling. It’s worth noting that Rust gives you similar benefits, but with a key difference: unlike the Rust equivalent, we don’t put the burden of ownership tracking on all users of the APIs. In many cases, code can be written in a more traditional style, with the garbage collector safely taking care of deallocations when it’s safe to do so.\n\nThis connects to [OxCaml’s larger design goal](https://oxcaml.org/) of providing control\nover performance-critical aspects of program behavior, but only where you need it. We\nthink that the resulting improvement in ergonomics is worth quite a lot!", "url": "https://wpnews.pro/news/using-oxcaml-to-implement-type-safe-reference-counting-between-ocaml-and-python", "canonical_source": "https://blog.janestreet.com/oxcaml-typesafe-reference-counting-python/", "published_at": "2026-06-15 23:24:05+00:00", "updated_at": "2026-06-15 23:48:57.615504+00:00", "lang": "en", "topics": ["developer-tools"], "entities": ["Jane Street", "PyOCaml", "OxCaml", "OCaml", "Python"], "alternates": {"html": "https://wpnews.pro/news/using-oxcaml-to-implement-type-safe-reference-counting-between-ocaml-and-python", "markdown": "https://wpnews.pro/news/using-oxcaml-to-implement-type-safe-reference-counting-between-ocaml-and-python.md", "text": "https://wpnews.pro/news/using-oxcaml-to-implement-type-safe-reference-counting-between-ocaml-and-python.txt", "jsonld": "https://wpnews.pro/news/using-oxcaml-to-implement-type-safe-reference-counting-between-ocaml-and-python.jsonld"}}