{"slug": "the-power-of-constraint-solvers", "title": "The Power of Constraint Solvers", "summary": "A developer with a career in constraint solving and constrained optimization surveys classical AI techniques — SAT, SMT, constraint programming, and derivative-free/surrogate-based optimization — and demonstrates them by building a minimal-depth 1-bit full-adder circuit from NAND gates. The writeup walks through real-world applications including verifying distributed program specifications for race conditions and deadlocks, generating mechatronic system architectures, and tuning bicycle aerodynamics and boiler modulation ratios.", "body_md": "With my kids[1](#footnote-1) having derailed every single one of my side projects (including this blog), I thought to revisit a topic very dear to me: **constraint solving and constrained optimization**. I already wrote an article about “[Machine Reasoning](https://btmc.substack.com/p/machine-reasoning-the-forgotten-side)”[2](#footnote-2), which is a made up umbrella term for all sorts of classical AI techniques (in contrast to modern [Machine Learning](https://en.wikipedia.org/wiki/Machine_learning)-based approaches), but it was a very high level overview focused on listing different types of constraint solvers that didn’t really provide any particularly useful piece of information. Not very interesting, so here’s a better one.\n\nMy entire professional career and academic research has involved constraint solving or constrained optimization in some capacity. Here are some use cases the tools I worked on have been applied to:\n\n- Verifying that distributed program specifications were free of race conditions and deadlocks, and then generating correct-by-construction program skeletons (containing just the network communication) from those specifications.\n- Generating mechatronic system architectures, including electrical power systems in aircraft and gearboxes in cars.\n- Parameter optimization, including improving bicycle aerodynamics and greatly improving the [modulation ratio](https://homesteadenergy.co.uk/what-is-boiler-modulation-and-how-does-it-save-money/) of a domestic water boiler.\n\nEach of the three bullet points used a different type of solver. The first used a chain of verification tools that ultimately invoked a [Satisfiability Modulo Theories](https://en.wikipedia.org/wiki/Satisfiability_modulo_theories) (SMT) solver. The second used a combination of a [Boolean Satisfiability](https://en.wikipedia.org/wiki/Boolean_satisfiability_problem) (SAT) solver and a [Constraint Programming](https://en.wikipedia.org/wiki/Constraint_programming) (CP) solver. The last one used a custom solver implementing a large portfolio of [Derivative Free](https://en.wikipedia.org/wiki/Derivative-free_optimization) and [Surrogate-based](https://en.wikipedia.org/wiki/Surrogate_model) optimization algorithms.\n\nMost programmers are completely unfamiliar with these sorts of tools and the wide variety of very hard problems they can tackle.\n\nIf you are into game dev you might be familiar with [Wave Function Collapse](https://github.com/mxgmn/WaveFunctionCollapse), which is basically a [Constraint Programming technique](https://en.wikipedia.org/wiki/AC-3_algorithm) applied to the procedural generation of textures and tile-maps.\n\nAll those LLM math proofs coming out are using [Lean](https://lean-lang.org) (a proof assistant) and are probably making very liberal use of the ‘[Grind](https://lean-lang.org/doc/reference/latest/The--grind--tactic/)’[3](#footnote-3) tactic for automated proof search, which uses SMT solver techniques under the hood.\n\nBut this is all a little too abstract, so let’s solve a cool problem together.\n\n# The Problem: An Adder Circuit\n\nHere’s what we want to accomplish: Using nothing but [NAND-gates](https://en.wikipedia.org/wiki/NAND_gate), a “functionally complete” logic gate that can be used to implement any boolean formula, implement a 1-bit [Full-Adder](<https://en.wikipedia.org/wiki/Adder_(electronics)#Full_adder>) circuit.\n\nA NAND-gate takes 2 input bits A and B and outputs one bit C that is the negation of the conjunction of A and B:\n\nA Full-Adder takes 2 input bits A and B plus an input carry bit C<sub>i</sub>, and outputs a sum bit S plus an output carry bit C<sub>o</sub>[4](#footnote-4), behaving as follows:\n\nSo for example, if A=1, B=0, and C<sub>i</sub>=1, then the output is C<sub>o</sub>=1 and S=0. Your CPU and GPU are full of adder circuits (pun intended), though not made up of only NAND-gates.\n\nWe want to use the bare minimum amount of NAND-gates necessary, and we want the configuration with the least depth as an approximation of propagation delay (less delay = better performance). Here’s a solution:\n\n9 NAND gates. Depth 6. Would you be able to come up with this solution by hand? Very likely, it’s not a hard problem. *But can you write a program that finds it*? \n\nAlso, is this the best we can do? If we increase the depth can we get away with less gates? If we use more gates can we pull off a lower depth? Or maybe can we improve both metrics at the same time?\n\n**Give that some thought before proceeding with the rest of the article.**\n\n# Constraint Satisfaction Problems\n\nThe problem above, of finding that Full-Adder circuit, can be modeled as a *[Constraint Satisfaction Problem](https://en.wikipedia.org/wiki/Constraint_satisfaction_problem)* (CSP). Any [NP problem](<https://en.wikipedia.org/wiki/NP_(complexity)>) can be modeled as a CSP because constraint satisfaction is itself an [NP-Complete](https://en.wikipedia.org/wiki/NP-completeness) problem.\n\nIf we add the objective of finding the lowest amount of NAND-gates necessary and/or the lowest depth required, then it becomes a [Constrained Optimization Problem](https://en.wikipedia.org/wiki/Constrained_optimization) (COP). Multi-objective optimization is a can of worms[5](#footnote-5), so we’ll do a little trick to have a single objective: For a given fixed number of NAND-gates, find the lowest depth. If there is no solution, increase the number of NAND-gates.\n\nUnless [P=NP](https://en.wikipedia.org/wiki/P_versus_NP_problem), any algorithm that can solve this problem must have worst-case exponential complexity: **O(2<sup>n</sup>)**. That might sound really bad, but it’s worst-case complexity. In practice, state-of-the-art solvers can eat a problem like the above for breakfast, as you’ll soon see.\n\n# Modeling the Problem\n\nA constraint satisfaction problem consists of decision variables, the domains of those variables (what values they can take), constraints relating those variables, and in the case of constrained optimization, an objective.\n\nWe’ll make use of the excellent Python package [CPMpy](https://cpmpy.readthedocs.io/en/latest/) because it has a super nice API and can invoke a ton of different solvers under the hood.\n\nHere’s our starting point:\n\n``` python\nimport cpmpy as cp\n\ndef main():\n  model = cp.Model()\n  # We'll model the problem here\n  solver = cp.SolverLookup.get(\"ortools\", model)\n  has_solution = solver.solve()\n  print(f\"Status: {solver.status()}\")\n  if has_solution:\n    # We'll output the solution here\n    pass\n\nif __name__ == \"__main__\":\n    main()\n```\n\n[OR-Tools](https://developers.google.com/optimization) is Google’s suite of constraint solving and optimization software. The solver lookup above is actually loading a specific solver from that suite, [CP-SAT](https://developers.google.com/optimization/cp/cp_solver), which has won a [constraint solving competition](https://www.minizinc.org/challenge/) every year for 13 years straight, it’s that good. If you run this program you’ll get the following result:\n\n```\nStatus: ExitStatus.FEASIBLE (0.019812 seconds)\n```\n\nThere are no variables and no constraints, so the problem is trivial. Other statuses we can get include:\n\n- OPTIMAL: The solver found a solution and proved it is the best possible one.\n- UNSATISFIABLE: The solver proved there is no solution.\n- UNKNOWN: The solver was not able to solve the problem in the time given.\n\nNow let us start setting up the problem. First, we’ll just model the inputs and outputs of the system and impose that it does indeed add the bits:\n\n```\na = cp.boolvar(name=\"a\")\nb = cp.boolvar(name=\"b\")\nc_i = cp.boolvar(name=\"c_i\")\ns = cp.boolvar(name=\"s\")\nc_o = cp.boolvar(name=\"c_o\")\nmodel.add(2 * c_o + s == a + b + c_i)\n```\n\n(CPMpy uses a lot of operator overloading abuse to let you write the above)\n\nIf we run the program we’ll get FEASIBLE again and it’ll take about the same time. We can also print the solution it found (within `if has_solution`):\n\n```\nif has_solution:\n  print(f\"{a.name}: {a.value()}\") \n  print(f\"{a.name}: {b.value()}\") \n  print(f\"{c_i.name}: {c_i.value()}\")\n  print(f\"{s.name}: {s.value()}\")\n  print(f\"{c_o.name}: {c_o.value()}\")\n```\n\nI just get false on all of them, which makes sense, 2 * 0 + 0 = 0 + 0 + 0. Try adding a constraint setting `c_o == True` and see what happens (make sure to remove it after).\n\nBut we’re just directly relating the outputs to the inputs here, this tells us nothing useful. What we want is to propagate those inputs through some number of NAND gates, such that, *for all possible inputs*, the output is as expected. We’ll do this in multiple steps.\n\n# Modeling NAND gates\n\nEach NAND gate will be modeled as 4 variables: 2 inputs, 1 output, and one variable tracking the depth where this NAND gate sits:\n\n``` python\nNUM_GATES = 9\n\nclass NANDGate:\n  def __init__(self, id: int):\n    self.id = id\n    self.a = cp.boolvar(name=f\"nand_{id}_a\")\n    self.b = cp.boolvar(name=f\"nand_{id}_b\")\n    self.c = cp.boolvar(name=f\"nand_{id}_c\")\n    # we need to give lower and upper bounds to integer variables\n    # depth can never exceed the number of gates, so max is NUM_GATES\n    self.depth = cp.intvar(1, NUM_GATES, name=f\"nand_{id}_depth\")\n    self.connections_to_a = []\n    self.connections_to_b = []\n    self.connections_from_c = []\n\n  def constrain_value_computation(self, model):\n    model.add(self.c == ~(self.a & self.b))\n```\n\nThe class itself is just a way to bundle the data together, it serves no other purpose.\n\nWe’ll tackle those connection lists later (more variables!). The depth will also come into play later. For now, we just want to add some NAND gates to the problem and enforce that they compute the right value, meaning output C has to be the negation of the conjunction of inputs A and B.\n\n```\nnand_gates = []\nfor i in range(0, NUM_GATES):\n  gate = NANDGate(i+1)\n  nand_gates.append(gate)\n  gate.constrain_value_computation(model)\n```\n\nWe can also print the values of these new variables in the solution, as a sanity check. It’s not really the information we ultimately care about, but it tells us if we screwed something up in the modeling, you can remove this later if you want:\n\n```\nif has_solution:\n  # ...\n  for gate in nand_gates:\n      print(f\"{gate.a.name}: {gate.a.value()}\")\n      print(f\"{gate.b.name}: {gate.b.value()}\")\n      print(f\"{gate.c.name}: {gate.c.value()}\")`\n```\n\n# Modeling Connections\n\nNow we want these NAND gates to connect to each other. Meaning, if we decide that NAND gate 1’s output C is connected to NAND gate 2’s input A, then the values of C and A must be equal. We’ll model connections as follows:\n\n``` python\nclass GateConnections:\n  def __init__(self, from_gate: NANDGate, to_gate: NANDGate):\n    self.c_a = cp.boolvar(name=f\"nand_{from_gate.id}_c_to_nand_{to_gate.id}_a\")\n    self.c_b = cp.boolvar(name=f\"nand_{from_gate.id}_c_to_nand_{to_gate.id}_b\")\n    self.from_gate = from_gate\n    self.to_gate = to_gate\n\n  def constrain_value_propagation(self, model):\n    model.add(self.c_a.implies(self.from_gate.c == self.to_gate.a))\n    model.add(self.c_b.implies(self.from_gate.c == self.to_gate.b))\n\ndef compute_connections(gates: list[NANDGate]):\n  connections = []\n  for i in range(len(gates)):\n    for j in range(i + 1, len(gates)):\n      con = GateConnections(gates[i], gates[j])\n      connections.append(con)\n      gates[j].connections_to_a.append(con.c_a)\n      gates[j].connections_to_b.append(con.c_b)\n      gates[i].connections_from_c.append(con.c_a)\n      gates[i].connections_from_c.append(con.c_b)\n  return connections\n```\n\nEach `GateConnections` instance is actually modeling 2 different connections between 2 gates, one for each of the input ports of the target. We could have kept them separate but this shortens the blog post. Each of the connections from output to input is modeled as a boolean variable signaling that decision. If `c_a` is true, then that means `c` is connected to `a`. Note that an output port can connect to more than one input port, so having both `c_a` and `c_b` be true is perfectly fine.\n\nThe `constrain_value_propagation` method adds constraints that ensure that the value of the output is propagated to the target input if the connection exists.\n\nThere are no backward connections in an adder, so the loop that creates connections (and the respective connection variables) only creates them from lower ID gates to higher ID gates. This cuts a lot of the search space that consists of nothing but swaps of gate IDs, a form of [symmetry breaking](https://en.wikipedia.org/wiki/Symmetry-breaking_constraints).\n\nNow, the connection constraints:\n\n```\nconnections = compute_connections(nand_gates)\nfor con in connections:\n  con.constrain_value_propagation(model)\n\nfor gate in nand_gates:\n  model.add(cp.sum(gate.connections_to_a) == 1)\n  model.add(cp.sum(gate.connections_to_b) == 1)\n  model.add(cp.sum(gate.connections_from_c) >= 1)\n```\n\nThe first loop should be obvious, but for the second loop we’re saying the following: for each NAND gate, each of its input ports must be connected to exactly 1 other port (exactly one of the connection variables involving it must be true), and the output port must connect at least once, but can connect any number of times (at least one of the connection variables involving it must be true).\n\nIf you actually try to run the program now, you’ll get `ExitStatus.UNSATISFIABLE`. The reason is that there’s no way to connect all the ports with 9 NAND gates that can only connect forward: the first NAND gate’s inputs can’t connect to anything and the last NAND gate’s output can’t connect to anything!\n\n# Modeling Mappings\n\nWe “forgot” (pedagogically) about the input and output ports of the adder itself. We need to implement a mapping from them to the NAND gates’ ports. We model it similarly to connections, but a mapping is input to input or output to output. First a small refactoring:\n\n``` python\nclass Adder:\n  def __init__(self):\n    self.a = cp.boolvar(name=\"a\")\n    self.b = cp.boolvar(name=\"b\")\n    self.c_i = cp.boolvar(name=\"c_i\")\n    self.s = cp.boolvar(name=\"s\")\n    self.c_o = cp.boolvar(name=\"c_o\")\n    self.a_mappings = []\n    self.b_mappings = []\n    self.c_i_mappings = []\n    self.s_mappings = []\n    self.c_o_mappings = []\n\n  def constrain_value_computation(self, model):\n    model.add(2 * self.c_o + self.s == self.a + self.b + self.c_i)\n```\n\nThe loose variables that represented the adder are now also bundled together, and we also track their mappings much like we tracked connections in the `NANDGate` instances. There is only one `Adder` instance but now we can pass it around. Our main function now starts as follows:\n\n```\nmodel = cp.Model()\n\nadder = Adder()\nadder.constrain_value_computation(model)\n\nnand_gates = []\n# ...\n```\n\nWith this small refactoring in place, we can implement mappings:\n\n``` python\nclass GateMappings:\n  def __init__(self, gate: NANDGate, adder: Adder):\n    self.gate = gate\n    self.adder = adder\n    self.a_a = cp.boolvar(name=f\"adder_a_nand_{gate.id}_a\")\n    self.a_b = cp.boolvar(name=f\"adder_a_nand_{gate.id}_b\")\n    self.b_b = cp.boolvar(name=f\"adder_b_nand_{gate.id}_b\")\n    self.b_a = cp.boolvar(name=f\"adder_b_nand_{gate.id}_a\")\n    self.c_i_a = cp.boolvar(name=f\"adder_c_i_nand_{gate.id}_a\")\n    self.c_i_b = cp.boolvar(name=f\"adder_c_i_nand_{gate.id}_b\")\n    self.s_c = cp.boolvar(name=f\"adder_s_nand_{gate.id}_c\")\n    self.c_o_c = cp.boolvar(name=f\"adder_c_o_nand_{gate.id}_c\")\n\n  def constrain_value_propagation(self, model):\n    model.add(self.a_a.implies(self.gate.a == self.adder.a))\n    model.add(self.b_b.implies(self.gate.b == self.adder.b))\n    model.add(self.a_b.implies(self.gate.b == self.adder.a))\n    model.add(self.b_a.implies(self.gate.a == self.adder.b))\n    model.add(self.c_i_a.implies(self.gate.a == self.adder.c_i))\n    model.add(self.c_i_b.implies(self.gate.b == self.adder.c_i))\n    model.add(self.s_c.implies(self.gate.c == self.adder.s))\n    model.add(self.c_o_c.implies(self.gate.c == self.adder.c_o))\n```\n\nThese mappings act much like connections, and we can treat them as such:\n\n``` python\ndef compute_mappings(adder: Adder, gates: list[NANDGate]):\n  mappings = []\n  for gate in gates:\n    mapping = GateMappings(gate, adder)\n    mappings.append(mapping)\n    gate.connections_to_a.append(mapping.a_a)\n    gate.connections_to_a.append(mapping.b_a)\n    gate.connections_to_b.append(mapping.b_b)\n    gate.connections_to_b.append(mapping.a_b)\n    gate.connections_to_a.append(mapping.c_i_a)\n    gate.connections_to_b.append(mapping.c_i_b)\n    gate.connections_from_c.append(mapping.s_c)\n    gate.connections_from_c.append(mapping.c_o_c)\n    adder.a_mappings.append(mapping.a_a)\n    adder.a_mappings.append(mapping.a_b)\n    adder.b_mappings.append(mapping.b_b)\n    adder.b_mappings.append(mapping.b_a)\n    adder.c_i_mappings.append(mapping.c_i_a)\n    adder.c_i_mappings.append(mapping.c_i_b)\n    adder.s_mappings.append(mapping.s_c)\n    adder.c_o_mappings.append(mapping.c_o_c)\n  return mappings\n```\n\nNote that we’re also adding mappings to the lists of connections in the gates. That’s because if an input port is mapped, it cannot also be connected to another port, the constraint that is it connected exactly once must also consider the option of mapping it instead (similar logic applies to output ports).\n\nThen we can add the constraints:\n\n```\n# add this before the connection sum constraints so the mappings are included there\nmappings = compute_mappings(nand_gates, adder)\nfor mapping in mappings:\n  mapping.constrain_value_propagation(model)\n```\n\nThe problem is solvable now but the configuration I got most definitely does not compute an addition:\n\nWhat’s going on here?\n\n# For All Possible Inputs\n\nThe issue is this constraint:\n\n```\nmodel.add(2 * self.c_o + self.s == self.a + self.b + self.c_i)\n```\n\nThis constraint just says the solver needs to find a value for these variables that makes the constraint true, it doesn’t say that for all possible inputs the correct sum is computed. So it can simply pick 1 input/output pair where the configuration of NAND gates happens to give the right result, even if the result is wrong for all other inputs.\n\nImagine we were doing something much simpler: implementing AND. When both inputs are True or both inputs are False, AND and OR agree, so an implementation of “OR” would pass as a solution for “AND” for certain inputs. Does this make sense? We need it to give the right solution *for all* inputs, simultaneously.\n\nThat “for all” there in the sentence above is quite important, it is the [universal quantifier](https://en.wikipedia.org/wiki/Universal_quantification) (∀). The problem we’re trying to solve is actually of the form:\n\nWhich is a synthesis problem, it states: there exists some program *p* such that, for all inputs *i*, the program constraints *P(p)* imply the specification constraints *S(p, i)* for those inputs.\n\nThere are various ways to deal with that universal quantifier:\n\n- Some solvers have native support for quantifiers, but performance tends to be rather dubious (it’s a very hard problem in general).\n- Two solvers can work together in a [Counter-Example Guided Inductive Synthesis](https://en.wikipedia.org/wiki/Program_synthesis#Counter-example_guided_inductive_synthesis) (CEGIS) loop, each handling one of the quantifiers.\n- We can just unroll the universal quantifier for all possible inputs.\n\nWe only have 8 possible inputs, so the latter option is easily the best choice for this Full-Adder use case.\n\nBut how do we actually model that? Each variable can only take one value, and we want to propagate 8 different inputs through them. The answer is to create 8 different sets of the “value” variables, and each input will be routed through one of those sets. The connection variables will then route all 8 sets of value variables simultaneously. We’ll start by making the following changes:\n\n``` python\nclass NANDGate:\n  def __init__(self, id: int):\n    self.id = id\n    # Note the 8 as the first input, a is now an array of 8 boolean variables\n    self.a = cp.boolvar(8, name=f\"nand_{id}_a\")\n    self.b = cp.boolvar(8, name=f\"nand_{id}_b\")\n    self.c = cp.boolvar(8, name=f\"nand_{id}_c\")\n# ...\nclass Adder:\n  def __init__(self):\n    self.a = cp.boolvar(8, name=\"a\")\n    self.b = cp.boolvar(8, name=\"b\")\n    self.c_i = cp.boolvar(8, name=\"c_i\")\n    self.s = cp.boolvar(8, name=\"s\")\n    self.c_o = cp.boolvar(8, name=\"c_o\")\n```\n\nAll of the value variables get an 8 as the first argument, turning them into arrays. CPMpy has an array-oriented API, so `a + b` is the same as `[a[0] + b[0], a[1] + b[1], a[2] + b[2], …]` . We do need to change the connection implication constraints because they don’t work directly on arrays, we need to say that we want the conjunction of those expressions, using `cp.all()`:\n\n``` python\nclass GateConnections:\n  # ...\n  def constrain_value_propagation(self, model):\n    model.add(self.c_a.implies(cp.all(self.from_gate.c == self.to_gate.a)))\n    model.add(self.c_b.implies(cp.all(self.from_gate.c == self.to_gate.b)))\n\nclass GateMappings:\n  # ...\n  def constrain_value_propagation(self, model):\n    model.add(self.a_a.implies(cp.all(self.gate.a == self.adder.a)))\n    model.add(self.b_b.implies(cp.all(self.gate.b == self.adder.b)))\n    model.add(self.a_b.implies(cp.all(self.gate.b == self.adder.a)))\n    model.add(self.b_a.implies(cp.all(self.gate.a == self.adder.b)))\n    model.add(self.c_i_a.implies(cp.all(self.gate.a == self.adder.c_i)))\n    model.add(self.c_i_b.implies(cp.all(self.gate.b == self.adder.c_i)))\n    model.add(self.s_c.implies(cp.all(self.gate.c == self.adder.s)))\n    model.add(self.c_o_c.implies(cp.all(self.gate.c == self.adder.c_o)))\n```\n\nNow, we’re ready for the input constraints, we just need to define the truth table:\n\n``` python\ndef constrain_truth_table(model, adder: Adder):\n  table = [\n    (0, 0, 0, 0, 0),\n    (1, 0, 0, 1, 0),\n    (0, 1, 0, 1, 0),\n    (0, 0, 1, 1, 0),\n    (1, 1, 0, 0, 1),\n    (1, 0, 1, 0, 1),\n    (0, 1, 1, 0, 1),\n    (1, 1, 1, 1, 1),\n  ]\n  for i in range(8):\n    array = cp.cpm_array(\n      [adder.a[i], adder.b[i], adder.c_i[i], adder.s[i], adder.c_o[i]])\n    model.add(array == table[i])\n\ndef main():\n  # ...\n  # right before we lookup the solver:\n  constrain_truth_table(model, adder)\n```\n\nWe now have a working full-adder generator. We didn’t write a visualizer so it’s hard to tell on your end (I have a vibe coded one that exports the solution to [netlistsvg](https://github.com/nturley/netlistsvg)), but we could also write an exporter to some logic circuit emulator and test it there.\n\nAnyway, just believe me for now. Run the program and it’ll very quickly tell you the problem is feasible and provide a solution. Here’s what mine looks like:\n\nLooks rather… familiar? It’s the same as the one we were looking for, just shifted around. Running the program a bunch of times seems to generate very similar images each time, but that’s no confirmation. If we ask for all solutions using `solver.solveAll` instead of `solver.solve`, we get that there are 24576 solutions, but many of those solutions are *symmetric* (meaning equivalent), they just have elements swapped around that don’t make any difference to the result. Dealing with symmetries and [graph isomorphisms](https://en.wikipedia.org/wiki/Graph_isomorphism) is out of scope for this article, however.\n\nBut here’s the million dollar question: Is there a way to make a Full-Adder with only 8 NAND-gates? We just have to set `NUM_GATES` to 8 and solve again.\n\nIn just 99 milliseconds on my dying M1 MacBook Pro I’m told in no uncertain terms that nope, you cannot make a Full-Adder with only 8 NAND-gates:\n\n```\nStatus: ExitStatus.UNSATISFIABLE (0.099592 seconds)\n```\n\nAs a reminder, this doesn’t mean the solver failed to find a solution, it explored the entire design space in 99 milliseconds. *There is no solution*.\n\nSome solvers can even produce machine-verifiable (albeit unreadable) mathematical proofs that there really is no solution. If we remove the truth table constraints we get nonsensical solutions again, but now with 8 gates instead of 9.\n\nWhat about depth though? Can we do better than 6?\n\n# Modeling Depth\n\nSpoiler alert: no, 6 is the best we can do, there is in fact only 1 way to make a Full-Adder out of NAND-gates (those 24576 solutions are all equivalent). But let’s get confirmation anyway.\n\nWe already added the depth variables, but we didn’t relate them in any way. The easiest way to fit depth into our existing constraints is to enforce that if two gates are connected, then the depth of the target gate is greater:\n\n``` python\ndef constrain_depth(model, connections: list[GateConnections]):\n  pairs = set()\n  for con in connections:\n    if (con.from_gate.id, con.to_gate.id) in pairs:\n      continue\n    model.add(con.c_a.implies(con.to_gate.depth > con.from_gate.depth))\n    model.add(con.c_b.implies(con.to_gate.depth > con.from_gate.depth))\n    pairs.add((con.from_gate.id, con.to_gate.id))\n```\n\nThen our objective is to minimize the max depth, simple enough:\n\n``` python\ndef main():\n  # ...\n  constrain_depth(model, connections)\n  model.minimize(cp.max([gate.depth for gate in nand_gates]))\n  # ...\n  if has_solution:\n    print(f\"Depth: {model.objective_value()}\")\n```\n\nAnd the result is, sadly:\n\n```\nStatus: ExitStatus.OPTIMAL (0.476214 seconds)\nDepth: 6\n```\n\nDepth 6 is as low as we can go with NAND gates.\n\n# Neat! How do I implement a Solver?\n\nThis is normally the part of the blog post where I tell you how to implement some crazy thing in C, in this case a constraint solver, but sadly this time I won’t. The article is already huge and a competitive constraint solver requires quite a lot of engineering. It wouldn’t be an article, more like a book. Here are some starting points for different solver types:\n\n- [Conflict-Driven Clause Learning](https://en.wikipedia.org/wiki/Conflict-driven_clause_learning) (CDCL, used in SAT).\n- [Arc-Consistency Algorithm #3](https://en.wikipedia.org/wiki/AC-3_algorithm) (AC-3, used in CP)\n- [Simplex Algorithm](https://en.wikipedia.org/wiki/Simplex_algorithm) (used in[Linear Programming](https://en.wikipedia.org/wiki/Linear_programming) ).\n\nMaybe I’ll make a “Crafting Solvers” in honor of Bob Nystrom’s “[Crafting Interpreters](https://craftinginterpreters.com)” someday.\n\n# Conclusion\n\nConstraint Solving is an extremely powerful approach for tackling hard problems that would otherwise require highly-specialized algorithms and heuristics.\n\nDespite having exponential complexity in general, state-of-the-art solvers can chew through (many) real-world problems with ease. Sadly with larger adders it begins to struggle (2-bit is still fair game), though you can always make larger adders out of Full-Adders as opaque blocks to make it tractable (though the optimal solution then involves adding extra gates to handle the carry computations in parallel, to minimize propagation delay).\n\nHere’s a [gist](https://gist.github.com/sirwhinesalot/a05f772e1d0b8a39902af90b8a69c890) with the complete code above.\n\nSee you on the next one.\n\n[1](#footnote-anchor-1)\n\nTotally not my atrocious time management skills, it is 100% the kids’ fault.\n\n[2](#footnote-anchor-2)\n\nNot to be confused with “reasoning” in LLMs.\n\n[3](#footnote-anchor-3)\n\nThe Isabelle theorem prover calls the same sort of tactic ‘sledgehammer’. Now that’s a name.\n\n[4](#footnote-anchor-4)\n\nA half-adder would lack the input carry.\n\n[5](#footnote-anchor-5)\n\nMany solvers do not natively support multiple objectives, and there are many different ways of setting up the problem when there are multiple competing objectives. Further reading: [Multi-objective Optimization](https://en.wikipedia.org/wiki/Multi-objective_optimization#Solution)", "url": "https://wpnews.pro/news/the-power-of-constraint-solvers", "canonical_source": "https://btmc.substack.com/p/the-awesome-power-of-constraint-solvers", "published_at": "2026-10-10 07:54:11+00:00", "updated_at": "2026-10-10 08:10:06.005189+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-research", "developer-tools"], "entities": ["Lean"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/the-power-of-constraint-solvers", "markdown": "https://wpnews.pro/news/the-power-of-constraint-solvers.md", "text": "https://wpnews.pro/news/the-power-of-constraint-solvers.txt", "jsonld": "https://wpnews.pro/news/the-power-of-constraint-solvers.jsonld"}}