OpenAI's SWE-bench harness relies on unisolated host Docker sockets

OpenAI's SWE-bench evaluation harness relies on unisolated host Docker sockets, according to a bug report filed against the project. The report details errors encountered while running the harness on Docker Desktop with 8 CPUs on an M3 chip and a 15.93 GB memory limit, despite 300 GB of available disk space, and states the evaluation failed to run as expected.

Describe the bug Ran into a few errors I am especially not sure why I get the second one, i have 15.93 gb allocated memory and 300gb space on my disk. Steps/Code to Reproduce Using docker desktop with 8 CPUs available M3 chip and 15.93gb memory limit Expected Results evaluation is ran Actual Results