{"slug": "avoiding-the-corporate-treacherous-turn-crowd-sourcing-design-ideas", "title": "Avoiding the corporate treacherous turn: crowd-sourcing design ideas", "summary": "OpenAI's failure to maintain its non-profit mission lock after the removal of Sam Altman has prompted a LessWrong user to propose a new commercial venture, Prealign, aimed at solving value generalisation and AI alignment. The founder is crowd-sourcing design ideas to ensure the company stays ethical, including an unconventional 'pivot to profit' mechanism where an AI ethics board could impound dangerous research and force the company to monetise safer, prior work instead of fighting profit motives head-on.", "body_md": "OpenAI was founded as a non-profit with a clear commitment to avoiding AI dangers and a unique structure to prevent investor control or excessive pressure from profit motives.\n\nThis [failed](https://en.wikipedia.org/wiki/Removal_of_Sam_Altman_from_OpenAI).\n\nI'm considering [launching a commercial venture](https://www.lesswrong.com/posts/58zFSWp8Tmxij6ckK/value-generalisation-1-a-research-and-deployment-program) (possibly named \"[Prealign](https://www.lesswrong.com/s/EYgCdcxsn73fKWWra/p/uMKGaEKRDpoqnZyBh)\") to solve [value generalisation](https://www.lesswrong.com/s/EYgCdcxsn73fKWWra/p/TZgezuYjkfMQxyqJC) and contribute to AI alignment, while avoiding anything that increases AI capabilities relative to alignment and values.\n\nI want a strong mission lock to keep the company ethical and an AI ethics board with real teeth and power to enforce that.\n\nBut seeing past failures, it's obvious we have to be careful here. So I'm asking for ideas to make sure that \"Prealign\" or whatever the organisation ends up being called, actually does stay aligned.\n\nI've been mainly thinking along conventional structural lines - maybe the ethics board should own the company, different share classes, incorporating in locations where a mission lock in the company's founding documents is respected and enforceable, choosing the right corporate design, etc.\n\nAny specific idea of that sort is of value, as are unconventional or original ideas as well. I'll just share one unusual design thought I had: the pivot to profit.\n\nThe OpenAI situation highlighted that it's not just a question of the leadership holding out against immense financial and social pressures; you have to consider the employees and partners as well. If the company holds out but the employees jump ship [1] to a deep-pocketed partner or rival, carrying their knowledge with them, little will have achieved.\n\nNow, in the normal course of operation, a company focused on deep research will turn up lots of designs, results, and customer needs that are valuable but not worth working on, because the deep research is the focus and priority.\n\nSo I suggest that if the deep research starts looking dangerous, the ethics board has the power to impound it and announce a pivot to immediate profit. The cutting edge research is taken off the table; and the company has to turn instead to monetising all those previous designs, results, and connections.\n\nSo instead of fighting profit head-on, it would go with it. Trading future profits and future immense risks for much more certain short-term cash. Now, people might certainly be disappointed (I know I would be) but it shouldn't provoke the all out fight that shutting down the company would.\n\nHow does that idea sound?\n\nOf course we'd want to only employ moral and dedicated people; but here's an omnipresent self-serving bias in everyone. It's easy to convince yourself (yes, you, yourself, and also me, myself), in ambiguous situations, that things are \"not quite so bad\" and that the AI ethics board are being unreasonable or overcautious. If the design depends on everyone being saintly and resistant to pressure, it will fail.", "url": "https://wpnews.pro/news/avoiding-the-corporate-treacherous-turn-crowd-sourcing-design-ideas", "canonical_source": "https://www.lesswrong.com/posts/XdCgGadph2f25Kya3/avoiding-the-corporate-treacherous-turn-crowd-sourcing", "published_at": "2026-08-14 13:10:18+00:00", "updated_at": "2026-08-14 13:50:06.682986+00:00", "lang": "en", "topics": ["ai-safety", "ai-ethics", "ai-policy", "ai-startups"], "entities": ["OpenAI", "Sam Altman", "Prealign", "LessWrong"], "alternates": {"html": "https://wpnews.pro/news/avoiding-the-corporate-treacherous-turn-crowd-sourcing-design-ideas", "markdown": "https://wpnews.pro/news/avoiding-the-corporate-treacherous-turn-crowd-sourcing-design-ideas.md", "text": "https://wpnews.pro/news/avoiding-the-corporate-treacherous-turn-crowd-sourcing-design-ideas.txt", "jsonld": "https://wpnews.pro/news/avoiding-the-corporate-treacherous-turn-crowd-sourcing-design-ideas.jsonld"}}