Need Unbiased and Real Assessment

A developer claims to have built a technology that helps smaller, locally run open-source AI models produce answers comparable to frontier models, tested across more than 30 prompts with and without the technique. The developer reports the results were promising overall, but 2 tests performed worse and 4 showed no improvement, and is asking whether the approach is worth pursuing further or has practical commercial applications.

I've developed a technology that appears to help smaller, locally run open-source AI models produce answers comparable to those of frontier models. I get conflicting answers from LLMs depending what day it is. No one around me like talking about this kind of stuff. I've tested it on over 30 prompts, with and without the technology. The results have been promising overall, although 2 tests performed worse and 4 showed no improvement. Is this worth pursuing further? Could this technology have practical commercial applications? I think it's got a viability but I've been wrong before. Tell me, if I am thinking out of my ass or otherwise. Thanks in advance Comments URL: https://news.ycombinator.com/item?id=50029811 https://news.ycombinator.com/item?id=50029811 Points: 1 Comments: 0