Many projects require replicatable builds. Generative AI is driven by noise so I am not sure if it has any role in optimization or code gen. I guess we could share an RNG seed with source but honestly current tool chains are too slow as it is and ML would likely make it much worse. I want faster edit/compile/debug cycles and compilers already do insane optimisations. Ken Thompson’s Reflections on Trusting Trust becomes much scarier if your code generation is being driven by a black box of billions or parameters nobody understands. An adversary could embed anything in there.

The machine translation origins of LLMs make them really good at inspecting text and looking for things that look like known bugs. It must be a game changer for security researchers who already know what sort of problems to look for but don’t have enough hours in the day to search the haystack manually. If some of that knowledge was available in a good quality FOSS local code analysis tool it might be irresponsible not to use it. We can’t be dependent on commercial vendors and closed models.