You know, compiler based AI (not just LLMs) seem like an idea that might be pretty interesting. The option to disable or completely omit them from the toolchain would be mandatory however. But, even then we would have to consider how many pieces of software that then rely on this proverbial black box built into compilers would then get integrated into critical supply chain? We soon would have to start running behavior based “background checks” (just an analogy) on our software much like we do with people due to the unpredictability of such complexity.
Side note, I can’t wait to see just how complex the field of game theory gets with AI. I would love to see any video essays on the topic if they do exist already.
Many projects require replicatable builds. Generative AI is driven by noise so I am not sure if it has any role in optimization or code gen. I guess we could share an RNG seed with source but honestly current tool chains are too slow as it is and ML would likely make it much worse. I want faster edit/compile/debug cycles and compilers already do insane optimisations. Ken Thompson’s Reflections on Trusting Trust becomes much scarier if your code generation is being driven by a black box of billions or parameters nobody understands. An adversary could embed anything in there.
The machine translation origins of LLMs make them really good at inspecting text and looking for things that look like known bugs. It must be a game changer for security researchers who already know what sort of problems to look for but don’t have enough hours in the day to search the haystack manually. If some of that knowledge was available in a good quality FOSS local code analysis tool it might be irresponsible not to use it. We can’t be dependent on commercial vendors and closed models.
I do wonder if Ai driven reverse engineering could have reproducible builds based on a similar principle so as to avoid needing to distribute any part of the original program
You know, compiler based AI (not just LLMs) seem like an idea that might be pretty interesting. The option to disable or completely omit them from the toolchain would be mandatory however. But, even then we would have to consider how many pieces of software that then rely on this proverbial black box built into compilers would then get integrated into critical supply chain? We soon would have to start running behavior based “background checks” (just an analogy) on our software much like we do with people due to the unpredictability of such complexity.
Side note, I can’t wait to see just how complex the field of game theory gets with AI. I would love to see any video essays on the topic if they do exist already.
🧐
Many projects require replicatable builds. Generative AI is driven by noise so I am not sure if it has any role in optimization or code gen. I guess we could share an RNG seed with source but honestly current tool chains are too slow as it is and ML would likely make it much worse. I want faster edit/compile/debug cycles and compilers already do insane optimisations. Ken Thompson’s Reflections on Trusting Trust becomes much scarier if your code generation is being driven by a black box of billions or parameters nobody understands. An adversary could embed anything in there.
The machine translation origins of LLMs make them really good at inspecting text and looking for things that look like known bugs. It must be a game changer for security researchers who already know what sort of problems to look for but don’t have enough hours in the day to search the haystack manually. If some of that knowledge was available in a good quality FOSS local code analysis tool it might be irresponsible not to use it. We can’t be dependent on commercial vendors and closed models.
I do wonder if Ai driven reverse engineering could have reproducible builds based on a similar principle so as to avoid needing to distribute any part of the original program