It has become impossible to tell managers mesmerised by artificial intelligence that the tools are not, in fact, helpful. So employees just play along with the fiction to keep their jobs, writes our tech columnist
The amount of text they learn from is so big and their structure is complex enough that they can actually learn underlying patterns about WHY words are usually arranged in a certain pattern.
Quantity of training material doesn’t confer new abilities. It makes the resulting weights more representative of the language of the materials, but it doesn’t give something the text doesn’t have.
This fallacy is why I say philosophy should be more widely taught: the relationship between symbols and semantics isn’t quite so trivial. In the specific context of computational conscience, the Chinese Room is a well-discussed argument that demonstrates how command of language doesn’t necessarily require or produce understanding of the same. We can argue about the implications for human consciousness (I’d rather not), but the critical part is that processing a foreign language doesn’t translate it into mine.
For a more practical example, consider the issue of legal arguments citing made-up or irrelevant precedence cases. It is trivial to check whether a given case reference actually correlates with an actual case. It is critical that your citation both refers to an actual case and correctly reflects the contents. Someone who understands the nature of legal arguments knows why a certain arrangement needs to be an extant item in a finite set of instances of that arrangement (namely, a topically relevant subset of all legal cases in history, which is also a finite set).
Yet LLMs get it wrong. They get the shape right, but the filling is a game of Russian Roulette. When drawing a semantic connection between the current context and a related case, it should be a no-brainer to correctly write down the reference to that related case, but they don’t do that. They draw on the trained set of symbol correlations to produce something likely.
The same goes for essay prompts in exams where a human should recognise that an instruction to include references to Madagascar, in white font on a white background, isn’t actually part of the question but rather a trap to catch blind copy+paste into AI. The AI doesn’t understand that context or that it should disregard that part. It doesn’t actually know why the instruction is there, it just processes it into part of the context.
Funnily, the mistakes seemed similar to mistakes humans usually make, like forgetting carryovers in additions or subtractions.
That is a damning verdict for a machine literally invented for computing. If there is one thing a computer should be good at, it should be the thing it was built for. Carryovers (overflow flags) are part of the most fundamental ALU design. The fact that it reproduces human error shows that it doesn’t actually understand the assignment, it just imitates the training material. If it understood that the reason a certain pattern is there is because the human writing it made a mistake, it should be able to correct it instead.
Otherwise, it is a parrot, or perhaps a really studious child that’s great at imitating adults, without any care for the actual semantics. A computer making as many or even more mistakes than humans is useless (for that task; we agree that they can do some tasks just fine). If AI should be useful universally, it needs to understand these semantics.
Language is a tool for communicating thoughts and perceptions, but that doesn’t work the other direction. Words do not imply reasoning.
And most men in my life regularly spit out false knowledge with a confidence only a mediocre white man and an LLM can have. ;P
Yeah, I wonder what material LLMs developed by companies with white, male CEOs are dominantly trained on…
So maybe trying to somehow decelerate technological progress really can be a reasonable way to take pressure out of that system.
I’d say it’s less about deceleration itself, more about diversifying the efforts and exploring alternate avenues to achieve the things they’re lacking rather than pouring those resources exclusively into LLMs. The deceleration of LLM development doesn’t have to mean a total deceleration of progress.
Quantity of training material doesn’t confer new abilities. It makes the resulting weights more representative of the language of the materials, but it doesn’t give something the text doesn’t have.
This fallacy is why I say philosophy should be more widely taught: the relationship between symbols and semantics isn’t quite so trivial. In the specific context of computational conscience, the Chinese Room is a well-discussed argument that demonstrates how command of language doesn’t necessarily require or produce understanding of the same. We can argue about the implications for human consciousness (I’d rather not), but the critical part is that processing a foreign language doesn’t translate it into mine.
For a more practical example, consider the issue of legal arguments citing made-up or irrelevant precedence cases. It is trivial to check whether a given case reference actually correlates with an actual case. It is critical that your citation both refers to an actual case and correctly reflects the contents. Someone who understands the nature of legal arguments knows why a certain arrangement needs to be an extant item in a finite set of instances of that arrangement (namely, a topically relevant subset of all legal cases in history, which is also a finite set).
Yet LLMs get it wrong. They get the shape right, but the filling is a game of Russian Roulette. When drawing a semantic connection between the current context and a related case, it should be a no-brainer to correctly write down the reference to that related case, but they don’t do that. They draw on the trained set of symbol correlations to produce something likely.
The same goes for essay prompts in exams where a human should recognise that an instruction to include references to Madagascar, in white font on a white background, isn’t actually part of the question but rather a trap to catch blind copy+paste into AI. The AI doesn’t understand that context or that it should disregard that part. It doesn’t actually know why the instruction is there, it just processes it into part of the context.
That is a damning verdict for a machine literally invented for computing. If there is one thing a computer should be good at, it should be the thing it was built for. Carryovers (overflow flags) are part of the most fundamental ALU design. The fact that it reproduces human error shows that it doesn’t actually understand the assignment, it just imitates the training material. If it understood that the reason a certain pattern is there is because the human writing it made a mistake, it should be able to correct it instead.
Otherwise, it is a parrot, or perhaps a really studious child that’s great at imitating adults, without any care for the actual semantics. A computer making as many or even more mistakes than humans is useless (for that task; we agree that they can do some tasks just fine). If AI should be useful universally, it needs to understand these semantics.
Language is a tool for communicating thoughts and perceptions, but that doesn’t work the other direction. Words do not imply reasoning.
Yeah, I wonder what material LLMs developed by companies with white, male CEOs are dominantly trained on…
I’d say it’s less about deceleration itself, more about diversifying the efforts and exploring alternate avenues to achieve the things they’re lacking rather than pouring those resources exclusively into LLMs. The deceleration of LLM development doesn’t have to mean a total deceleration of progress.