It has become impossible to tell managers mesmerised by artificial intelligence that the tools are not, in fact, helpful. So employees just play along with the fiction to keep their jobs, writes our tech columnist
It’s a text generator that predicts a likely series of words, based on the language patterns it learned from its training material.
Yes and no. While I do absolutely agree about the problem of reliability and false confidence, I do not agree on the general understanding of LLMs. You make it sound like it’s merely a predictor of likely words, like your phone keyboard may suggest words you usually use in sequence. But it is really not that simple. I disagree with the - especially here on Lemmy - very widespread “parrot” analogy in the sense that imho people have a very oversimplified idea of what large language models actually are. They don’t just count statistics of word sequences and then spit out the most likely combination. The amount of text they learn from is so big and their structure is complex enough that they can actually learn underlying patterns about WHY words are usually arranged in a certain pattern. That way, they can absolutely learn concepts and “skills” like logical reasoning, to a certain extent. The exact patterns they learned aren’t even fully understood, but obviously it does work. Even early GPT models learned to do arithmetic - there are publications that show that GPT was able to calculate arithmetic problems that were not in the training data, although it did make mistakes. Funnily, the mistakes seemed similar to mistakes humans usually make, like forgetting carryovers in additions or subtractions. Still, it had apparently learned the concept of arithmetics, just by seeing examples (and maybe explanations - we don’t really know). Modern LLMs absolutely capable of a certain level of logic. You can give ChatGPT a task it’s never seen before and chances are it can solve it. Even if it can’t - that doesn’t make its reasoning qualitatively different from how a human brain works. My argument is that humans learn how to think “logically” by seeing examples too - and they fail at logic all. the. damn. time. Go to a city center and ask people logic puzzles - see how many will be able to solve them correctly (and maybe compare the result to an LLM). I wouldn’t even be surprised if by now, even asking medical questions to doctors would, on average, yield worse results than a state-of-the-art LLM. I did a PhD in computer science a few years ago and I can tell you, I would probably rather trust the latest ChatGPT model on most questions regarding my field than myself. It has just become that incredibly good. I’m absolutely not saying it’s perfect, it still makes stupid mistakes and you have to be aware of that, but so do people. I have not seen a single convincing argument why the concept of logic or reasoning that humans have is fundamentally different from that of LLMs. Most people suck at logic. And most men in my life regularly spit out false knowledge with a confidence only a mediocre white man and an LLM can have. ;P
In all seriousness - LLMs lack sensorimotor interaction with the world, and even if we give them robot bodies, they will never know what it’s like to be human. In that sense, they will not replace us, and they will never be human. I would even agree that they will probably never be conscious (although of course we can’t be sure, especially since we haven’t understood consciousness yet in the first place). Their biases and their “way of thinking” will also most probably always be different from humans. But I do not agree that they cannot be “intelligent” in a logic/reasoning sense and that they are merely “statistical parrots”.
The various executives of the big AI vendors are obviously capitalising on that, stoking the hype with grand and utopic visions because they want to sell their product.
It’s not that it’s useless. It’s that the utility is far less than the lofty promises made by people milking the hype for all it’s worth. And that little utility comes at a terrible social, economic and ecologic price.
Agreed. I’m fascinated by how far AI has become, but it just isn’t where marketers claim it is, and we can’t be sure it will get there anytime soon.
I don’t think we should dismiss the consequences our use of US infrastructure has. Worse yet, I don’t think we should dismiss the political dependencies that creates.
True. Political dependencies are a serious problem and I’m pretty sure they play a big role in why the US actually lets AI companies get away with all the shit that they do.
I’ve seen people trying to argue with experts because “ChatGPT said” because they genuinely do not understand that ChatGPT is a parrot, not an expert.
Yeah. The point is: ChatGPT will not make you an expert. ChatGPT may act as an expert and even provide accurate information, but citing its answers will not magically make you an expert yourself. That is something that needs to be hammered into people’s heads.
I’m not fighting it. I’m trying to pull it out of the pit that the current obsession with imitation has dug. When college students, the next generation or scientists, trades their scientific understanding for the convenience of high-tech parrots, that is the opposite of progress. It’s stagnation, fostered by those few people that are happily trading our future for their present profits.
Fair enough. We need to find ways to use AI as a useful tool, not a tool to make people lazy and stupid. Tbh, I think it will happen. It happened with Google as well. People used to use Google wrong all the time when it was still new. They clicked the first search result and believed absolutely everything it said. I’m kinda old, I remember that time. :P
I’m pretty sure it will be similar with LLMs. By using them, we will learn their limitations and shortcomings, and many of us will learn the hard way. The marketing strategies of AI companies are NOT helpful with that.
We’ll need to understand the social dynamics of such technology to better prevent the disastrous side-effects. We’ll need to compare historical developments with present circumstances and plans to account for future developments. We’ll need to study the psychological effects of interacting with human-like machines to be able to correct course where needed.
I absolutely 100% couldn’t agree more. AI has already had a lot and probably will have even more impact on us, on other technologies like medicine, biotechnology, probably chemistry, material sciences, and loads of other key technologies. We will have a LOT of trouble keeping up with that acceleration of technological progress as a society. In a world where even the existence of the internet doesn’t seem to have been digested fully, AI has the potential to completely wreck everything. And certain people will do all they can to exploit that to gain power and money. And we will have to be faster than them.
In that sense, I agree that
Also, we really, really should spend less time thinking about whether we could and more about whether we should.
is probably right. The EU’s AI Act is a good first step, but the world is in this together, and with the current geopolitical situation I honestly don’t see us working together here for the good of humankind. So maybe trying to somehow decelerate technological progress really can be a reasonable way to take pressure out of that system. I’m not sure it’s possible though.
The amount of text they learn from is so big and their structure is complex enough that they can actually learn underlying patterns about WHY words are usually arranged in a certain pattern.
Quantity of training material doesn’t confer new abilities. It makes the resulting weights more representative of the language of the materials, but it doesn’t give something the text doesn’t have.
This fallacy is why I say philosophy should be more widely taught: the relationship between symbols and semantics isn’t quite so trivial. In the specific context of computational conscience, the Chinese Room is a well-discussed argument that demonstrates how command of language doesn’t necessarily require or produce understanding of the same. We can argue about the implications for human consciousness (I’d rather not), but the critical part is that processing a foreign language doesn’t translate it into mine.
For a more practical example, consider the issue of legal arguments citing made-up or irrelevant precedence cases. It is trivial to check whether a given case reference actually correlates with an actual case. It is critical that your citation both refers to an actual case and correctly reflects the contents. Someone who understands the nature of legal arguments knows why a certain arrangement needs to be an extant item in a finite set of instances of that arrangement (namely, a topically relevant subset of all legal cases in history, which is also a finite set).
Yet LLMs get it wrong. They get the shape right, but the filling is a game of Russian Roulette. When drawing a semantic connection between the current context and a related case, it should be a no-brainer to correctly write down the reference to that related case, but they don’t do that. They draw on the trained set of symbol correlations to produce something likely.
The same goes for essay prompts in exams where a human should recognise that an instruction to include references to Madagascar, in white font on a white background, isn’t actually part of the question but rather a trap to catch blind copy+paste into AI. The AI doesn’t understand that context or that it should disregard that part. It doesn’t actually know why the instruction is there, it just processes it into part of the context.
Funnily, the mistakes seemed similar to mistakes humans usually make, like forgetting carryovers in additions or subtractions.
That is a damning verdict for a machine literally invented for computing. If there is one thing a computer should be good at, it should be the thing it was built for. Carryovers (overflow flags) are part of the most fundamental ALU design. The fact that it reproduces human error shows that it doesn’t actually understand the assignment, it just imitates the training material. If it understood that the reason a certain pattern is there is because the human writing it made a mistake, it should be able to correct it instead.
Otherwise, it is a parrot, or perhaps a really studious child that’s great at imitating adults, without any care for the actual semantics. A computer making as many or even more mistakes than humans is useless (for that task; we agree that they can do some tasks just fine). If AI should be useful universally, it needs to understand these semantics.
Language is a tool for communicating thoughts and perceptions, but that doesn’t work the other direction. Words do not imply reasoning.
And most men in my life regularly spit out false knowledge with a confidence only a mediocre white man and an LLM can have. ;P
Yeah, I wonder what material LLMs developed by companies with white, male CEOs are dominantly trained on…
So maybe trying to somehow decelerate technological progress really can be a reasonable way to take pressure out of that system.
I’d say it’s less about deceleration itself, more about diversifying the efforts and exploring alternate avenues to achieve the things they’re lacking rather than pouring those resources exclusively into LLMs. The deceleration of LLM development doesn’t have to mean a total deceleration of progress.
Same here. It’s the best way to test my current views and beliefs and reshape or refine them.
Ah yes, hello there, fellow German. :D winkt fröhlich auf Deutsch ;)
Yes and no. While I do absolutely agree about the problem of reliability and false confidence, I do not agree on the general understanding of LLMs. You make it sound like it’s merely a predictor of likely words, like your phone keyboard may suggest words you usually use in sequence. But it is really not that simple. I disagree with the - especially here on Lemmy - very widespread “parrot” analogy in the sense that imho people have a very oversimplified idea of what large language models actually are. They don’t just count statistics of word sequences and then spit out the most likely combination. The amount of text they learn from is so big and their structure is complex enough that they can actually learn underlying patterns about WHY words are usually arranged in a certain pattern. That way, they can absolutely learn concepts and “skills” like logical reasoning, to a certain extent. The exact patterns they learned aren’t even fully understood, but obviously it does work. Even early GPT models learned to do arithmetic - there are publications that show that GPT was able to calculate arithmetic problems that were not in the training data, although it did make mistakes. Funnily, the mistakes seemed similar to mistakes humans usually make, like forgetting carryovers in additions or subtractions. Still, it had apparently learned the concept of arithmetics, just by seeing examples (and maybe explanations - we don’t really know). Modern LLMs absolutely capable of a certain level of logic. You can give ChatGPT a task it’s never seen before and chances are it can solve it. Even if it can’t - that doesn’t make its reasoning qualitatively different from how a human brain works. My argument is that humans learn how to think “logically” by seeing examples too - and they fail at logic all. the. damn. time. Go to a city center and ask people logic puzzles - see how many will be able to solve them correctly (and maybe compare the result to an LLM). I wouldn’t even be surprised if by now, even asking medical questions to doctors would, on average, yield worse results than a state-of-the-art LLM. I did a PhD in computer science a few years ago and I can tell you, I would probably rather trust the latest ChatGPT model on most questions regarding my field than myself. It has just become that incredibly good. I’m absolutely not saying it’s perfect, it still makes stupid mistakes and you have to be aware of that, but so do people. I have not seen a single convincing argument why the concept of logic or reasoning that humans have is fundamentally different from that of LLMs. Most people suck at logic. And most men in my life regularly spit out false knowledge with a confidence only a mediocre white man and an LLM can have. ;P
In all seriousness - LLMs lack sensorimotor interaction with the world, and even if we give them robot bodies, they will never know what it’s like to be human. In that sense, they will not replace us, and they will never be human. I would even agree that they will probably never be conscious (although of course we can’t be sure, especially since we haven’t understood consciousness yet in the first place). Their biases and their “way of thinking” will also most probably always be different from humans. But I do not agree that they cannot be “intelligent” in a logic/reasoning sense and that they are merely “statistical parrots”.
Agreed. I’m fascinated by how far AI has become, but it just isn’t where marketers claim it is, and we can’t be sure it will get there anytime soon.
True. Political dependencies are a serious problem and I’m pretty sure they play a big role in why the US actually lets AI companies get away with all the shit that they do.
Yeah. The point is: ChatGPT will not make you an expert. ChatGPT may act as an expert and even provide accurate information, but citing its answers will not magically make you an expert yourself. That is something that needs to be hammered into people’s heads.
Fair enough. We need to find ways to use AI as a useful tool, not a tool to make people lazy and stupid. Tbh, I think it will happen. It happened with Google as well. People used to use Google wrong all the time when it was still new. They clicked the first search result and believed absolutely everything it said. I’m kinda old, I remember that time. :P I’m pretty sure it will be similar with LLMs. By using them, we will learn their limitations and shortcomings, and many of us will learn the hard way. The marketing strategies of AI companies are NOT helpful with that.
I absolutely 100% couldn’t agree more. AI has already had a lot and probably will have even more impact on us, on other technologies like medicine, biotechnology, probably chemistry, material sciences, and loads of other key technologies. We will have a LOT of trouble keeping up with that acceleration of technological progress as a society. In a world where even the existence of the internet doesn’t seem to have been digested fully, AI has the potential to completely wreck everything. And certain people will do all they can to exploit that to gain power and money. And we will have to be faster than them.
In that sense, I agree that
is probably right. The EU’s AI Act is a good first step, but the world is in this together, and with the current geopolitical situation I honestly don’t see us working together here for the good of humankind. So maybe trying to somehow decelerate technological progress really can be a reasonable way to take pressure out of that system. I’m not sure it’s possible though.
Quantity of training material doesn’t confer new abilities. It makes the resulting weights more representative of the language of the materials, but it doesn’t give something the text doesn’t have.
This fallacy is why I say philosophy should be more widely taught: the relationship between symbols and semantics isn’t quite so trivial. In the specific context of computational conscience, the Chinese Room is a well-discussed argument that demonstrates how command of language doesn’t necessarily require or produce understanding of the same. We can argue about the implications for human consciousness (I’d rather not), but the critical part is that processing a foreign language doesn’t translate it into mine.
For a more practical example, consider the issue of legal arguments citing made-up or irrelevant precedence cases. It is trivial to check whether a given case reference actually correlates with an actual case. It is critical that your citation both refers to an actual case and correctly reflects the contents. Someone who understands the nature of legal arguments knows why a certain arrangement needs to be an extant item in a finite set of instances of that arrangement (namely, a topically relevant subset of all legal cases in history, which is also a finite set).
Yet LLMs get it wrong. They get the shape right, but the filling is a game of Russian Roulette. When drawing a semantic connection between the current context and a related case, it should be a no-brainer to correctly write down the reference to that related case, but they don’t do that. They draw on the trained set of symbol correlations to produce something likely.
The same goes for essay prompts in exams where a human should recognise that an instruction to include references to Madagascar, in white font on a white background, isn’t actually part of the question but rather a trap to catch blind copy+paste into AI. The AI doesn’t understand that context or that it should disregard that part. It doesn’t actually know why the instruction is there, it just processes it into part of the context.
That is a damning verdict for a machine literally invented for computing. If there is one thing a computer should be good at, it should be the thing it was built for. Carryovers (overflow flags) are part of the most fundamental ALU design. The fact that it reproduces human error shows that it doesn’t actually understand the assignment, it just imitates the training material. If it understood that the reason a certain pattern is there is because the human writing it made a mistake, it should be able to correct it instead.
Otherwise, it is a parrot, or perhaps a really studious child that’s great at imitating adults, without any care for the actual semantics. A computer making as many or even more mistakes than humans is useless (for that task; we agree that they can do some tasks just fine). If AI should be useful universally, it needs to understand these semantics.
Language is a tool for communicating thoughts and perceptions, but that doesn’t work the other direction. Words do not imply reasoning.
Yeah, I wonder what material LLMs developed by companies with white, male CEOs are dominantly trained on…
I’d say it’s less about deceleration itself, more about diversifying the efforts and exploring alternate avenues to achieve the things they’re lacking rather than pouring those resources exclusively into LLMs. The deceleration of LLM development doesn’t have to mean a total deceleration of progress.