They are not only using human written data any more. Reinforcement learning through human feedback (RLHF) is a big part of it now, that’s the AI running through a problem multiple times with a human picking the best attempt and then using that best attempt as training data going forwards. The model collapse stuff from a few years ago was from an AI repeatedly ingesting its own input with no guidance over many training iterations.
The fingerprinting they’re doing now is meant to help with this. If they see a fingerprint, it was probably not human generated so don’t ingest it.
They are not only using human written data any more. Reinforcement learning through human feedback (RLHF) is a big part of it now, that’s the AI running through a problem multiple times with a human picking the best attempt and then using that best attempt as training data going forwards. The model collapse stuff from a few years ago was from an AI repeatedly ingesting its own input with no guidance over many training iterations.