silence7@slrpnk.net to Technology@lemmy.worldEnglish · 13 hours agoAI Companies Are Buying Tons of Old Books Because They're Free of AI Slopwww.404media.coexternal-linkmessage-square66linkfedilinkarrow-up1282arrow-down12
arrow-up1280arrow-down1external-linkAI Companies Are Buying Tons of Old Books Because They're Free of AI Slopwww.404media.cosilence7@slrpnk.net to Technology@lemmy.worldEnglish · 13 hours agomessage-square66linkfedilink
minus-squareDeacon@lemmy.worldlinkfedilinkEnglisharrow-up15·7 hours agoYou can only stave off model collapse so long. It’s only a matter of time before most of the data going in to LLMs came out of one too. Like I’m not even good at math or anything, much less computer science, but that seems obvious and inevitable to me. Maybe someone smarter than me can chime in.
minus-squareUairhahs@lemmy.worldlinkfedilinkEnglisharrow-up7·edit-26 hours agoIt’s kind of sort of already happening. Emdashes are fragments of it, a token saving strategy introduced by already trained on old books in their data sources. Model collapse is inevitable.
minus-squareEvotech@lemmy.worldlinkfedilinkEnglisharrow-up1·10 minutes agoIdk. Prose books generally have tons of em dashes.
You can only stave off model collapse so long.
It’s only a matter of time before most of the data going in to LLMs came out of one too.
Like I’m not even good at math or anything, much less computer science, but that seems obvious and inevitable to me.
Maybe someone smarter than me can chime in.
It’s kind of sort of already happening. Emdashes are fragments of it, a token saving strategy introduced by already trained on old books in their data sources. Model collapse is inevitable.
Idk. Prose books generally have tons of em dashes.