sanitation@lemmy.today to Programmer Humor@programming.dev · 4 days agoNew compression techniquelemmy.todayimagemessage-square160linkfedilinkarrow-up11.21Karrow-down115file-text
arrow-up11.2Karrow-down1imageNew compression techniquelemmy.todaysanitation@lemmy.today to Programmer Humor@programming.dev · 4 days agomessage-square160linkfedilinkfile-text
minus-squareShellMonkey@piefed.socdojo.comlinkfedilinkEnglisharrow-up38·4 days agoOn a similar note, I saw a story a bit back of someone saving input tokens by feeding the bot an image of a wall of text rather than the text itself and having it read the image via OCR. Satire and reality are too hard too distinguish these days.
minus-squareskisnow@lemmy.calinkfedilinkEnglisharrow-up8·4 days agoI see this one all the time on LinkedIn, the management rubes love it
minus-squareSorryQuick@lemmy.calinkfedilinkarrow-up3·3 days agoSaves input tokens and uses more output because of OCR, and output is priced way higher.
minus-squareRooster326@programming.devlinkfedilinkarrow-up3·3 days agoIt’s an accounting trick. It uses a different budget…
minus-squareboonhet@sopuli.xyzlinkfedilinkarrow-up1·3 days agoAlso if you’re doing anything multi-prompt, that output is re-used as (luckily cached) input tokens for the next prompt anyway lol
minus-squaretopperharlie@lemmy.worldlinkfedilinkarrow-up3·3 days agoI mean, back in my days (👴) people sent a photo pasted into a word document, so things haven’t change that much…
minus-squarebrian@programming.devlinkfedilinkarrow-up1·3 days agothis is the default compaction method in omp now. afaik it’s exploiting the pricing structure of the big providers more than anything
On a similar note, I saw a story a bit back of someone saving input tokens by feeding the bot an image of a wall of text rather than the text itself and having it read the image via OCR.
Satire and reality are too hard too distinguish these days.
I see this one all the time on LinkedIn, the management rubes love it
Saves input tokens and uses more output because of OCR, and output is priced way higher.
It’s an accounting trick. It uses a different budget…
Also if you’re doing anything multi-prompt, that output is re-used as (luckily cached) input tokens for the next prompt anyway lol
I mean, back in my days (👴) people sent a photo pasted into a word document, so things haven’t change that much…
They still do
this is the default compaction method in omp now. afaik it’s exploiting the pricing structure of the big providers more than anything