Now just get AI pollsters and AI readers and leave us alone.
Ah yes, “synthetic users.” This is being pushed at my job as well. We’re supposed to use AI to design the next feature for our website, then ask AI “users” what they think of it.
That’s not our entire vetting process - it’s supposed to replace someone just writing down an idea and saying “I think this is good.” And I agree that just firing from the hip like that is dumb. We want our product managers to do more research into their ideas before they get greenlit to be built.
The question is whether AI “synthetic users” add anything of value. The team that put this tool into service noted it has a “positivity bias,” aka “you’re absolutely right!” So we feed it an idea we think is good, and it says oh yes it’s very good.
It’s read every customer email we’ve ever received and every user research report ever conducted by our human UX researchers. But it’s still just not that useful. I think AI is very useful for summarization, searching, and collation of information, but this goes beyond that, asking AI to imagine it is a person and then come up with things to say about an entirely novel concept. And AI is not good at that.
You might as well just put all those emails into a hat and pull out random ones. Or maybe categorize them first and pick from the hats your feature falls under.
Try this: ask the AI how useful it is to ask an AI for “synthetic user feedback” and it will probably even tell you why this particular task is particularly stupid for an LLM. Ok, I tried it with Haiku, you might need to follow up with a question that mentions that experience and implementation specifics matter but aren’t going to be in the context window before it will give an in-depth explanation about why this approach is a waste of resources, though using an AI to help summarize the important problem areas users want addressed can work, it just won’t be able to tell you how you did.
I used to work for one of the nation’s largest survey marketplaces. Y’all have no idea how deep this hole goes.
Surveys\polls are largely requested by political polling groups, research teams, and ad agencies. They put those up on an auction block just like ads, and then we would route traffic into it from various places. Mostly the survey takers come from mobile games (take this 3 question survey for 20 Blorp Points kind of stuff) or survey taker apps that give you points for gift cards and such.
So even before bots, most polls are taken by “professional” survey takers who use banks of phones to maximize their point earnings. We spent a lot of energy on “proving” to the survey provider side that real humans were answering, and not using scripts or bots to just rapid finish them (answer B to everything kind of stuff). Using sophisticated bots to randomly answer was super common.
They were super ready for AI. We talked about it everyday, game planned how it would work, designed systems around it. “Synthetic survey” was the buzz word. Why ask humans for answers if the statistics machine can convincingly predict the answer for you? We proposed ideas like generating the prediction fast and early, then using actual polls to adjust the result towards reality over time. We had tools to track people and connect their spending to poll questions so we could ask follow up questions on purchases, to provide “lift” metrics to agencies on if their ads were working. We were working on the “verification can” tech, only it would have been “Answer this 10 question survey to continue watching your movie.”
I was so glad to leave that place. They got bought and consolidated into the world’s largest survey company a year later and they fired everyone else that had been left. All they wanted was the tech and the customers.
Thanks for the inside view. I never knew a lot of the survey takers came from mobile games or that spending could be tracked.
Survey Monkey?
Sounds like something the GOP and O&G industry would do, with all their bots commenting horse dumpings all the time.
“According to the latest polls, Americans favor Republican policies UNANIMOUSLY!”
Just as useless as product reviews on Amazon.
Some products give a sort of cashback promise for a positive review on a card that comes with their product sometimes.
“I should base my surveys about human behavior solely on responses from non-human machines” said… someone, apparently? Damn. 💀
It’s very funny they even bother to do that. Why not just lie from the beginning? Why bother building a bullshit pyramid everyone can smell??
But people could have replied that.!
Every single answer was indistinguishable from that of a human!
Just one more thing losing contact with reality.
Good news, maybe this means people will finally stop trusting polls so those of us who still have some semblance of democracy can go vote for the things we actually want to see changed instead of having our choices prejudiced by polls that tell us we must “strategically vote” so we can’t have nice things.
Voting for the lesser evil is still evil.
nonoNoNoNO
Not voting is voting. No politician is going to agree with you on everything and some are much much worse than others.
This is the hill I die on.
Not voting is voting.
Of course it is. It’s the “I don’t care enough to even choose the lesser evil, so I’m ok with whatever result gets out of the ballot” vote.
Its the hill we all die on, since it affects so many. Even those who cant vote in those elections.
“why do they vote for lizards? cause if they didn’t then a worse lizard might get in”
I could never down vote Douglas Adams.
Having said that, when was the last time you had two candidates with exactly the same policies. Keeping literal Nazis out is a start, then you can participate in primaries, local politics or whatever for faster change. If you don’t participate then you are allowing the dickheads to choose.
PS by you I don’t mean you personally
the establishment dems ran republican endorsements at the DNC…keeping nazi’s out isnt on their platform
Finance fascism.
Then I hope you enjoy the system you have because it will never change.
And not voting changes the system how, exactly?
Not voting makes election fraud much easier and therefore getting people you definitively don’t want in power.
You are not happy who you get to vote for in general election? Why not take part in elections that influence it. Local elections, midterms.
Why not volunteering in campaigns for candidates that you actually support? Maybe even running yourself?
Ajay promote ranked choice voting.
We recently had victories for multiple candidates that establishment didn’t want to win.
And some are almost exactly the same but painted with two different colors of evil. Strategic voting forces you to choose one. If you think strategic voting is the answer, then that certainly is the hill you are going to die on because the false dichotomy of Kang and Kodos is absolutely going to kill you.
Strategic voting at least staves off the worst for a while. It’s not the solution, but it is part of a solution.
There’s no one single thing that will fix everything. Not protesting, not violent action, not voting. They are all part of a whole that is necessary to affecting lasting and positive change. Advocating that people not do one and only do the other lessens all action.
Absolutely agreed, my only point is that people treat it like it’s a victory and celebrate like they’ve won the superbowl, when it’s just death by a thousand cuts. People need to understand that strategic voting is not a victory even when it’s successful, it’s a “we haven’t lost yet”. The fighting doesn’t stop there. There is so much more work to do and the people you voted into office are not going to do it no matter what party they are. The corruption is on both sides of the aisle. The corruption doesn’t care what your personal politics are.
Nothing is a victory by that metric. Nothing is ever perfect, and criticizing the methods instead of the negativity is completely counterproductive.
If you want to make the argument that people shouldn’t count their eggs before they’re hatched, make that argument. Don’t make a different argument then chastise people for not getting what you “mean” instead of what youre actually saying.
Sometimes “we haven’t lost yet” is better than “we’ve already lost.”
What are you suggesting? Because nothing short of nationwide militant revolution is going to change the facts for any country. “Both sides are the same” is the kind of rhetoric that got the US in the shit it’s in now, for example. Yes, the system needs to be completely overhauled but that’s not going to happen overnight. Nobody’s saying strategic voting is the answer, it’s making the best of a bad situation. Sometimes you need to make incremental progress by choosing the least bad option, because the alternative is worse. No, Kamala would not have been the best pick to be US president, but if you are honestly saying she’d have been the same as Trump you either haven’t been paying attention for the last decade or are actively trying to disenfranchise voters. Either way, keep that shit to yourself.
That’s nonsense, you need to keep your militant revolution shit to yourself. Protests and civil disobedience are extremely powerful motivators that can affect real change, yes, but they are not a militant revolution, and there are grassroots and progressive options for democratic change. No, the US may never lose the two-party system, but voting is not just something you do for a president, and it does not always mean simply walking into a voting booth, casting your vote and going home and shrugging if the result isn’t the one you voted for.
Desegregation and women’s suffrage were both accomplished with great effort by accepting neither party’s position on the issues and actively forcing a third option onto the table. This was not accomplished by simply “voting for the democratic party a bunch of times”.
I’m not suggesting violence, I’m saying that’s the only thing that would change things overnight. Lasting change takes time. Desegregation and women’s sufferage didn’t magically spawn a third party, they were both accomplished by years of hard work forcing the two existing parties to acknowledge them as genuine issues. Throwing your hands up and saying “they’re all the same anyway” does nothing but make way for the people working very hard to make things worse.
Try telling that to Alex Pretti and Renee Good.
Good point, I’ll vote for the biggest anti-ICE candidate in the next general election.
As you should.
Even if they don’t promise to abolish the police, “Defund ICE” should be a winning proposal.
There’s is no such thing as absolute good in this world, you will always be choosing the lesser evil
To be fair, they’re suggesting we pick the best available, not the second worst.
Your problem isn’t with stats, polls are still valuable.
Your problem is political think tanks that pay for biased polling that reflects what they want instead of reality. And billionaire owned media presenting those biases stats with a straight face and hoping no one notices.
Imagine your back in college and the water bottle you just chugged had vodka in it.
That’s a bad bottle, but the take away should be “verify it’s water first” and not “never try to drink water again”.
Meaning you shouldn’t disregard all polls, it’s just responsible to take a real.looknamd not just believe headlines or even articles.
Voting for the lesser evil is still evil.
Even if you’ll never vote D in a general, there is literally no downside for voting for the left most candidate in the next Dem primary. Hell, you could even try voting for the left most candidate in the Republican primary instead, I don’t think that would be as effective though.
After all, it’s the first step in Marxism-Lenism:
Marxism–Leninism holds that a two-stage communist revolution is needed to replace capitalism. A vanguard party, organized through democratic centralism, would seize power on behalf of the proletariat and establish a one-party communist state. The state would control the means of production, suppress opposition, counter-revolution, and the bourgeoisie, and promote Soviet collectivism, to pave the way for an eventual communist society that would be classless and stateless.[12]
https://en.wikipedia.org/wiki/Marxism–Leninism
Personally I want to exit ramp before all the Stalin stuff, but you can’t argue that it didn’t work for him.
Yeah, it’s the whole “controlled by the state” thing I’ll never trust about marxism-leninism. You dont get an informed and organized population by subverting them and taking away their mobility.
Authoritarianism doesn’t lead to freedom.
Yeah, Lenin seized power by having all the soviets assassinated by bolsheviks after he lost an election to them. Then the USSR descended into widespread famine, surveillance, and state-sponsored massacres.
It’s wild how many people still swear by marxist-leninism, as if “just try it one more time, this time it’ll work, I swear!” Or even wilder “actually, widespread famine, surveillance, and monopolized violence in the USSR was good!”
And then they just smugly tell you to “read theory” because they assume you haven’t already, because they can’t imagine anyone would actually read it and think critically about it, since they sure as hell didn’t…
The problem with economics as a science is that testing some theories can be very expensive.
I’ve always viewed economics as a pseudo-science.
There might be some ways to apply the scientific method to economic systems, but even if so it would be a soft science at best.
But that’s not even what’s happening in conventional economic theory, and they try to treat it like a hard science.
“The idea behind silicon sampling is simple and tantalizing,” they write. “Because large language models can generate responses that emulate human answers, polling companies see an opportunity to use AI agents to simulate survey responses at a small fraction of the cost and time required for traditional polling.”
Somebody invested money into this company. And there’s at least hundreds, maybe thousands, of other businesses with these asinine ideas about how to use AI. They’re all getting capital from someone who’s supposed to be smart because they have capital. Remember that when llm providers cost correct token prices.
That someone is Peter Thiel according to this: https://corbettreport.com/the-stupidest-poll-of-all-time/
We really need to kick this idea that rich people are smarter. The vast majority were born on third base and think they hit a home run.
I inherited my grandfather’s robber baron fortune, I’m a GENIUS!
The whole venture capitalism thing is bullshit. It’s just vibes and rich man hubris.
Seeing as the vast majority of “polls” I’ve seen in the last 5-6 years have been “this was a poll done online, so we can’t assign any certainty or margin of error, cause we have no idea who actually responded and it could’ve been just like, two dickheads with bots spamming nonsense, but the results were click baity enough for us to run a story” … I don’t see how them cutting out the two dickhead middle-men, and just using their own bots, is really that much different.
The only math class I ever enjoyed was a college statistics class which actually made sense to me. So I spent my life reading polls, always checking the sample size and the margin of error, because I knew how important those are to accuracy, etc. But that same knowledge also served to let me know that modern polls are becoming horsehit.
I remember hearing Rush Limbaugh telling his listeners to either refuse to take polls, or to lie on them and say the opposite. He, and others, taught MAGAs to disrespect polls (cuz polls are the enemy of predatory politics).
Also, many of the “pollsters” are MAGA operatives in disguise. Add unreliable pollsters to unreliable respondents, and you end up with a weird poll that doesn’t reflect reality at all.
I no longer enjoy tracking polls. Too many of them have been games.
Yeah – I think a lot of people who took even just one stats course are in a similar boat. Though I think it’s a bit easier to understand the shift if you frame it within the context of Social Media sites controlling the population’s opinions / propaganda.
Most govts understand at this point, internally at least, that if a message is repeated often, loudly, and it saturates a people’s media, they start to believe it / agree with it. The survey, and the reporting storm surrounding a survey, isn’t so much about showing people an accurate representation of how people’s viewpoints vary, but rather a vehicle for govts/companies to tell people how to think. Sites like Facebook don’t so much as sell advertising, as they sell the ability to socially engineer its users to like your product / political stance: make enough general noise about a niche position, and people will think it’s a majority opinion.
Where the bots get used in the workflow, isn’t really that big of a concern.
Valid. Media Manipulation is the name of the game right now, but that’s so inefficient. You throw out a bunch of propaganda, and hope something gets traction.
Soon, Data Manipulation will be more important. They’ll start tracking all your date points, and soon they’ll find something they can manipulate you over. They’ll find some obscure crime to leverage you with, like a wrong statement on a Federal form interpreted as a deliberate lie, rather than a confused wrong answer. Or they’ll identify patterns, and create laws to make those patterns a crime, and get you on that.
They’ll threaten your family, job, your healthcare, your money, your home, your freedom, and they’ll eventually get you to dance to their tune.
Humans barely answer surveys as if they were human, this is some real derivative shit.
Republicans are going to love this
they love AI generated art/porn festishizing POC.
This is hilarious. And when it inevitably doesn’t work, they will have a human tweak the statistics of the “AI”. The irony of it all.
They could have an AI tweak the statistics if they seem unrealistic
