I do think models currently don't have enough source skepticism.
If you look at agent traces when asked to compare two options to help inform a decision, many of the comparison pages cited in research are often hosted by one of the companies being compared; nearly all are AI-generated AEO plays. Not deeply considering the motive of published information is currently a glitch that can be exploited, but the window will close.
I'm sure model providers will set up some crappy pay for play verification system for "trusted" product information, comparisons, and reviews.
Yes 100%. I see this all the time. You’re asking about a product and the LLM will cite a source from a competitor where the competitor will review the source and list a few positives about the product but lots of negatives. Then the LLM uses them in the response. So cheeky
There are a lot of ways to make this a lot better easily. First of all, they could use a blacklist of sites that sell guest posts on adsy/etc.Also, if an article only links to one of the products listed, or only one is a dofollow link, they should also be excluded.
That'd probably cut down on a huge portion of spam by itself.
Sounds like a tricky problem that will get a low-tech solution like a blacklist, whitelist, or chatGPT-approved vendor list.
Or reddit - lies writen by other LLMs or by paid shills.
Reminds me of this ad from the pre-LLM days! https://en.wikipedia.org/wiki/Burger_King_Google_Home_advert...