The buyer of this technology is not shoppers, it's retailers. The measurement of quality is "does it make us more money?" not "does it help me make better buying choices."
Retailers do not want you to make better choices. They want you to buy the widget.
A lot of evidence suggests that also shoppers aren't that interested in making the best choice either. They want to make a tolerable choice with as little effort as possible. There is no basically no consumer market for "power shopping" outside of weird niches like pcpartpicker.com etc.
Is there a way to measure users "making the best choice?" You could measure the amount of time spent comparison-shopping, but most people are terrible at that anyway; it's an acquired skill for sure. Besides a willingness to spend time, it seems like an impossible-to-quantify metric even in the abstract.
Maybe the best proxy metric is whether the customer returns the product. But the store will also be willing to eat more returns on a higher margin item if they make more profit at the end of the day.
I don't think I agree. If I overpay by 10%, I'll never know it and probably wouldn't return it even if I did know--once the shrinkwrap is off, too late. If a superior product exists but I don't find it, by definition I wouldn't know and wouldn't return the thing I did buy.
Cynically, the customer might not know if they overpaid but the retailer doesn’t care about that. Where “making the best choice” actually cashes out is the customer DAU dropping (rare) or product returns increasing.
That's a cynical way to look at it. Most likely the LLM will take a cut of sales and they'd be more or less indifferent who cuts the check. There's a market for this sort of thing. People will go to the best LLM for shopping. If the LLM is a shitty product, people will switch. LLMs are increasingly commoditized.
All you say is true for an aggregator like Amazon. But Amazon is better than Nike.com because as an aggregator they go from 1 to many retailers. LLMs will go from 1 aggregator (Amazon) to many so it will be better. And they don't have to invest a lot in UI/UX as chat is the interface.
Retailers do not want you to make better choices. They want you to buy the widget.
A lot of evidence suggests that also shoppers aren't that interested in making the best choice either. They want to make a tolerable choice with as little effort as possible. There is no basically no consumer market for "power shopping" outside of weird niches like pcpartpicker.com etc.