> Prediction markets have also had some high-profile misfires, however — such as giving the odds of a Brexit ‘stay’ vote as 85% on the day of the referendum
I'm continuously annoyed by this. The prediction was 85%, not 100%. I'm guessing if you looked at all the 85% predictions, about 15% of the time the other outcome happened. That's what 85% means, after all.
You can't just round all predictions to the nearest integer and call it a "misfire" when the unlikely events happen. 1 in 7 events happen all the time! The misfire would be if 85% events always happened.
I agree with your logic but perhaps they consider the magnitude of the actual Brexit referendum vote to refute the prediction 'stay' likelihood of 85%. I don't know how close the election was (IIRC it was fairly close), I'm just stating that many people might consider a landslide in the other direction to mean that the prediction was flawed.
OTOH maybe people misinterpreted the 'stay' likelihood itself as the ratio of the predicted popular stay vote and that would explain it too.
It was very close, something like 51:49. The predictions circulating before the vote may have influenced the result. Apparently more Remainers would have bothered to vote if the predictions had not been so clearly in favor of Remain. It's a tragedy, really, that something so important would be decided on so flippantly.
This, by the way, is another weakness (or limitation) of prediction markets. They should probably not be applied to matters where the prediction interacts with the "predictee" lest you enter a "Keynesian beauty contest".
Biased sample here, but I spoke to multiple people with 'bregret' - they told me they only voted for brexit because they thought it wouldn't happen and wanted to stick it to the man. The stupidity of these people boggles my mind, but there you have it.
And nobody predicted a big loss - it was going to be close either way.
The nuance though is the direction in which the market moved. On the day, the market went from 20% for Brexit in the morning to 10% by the evening for Brexit. And this trend had been in place for about 10 days (ie Brexit losing ground). So you'd think that as an event draws closer, the prediction odds would tend towards the outcome, not diverge from it.
One place I'd love to try prediction markets is with product changes undergoing A/B tests. A product manager could submit an experiment and people at the company could bet on what the product change will do to engagement/revenue/growth metrics. This framework could eventually be used to weed out product changes before they are built. It would be particularly useful when the product change is driven by the leadership and most of the rank and file think the direction is wrong.
Enterprise Prediction Markets have been a thing for a long time. There are existing enterprise software vendors [1], it is known that Google has an internal prediction market [2], and there is a wealth of academic literature on corporate prediction markets [3]
This doesn't seem useful. Innovation has an always unpredictable character, in the sense that it might occur in unlikely places. Unforeseen consequences might make workers no better than chance in predicting the outcome of the majority of product changes.
On the other hand, if something which is obviously wrong is being pushed by management despite all evidence and opposition, in all likelihood, that's because neither one has any bearing on the it's advocates ability to push it nor it's willingness to push it. It might be less an issue of right or wrong, and more one of shameless corporate politics.
I'll look a Taleb reference for the innovation bit.
The "guess the number of jelly beans in a jar" game gave its name to one of the startups whose failure disappointed me the most, Beansight. In 2010, it was asking people their opinion on various topics, attributing them experience points when they guessed right, and using this information about the expertise of its users to improve its predictions.
I love the name! Sounds like insight, and when you need to go and clarify the spelling to someone familiar with the company, you have a great story to tell that is completely related to what the company does. Genius.
According to this, after last night's performance, Trump has an 8% probability of becoming the 45th President of the United States of America, while Clinton has a 92% probability.
Robin Hanson proposed a type of government based on prediction markets that he calls Futarchy [1]. Vitalik Butalin has suggested it as a governance structure for DAOs [2].
If you knowingly make a wrong bet, then other people can exploit it and pump money out of you.
But yes, I think any realistic implementation would have to be tightly controlled. With the traders being whitelisted and audited after large bets (to test for conflict of interests), make their bets public, and force them to give reasons for their predictions, etc.
Personally I like the idea of not using a market at all. Have individuals bet with play money, or just assign odds directly and skip betting. Then you can find a set of individuals that are very good at making predictions, and average their predictions together. This is basically what they did in the book "superforecasters" and the CIA now uses something like this. They found it was superior to prediction markets (which themselves were much superior to predictions made by 'experts' with access to classified information.)
Tough players in prediction market will eventually start techniques like this, if they work. But it shows that just the incentive of betting alone isn't enough to get the most accurate predictions from people. You really need to filter for the people who are surprisingly good at it.
* It's not an open panel. You can apply for an account, but you will be screened. This removes the noise from people who just want to try out prediction markets.
* Contestants don't play for real money. I believe this improves predictions by removing biases due to having real financial stakes in the outcomes.
I also like the fact that it's a real prediction market based on trading and not one of those systems where you have to update your predictions all the time. In a prediction market, you just trade when you see a potential benefit, and the power of the market does the rest.
The only issue I have with it is that the site is so damn slow...
If you're not trading for real money then what are you trading for? Wouldn't people end up taking big risks (lowering the overall accurancy) if what they're buying/selling isn't actually worth anything?
Without real money, it's useless imo. At least with something like betfair, one vote = £1 so you only vote big when you know something, which moves the market and incorporates your opinion the appropriate amount.
Quite the contrary, actually. Real money markets give way too much importance to clueless gamblers. Also experience shows that the prospect of losing your rank is a sufficient deterrent to keep people from making large dubious bets.
It's worth pointing out that X% of people betting an event will happen is not the same thing as an event having X% chance of happening. The former is an estimate of occurrence and the latter is a measure of the occurrence. This is a common conflation with statistics I hear when people talk about predictions.
You may be well aware of this, but in case you aren't, that isn't the value that prediction markets give.
The value that prediction markets give is based on the odds people are willing to bet at, not the proportion of people willing to bet in what direction.
If any risk neutral (in terms of currency) individual believes that their predictions are better than the prediction market, and the transaction costs and time costs are low enough, etc., it should be rational for them to participate in it. And if they are better at predicting, the prediction market prediction should improve.
So, yes, prediction markets might not always make the best possible predictions, but if anyone claims to be better than them, it opens the question of why they aren't participating.
So, if I don't have any special knowledge about the possibility of something, and think that a substantial amount of people know better than I do, I generally expect the market's prediction to be better than mine.
(Assuming the people better informed than me make up a reasonably large part of the market, which I think they usually do.)
I'm continuously annoyed by this. The prediction was 85%, not 100%. I'm guessing if you looked at all the 85% predictions, about 15% of the time the other outcome happened. That's what 85% means, after all.
You can't just round all predictions to the nearest integer and call it a "misfire" when the unlikely events happen. 1 in 7 events happen all the time! The misfire would be if 85% events always happened.