Our society is built on Trust. You have to trust your bank to keep your money in your name and allow you to withdraw when you want it. You have to trust that the fire truck will come when you have a fire. You have to trust that your neighbours won’t randomly wake up one day become zombies thirsty for your blood.
Same way you ought to trust, but not blind trust. Trust but verify.
I think the self driving game will be won with someone who as a transparency mindset. Here’s the car. Here’s the software, here’s the guarantee that it hasn’t been modified and here’s the extensive tests that we’ve run on it, verified by 3rd parties vouching for its safety and quality.
Same with any technology company. You win by making solid technology backed with real rigorous testing vouched by trusted 3rd parties and general public.
I agree with you. For folks reading along, there is an entire discipline around this stuff, it is call vulnerability analysis and surety systems. There are a lot of good papers published by Sandia National Laboratory on these topics. They were responsible for developing the US surety system around access to nuclear weapons[1]. When I worked at Sun they gave a talk at an e-comerce payment processing forum that Sun was participating in to discuss how you approach the problem of securing something in presence of known and unknown bad actors.
In particular they discussed the systems around banking which prevent your bank from stealing your money from you. A topic that I found quite interesting.
But one of the things that has always stuck with me was the discussion of the trade-off between the "cost of effort" to "actualize" a vulnerability. It is the difference between having something that could be done in theory versus having doing it having a high enough payoff to actually do it. When you look at things like dye packs in money and silent alarms and time locked safes, those are parts of a system that minimize the amount of money you can expect to make off with in a bank robbery. They are part of a surety system that is protecting the money in the bank. And they don't make it impossible to rob the bank, they make the likely-hood that you'll have enough profit from it to risk it low enough that people don't do it.
[1] They are designed, in part, to prevent anyone from detonating a US nuclear device without specific authorization from the President.
Same way you ought to trust, but not blind trust. Trust but verify.
I think the self driving game will be won with someone who as a transparency mindset. Here’s the car. Here’s the software, here’s the guarantee that it hasn’t been modified and here’s the extensive tests that we’ve run on it, verified by 3rd parties vouching for its safety and quality.
Same with any technology company. You win by making solid technology backed with real rigorous testing vouched by trusted 3rd parties and general public.