engineers are often not aware of basic principles of fail safe design. I mentioned Toyota, and this article confirms it.
Not mentioned in this article is the most basic fail safety method of all - a mechanical override that can be activated by the driver. This is as simple as a button that physically removes power from the ignition system so that the engine cannot continue running.
I don't mean a button that sends a command to the computer to shut down. I mean it physically disconnects power to the ignition. Just like the big red STOP button you'll find on every table saw, drill press, etc.
Back when I worked on critical flight systems for Boeing, the pilot had the option of, via flipping circuit breakers, physically removing power from computers that had been possessed by skynet and were operating perversely.
This is well known in airframe design. As previously, I've recommended that people who write safety critical software, where people will die if it malfunctions, might spend a few dollars to hire an aerospace engineer to review their design and coach their engineers on how to do fail safe systems properly.
One interesting thing I've noticed is that subway systems, long home of one of the more iconic failsafes, the big red lever behind glass connected directly to hydraulic brakes, are moving away from that. An old-style subway (like NYC) has emergency-stop-in-place levers that cause the train to screech to a halt immediately. But automatic train systems, like the Copenhagen Metro, have a similar looking lever that is just a computer signal that signals an emergency condition. The default response (according to the fine-print under it that I recently read) is that the train will continue on to the next station, open its doors, and then hold until further instructions.
That part I assume is on purpose, because even a computerized system could have "stop immediately" as the default policy when the emergency lever is pulled. Would be interesting to read the analysis that led to the decision. My guess is that it's because on-train issues are statistically the most likely emergency situation a passenger would signal (heart attacks, fights, etc.), in which case continuing to the next station (typically 30-90 seconds) where emergency staff can meet the train and access it, rather than stopping in the middle of a subway tunnel or elevated rail segment, is the most sensible policy.
NYC subways have additional safety features - if the track signals indicate stop (due to imminent collision with another train, for example), there are tripcocks. These raise from the tracks, and catch on the bottom of the train, causing it to hard break immediately.
Failsafes don't always fail safely. Take the unmanned Chicago subway train that cruised through the failsafes and crashed into another train, two weeks ago. Another article quotes a CTA official as not knowing how it could have escaped the yard without the brakes tripping.
Many of the newer NYC trains, I am told, are more complicated - an emergency lever pull (someone caught in the doors) should hard-brake the train. If it's travelled more than 1000 feet, it is assumed the train is already in the tunnel, and the lever just signals the conductor, who presumably will radio for help and tell the operator to hold at the next station. I guess this implies that there's a computer inline.
The tripcocks, I hope, are still connected directly to the brakes.
You already have that, it is called the ignition key. The fact that the driver didn't think to either switch off the engine, put the gear in neutral, or slam on the brakes, makes me think he also wouldn't think to press the big red button.
Also pilots get proper training to handle their vehicles, car drivers not so much.
On my (older) Prius, when I insert my mystical key fob into the dashboard and turn on the car, I can't remove it until I've turned the power off: it's held physically in the slot. (I haven't tried yanking hard.) As far as I know, the "On" button is electronic rather than physical. The gearshift is also just sending instructions to a computer: it doesn't even stay in position after you've used it. And I honestly don't know how the break pedal works: it somehow swaps over from magnetic regenerative breaking to traditional friction brakes at some point, but I'm not sure to what degree that's electronic vs. mechanical. (Maybe the parking break is purely mechanical: it's definitely on my "in case of emergencies, try this" list.)
The point is, most of the options you've listed there really may be computer-mediated in modern cars. (And yes, I've heard that there's a strong correlation between unintended acceleration and older drivers, and that a lot of those cases really are driver error. But I don't think you're making that case here.)
The Prius uses hyraulic brakes, which are at times assisted by the electromechanical motor.
The drivetrain spins the electromechanical motor at all times, adding drag. The drag is not just from the added mass of rotation, but also a dynamic resistance caused by electrical properties of the motor being varied in different ways so that the computer can achieve either regeneration (by temporarily changing modes to allow the motor and circuitry to act as a generator, usually during a coast downhill or to a stop), or additional braking (by electrically braking the motor, using the energy stored in the batteries, to add further resistance to the drive train at the cost of heat generation and range reduction).
If a check engine light that has to do with the electromechanical subgroups of your prius comes on (indicating a fault) those systems are disabled, meaning that the car is more or less non-hybrid during those times. Braking will feel stiff, and the car sluggish, but it is by no means dangerous to drive (unless you consider the new learning curve for the cars' performance profile to be dangerous, which it is.)
Also : Your emergency brake is indeed fully mechanical, but on newer models they may be released electromechanically via a command, i'm unsure. I haven't worked on one since the second generation.
p.s. you forgot a sub group. Your steering rack is also electromechanical. One of the first of its' kind in production. Meaning, if you ever experienced a total blackout, your steering would, too, become much more resistant. This , however, isn't considered to be a safety hazard, because at speed the steering rack does little to assist the driver. the forward momentum takes care of that. The steering assistance is mostly there for parking lot situations.
(source : I was at one time a toyota technician, and my back still remembers the recall on first generation prius battery packs, they weighed 124lbs and were way awkward to remove.)
As far as I know, the "On" button is electronic rather than physical. The gearshift is also just sending instructions to a computer: it doesn't even stay in position after you've used it. And I honestly don't know how the break pedal works: it somehow swaps over from magnetic regenerative breaking to traditional friction brakes at some point, but I'm not sure to what degree that's electronic vs. mechanical.
All of these functions are electronic on the Prius (and indeed on every hybrid car that I know of that's on the roads). The balance between regenerative and regular friction braking in particular requires quite a bit of computer code and calibration to get right.
Vehicle tests confirmed that one particular dead task would result in loss of throttle control, and that the driver might have to fully remove their foot from the brake during an unintended acceleration event before being able to end the unwanted acceleration.
Every one of those approaches you suggested are, in many modern cars, fully software driven. And the article even shows an example of how a bug in the software can only be resolved through the exact opposite of what a rational person would do in a crisis.
I think the only actual mechanical failsafe left is the handbrake. Please tell me that's still sacred...
Handbrakes are almost always mechanical cables, but they're almost definitely not enough to stop a car under high engine output. They're mechanical and not power-boosted (see comments on how much force is needed on the brake pedal without power assistance) and plus, most people have at some point driven around for a few miles before they realized that beeping sound was the parking brake stuck on the whole time.
Even if you are able to, fully engaging the handbrake in a car at highway speeds, while the drivetrain is in runaway, wouldn't be helpful.
source: I have tested this at ~30-40 mph and nothing about that experience leads me to believe that it would be safer if I had been going faster, and at full throttle.
Oh, of course - I've left the "emergency 'smell funny' lever" on before (thanks Mitch Hedberg).
So you're probably right, it's a stretch to call the handbrake something useful in emergencies when in reality it probably wouldn't perform that function.
It might be sensible, though, to couple a killswitch to the handbrake, so that engaging it switches the engine off, or cuts off all electronic control, or something.
I don't drive cars, so i have no idea if that would conflict with normal uses of the handbrake. Perhaps it could have a position beyond the normal brake-engaging position that did this? So that if someone panics and yanks on it as hard as they can, they get the result they probably want.
Very well said. The focus on convenience and cool features over safety makes me really sad, and want to force automotive engineers to watch some Alan Kay talks. He loves to talk how people who don't know the history and basics of their craft will arrive at inferior solutions, for example in this one:
http://www.youtube.com/watch?v=FvmTSpJU-Xc
Electric hand brake is very common in modern cars. Luckily most the features you listed are usually implemented in separate ECUs. Neutral probably goes through the gearbox ecu, handbrake through the brake ecu etc.
>Electric hand brake is very common in modern cars.
Yuck!
>Luckily most the features you listed are usually implemented in separate ECUs.
What concerns me would be how the systems handle unexpected inputs.
In the article it notes that the only way to end one unexpected acceleration event was to stop using the brakes. I'm not sure if the vehicle in question has separate controllers, but if it doesn't that's a real concern that unexpected input from one tickles a bug in another.
The previous sentence in the article seems to indicate that a "dead task" in this context will only happen if a specific bit in the controller's RAM is corrupted.
No, you don't. Many modern cars use contactless/electronic ignition keys and start buttons connected to software. Braking is a software function (I.e. ABS) and most Americans drive automatic (software) transmissions.
Are you sure? It seems sensible to me that accelerator inputs would factor heavily into the braking system, so it seems very sensible to me that an unexpected condition in one could translate over to the other - as the article noted the only way to undo one unexpected acceleration condition was to completely remove your foot from the brake pedal. Sounds like cross-over to me...
>you still have a neutral position.
... which is likely just a software input to the transmission computer.
In all likelihood the only non-electronic failsafe is the handbrake, which I still think is a direct mechanical connection in almost all cars.
> In all likelihood the only non-electronic failsafe is the handbrake
Not on my Nissan Leaf. The brake lever is a switch that turns what I assume (based on the noise) is a small motor to engage the rear brakes. As far as I can tell, everything is electronically controlled. Brakes, accelerator, "ignition", "transmission" (both in quotes because the Leaf really has neither), parking brake. If there's a firmware failure, there's not a mechanically-operated fail safe to save me.
I was a passenger in a 2005 Prius going at ~65 mph when the driver tried it. It didn't work. Needless to say, you can switch into neutral when stopped, so I gather there is some speed threshold above which it doesn't let you switch into neutral.
Edit: this made me curious, so I did some cursory research and found this:
According to one commenter, to shift into neutral when driving you can do one of the following:
1. Press the park button
2. Shift into reverse
3. Hold the shifter in the neutral position for 3 seconds
A video posted by a different commenter shows the driver holding the shifter in the neutral position for not quite 3 seconds, but still longer than is required to shift the car into other gears (and if my memory is serving me right, longer than is required to shift into neutral when stopped).
In any case, the most obvious way to shift the car into neutral did not work for us, and it's unlikely a panicked driver would think to try any of the methods listed above.
The cost of the level of training airline pilots get (around 100kUSD) would be prohibitive for individual driving. Also, they fly with an equally well trained colleague who will run the checklists in an emergency and correct their mistakes. I assume private pilots make as stupid mistakes as individual drivers.
Aircraft investigation is really good at overcoming hindsight bias and looking at human factors in a more objective way. What seems absolutely logical for you to type, having read about these incidents before, might not be as obvious to a driver who hasn't read about unwanted acceleration but is suddenly experiencing it.
My car doesn't have an ignition key, it has a button. If you press the button while in motion, I'm not sure what happens, but I'd bet it would ignore it.
2013 Prius here. So, I tried it on the way home. A quick push of the button is ignored, but if I hold it down the car shuts off. It takes power steering with it though, so I don't recommend it under uncontrolled circumstances. I had to come to a stop to restart as well.
Of course, if the computer is busted, I'm assuming the long button press will be sent to /dev/null.
I had my old Caravan chew threw a serpentine belt once...
Getting that boat of a minivan around the next corner in traffic was both entertaining, dangerous, and probably the best upper-body workout I got that year.
A number of cars these days are actually removing the ignition key entirely. Just look up the cars with "push button start" Most of them no longer have key's but instead the keyfob that opens the doors also acts as a signal that it's ok to start the car. It's one big reason that I don't want to buy any car that doesn't require a key. I know they're easy to duplicate, but i don't believe them when they say that the fobs are hard to duplicate.
Yes, better driver training could have made some of these faults less serious. Either by fully braking properly, switching into neutral, or other techniques. That doesn't excuse the faults though.
My wife's old Camry once got its starter motor stuck on – she could remove her key and it kept running until it smoked itself out. Needless to say we don't own that car any more.
Software that ensures safety like this really ought to be mandated to be open-source.
WalterBright wrote: "engineers are often not aware of basic principles of fail safe design."
I would suggest they should not be called "engineers" then. And in many countries, they're not. Part of the problem is that the tech community includes a lot of different people. Some are programmers, some are program managers, some went to engineering school, some are licensed engineers (in some other discipline). In the US, these are all commonly called engineers. Sadly, I think a lot of web programmers just don't know the true scope of the software industry and its practices.
If you want to design/build a bridge, you need a state license and insurance. The software industry isn't regulated like that. Anyone can design the software that controls a car. That's probably OK since web apps are non-critical systems. But I can't help but wonder if net security wouldn't be better if more programmers had better training in recognizing and improving the total impact of a system.
It is only the reputation of the company and potential damages in a lawsuit such as this one that put pressure on the car manufacturer and web-app startup to test their code in depth. Actually, I do wonder how much the US auto safety regulations are involved with firmware--or do they just test the macro behavior of the car?
Failsafe design flaws are not uncovered by testing code.
Failsafe systems are designed not by "the code works therefore it is safe", but by "assume the code FAILS". Regardless of how much testing is done, you still ASSUME IT FAILS AND ACTS PERVERSELY. Then what?
(Note that acting perversely is hardly farfetched in these days of ubiquitous hacking.)
I have a quick question for you that's a matter of personal curiosity and one I think you might be delighted in answering: What sort of failsafes are there in a fly-by-wire system? Is it a matter of redundancy or another mechanism that ensures pilot inputs yield expected outputs?
I've really been enjoying the posts you shared relating to your time in aerospace. I think there are a lot of lessons the entire software industry should learn from...
All I know in detail is the 757 system, which uses triply-redundant hydraulic systems. Any computer control of the flight control systems (such as the autopilot) can be quickly locked out by the pilot who then reverts to manual control.
The computer control systems were dual, meaning two independent computer boards. The boards were designed independently, had different CPU architectures on board, were programmed in different languages, were developed by different teams, the algorithms used were different, and a third group would check that there was no inadvertent similarity.
An electronic comparator compared the results of the boards, and if they differed, automatically locked out both and alerted the pilot. And oh yea, there were dual comparators, and either one could lock them out.
This was pretty much standard practice at the time.
Note the complete lack of "we can write software that won't fail!" nonsense. This attitude permeates everything in airframe design, which is why air travel is so incredibly safe despite its inherent danger.
The shuttles had similar concepts - various flaps had multiple redundant hydraulic pumps to control them so that even if one went nuts and started going in reverse that other pumps would over power it, and the result would simply be slower response times.
Gosh, this is an incredible comment. I see in greater detail what is meant by your illustration of "dual path." I had no idea the systems-level design was so thoroughly isolated.
Thank you very much for taking the time to share and answer my question!
I'd be surprised if there was a single question on a state licensing exam on failsafe design.
The sample tests I looked at had none. The GRE exams I took had none. The engineering courses I took never mentioned it. I don't recall ever seeing an engineering textbook discussing it. I've never seen it brought up in engineering forums or discussions about engineering disasters.
And, I see little evidence of awareness of it outside of aerospace - Toyota, Fukushima, and Deep Water Horizon being standout examples of such lack. You can throw in New Orleans where hospitals (and everyone else but one building) put their emergency generators in the basement. And in a NYC phone company substation was entirely destroyed because a vital oil pump was in the basement that got flooded during Sandy.
I looked into licensing at one point when it seemed like my career would be heading in a different direction. As I understand it, my state defines "engineering" as anything that could affect public safety, and says that all engineers have to be licensed, but there are only exams in certain subject areas.
I think the guiding principle of engineering regulation would lead one to believe that software controls for a car should be covered by licensing, but that this has not occurred in practice due to the regulations not keeping up with technological change.
Everything I learned as an engineering undergrad had an underlying safety mindset. It was made pretty clear that redundancy and safety are foremost in design procedure. If most engineers these days aren't receiving that in their education then we need to drastically shift the education paradigm. As the people designing all these important systems, they absolutely must be focused on safety.
I worked on the 757, and got it hammered into me how to do failsafe design.
I don't know the failsafe design of the 787, but I have faith that Boeing, the FAA, and the aerospace engineers know what they're doing with failsafe design.
WalterBright I have a great deal of appreciation and respect for nearly everything you've said in this thread up to the point where you declare to have "faith" in Boeing and the FAA. Engineering critical systems and faith of any kind don't belong together.
I often point out in aviation-related threads the striking difference between aircraft engineering in the past and today.
In 1989, a DC-10 operating United flight 232 suffered an uncontained engine failure which damaged the tail and disabled flight controls, resulting in 111 deaths (and could have been more, but in a freak of chance, a DC-10 flight instructor was on board and was able to assist the crew in landing, which may have made the difference for the other 185 people on board). In 2010, an Airbus A380 operating Qantas flight 32 suffered an uncontained engine failure which damaged a wing, disabled a hydraulic system and braking systems, and disabled some flight controls while starting a fire. It resulted in... zero deaths.
In the early 1990s, Boeing 737-200 and 737-300 aircraft had a variety of uncommanded rudder movement issues, resulting in at least 157 deaths. In 2012-2013, the 787's battery fires resulted in... zero deaths.
In other words, the main difference between "then" and "now" is exactly the opposite of the usual arguments against modern aircraft development: more recent aircraft, when they have serious issues, result in fewer injuries and deaths than older aircraft when they experienced serious issues.
This track record of improvement gives justifiable faith that modern aircraft development is safer.
I don't think anyone should be especially reassured that no airframes have been lost due to battery fires. If any of the 787 battery fires that occurred on the ground had occurred in flight it isn't clear the aircraft would have survived.
If any of the 787 battery fires that occurred on the ground had occurred in flight it isn't clear the aircraft would have survived.
There was an in-flight fire, on an ANA 787. It made an emergency landing and the plane was evacuated. No lives were lost, and the airframe was not lost.
Even in the 787, the spoilers and horizontal stabilizers can be operated electrically independently of hydraulics and flight computers: http://en.wikipedia.org/wiki/Fly-by-wire.
If you have computer-assisted hill starts, collision avoidance, or computer assisted braking [through ABS, certain traction & stability control systems, etc] your computer has control of your brakes.
If you have range-assisted cruise control, or early collision alert, your computer likely has total control over your throttle.
If you have parking assist, lane departure warning, lane following assist, or electric power steering etc: your computer has control over your steering.
If your car has a DCT: your computer has control over your shifting _as well as both clutches._ -- Meaning you have _no mechanical interface_ to disengage the motor from the transmission. This is similar to your Prius example: the shifter is not mechanical.
On many new cars: there's no ignition key to remove. You likely have a smart "keyfob" that simply needs to be within X-feet of the car, and then you have a push button ignition.
I'm sure there's some override for the button [push and hold for three seconds], but it's still going to go through some electronics to figure that out.
---
The only bit that scares me is that all these systems potentially share the CAN bus with the horrendous "Infotainment" systems that every manufacturer loves to install. shudders
Mfrs have started using LIN for slow-control accessories like power windows, door locks, wipers, etc. A separate CAN for critical functions of the auto, and another (sometimes CAN) system for non-critical things like GPS integration, entertainment, and (apparently) MOSTbus has finally taken off after much teeth gnashing. MOST is a higher bandwidth than CAN bus system designed for 'infotainment' systems in autos. https://en.wikipedia.org/wiki/MOST_Bus
I have to admit though I'm surprised that some of those accessories even require a bus as they're typically operated by simple switches.
I guess door locks, wipers, etc. make some sense: with the prevalence of central locking, as well as wipers that sense rain and things like that.
Windows are a bit odd, as they just need a very simple switch. I suppose having all the accessories on a common bus must simplify the wiring harness though.
---
I'm curious how TESLA integrates their in-dash system with the rest of the car.
I thought their in-dash console could control some rather safety-critical "preferences" of the car ... for instance I thought you could adjust the level of regenerative braking.
(Other manufacturers are also offering "sport modes", etc -- though they're not always controllable through the dash.)
One example of where a CAN/LIN bus connected power window is good is my old Jetta. You could insert the key in the driver door lock and turn it backwards to make the car roll down all of the windows and open the sun roof. As someone else mentioned, you can likely implement features like this via hard wiring, bu then you get massive wiring harnesses.
>I have to admit though I'm surprised that some of those accessories even require a bus as they're typically operated by simple switches.
They certainly don't require a bus in principle. However, if you want to understand both the business case and some very good engineering reasoning behind the use of data buses to control simple automobile accessories you should have a look at the wiring harness for an early 90's Mercedes or similar luxury car. Demand for fancy features drove the number of wires running to and fro to an unmanageable level. Having a mile of wires in a car is expensive for many reasons I'm sure you can imagine, as well as each connection point becoming an opportunity for something to go wrong.
I'm also curious how electric cars in general will manage safety systems going forward, especially in light of this Toyota court decision.
That's why you buy a car with a manual transmission. Those haven't been electronic-ized yet and are unlikely to be as the forces involved in depressing the clutch are substantial and unlikely to go away.
Once the clutch plates are no longer in contact the engine can do whatever it wants but none of that makes it to the wheels. And then you can get the car out of gear to coast, turn the whole car off and then back on.
There's no substitute to having a human being in the loop to execute high-level "executive" functions, especially when things aren't done to the very high standards of aerospace. You know, like cars.
There exist "manual transmission" that lack an operator controlled clutch. DCTs are very much manual transmissions, but many of them rely on a TCM to disengage the clutches.
I do agree that having a hydraulic (or cable)-clutch car eliminates the ECU from a critical path (power to wheels), but there are plenty of electronic-ized manuals.
There are all sorts of advanced DSG / dual-clutched transmissions in many newer cars, but even some older cars like the MR2 and Smart cars have a http://en.wikipedia.org/wiki/Electrohydraulic_manual_transmi... : quite literally a manual transmission where the clutch has been actuated electronically.
I agree. With a Camry, shifting into N would have immediately remedied the problem. I remember hearing reports claiming: "Shifting to N didn't do anything".
The explanation always seemed obvious to me: Non-technical driver experiences UA. In a panic shifts into neutral. Hears engine scream. Thinks, "The car is still going!" and doesn't realize that despite the sound, the car is actually slowing down.
I know this to be a common misconception in my experience. People commonly misinterpret high revs with "going fast".
https://news.ycombinator.com/item?id=6615457
engineers are often not aware of basic principles of fail safe design. I mentioned Toyota, and this article confirms it.
Not mentioned in this article is the most basic fail safety method of all - a mechanical override that can be activated by the driver. This is as simple as a button that physically removes power from the ignition system so that the engine cannot continue running.
I don't mean a button that sends a command to the computer to shut down. I mean it physically disconnects power to the ignition. Just like the big red STOP button you'll find on every table saw, drill press, etc.
Back when I worked on critical flight systems for Boeing, the pilot had the option of, via flipping circuit breakers, physically removing power from computers that had been possessed by skynet and were operating perversely.
This is well known in airframe design. As previously, I've recommended that people who write safety critical software, where people will die if it malfunctions, might spend a few dollars to hire an aerospace engineer to review their design and coach their engineers on how to do fail safe systems properly.