Is 12 enough of a sample size to make a statistical judgement? What if there were 20 more which didn’t have a loss of life? Is it then 1/30? What if there were 20 more?
The risk factor is calculated _per mission_ from what I understand. You can have three accidents in a row and nothing for decades but the risk itself can still be 1 in 30.
Your point is fair and and important distinction. I think when estimating a risk factor though, this empirical data, while a low sample size, is a valuable statistic because it's empirical, and not that small of a sample size. Maybe going forward, we have 3 risk levels:
- Historical. Low N as you say. (Even though each mission and spacecraft is different and they're spread out over time, there's value in this)
- Bureaucrat number; absurdly low, but looks good to politicians etc
- Engineering estimate
Yes, actually. This is similar to having a 100 year flood five years in a row. It doesn’t mean that the flood occurs only once in 100 years. _On average_ it’s 1/100 probability of occurring in any given year.
But then, Apollo 1 was after all the first mission on the Saturn V. I think we should assess even its pre-launch risk much higher than the rest of them. Similarly Artemis II has a much higher risk than the subsequent ones will have.
But we’re talking about the risk of a defined set of events that have concluded, not a prediction of the future.
Of course Apollo would have likely had a better average if it had continued, but the risk of the Apollo program, as executed, included things like the first flight of the Saturn V.
If the final empirical mortality result of the Artemis program is 1/30 or less, it will be better than Apollo in that statistic.
A comparison of acceptable mortality is where this discussion began. If Apollo was acceptable at 1/12 (We did it, it was apparently acceptable as the program was not cancelled due to mortality rate) then an acceptable mortality of 1/30 is stronger than Apollo, not weaker.
If I toss a coin four times and it comes up heads three and tails once, it doesn’t mean that there’s a 75% chance that this coin lands heads up. Be careful about conflating risk factor and mortality rate.
But you doing better is independent of the risk involved. The chances of you getting 3/4 heads or better is around 31%, so theres ~69% chance you’ll do worse next time round. Doesn’t change the fact that each coin toss is still 50/50.
> Doesn’t change the fact that each coin toss is still 50/50.
That assumes a fair coin. The fact is you don't know what the odds were of getting heads or tails for that particular coin, all you know is that you got 3/4 heads. And in this analogy, a few hundred coins have every been made, in maybe a dozen styles, none of which have been fair, so you have no good reason to believe that this particular coin should have 50/50 odds of landing heads up.
And it may be, but the important thing is we don't have priors that lead us to expect it to be fair.
We are not dealing with the tautologically true statement that we are assuming the 1/1000 estimate is correct and thus the odds are 1/1000 no matter what we measure. We are dealing with whether or not we can safely reject the hypothesis that the true odds are 1/1000 based on the actual observation of 1/12.
Billions of coins have been minted, and flipped a countless number of times, and we can do the physical analysis of coins such that we know the odds of a coin not being fair, without deliberate intervention to make them such, are astronomically low. As such no one is going to reject the hypothesis that a coin is fair based off of a small number of coin tosses. Hell even if you got 10 heads in a row, while the odds of that sequence is 1 in 1024, we would probably conclude it was luck rather than that the coin was flawed.
For spaceships on the other hand, those priors don't exist. We need to look at just the data from this particular test. The odds of a 1/1000 event occurring in the first 12 attempts is 1 in 84. For rejecting the hypothesis that a mass produced coin is fair, those odds aren't bad; but for rejecting the null hypothesis that the apollo capsules were just unlucky it's way over the reasonable threshold.
The original discussion was about acceptable mortality rate. Artemis's target is 1 in 30, which is better than the empirically observed mortality rate of the actual Apollo missions. The mortality rate is a target. And if that target is an improvement over the actual outcome of the Apollo missions, I think it's difficult to say that the target is weaker than Apollo's, which was the claim up the thread that I was responding to.
The public doesn't care if Apollo had a theoretical risk rate lower or higher than 1/12, what they saw was that 1/12 missions resulted in the death of the crew. The NASA administrator explaining that their estimated risk was only 1/1000 doesn't change the real-world perception or outcome.
The risk factor is calculated _per mission_ from what I understand. You can have three accidents in a row and nothing for decades but the risk itself can still be 1 in 30.