Friday, February 26, 2010

Mars Sample Return and Cumulative Risk

I listened in to parts of the Decadal Survey's steering committee earlier this week.  A full morning was dedicated to the proposed Mars Sample Return (MSR) mission. (Presentations haven't been posted yet.)  In its current form, the mission would consist first of the delivery of a rover that would collect and cache carefully selected samples.  Four to six years later, an orbiter would be launched that would eventually receive samples from the surface and then return them to Earth.  Two years after that, a lander would deliver a small rover that would fetch the cached samples and an ascent vehicle that would blast off from the surface and deliver the samples to the waiting orbiter.  Total cost for the series of missions is current estimated to fall between $6-7B.  (See Bruce Moomaw's descriptions of the current plan part one and part two or read a recent presentation at http://www.spacepolicyonline.com/pages/images/stories/PSDS%20Mars2%20Li-MSR.pdf.)

The most interesting part came in the discussion by the committee members after the final presentation.  The point was made that at this cost, MSR would be NASA's mission of the decade and therefore could not fail.  Another member described the proposed plan as too fragile because everything must work for the samples to finally return to Earth. 

These comments got me to thinking.  The MSR proposal involves three launches from Earth, four interplanetary cruises, three atmospheric entry and landings (on Mars) or recovery (on Earth), two rovers operating on Mars, a launch from the surface of Mars, and the transfer of the sample in Mars orbit from the Mars ascent vehicle to the waiting orbiter.  I did a simple spreadsheet that looked at the effect of cumulative probabilities.  The challenge is to know what probability of success to assign to each individual element.  Since I don't know, I did a sensitivity analysis that first assumed 99% probability of success for each stage of the missions, resulting in a respectable 87% probability that all would succeed.  But reduce the probability of success for each stage to 97%, and the overall probability drops to 65%.  The odds of overall success drop to less than half if the probability of each element completing successfully drops to 95%.  I also tried a version that assigned different probabilities to different mission stages based on a weak sense of how risky each one might be.


Editorial Thoughts: Unfortunately, assigning probabilities of success to mission elements is tricky.  The goal is to have overall mission success in the high 90 percents.  Those probabilities are modeled, not calculated from statistics.  Except for launches (of which there are many) these are one of a kind experiments.  (According to Wikipedia, the Delta II family had a 95% launch success rate over 300 launches.)  What if Spirit's early memory problems had proved fatal?  What if the fetch rover finds a sand trap it cannot get out of?  What if winds cause a crash landing?  A Mars sample return mission has at least two never tried before stages: launch off the Martian surface into an orbit matching the waiting orbiter and then successful rendezvous and transfer of the samples.  Not all elements of these complex missions can be fully tested on Earth.  There will be, for example, no full end-to-end testing of the Mars Science Laboratory's entry, landing, and descent until the craft reaches Mars.

Balancing this is the careful attention the mission designers put into trying to minimize the chance of failure partially by over designing and testing the systems and partially be operating them conservatively.

The most obvious solution to the problem of cumulative risk for MSR would be to duplicate each element.  That strategy, however, probability would raise the overall program cost to over $10B and perhaps even to $12B. 

An alternative strategy would be an adaptive approach.  Hold off on building the orbiter and Mars ascent/fetch rover elements until you know that the caching rover has succeeded.  Also have the caching rover leave duplicate caches. If the caching rover fails, build and fly a second copy.

Once you know that the samples are waiting to be picked up, fly the orbiter with enough resources to last in Martian orbit for many years.  When building the Mars ascent/fetch rover, buy duplicates of key parts.  Then if the first attempt fails (and possibly loses one of the cached sets of samples), figure out what went wrong and then build a second version.

Designing missions is not my area of expertise.  JPL's engineers know probability statistics far better than I do, and so I'm sure they are looking for ways to address the problem of cumulative probability of success.  They probably have a more clever approach in mind than what I've suggested.  So think of this blog entry as raising a key problem that we haven't heard the solution to yet.

5 comments:

Al Schmitt said...

I have a couple of comments:

1. Why can't the 2011 MSL mission cache the samples? It has a large rover that technically should be able to do the job. That way some of the risk and cost associated with MSR could be mitigated.

2. With a cost of $6-7 billion, would MSR compromise the proposed JEO and Enceladus outer planet missions? I consider those two missions to be of high scientific value, even more so than MSR. Any negative impact on those missions would be unwise, IMO.

vk said...

Having the 2011 MSL cache samples was looked at. Part of the issue, if I recall correctly, was that the caching mechanisms done right are not simple. Another issue was that MSL's drill powders the sample, while you want samples returned to Earth to be solid cores so that you can reconstruct the stratigraphic relationships. The idea was dropped after a review panel concluded that it wasn't worth the effort. If it had been part of the plan from the beginning, it might have been reasonable.

The impact of MSR on other programs depends on when each element is flown and how much other space agencies contribute. Right now, it appears that if the Mars program is funded at current levels, then JEO could be afforded only if other programs such as the lunar program and either the New Frontiers or Discovery program. An Enceladus mission could be afforded only if an additional program was raided (whichever of New Frontiers or Discovery that hadn't already been used for JEO) or reducing the Mars program

There are going to be some tough choices to be made. Too many really high priority missions and a budget that is only about half of what it was a few years ago.

Al Schmitt said...

vk:

"...Right now, it appears that if the Mars program is funded at current levels, then JEO could be afforded only if other programs such as the lunar program and either the New Frontiers or Discovery program."

Is there a phrase missing at the end of the sentence above?

"...There are going to be some tough choices to be made. Too many really high priority missions and a budget that is only about half of what it was a few years ago..."

Do you know when the Decadel Survey intends to make their final selections public? I for one will be anxiously waiting for the results!

vk said...

It should have read:

Right now, it appears that if the Mars program is funded at current levels, then JEO could be afforded only if other programs such as the lunar program and either the New Frontiers or Discovery programs are eliminated or reduced. (The lunar funds plus the funding for either program or half of both would be in the right ballpark.)

The Decadal Survey has a March 2011 deadline for the final recommendations. I think the panels (Mars, outer planets, etc.) complete their reports this spring. I'm not sure when the draft complete report is due. Late Fall?

Anonymous said...

The total cost for the three Mars sample return components would be 10G$ or more, not 6G$. In the current budget climate that will probably kill any other flagship mission, including the compromised Europa mission. Unfortunately, the cheapest and potentially most interesting mission, an Enceladus flyby with sampling, is not likely to beat out the more politically entrenched Mars and Europa missions.