NASA Johnson Space Center
Oral History Project
Tacit Knowledge Capture Project
Edited Oral History Transcript
Robert
C. Doremus
Interviewed by Rebecca Wright
Houston, Texas – 2 July 2008
Wright: Today is July 2nd, 2008. We are at the NASA Johnson Space
Center to speak with Bob Doremus, Deputy Manager of the Space Shuttle
Safety and Mission Assurance Office. The interview is being conducted
for the JSC Tacit Knowledge Capture Project for the Space Shuttle
Program. Interviewer is Rebecca Wright assisted by Sandra Johnson.
Thanks for taking time today. We’d like for you to start by
telling us how you first came to work for the Space Shuttle Program,
and then how your duties evolved.
Doremus:
For the Space Shuttle Program I joined in May of 2004. I had previously
worked in the Orbital Space Plane Project. I worked there for about
ten months, a very brief and intense exposure to program and project
management. Prior to that, from 1984 to 2003, I was a flight controller
in Mission Operations in the Systems Division in the Mechanical Systems
Group. I had started in 1984. I had graduated from Texas A&M University
with a mechanical engineering degree in 1982 and had worked on the
South Texas Nuclear Project for Bechtel Energy Corporation for a year
after college. Then in ’84, I was hired by Ford Aerospace to
come work here as a flight controller.
My flight control experience, you can see up on the wall I have pins.
There’s 51 of those. I supported 51 missions as a flight controller.
It was all in the Mechanical Systems Group. We had—we, I’m
speaking as we the Mechanical Systems Group—had responsibility
for all the mechanisms on the Orbiter, the structure. We had responsibility
for auxiliary power units, which are these little gas turbines that
burn rocket fuel that provide power for the hydraulic system that
the Orbiter needs in order to operate like an aircraft. We had responsibility
for the landing gear, the brakes, the nose wheel steering, the drag
parachute.
Hydraulics system does a whole lot of things, runs all the aerosurfaces,
and works the brakes and nose wheel steering. The mechanisms included
payload bay doors, external tank doors, vent doors, Ku-band antenna.
We also had responsibility for the docking mechanism when that was
added. We had responsibility for crew systems, all the stuff the crew
used on orbit that would frequently break down, the cameras. In fact,
it was usually pretty terrifying when we’d hear a call about
a camera because those were tricky.
In all the areas, being a flight controller you’re expected
to be an expert in all those systems. And that’s the minimum
expectation, is that you have expert level knowledge. Then in order
to operate as a member of the flight control team you need communication
skills. You need to understand how to work as a member of a team and
how to communicate effectively because your expertise doesn’t
do anybody any good if you cannot pass that on in an understandable
way to the rest of the team. You learn how to do that very quickly.
I began work in ’84, and I supported the first mission in 1985,
STS 51-D. OJT [On-the-job training], my first certification was [STS]
51-J, which was a classified mission for the Department of Defense.
I worked five or six of those. I don’t remember exactly. But
those were really different also.
Working as a flight controller, what was that like? It surprised me
coming from Bechtel to NASA, the office environment itself. I felt
like I’d stepped back in time from a modern up-to-date cubicle
environment to old-fashioned desks and binders and things like that.
But the training was just outstanding, and the way we trained was
through personal study. Studying drawings, studying briefing materials
about all the different systems and about spaceflight, about how launch
and landing and orbit worked. Also we learned from our fellow flight
controllers. The senior flight controllers would train the junior
flight controllers.
Then a huge component of the training was integrated simulations,
which took place in Mission Control [Center]. We would go into Mission
Control, and the flight control team would be on console. The flight
crew, the astronauts, would be in one of the simulators training,
and there would be a computer driving the data that the flight crew
saw and the data that the flight controllers saw on console. These
very devious-minded training personnel would be putting malfunctions
into the system so that we saw the same data at the same time and
practiced how to handle situations in flight.
We spent almost half our time on console training. So you’re
in an office environment half the time, and the other half the time
you’re over there in Mission Control. I will never forget the
first time I went in Mission Control. You get this sense when you
go in there. The Mission Control that I’m referring to—this
is 1984, this is the same Apollo era control room that we used in
the early part of the Shuttle Program. The first thing you notice
is there’s a lot of people, you can feel the intensity, because
there’s a lot of people working. But you’re struck by
how quiet it is. There’s very little sound. That’s really
a false impression. Because once you put a headset and plug into the
coms system, you hear a lot of sound. You hear more sound coming at
you than you can handle really. You hear the crew’s voice, you
hear the voices of other flight controllers—controllers that
are on your team, controllers that are on other teams. You have to
be able to sort all that out and learn what’s important and
learn what’s not. It blew me away when I plugged in. “Oh
my gosh, how do I separate all that?”
So you start out—the flight control team is very structured
and very organized. It is a team. The leaders are in what we call
the front room, which is the room you see on TV. The consoles are
there. Every person there is leading a team comprised of multiple
individuals in what we call the back room, which is where the MPSR
is, Multipurpose Support Room I think is what the acronym means. You
start off as a MPSR person in the back room, and you work your way
up through various certifications until you reach a point where you
are able to take control of the team in the front room.
Training—you have mandatory reading materials and mandatory
number of simulations that must be completed. The whole time you’re
doing that, the instructors and the senior flight controllers are
watching you. When you feel like you’re ready, and they feel
like you’re ready to become certified, you get a very special
simulation that was designed just for you to test your ability. They
call that the certification sim [simulation]. And it’s a tough
exam. What’s really neat is you go in there and you’ve
trained and you’ve learned your systems, and you get this feeling
that, "Bring it on, I want to see what you got, I’m ready."
That is exactly the kind of feeling that you have to have when you’re
a flight controller. That you have to be prepared and confident, because
what you see in Mission Control is usually really different from what
you saw in the sims, but you’ve learned in the simulations how
to solve problems, how you fit in with everything else that’s
going on. I had to know when mechanical systems were important and
when they weren’t. It was equally vital to know both those things.
I had to know when to shut up and let the people with the important
stuff talk.
So yes, it was a job I really enjoyed. It kept me there from ’84
to the middle of 2003. I really thoroughly enjoyed it. The systems
for which we had responsibility evolved, changed. We went through
two really horrific tragedies that shaped us as well, the [Space Shuttle]
Challenger [STS 51-L] disaster and the [Space Shuttle] Columbia [STS-107]
accident. It’s funny how we ended up feeling really responsible
for that. Because you have a feeling when you’re a flight controller
that, "Nothing’s going to happen on my watch." You
really zealously guard the crew’s safety and the safety of the
vehicle.
The feeling you get when you see, for example, a heater that’s
failing. Heater operations are something we watch. The environment
of space is very hostile to a lot of things. If there’s a fuel
line that has to be kept warm or it’ll freeze and damage the
fuel line, damage the system, or have that system be unavailable,
you’re watching that closely. If you can see that heater is
failing and tell the crew switch to another heater, and you see the
backup heater start to work and the system is working again you feel
like, "Wow, I just"—that little call that we were
able to make helped save a multimillion-dollar piece of hardware and
probably justified our salary for the year.
So yes, it was a very good feeling, very rewarding feeling, and it
was a very humbling job because it was easy to make mistakes and find
out we were wrong. I’m sure there are still indentation marks
on the console where I pounded my fist, from when I pounded my fist
and said, "That will never happen again. I will never make that
mistake again."
So let’s see. I was able to obtain certification and move into
the front room for STS-26 in 1988 in September. That was really a
great feeling. That was my first time in the front room on console,
and eventually—as a flight controller the greatest responsibility
for our particular discipline was during the ascent and entry phases.
That was the highest level of certification you could attain, and
I was able to attain that in 1990, a couple years later. So I worked
orbit in the front room for the first several flights and then ascent
and entry.
What happened, after Challenger we had several people leave the area,
and we were short on flight controllers. So they had to accelerate
the training of some folks. We had just certified someone in April
of 1988, and that individual left. Flight controllers get hired for
other positions, and he went to another position and got hired. So
they approached me and they said, "Well, now it’s June,
and we’re launching in September, and we want you to get ready
and be trained to work in the front room." I said, "Wow,
are you sure? That’s really accelerating everything." My
boss at the time told me, said, "We will make every class available
to you, and we will make all the simulations available to you. But
it will be up to you to be ready in three months to work in the front
room." I was green as all get-out when I finally made it there,
but I was able to do it. What a great feeling.
Wright:
What a great flight to be able to be on.
Doremus:
Absolutely. That was so cool. And my oldest daughter was born two
months later, two months after that. But yes, it was really cool.
So two years later, I told you, working launch and entry. When you’re
working a launch, it’s game on. Any time you’re on console,
but particularly then. At the time the director of mission operations
was Gene [Eugene F.] Kranz, and of course we really looked up to him.
He would sit in the viewing area right behind our console. Our mechanical
console was in the very back of the flight control room. I used to
call it the cheap seats. The back left. It was a great vantage point.
He would sit right behind us and be listening to the voice loops.
That was just a little extra incentive to do your job well, because
he’s right back there, and wanting him to think that you were
worthy of sitting out there. That was cool.
The other thing that used to blow my mind when I was sitting there
was I would look at the console hardware. This is in the 1980s and
technology had advanced since the mid ’60s, but there we were
looking at black-and-white screens and plots where it would plot a
single character on a black-and-white screen to show us the trends
in different pressures and temperatures. I would look at the buttons
that we would push, and it was all ’60s technology. So on one
hand you’re looking at it and saying, "You know, this stuff
is old." On the other hand, you’re looking at it and you’re
saying, "This is where they sat when they landed on the Moon,
this is where they sat during Gemini, and what am I doing sitting
here? How in the world is it possible that I’m sharing this
console with those men that did that?" So that was really cool.
But again, you realize, "Hey, by the way, there’s spaceflight
going on." You’re able to really snap to it. So anyway,
it was really fun working missions. It was great.
I really enjoyed that. We had several big problems that we had to
work. One of the fun things we got to do—and I know it sounds
like it’s all fun, and it all was—one of the particularly
fun things that we got to do was working in-flight maintenance procedures
or IFMs, and what those really were is something would break on the
ship where the crew could access it, and so we would utilize the tools
available onboard to develop a procedure for them to use to fix it.
That was a lot of fun. That was great. We had to put an official procedure
and uplink it to the crew. And sometimes it was rather complicated,
like building a power cable when the refrigerator they were using
for some science experiments failed, and working around that. Or building
a data transfer cable for a payload data unit that was not transmitting
the data, for which we had launched and gone on orbit for, properly
to the ground. Fixing that was very rewarding too, being able to figure
out an alternate way to save the data.
Sometimes it was really simple, like one time a Japanese Spacelab
flight. For some reason, they had brought newts on orbit to study
whatever one needs to see about their physiology in a space environment,
and one of the poor creatures died. So what do you do with a dead
newt on orbit?
Wright:
Not a common question we ask very often, but that’s a good one
to know.
Doremus:
We were asked that question.
Wright:
What do you do with a dead newt?
Doremus:
What do you do with a dead newt? We had to develop a dead newt disposal
IFM. So I’m very proud to have been part of that. I actually
made a change to the procedure too. Because what we did is we told
them to put the newt in a Ziploc bag and double-bag it and put it
in the trash. We did a drawing showing how exactly to do that. I remember
changing on the drawing—there was a picture of the newt, and
I remember putting a cross, an X over his eye to show that he was
actually deceased to go in there.
But yes, that’s some of the stuff we would do. That was it.
It was really fun during STS-66 on entry. We had what looked like
a problem in an APU [Auxiliary Power Unit] fuel line that could have
been—APU, this is the gas turbine I talked about that burns
rocket fuel. That rocket fuel that it burns is hydrazine, and there’s
a reason we use it for the APU. It’s a really good fuel. It
just packs a lot of energy per unit mass. But the very thing that
makes it a good fuel makes it really bad if any of it should leak
out and get onto a hot surface, because it’s very flammable.
It tends to react with anything it touches. It may even react with
itself, it’s so volatile.
We saw an indication really similar to one that had happened on STS-9
where two APUs had leaked fuel and had blown up—after landing,
thank God. We saw a similar indication during STS-66 entry that there
might be a fuel leak as well, and we watched that the whole time,
and fortunately it ended up being just one temperature sensor indicating
a decrease, which, not to get too technical, but a decrease in temperature
can be because fuel is leaking onto a line and flash vaporizing in
a vacuum environment. That’s what the liquid will tend to do,
and that takes energy away from the line and brings the temperature
down.
We were watching that, and it looked awful similar to what had been
seen prior to landing on STS-9. So we watched it all the way down,
and worried about it all the way down, and actually shut the APU down
early, but what they train you to do in Mission Control is you don’t
take action based on just one sensor. You make sure there’s
multiple sensors that agree with what you’re seeing, because
it could be a failure of a single piece of instrumentation. That’s
what it was we learned after the flight.
But again the training was absolutely right. That was the right thing
to do, so we worried about it all the way down. But the crew never
had to. They got to fly and not be distracted by something that did
not need their attention.
Wright:
That sounds like one of the areas that we certainly are interested,
the best practice of knowing that that’s one of those elements
to do. While you were thinking that, I wanted to ask you, because
you had mentioned about you have to basically train to know what’s
important and then what’s not. Are there some things you can
share with us to help us gauge how you know that? How do you determine
that?
Doremus:
Yes, we saw people struggling with that. Let me define what I mean.
In 1995, I became the lead for my group, and I served in that capacity
until 2003. So I was lead for a group of flight controllers. It was
like the player coach. Because you work missions, and you’re
everybody’s boss, and you’re responsible for their training.
So we’re watching people struggle, and they don’t seem
to be understanding too well what the big picture is and how things
like that fit in. So what we did is I got the senior controllers together,
and we sat down and made a set of expectations for every position.
It was things that are along the lines of, "Know your system
and how it fits into the big picture. Try to get inside the mind of
the Flight Director. Understand what he or she needs to know at a
given situation. Understand what the other controllers’ priorities
are." We just sat down and wrote these things down.
In the mechanical systems group, there were five positions that all
required certification. So you worked your way up through all those
five certifications, and we defined those expectations at each point
and gave them to the people. I remember telling a young controller
that we just hired, I’d say, "Here are the answers in the
back of the book like you have in your textbook. Read these and understand
these."
It helped. It really did. Why not give them the answers, because they
will face the exam. I really did not enjoy the part of the job where
I had to bring someone to my office. Not everybody was able to be
certified, and when you tell someone, "I’m sorry, you’re
not cut out to be a flight controller, you’re going to need
to find another job," that is no fun. You think of the investment
we’ve made in this individual to get them to the point where
we try to certify them, and a failure there is very costly. The resources
over there are not so great to where that kind of thing needs to happen
very often. It’s very important that the people who are brought
in are good ones as well.
So giving them the expectations really helped. We did another thing,
trying to accelerate training, because you want to train people effectively.
They have to be ready. You set the bar, and the bar is there, and
you do not lower the bar, because we feel like where we have set the
bar in terms of aptitude, capability, and what a person can be able
to do when they’re on console—that’s a line in the
sand we do not cross. We don’t send someone up there who’s
unable to do the job the right way. So given that how do you get someone
trained effectively if they’re struggling?
We sat down and struggled with this because we saw folks floundering
a little bit. We were wondering is it because of the way colleges
are training these days. We wondered a lot of things. We decided that
it was not going to be because of something we were doing in training
that was ineffective. So we reviewed our training process. I took
a look at the baseball teams. They had a big article in the paper
about how the [Houston] Astros [baseball franchise] minor league system,
they’d redone their training, and how they had individualized
training at every level. This is an example of a best practice. We
saw that, and I brought the senior controllers together and gave them
each a copy of the article, and said, "Is this something we could
apply to flight controller training?” Why not? Because you’re
a ballplayer, you want to move from class A to AA, AAA, all the way
to the big leagues.
There were some things we saw there that we applied, and we changed
the way we trained to provide a lot more. We set up classes that senior
controllers taught just on specific systems to give to the folks.
We set up little mini-sims led by a senior flight controller to help
these folks come along. We gave them their expectations as well. So
we really, I think, made a significant contribution to training. It
really improved things. I think the people coming out of there were
way better trained than I was back in the ’80s.
Wright:
What do you think based on your almost 20 years there, and then you
just talked about revising the training—what do you think is
going to be needed to equip and train the next group coming in?
Doremus:
For controllers, I think they’re going to need to take a real
hard look at Shuttle. But I think the way the Shuttle missions are
conducted is a lot like the Apollo missions. And it is a two-week
period of intense activity and round the clock support. The new program
will be—if we do a lunar expedition mission and we bring the
crew right back, that’s going to be very similar to a Shuttle
or Apollo mission. But I think what we have planned for the lunar
missions and for the Mars missions is going to be a whole lot longer
duration. Mars, it takes a year almost just to get there, and you
want to stay there that long, and then another eight months to a year
to get back.
So I think they need to look at what we did with the understanding
that it does not apply completely to the nature of the missions they’re
going to be conducting. Look at the best practices, but if the mission
you’re working is going to last three years, what level of certification
do you need for the beginning? How do you manage the entire mission?
How do you manage the people? People working in Mission Control, you
work shift work, it wears you down. I’d had it with shift work,
especially working nightshift, by the time I left. When you’re
24, it’s not that big a deal. When you’re 43—
Wright:
Start to feel it.
Doremus:
Yes, a little bit, yes, and keeping your kids from preventing you
from sleeping during the day when you have to go in at night and things
like that. So to answer your question, look at what Shuttle did, but
look at it with a critical eye, because it isn’t all going to
apply. We should really look at our friends in the [International]
Space Station world. They’re doing things there where they’re
managing the people quite effectively. We could probably take out
Mission Control, the building, and be able to manage a mission at
another facility if we still had the people who could sit at the console.
They’re such a valuable resource, and we’ve got to take
care of the folks that support the missions. They’re vital,
and they need to have something that keeps them interested and motivated,
and you have to maintain their training. Are they going to be as sharp
three years from now? Do they need different skill sets during each
phase of the mission? Do you do that with different people? Do you
have enough people to do that? It’s going to be a real interesting
challenge for how they do that.
Wright:
The teamwork that you talked about certainly infers a definite level
of trust between the management and the team members. How is that
built, and how do you maintain that level of trust?
Doremus:
It’s easy to say in the Control Center, "You have to trust
your folks on your other teams." You have three shifts or sometimes
four. You can’t be there 24/7. They have to monitor the systems
while you’re not there, and they have to communicate to you
effectively what they’ve seen and what it means. You have to
do the same for your team members. So there’s an hour period,
and there are three of them each day, in which the oncoming team receives
a handover from the offgoing team, and that has to be done right.
We developed a checklist for how to do that. So part of it is, “My
buddy has told me everything I need to know, and I therefore trust
him.”
So that’s for the folks that are on the mechanical team. But
what about the people looking at electrical or propulsion or the robotics
person or the other? A big part of that is respect. I know you are
an expert in your system, and I trust you to say that, and I understand
you know that I have the expertise in my system. Conveying respect
to one another, avoiding the jealousies about who owns what. You get
that. There’s human beings over there sitting there, you get
those little things going on.
So mutual respect between team members, that’s really good.
The Flight Director sets such a critical tone. If the Flight Director
is listening to people and respecting their ability and their value,
the teamwork just flows from that. Really, from having observed probably
20 to 30 different Flight Directors operate, I got to see who did
a good job there and who struggled. I would take mental notes on what
I thought the aspects were of a good leader. One thing that is huge,
a good leader needs to be decisive. There are plenty of jobs down
at the Waffle House for people that can’t decide. Again, the
needs of the program define that.
A good leader is respectful of the people on the team and treats them
with respect. It sounds so easy to do, but the third aspect of a good
leader is a leader has to be able to communicate to the team, "We’re
going to make it through this, even though it’s difficult."
An ineffective leader will react emotionally, and people lose confidence
in their ability to be objective. A good leader will be there and
be a calming influence or a steadying influence or a motivating influence
towards getting the whole team to accomplish the mission. So giving
you examples of what I took from there and try to apply here.
By here I mean the Space Shuttle Program. I’ve had to head technical
panels here, and so I’ve tried to have people treat one another
with respect, and treat them with respect. And to get people—if
we have a difficult technical issue that we have to review to say,
"Hey, this is going to be hard, but we’re going to get
through this. This is not something we can’t handle." I’m
trying to give you stuff that’s helpful.
Wright:
It is. A lot of what you’ve had to do in the flight control
room and now, you’ve dealt with risk. Are there some lessons
and some aspects of risk management or risk mitigation that you could
pass on?
Doremus:
Yes, I could. That’s been one of the things I’ve enjoyed
being in the program S&MA [Safety and Mission Assurance] office,
part of our responsibility is risk management. What a great tool risk
management is for the program manager. I did not realize that. In
a way, everyone intuitively conducts risk trades. When you’re
going to pull out into traffic, you’re doing a trade on how
good is my vehicle versus how fast is that car coming, and what is
the road, and things like that. When you see a yellow light you do
a risk trade, “Should I punch it, should I slow down, is there
one of those cameras at this intersection?” Those kinds of things.
We talk risk a lot in the Shuttle Program. We talk about hazard reports.
They are a rigorous assessment of a hazard that could affect the safety
of the crew or the vehicle that gets compiled in a report and approved
by the program manager. We look at items called critical items lists,
or CILs, and those are areas in the design that don’t meet our
requirements—usually for really good reasons. Like we require
there to be redundancy in everything, and some things you can only
have one of certain pieces of hardware.
So they don’t meet our requirements for very good reasons, and
we say in the CIL why they don’t meet our requirements. And
why even though they don’t meet our requirements, it’s
still a good design. We have that all documented. So you have a hazard,
you have a CIL, we have these analyses that the people that do I guess
a bag of bones and they throw them out, and it comes up with numbers
on how probable a failure is. We have those. Probability of an issue,
of a bad thing occurring. We do a lot of work with that. We have folks
that are really good at that. When I say bag of bones, I’m only
saying it because I’m not smart enough to do that sort of thing,
but we have folks that are.
We have a risk system where we describe risks. So we have a hazard,
as I mentioned before. There is a risk associated with that bad thing
happening to us. For example, the debris coming off the external tank
and striking the Orbiter, that’s a risk. Lightning striking
the vehicle and causing damage that we don’t find out about
until in flight, that’s a risk. It represents a risk or a hazard.
So a hazard, there’s a risk associated with it.
The design problems I told you about, though they don’t meet
our requirements, so there’s a little bit of a risk associated
with that. Then there are these probabilities that it’s a one-in-X
likelihood of a bad thing happening. It may be one of those hazards
we talked about. It may not be. Then we have a risk system that talks
about just risks. We have to define a risk, and there’s a risk
in our risk system talking about debris hitting the Orbiter. There
are risks having to do with lightning. There’s a little bit
of double bookkeeping, but it’s a different perspective on each
one. So what I have learned—I’m sorry this is taking a
long time to answer the question.
Wright:
It’s a pretty difficult subject.
Doremus:
It is, and I hope I’m explaining it at an understandable level
because I’m trying to. I usually practice on my kids. With teenagers,
if you can keep their attention—so the idea I had, if you want
to call it that, was I should probably be comparing hazards and CILs
in this risk system that I have, and these probabilities, and comparing
between the two. So the program manager, when he has to decide where
to spend his money, which is a limited resource—“Does
PRA [Probabilistic Risk Assessment] tell me one thing and the hazard
tell me another? Do they agree? Well, why don’t they agree?
I have a CIL there, but the hazard says it’s probably okay.
But my probability says it’s very likely to happen. Therefore
do I want to spend money on preventing foam from coming off the External
Tank, or do I want to spend it somewhere else? Do I want to spend
money on lightning protection?”
The thing that seemed analogous to me was a flight controller sitting
in Mission Control will look at a temperature and a pressure or a
voltage measurement, and he or she will be trained to understand what
those mean. It won’t be just looking at one temperature measurement.
Like I told you about that fuel line. I can’t look at just one
little temperature. I was looking at the whole fuel line, the whole
system, and seeing if there was a sign of a leak. Well, can John [P.
Shannon], the [Space Shuttle] Program Manager, or [N.] Wayne [Hale],
look at the whole system of risks like a controller, and there’s
unlike components, and compare across the two and draw conclusions
about where to spend his money?
I think we’re just now understanding how to do that in the Shuttle
Program. Which is great, because we’ve got ten flights, and
we can do it, and we have limited resources, and maybe we can serve
the agency by figuring out ways to do that.
So really to summarize it, can you compare the ways that risks are
presented to a program manager—and they’re communicated
and presented in different ways—and look across those different
ways and bring trends or draw conclusions from the differences or
the similarities about how to spend your money? That’s what
we’re trying to develop here in our office. Again, I’m
not claiming we have this process nailed down. We’re really
learning how to do it. I think also I’ve seen the program start
to look at risk management as a useful tool, where the whole program
is buying into we can look at this risk management system and how
risks are elevated and how things get communicated to the program
manager.
So check back in a year and see what we’ve learned, because
we are learning right now. That’s one thing that I like about
the job, is we can still learn. We’re going to be looking at
shutting the program down, which hasn’t happened in a while
to a major program like this. We’d have to go back to Apollo
to see the last time a really significant program was shut down. We
got to do it right and prepare for the gap that’s going to be
going on.
One thing we are going to be trying to do is find efficiencies in
how we’re doing things with our limited resources. All those
items I described to you, CILs, hazards, risk management—those
all take people to do and other infrastructure resources. Can we do
them more efficiently? We’re going to be forced to do it more
efficiently as we’re winding the program down. So because of
the nature of the challenge we’ve been presented we may—not
we may—we will have the opportunity to find efficient ways of
doing things that can benefit the rest of the agency. So in a way,
you can look at what we’re doing in the Shuttle Program now,
and you look at it from that aspect and say, "Okay, they got
a chance to really figure out how to do things efficiently, because
they have to."
The other thing you look at is, "Boy, what they’re doing
is pretty cool." Finishing building Space Station. This has been
since ’04—this has been really a challenging time. Some
of the missions. Now instead of as a flight controller, I sit in the
Mission Management Team [MMT]. What we’re doing is amazing.
We had another APU fuel problem on STS-121, and I was asked to lead
a team to help assess the risk trades associated with how to operate
that APU on entry with a potential fuel leak. Thank goodness, it was
not a fuel leak, but a leak in the nitrogen system used to pressurize
the fuel. But we were prepared if it had been.
We had taken the right steps, and it was fun getting to work with
the flight control guys again and determine operationally what we
could do in the event that we did have a fuel leak, and we came up
with a really good set of risk trades. You could do A, you could do
B, here are the implications. It was in this case A, B, C, or D. Here
are the implications of each. So I was able to pull a team together
to go through those with the experts and then present that to the
MMT. The MMT made the right decision because they had a good view
of what the trades were involved. Just like you know coming into that
intersection if it’s got a camera or not and how pale green
or bright yellow the light is going in. If you have a good understanding
of it, it should be an easy decision, unless you’re not a logical
reasonable person.
Wright:
That was a very good explanation. So thank you for that.
Doremus:
Yes, it’s been intense in the Mission Management Team. We’re
really sensitive to anything that could damage the heat shield, for
obvious reasons. I’m going to jump back to 2003 [Columbia accident].
The mechanical systems team—boy, what an unusual time that was.
We had seen the video of the foam striking the wing, and at that point
in 2003 there was no instrumentation that the flight control team
could look at to tell what had happened to the wing. Really it was
a matter of the engineering evaluation team looking at that, determining
the magnitude of the impact and the extent of the resulting damage.
It was up to them to do that. Our flight control team would deal then—take
whatever steps were necessary based on the assessment of the engineering
team. The engineering team says, "That’s a bad hole in
the wing," flight control team would do what they had to do.
Our guys on console were therefore largely unaware of some of the
debates that were going on behind the scenes relative to there being
a problem with the wing or not. Some folks actually got e-mailed some
of the presentations. But again, on console, you don’t really
pay that much attention to e-mail, because you got plenty to do, and
you got the mission in front of you, and if you have time you read
e-mail—but generally you don’t have time. But they did
actually have some folks that had done some simulations and thought,
"Well, if the Columbia’s left wing was actually damaged,
it could affect the hardware that’s inside the wing well, including
the main landing gear area."
Some of the folks who have responsibility in engineering for dealing
with main landing gear actually did some simulations to say, "Hey,
if we had a burn-through, and it affected the tires, would we be able
to do a safe landing if one of the tires failed and the other one
did not?" You have two tires on each main landing gear. They
just sent an e-mail to the guys on console and said, "Hey, by
the way, we looked at this, and we think if one tire failed you stand
a reasonable chance of"—and there’s four main gear
tires, there’s two on each side. The question was if one of
the tires failed, would the other tire support rollout and landing.
If both tires fail on one side, then the Orbiter is going to pivot
on that. It wouldn’t be a good situation.
So they sent that information to the guys on the flight control team
and said, "Hey, we think probably you’ll make it if you’ve
lost one of the tires." The guys on the flight control team saw
that and talked about, "Well, if you had damage bad enough to
get into the wheel well and damage a tire, that failed tire is going
to be the least of your problems." So they sent a couple messages
around saying, "I think this is what we would see, burn-through
and temperatures going up, and eventually loss of the vehicle."
They described what was really going to happen, because they know
that. Again, these folks from engineering were just trying to be helpful,
but it really wasn’t useful at all.
But what the guys on console had heard was that the Mission Management
Team had looked at the failure and determined that the impact was
not going to cause burn-through. So in the minds of the guys on console,
everything was going to work. They just did this speculative, "If
it really were bad, what would it look like," and they put that
down in some e-mails that unfortunately got released later on to the
press.
So imagine you’re sitting console and you’ve been through
this in your mind, what would it look like, and then they saw that
scenario unfolding in front of them. It was just devastating to the
team. The first folks who noticed the tire pressure problems and saw
some of the alarms were our guys on console that had been involved
in that e-mail. So for our whole team, it was really difficult, I’ll
just say. We ended up with media attention because those e-mails got
released later on. That was along the lines of, "Why didn’t
you share your concerns?" when in fact there were no concerns,
it was just an understanding of if there really was an issue what
would have happened.
It’s funny, the way those things affect you. Between Challenger
and Columbia, when I went into the Control Center and talked to those
folks right after within an hour of the incident, all the group leads
came in and were sitting with their guys who were just in shock, and
I saw them. All they really knew how to do was fall back on their
training, pull out the procedure, and start gathering the data that
they needed to do. That somehow gave them—I guess just to sit
there and not do anything was too hard. They were able to pull together
as much information and data as they could. Then we were very deeply
involved after that in all of the investigation to try to determine
what had actually happened. But the observation I had was how they
were falling back on their training.
I actually had three brand new people in there who were OJT-ing, observing
their first landing, and each one of those folks is now a certified
console operator who has had a successful run since then and is now
a veteran of several flights. But yes, that was very hard. Again yes,
that same—you tend to blame yourself, “What could you
have done, what could you have done.” And really again the folks
on console, that was not—shouldn’t think that way, but
you tend to do that. May be just human nature, I don’t know.
So I brought that up just to let you know the way people react in
that kind of situation. If you’re really well trained, it helps
you quite a bit to have something to do and to focus on it. Those
of us who had been through Challenger were able to tell the folks,
"We’ll get through this." Like what I said, a manager
has to be able to do is say, "We’re going to get through
this. This is going to be hard, but you got to be strong right now,
and you got to focus on the investigation and moving forward to the
next step."
Wright:
What do you feel that you’d pass on as probably the best or
maybe the hardest lesson that you’ve ever had to learn during
your years here?
Doremus:
Wow. I think it would be being able to persevere through adversity.
Because when you’re in a really great job—did I mention
that this is a pretty cool job? When you’re in a really cool
job, there are a lot of really cool things you get to do. You get
to meet some of the finest people that you’ll ever want to meet,
the other controllers. And these outstanding folks, the astronauts,
you get to meet them and work for them sometimes now like I’ve
done the last three years. You get to go look at hardware, you get
to crawl around inside the vehicle. For me, when I do that, it’s
like time stands still. I’m enjoying that so much when I’m
out at the Cape [Canaveral, Florida] or somewhere else, you’re
doing that.
You get to sit on console, and one of the greatest feelings in the
world is sitting on console and the crew is asleep, everything’s
running fine, and you know it is, because your team is on duty and
it’s all running well. Sometimes at night you’re on a
graveyard shift, and they’ll run the cameras from the payload
bay down at the ground. And you’ll be looking at the tail of
the Orbiter silhouetted against the Earth or against the night sky,
and it’s a really peaceful feeling. You’ll watch South
America or Africa going underneath you, and you’ll watch that
and you’ll be looking at your data and you’ll know things
are running well, we’re ready. It’s a great feeling. So
you get to do stuff in a job that is just really cool. So that’s
why I say cool job, and I could go on.
With a great job comes the potential for great heartache as well,
and great risk—that goes along with great reward. It’s
hard to describe to new people coming on, but it is a real part of
the job. Your actions could have ultimate consequences. When you make
a mistake in there, you feel horrible. I’ve done my share. You
make a mistake, headlines happen. I used to show folks a headline
from a paper, from the old Houston Post, from a mistake I’d
made one time. You do. So the hardest lesson is that you’re
going to really enjoy the benefits of the job and they are sublime,
but sitting there in the office and having someone come in and say,
"Challenger just blew up," or getting the phone call at
home on a Saturday morning about Columbia and rushing in to be there
with the folks, or dealing with a mistake you made and the bad things
you feel. You can get through that, but that is a part of the job.
So the hardest lesson is how to persevere through that and continue,
because we’re going to encounter obstacles like that if we try
to go back to the Moon and when we try to go to Mars. We’ve
had big roadblocks in Shuttle.
What happened last year during the summer? We had a hole in the heat
shield again that the astronaut had to go repair. We had major computer
problems on Station. That was Apollo 13 again, and we handled it.
But it was pretty sporty while we did that. We thought the very next
flight, [STS-]118, that we might have a hole in the heat shield again,
and, “Do we go out and fix it or not?” These are really
tough problems, and we’re going to struggle with that again.
We may get another hailstorm on a tank and figure out what to do.
I had my whole family out to watch STS-122 launch in December. We
didn’t launch.
Dealing with the adversity that you face as a consequence of being
in a really neat job is the thing. You can train people to understand
the systems. You can train people to solve problems, although they
don’t come out of college necessarily knowing how to do that.
But dealing with adversity with the hardest part. You remember the
feeling. I still remember driving—were you all around during
Challenger? I remember driving off site that day and seeing all the
flags at half-mast. I lost it. I’d only been here a year and
three months. Then Columbia was so much deeper. That’s part
of the job. Having reporters come to the house, and having your teenage
son answer the door, and there’s a New York Times fellow wanting
to talk to you. That’s as much a part of the job as getting
to go to Kennedy Space Center [Florida] and see the launch. But the
other part of the lesson is the benefits of the job far outweigh the
difficulties and the trials.
Wright:
That’s good, and it might be a good place for us to stop.
[End
of interview]