NASA Johnson Space Center
Oral History Project
Edited Oral History Transcript
Robert
L. Carlton
Interviewed by Kevin Rusnak
Houston,
TX –
29 March 2001
Rusnak: Today is March 29th, 2001. This interview with Bob Carlton
is being conducted in the offices of the Signal Corporation in Houston,
Texas, for the Johnson Space Center Oral History Project. The interviewer
is Kevin Rusnak, assisted by Sandra Johnson and Tim Farrell.
I'd like to thank you for taking the time out this afternoon to participate
in the project. If we could begin, tell us some about your growing
up and maybe some of the interests or experiences you had that led
you into engineering and in aviation.
Carlton:
I guess, like most of us, as a young man I didn't have much of a plan
about where I was going. I ended up in aerospace as a sort of an accident
of a combination of events that sort of led me that way. If I pick
up about where I was started into the branch that brought me into
aerospace was a completely totally unplanned thing. As a teenager
and growing up as a young adult, I aspired to be a businessman, and
I was still in high school when I joined the [Alabama] National Guard.
All the fellow students went in the Guard. That was a weekend warrior
game. You got a little extra money out of it, and there's always a
glamour to that. So that's what sucked me in as something that you
just at the time would have never dreamed where it would lead, but
that's what led me to the aerospace world, unbeknownst to me at the
time.
The reason it led to the aerospace world is when the Korean War came
along, the National Guard got called up, and as it was being called
up, I decided I'd rather be in the Air Force. So I joined the Air
Force, and in doing that, that earlier decision for the weekend fun,
I was at a crossroads and didn't know it and ended up in the Air Force,
which really was the stepping stone into the aerospace world. When
you get involved in aircraft work, and I think the military sort of
grows you up in a hurry, at that point becoming involved in aircraft
for four years in the Air Force, well, that sort of sets your mind-set
to where you think aircraft, and then as aerospace come along, you
think aerospace.
I wasn't in the Air Force very long before I realized the lack of
an education was a big handicap. So while in the Air Force, I woke
up to the need to have an education and resolved to do that. After
getting out of the Air Force, I went on to Auburn [University, Alabama].
While I was in the Air Force, I worked on the flight line on aircraft,
and I can see now a great parallel in what I was doing there to what
I ended up doing in flight control. The aircraft mechanic's job was
to find problems and fix problems and repair problems. So when an
airplane came in, the pilot came down with a big long list of complaints,
this went wrong, that went wrong, and you went out on the flight line
then and tried to find where the problem was. So you started out with
the knowledge of how your system worked, air-conditioning systems
in the airplane and powerplants and the guidance and control. They
mostly had manual control in those days. You had fire control systems
on the guns.
In all of those, I could see, really, a one-for-one parallel to what
we did in mission control. You [had] problems occurring, you fixed
them. That was just a piece of mission control. In mission control
there's more to it than that. You not only have a problem and you
try to fix it, but you fix it with a view in mind of trying to maximize
the game downstream. You're going to try to make it limp along until
you get to a point where you can totally fix it. You rarely ever can
fix something on orbit, but on the ground, it was just repair and
fix.
There was another difference. On the ground with aircraft, it worked
according to spec, and if it didn't, you pulled the boxes out and
replaced them. When you're in orbit, you don't have that option as
a general rule. Now, what you try to do is make it limp along and
finish as much of the flight as you can, as long as it can be done
safely. But nevertheless, the diagnostic steps you went through to
ascertain something's wrong, do I have a leak in this system? Is there
a short in this system? Those diagnostic steps were the same. So that
whole four years in the Air Force, in reflecting back on it, I think
really couldn't have been a better primer and preparatory step to
subsequently working in flight operations.
After leaving the Air Force, I went to Auburn and got a degree as
a mechanical engineer. For me, that was a great struggle. I didn't
mention it earlier, but I quit school. Ninth grade was the last grade
of school that I finished. I started into the tenth, and about three
months into it, I decided I could become rich as a businessman and
didn't need an education. So I had an especially hard time in the
first year or two years of college. That was a pretty good handicap.
But it all worked out, and I ended up with a mechanical engineering
degree from Auburn.
When I got out of Auburn, it was just sort of natural to go back to
the aerospace industry. I ended up back at General Dynamics [Corp.],
Convair in the early days it was called, and then it changed its name
to General Dynamics. At the time I came back to General Dynamics,
they were working on the B-58 [Hustler] aircraft, which is a four-engine
jet bomber. It was put in service for a short period of time and then
retired.
At the time I went to General Dynamics, they were just on the verge
of finishing up the flight test and then beginning to go into production
to deliver aircraft to the Air Force. So I was fortunate there to
be involved in some of the flight test activities, partly with respect
to the flight test program that tested the B-58 to prove all of its
characteristics to the Air Force.
Then subsequently, as a part of the—this may be a misnomer to
call it flight tests, but essentially the same thing when we roll
an airplane off the production line. As it came off the line, you
rolled it out on the flight line and you went through a series of
flight tests to verify that everything was working right before you
delivered it. In both of those, there was, again, a similarity that
was kind of an extrapolation of what I'd done in the Air Force. Again,
with the exception of flight tests, that began to be a little closer
to flight control in NASA. When you were flight testing the airplane,
you started out with a flight with a set of objectives for this flight:
I'm going to test the flight control system, or this flight might
be to test the engines at this time. This one might be to test the
performance of the airplane. So you had different characteristics
that you were flight testing.
At the same time, you were monitoring the airplane from the ground
as well as the on-board crew, just very similar to what we did in
flight control, and if something went wrong, you were trying to sort
out, is this a deficiency in the airplane design or is this a malfunction
in the system or have we messed up in how we planned? You know, is
its overall performance just not what we thought it would be? Is the
design-deficient? And you tried to finish every flight test. You had
a certain repertoire of things that proved that this particular system
or tested it and you had planned so many flights to get that done
and you were trying to get it done within that number of flights.
Very similar operation to NASA. If you began to have problems, you
began to then look and say, well, what else can I do? I don't want
to waste this flight. So I see a pretty strong parallel to what we
did there. We did not have the same kind of facilities assisting us.
We had a much more simplified downlink and in general much less of
it was available to us to look at on the ground as was available in
NASA.
Rusnak:
Any parallels in dealing with flight crew on this?
Carlton:
I would say an exact parallel. I have a great respect for man's interface
and interaction in operations, and yet at the same time, we have a
dichotomy here. Man can do things and is a great value and gives you
a much better chance of accomplishing a mission. At the same time,
man is more prone to mistakes than machine. Somewhere I saw an article
on flight safety of airplanes, and I think, if I remember rightly,
it was over ninety percent of all aircraft accidents are human error.
So you think, well, the last thing you want is a man aboard. But that's
not true. When the hardware fails, man can diagnose it and can pick
up the pieces and go on with what's left and make do, whereas if you
tried to automate the process, when it don't do what it's supposed
to do, usually you're dead in the water, you lost that one.
But to more directly answer your question, the flight crew would—very
frequently the problem would be human error, you know, something wrong
with what they were doing rather than something wrong with the hardware.
In flight tests, that was particularly critical. You're trying to
qualify the airplane, and it's our human nature, if we've done something
wrong, we don't want to 'fess [confess] up. There was always a good
bit of give and take between the ground team, who were always suspicious
that this problem really was due to human error.
So there's always a good bit of little give and take between the crew
and the ground, and sometimes it brought us to animosity between ourselves.
Usually it didn't. Usually there was a great deal of respect about
what was going on. If you were on the ground and you heard a flight
under way and taking place and you heard the pilot report, "This
so and so is just not working. I'm doing oscillation," or, "I'm
doing this and the other," and you heard the ground flight controller
suggest to him, "Well, why don't you cycle this switch, cycle
it to the off position and back to the on," amongst the flight
control guys there'd be a little snicker, you know. He knew what was
wrong with it, but he was being polite to the pilot. He forgot to
put the switch where it belongs, dummy.
At the same time, you know, I'm sure they saw and got frustrated with
us when they'd have something that was really wrong and we'd refuse
to accept it. And especially if they brought it back on the ground
and told us what it was and the next flight it's still there. So it's
a two-way street.
Now, I went off on a tangent. Did that answer your—
Rusnak:
Yes.
Carlton:
The B-58, I think, was a great learning tool for something else that
had a very strong application of flight control, and that is, when
you're in a flight test of an airplane, even though it's not as critical
as space, there still is a criticality factor there that gets you
in the mind-set of you've got to work fast, you've got to solve something
quickly, resolve it, and you've got to take action based on it. You
don't have time to dally around. Even though it's an airplane and
it come back and land, and that does ease the stress, it costs a lot
of money to launch one of those beauties and bring it back, and it
was not a small thing. It took several weeks to turn around work on
it, to get the airplane turned around and back up airborne again.
So you tried to do all you could, and it cost the company a whole
lot of money if you failed to finish all the objectives of the flight.
From the B-58 flight test, we reached the point where we started to
deliver aircraft to the Air Force, and General Dynamics sent a team
of people to Bunker Hill and to Carswell Air Force Base [Fort Worth,
Texas] with two squadrons of aircraft that they sent out and deployed.
I went to Bunker Hill with that team, and there we had a group of
young Air Force mechanics. They were learning to maintain the B-58.
And a new group of pilots, they had already flown it and gone through
training of flight, but they were learning to live with it. So it
was a learning experience and quite a bit different from the flight
test environment and yet in a lot of ways similar.
But the main thing that happened there was, now, [I was] no longer
on line hands-on, I'm more looking over the shoulder, advising and
training. That was akin to the later stages, I guess, of flight control.
But still, I look back on those days, it still was so similar to what
I was doing, but it was less pressure. Air Force were less concerned
if a flight came back down. Now, every flight was a training flight,
I guess you'd say, in the early days of the deployment, so it was
more relaxed and a fun sort of thing from the pressure of trying to
get the flight test program over with.
From General Dynamics, I went to Lockheed Marietta—in Marietta,
Georgia, and worked on the C-141 [Starlifter], which was in the final
stages of design, I worked in the flight control area as a maintainability
advisor to the design group, and there I saw a different perspective
of aircraft. The shoe come on to the other foot, as the saying goes.
On the maintenance side of it, every time we'd run into a problem,
we'd cuss the designers, and you'd sit down and try to see what was
wrong and what they should have done and how they should have designed
it, and you didn't really have a lot of good dots for them when you
run into problems. It's kind of like in your automobile, you get in
to change a spark plug and you can't get to the plug, you know, you
cuss the designer.
Rusnak:
Yes.
Carlton:
That's what we did to the airplanes then. But now the shoe being on
the other side of the foot, that was my job on the C-141, was to design
in easy maintainability. The Air Force woke up to the fact of the
need for that and insisted on it, and when Lockheed won the design
for the C-141, well, that was one of the strong points. They had their
superior ability to design in easy maintainability.
I wasn't there very long, only six months, but it was in the crucial
time that design was gelling and finalizing. So it was a fun thing
to do. But it give you—it gave me, at any rate, a new perspective
and a new sympathy for the guy on the other side of the fence. It
wasn't as easy as I thought it'd be when you're trying to second-guess
what are these dumb bunnies going to do on the flight line? How will
they mess it up? And as I talked to those guys trying to design, I
saw things from their view.
I remember one time we were talking about the hydraulic system, and
we had the plumbing go back—and when you have a hydraulic system,
you have hydraulic power, liquid under high pressure, and it provides
the power to operate an actuator through some controls. Well, that
fluid goes into the actuator, but it's got to come back. It comes
back under low pressure. So the side that carries the high pressure
fluid down there is maybe 3,000 psi and the side coming back might
be 50 psi. Well, weight is critical. You don't have to have as big
and heavy and stout a line to handle 50 psi as you do 3,000.
But as they were putting this together, I remember them saying, "Now,
you've got to use a different-size fitting." They said, "Those
dummies will interconnect them wrong if you don't." And so you
would have a totally—where it was impossible for you to hook
it up.
They told some scare stories about in the past how these guys had
crossed them up. One case I recall them talking about the high pressure
line didn't fit the other one, so the mechanic decided something was
wrong and he went and had himself the hoses rebuilt down in the hydraulic
shop so they'd hook up where he thought they went. [Laughter] So you
only see it from your own viewpoint.
But it was interesting and fun, and it did give you the perspective
to look from both sides and to be a little bit more understanding
when it wasn't designed like you thought it should be from an operations
viewpoint.
From working at Lockheed, then, I went to the Army Missile Command
at Huntsville, Alabama, and worked on the Sergeant missile. There
we were involved in the engineering oversight of the design of the
Sergeant. It was designed and being built by Sperry Utah, and that
was more remote. It no longer was a hands-on and in the middle of
the hardware. The design was almost completed at the time I came on
board. They were still flight testing it, but that was by the flight
test [operations] guys out at White Sands [Missile Range, New Mexico].
So the role of the engineer overseeing the design work was more the
changes they wanted to do to the design, to oversee that and make
a judgment of whether it was necessary or not, were they doing it
the right way or not, and were they doing it in a way that would be
most economical to the government. So it was more a project management
hat, which I guess I never thought about to this very instant, but
I guess that gave you yet a third perspective of the design of a spacecraft.
That perspective was what does it cost and the time to get the production
schedule going. I guess deep down that sort of played a role in my
thinking from that point on, but I didn't appreciate what a role it
was.
After there, I came to NASA. When I worked at the Sergeant Project
Office, it came to an end, to a close, as the design was completed,
and from that point on, it was just production. So they began to power
down the project office, and they would move it over into a different
group that was just more oriented toward production advice, where
the project office was oriented toward design of a new vehicle, spacecraft.
About the time I was beginning to worry about where will I go next
and will they have a job for me here in the Army, I got this phone
call from NASA. It couldn't have been more in sync with where I was,
and it was a guy named Mel [Melvin F.] Brooks. He died last year.
He was my first boss. He was the guy that contacted me. He hired me,
and I came to work for him.
They were in the early part of the Gemini Program at that time. The
Mercury was still fresh in everybody's minds, and most of the Mercury
team of guys had come on board and just moved right on into Gemini.
They had hired a lot of new people, but those new people had been
on board long enough till they considered themselves pretty knowledgeable.
I came in from the outside. They were about the third flight, I think,
as I remember, somewhere along in that vicinity.
The first Gemini spacecraft were—well, all of the Geminis were
manned, but I got on the Agena, which was unmanned, and it didn't
come into the program until downstream.
[Telephone
interruption]
Carlton:
Let's see. Where was I?
Rusnak:
Well, you had just arrived at NASA and were talking about—
Carlton:
Oh, yes, the difference in the Agena missions.
As I came in, the Gemini was under way, and Gerry [Gerald D.] Griffin
was the Agena. He was heading up a little section of Agena guys that
worked on the flight control. We sort of had the Agena divided up
into two pieces. In fact, if you look at the way we manned the control
center, you see that carried forward. You had one group of guys that
got us in control and propulsion systems. Then you had another section,
and this was usually a section within a branch. There would be a branch
for a spacecraft with two sections, one section G&C [guidance
and control] with propulsion. The other section handled the instrumentation
systems, the electrical systems and so forth, and the environmental
control if it had that on it.
Gerry was responsible for the guidance and control, and he was being
assigned to the Gemini. They were hurting for people, and they wanted
him to go over there, and he was quasi-trained. I don't know how well
he was trained at that time. To me, he was very highly trained. He
was at NASA. He represented the only NASA I saw at the moment [except
me]. But anyway, he sat down, and he was going to give me—I
think I mentioned this in the little write-up—Gerry was going
to give me this—they told him, said, "Bob's going to come
in and work in your place. So you bring him up to speed on the Agena."
So we sat down, and he sat down and talked about the Agena guidance
and control system for about thirty minutes or an hour or so, and
the phone started ringing. He answered the phone, and he said, "Well,
you're on your own here." [Laughter]
I thought, "Oh, goodness." I told him, "Well, tell
me where your documents are that describe this bird."
But that was kind of characteristic of what took place. Now, they
did have some training classes that came on line sometime after that.
We did get some training classes, but by the time we got them, by
that time we'd learned the systems. But they helped. When somebody
came in and gave you a long detailed description of the system, he
saw stuff that you hadn't really appreciated, and also the training
people do it so much till they kind of tie it all together, and the
engineer tends to get all bogged down. He can't see the tree for the
leaves. As a matter of fact, he can't even see the leaves sometimes
for the veins in the leaves. Ask him what the forest is, and he don't
even know there is a forest there. But anyway, we spent quite some
time learning the Agena.
Those early months—I can't remember how long it was from the
time I came on board until we flew the first flight. It probably was
a year or so. I came in November of '64, as I remember, and I don't
remember when the first Agena flight was. You've probably got it in
your records. But my recollection is we had about a year to get up
to speed.
In that year, we hired some additional people. When I came on board,
there was myself and Ed [Edward L.] Dunbar [Jr.] in the Agena area
where I was. Ed Dunbar and Bill [William E.] Sturm, I think, came
on board at that time. He was a Philco [Philco Corp., later Philco-Ford
Corp.] contractor. Ed Dunbar was also a Philco contractor. The way
we were organized at that time, the Philco contractors manned, for
the most part, the remote sites, as far as the systems monitoring
was concerned. You had three people that went out there, a capcom
and then you had a Gemini systems man and you had an Agena systems
man. The capcom might be a NASA man. As I remember, some of them were
at Philco, or sometimes there might be an astronaut out there was
a capcom. We had Philco people that worked with us in the control
center. A lot of those people had been with them through Mercury,
so they were pretty proficient, pretty skilled in what they were doing.
Then we began to hire people. At that time, they were already looking
ahead to Apollo. Gemini was a stepping stone to Apollo, and from the
very first, from the time I got there, that was understood. So the
hiring was preparatory to Apollo, not really just to do Gemini.
We had a situation in retrospect seems kind of ludicrous to me, but
at the time it never dawned on me to question it, or anyone else.
The manning for Apollo was against the supposition of many, many flights
and a high flight rate, and I don't recall the number of flights,
but I remember over and over we were told, "Now, you're going
to have multiple flights coming back to back to back. You'll be working
people too hard. You need three full shifts of people to do the job
and maybe a fourth. They can be off line having a vacation."
Now, if you go back and look, we didn't have very many Apollo flights.
What happened was, the public began to say, "This costs too much
money," and the Congress reflected that. So the Congress began
to say, "Hey, shut this thing down. Shut this thing down."
So just came the cleaver, we shut down. We had planned to have a high
flight rate and power up with a lot of people to sustain that flight
rate. Fortunately, we didn't get powered up fully, so that saved our
bacon.
But back in the Gemini days, we were looking ahead to that and beginning
to try to bring people on board so we could give them training in
Gemini that would carry them on in to Apollo. In my section, we brought
on board John [A.] Wegener, and another young man named Paul [D.]
Nering. I'm having trouble remembering—about the time we finished
Gemini, they had meanwhile had hired an entourage of people who were
following Apollo, and they moved in with us. We had several Air Force
people came in and worked with us. I was just trying to remember if
they came in as a part of the Apollo group or we hired them, and I
think when I say we hired them, the Air Force would send them in to
train with us. So we didn't really hire them.
They had topnotch people, by the way. They must have skimmed the cream
of the crop off the Air Force people to send to us. I never saw an
Air Force guy that I just didn't really get impressed with. I had
Bruce [A.] Stach. He was there during Gemini. Ed [Edwin E.] Marzano,
Jerry [Jerome M.] Sisk, Hopkins—I know Hopkins, Carroll [E.]
Hopkins, came in as a part of the Apollo. It may be Marzano came in,
but I believe that the others came in directly assigned to us as they
came in from the Air Force, if I recall right.
But we were powering up at any rate, is the point I'm developing here.
During the Gemini we were constantly powering up, and that had several
effects on us. One thing was, in the front end of the program we were
just so swamped. That few bunch of guys, we were trying to put together
our drawings to fly the missions with. That was a very time-consuming
thing. Did you ever see any of the mission rules preparation in NASA?
Do you know what mission rules is? Have you been through that with
any of the other interviewers?
Rusnak:
Yes, on a basic level. But why don't you go ahead and explain it for
the record.
Carlton:
Mission rules took a tremendous amount of time, but it was a very
powerful tool that NASA had, and I think it was a unique tool. If
we'd had it back in the aircraft flight tests, it would have been
very, very helpful. It served a variety of purposes. What a mission
rule was, was you sat down prior to the mission and you said, "Let's
make some rules up of 'what ifs' this happened, what would we do to
react to it." "If this failure occurred" was usually
the way they were oriented. So you start at the front end. You say,
"Now, what's most important about this mission?" We want
to get as much as we can out of the mission. A lot of money is required
to put it up there, and a lot is at stake, and you don't want to waste
any of it. You want to maximize the return you're going to get out
of it, get every experiment done you can possibly do, every flight
test item done you can possibly do.
In conjunction with that, you have got to protect crew safety above
all else. So you're starting in, a mission rule would be that you
will never do anything to jeopardize crew safety. I mean, you didn't
have to really write a rule for that, but it was there, and it was
your starting point. You had a rule that was an offshoot of that,
to protect crew safety, you always tried to have redundancy in your
systems so that if one system failed, you still could bring them home
with the other systems. So you had a mission rule that said you will
always maintain yourself in a posture such that no single failure,
subsequent single failure, could put the crew at risk.
Now, to illustrate how you would apply that, if you had a reaction
control—RCS— system—you know what that is; that
controls the attitude of the spacecraft—and it had redundant
propellant supplies to it, and sometimes those two separate paths
would have redundancy in themselves, and you might have a failure
in that second set, a level of redundancy. Well, at that point, maybe
you had a plumbing line going from a propellant tank down into the
system, and maybe it went through two valves, you had two check valves.
When it flowed at this and stopped up, it would still flow on downstream.
If one of them stopped up, okay, I'm in a posture where one other
failure will kill this system. However, I've got another whole system.
So this failure don't get me in a posture where one other failure
would kill me.
Or it might be that maybe it would be in a pressurization system that
flowed gas to pressurize a propellant tank that in turn pushed the
propellant out to the engines. Well, I've got valves in that pressurization
system that's pushing gas down on top of the propellants. As I use
propellant during a system, the propellant tank, you know, it's a
big tank, so you empty it. Pretty soon it gets where it's over half
full of gas or two-thirds full of gas. Now, the gas coming into it
is regulated, and as you do burns and you use the propellant up, more
gas comes in and keeps the pressure.
Well, you reach a point, though, gas expands. So you reach a point
about halfway point or maybe a little more, where if you lost your
gas, you've got enough expansion ability in there that it would push
the rest of the propellant out. So at the front end of the mission,
you'd have a mission rule that said, "If I lose this, I come
home quick because I've got the other system, and that's all that's
got me. I've got one system failure between."
But later on in the mission, as you get more and more volume in that
tank, you'd reach the point where, "Aha. This tank will bring
me home. Now I've got two systems to bring me home." You can
just do anything you want with that. So mission rules you apply to
failures to help you take corrective actions that maximizes mission
success.
That's basically what they do, but as you sit down and start planning
how you'd react to a failure, say we've got three experiments, and
one of them takes pictures of the surface, and one takes pictures
of the sky, and one of them does something else, and we've got another
experiment that conflicts with that. This experiment to take pictures
wants you to do attitude controls, and I've got another experiment
that wants you to not do it, it wants to be real stable. So I have
to decide which one of these is most important.
If I have a failure that's going to make me come home early and I
can only do one experiment, here I am, a little poor flight controller
on a console, and I don't know which one of these two experiments
is most important. Which one do I give priority to? Well, that's above
my job level.
The manager that's managing the whole project makes that determination.
So he gives me a guideline set in the mission that becomes a mission
rule. Here's the priority of all of the things we do, not just experiments
but everything. So the mission rule becomes a mechanism for him to
tell the operators the priorities of what needs to be done. Then when
we little flight operators then begin to look at all these failures
and postulate what we'd do, it becomes a mechanism to go back and
review mission rules with the managers for them to see how we're going
to react to a particular failure. So it gives them insight into how
we're operating the mission, and it gives us insight into how they
want it operated, and it gives us guidelines as to how to quickly
react to an emergency. A very, very effective thing is a mission rule.
Another thing it does, it trains you. A newcomer coming into the system,
and he has it explained to him what a mission rule is all about and
what you're supposed to do with it, and the first thing I would do
with my guys I hired, I'd say, "Okay. You're in charge of the
RCS system. You're an RCS expert. Here's our mission rules for this
next flight. Go through and tell me all the failures, now, and how
we're going to react to them, and I expect you to understand what
we're doing in this flight, and these mission rules tell you what
we're doing."
Now, he also had the flight plan that outlined every single thing
we're going to do. We'd already gone through it and laid out a total
flight plan, and everybody had the mission rules available to them,
and they worked together with the flight director and with mission
management to put together a flight plan that accomplished the most.
And then the mission rules allowed you to change that flight plan
according to failures that were incurred.
So I'd tell this new guy, "Okay. What if you lose this propellant
tank of gas?," you know, what if, what if, what if, what if,
and he would come back and propose a mission rule, "Here's our
corrective action if we'd had this failure." I could look at
what he proposed, you know, and I could critique the heck out of him.
And in that critique, I'm teaching him flight control. I'm teaching
him what a mission rule is all about. I'm teaching him to orient his
thinking toward maximizing the mission's success, but don't ever jeopardize
crew safety. So it's a very effective training tool, as well as that
communication, and it allows the manager to control what's happening,
even when he's not there.
Then last, when you go through mission rule review after review after
review and planning and planning and trying to decide what you're
doing and in simulations, applying that mission rule, then comes the
day of flight and a failure occurs, and I've never seen any statistics,
but I'll bet you that probably somewhere between a third and a half
of all failures were not failures we'd thought of, that we had no
mission rule for them. But having gone through them so many times,
you got to where you'd think along the same lines, and not only that,
as a team.
This is something else that gradually comes out, and it takes a long
time to absorb it and to appreciate it; the flight control team works
like a team. When it don't, it's not very effective. It has to work
as a team. When all of the guys that work together as a team, they
all think alike on what we're trying to do, this mission rules helps
mold us into a team as we applied it in simulations and as we then
applied it in the flight itself.
Well, the mission rules was something that helped us make that transition
in the Agena to becoming knowledgeable about what we're supposed to
do. We finished our drawings, we had the mission rules, we worked
on the flight plan, we worked on the ground systems where it was being
built, they brought in the mission control system Houston during the
Agena Program. The early days, they were flying from the Cape and
they were working furiously, the ground guys were, to get the MCC
[Mission Control Center] ready.
And what did they design in the MCC? It was what the flight controllers
wanted. We gave them a set of requirements, said, "This is the
way we want the console laid out." Now, when I came on board,
they already had the basic concept set in concrete. You would have
TV tube systems. You'd have these little event lights. I don't know
if you've looked through some of those consoles, but around the perimeter
of the TV tube you had a whole bunch of little event light panels.
The first sets had thirty-six little event lights, and very quickly
we filled those up. Somebody went out and got them where you split
each one, and I've been able to have seventy-two. That was great.
Learning to use the TV tubes was something that was a learning experience.
In flight test work, you had banks of needles. Each needle represented
a particular parameter. Arnie [Arnold D.] Aldrich, he'd been using
the needles down at the Cape. They had needles, or little gauges.
Well, displaying your system on the TV tube was a gigantic leap forward.
On my system, I had it laid out in a schematic, just like you was
looking at your schematic drawing. The fuel tank's on the left, there
was a little bubble. Inside the tank was a little measurement that
said 50 psi or whatever the pressure was. You'd just look in there,
and there it is inside the tank. Coming out of the tank was a line
that represented the gas line going to the propellant tank. This was
gas that was under high pressure, 3,600 psi, and it went through a
valve, and then it went through a pressure regulator that regulated
it down to about 200 psi. From there it went into the tank. It kept
a blanket pressure on the tank of 200 psi. As you burn propellant
and as the propellant emptied the tank, it just kept putting more
pressure, and then you used that pressure—you didn't have a
fuel pump—you used that pressure to blow it out. Well, you had
that schematic on your TV tube.
Now, Arnie—I almost hate to use the name, because, you know,
we've all got something downstream we did that's funny. It wasn't
funny to us at the time. Arnie, I looked at his TV tube, and first
off, they told him, "You've got to take all those gauges off
at the console."
"No. I won't do that."
I could empathize with what he was saying. That gauge was instantaneous.
It was high data rate, very sensitive. If the pressure was doing this
[gestures up and down], that little needle on the gauge is doing this.
The TV tube, the data come into the big mission control center computer,
and it put it in a database, and then plop, it updated your number.
One second later, plop, the number changed. So it was very slow compared
to the dynamic response of the needles. Well, Arnie said, "No,
I don't want that. I want the needles, make them on all of my instruments."
"No, you can't have it, Arnie."
Well, the first configuration, he had both, and guess what was on
his TV tube: pictures of the needles. [Laughter] Well, he could only
do about five parameters on his console. So what did he have? He had
a very degraded—you know, it didn't even compare to his needles,
but it only had four or five parameters. I had 300 and something on
my display. And you could get a lot of data on it once you began to
appreciate it.
Well, after he got to using it, you know, he made this transition,
but it was kind of funny. I cackled at him many a time looking over
there at that, and there's that big old TV, or that big old ugly meter.
But he got swamped by progress and learned how to use his system.
Well, as we started on in to our first missions, we finally got ready,
and our first Agena mission was the Gemini VI spacecraft, Gemini VI
mission. It was the first Agena spacecraft. And this little beauty,
we launched ahead of them. We got in orbit and we were their target.
Then when they'd come up with the Gemini spacecraft, they'd launch
behind us, because I remember they used the next rev [revolution],
if I remember rightly. We launched, and we went around one rev, and
when it got in the right position, coming overhead and zoom, here
they went after it, chased us, and did a rendezvous with us and docked
with us, etc.
Well, we started into orbit. We launched, and we started toward orbit,
not into orbit yet, and we were sitting there watching the data, you
know, a bunch of green—first flight for any of us. We'd had
a lot of simulations. Bernie [Bernard R.] Brabant was the electrical
guy in the back, I was the Systems guy, and Mel Brooks was the Control.
He was the guy that sent the commands, and he was sort of the boss
of the—we were, two of us, on the front room.
With the Agena, I had all the systems. I guess it was the only spacecraft
where we didn't split it up into guidance and control and electrical
systems, and he was the command. He took care of the command. The
Agena we commanded from the ground. It was unmanned.
Well, as it went up into orbit, what happened was, I'd have to draw
you a picture for you to see it, but fundamentally what happened was
that when we bought the Agena, we changed the way the propellant was
put in the rocket engine.
Now, the way the engine was designed to work was in the inside—are
you familiar with a rocket hypergolic? The hypergolic propellants,
if two propellants touch each other, they burn. You don't need a sparkplug
like you do in a car. So the way this rocket engine worked is you
had a big flat plate about this big around, about this thick, and
then the engine bell is around that. This is the injector fuel. It
had a jillion holes drilled in it.
And then behind it, you had—I guess you'd think of them as feeder
tubes drilled in it for oxidizer, a big hole. So the oxidizer would
come squirting out. If I just took one of those—there's three
holes to each thing. You'd have one oxidizer, and then you'd have
the fuel come in at an angle and impinge on this jet oxidizer, and
that was repeated over this whole housing. I don't know how many holes
was in it, but a bunch, and you had little manifolds behind it drilled
through the plate. So these feeder holes penetrate in that manifold.
So in the manifold holes behind it, you had this oxidizer with a bunch
of oxidizer holes coming out. Here come a jet oxidizer, and then the
fuel would impinge on it.
Well, what happened was, the Air Force, when they cranked the engine
up, they squirted the oxidizer out first, and then milliseconds behind
that come the fuel, and you got a good steady stream of oxidizer going
and then the fuel starts impinging on it, and as it does, of course,
it starts to burn and create big pressure and then blows out the engine,
you know, and you've got combustion and you've got thrust.
Well, for some reason, it was before my shift, but, you know, this
was already designed when I came on board, when Agena went into orbit,
when the engine cracked, the telemetry went dead, just like that.
Now, it took a long time going back and looking at strip charts to
figure out why it went dead, and I didn't understand why it went dead
till years later.
What happened was, NASA changed it to be what's called a fuel lead.
Those two jets of fuel turned on first, then come the oxidizer out.
Now, I'm explaining something to you that I'll bet you there's not
three people in this universe really understands why that engine blew
up. If you take two squirts of the liquid and squirt them here, you
know, right against each other, they'll hit and go out, won't they?
You can see that. They'll realign and come out. Now, if I start changing
the angle on them, it still will squirt out. Finally, you'll reach
an angle where there's no squirt back this way. You see what I'm describing
there? It's all this way. Okay. I'm not to that angle yet, okay? I'm
on an angle about like this. I'm still squirting some this way and
some this way. So here comes my fuel. Squirt, splatter, splatter.
Now, what's down here?
Rusnak:
Oxidizer.
Carlton:
Oxidizer hole. No oxidizer there yet. So I squirt that hole full of
fuel down in the manifold. Now I've got a bunch of fuel down in the
manifold, no oxidizer yet. Now, when oxidizer comes rushing down that
manifold, what's going to happen? It's going to blow up. It's going
to blow up. That's what happened.
At the time, when we went back and looked through it, looked at the
data afterwards, I didn't know what I've just explained to you. I
knew that we had an explosion in the manifold, but I didn't know how
the fuel got there. I thought probably we had a crack in these manifold
lines, was what I really thought, but I didn't know why. But I saw
the explosion. We had a bunch of transducers, and I could see the
explosion hit this transducer. It popped out. Milliseconds later,
this transducer popped out. Milliseconds later, this—and I could
backtrack it all the way back up, you know, pop, pop, pop, pop, pop.
You could see the shockwave go ricocheting up through there, and this
is milliseconds.
Now, it didn't take long for the shrapnel to kill the electrical power.
Now, it was still milliseconds. You know, if you were looking at it
real time, it's like everything just turned off. It took maybe three
seconds or four before we lost total telemetry, but so much happened
so fast. And what happened, the power started down, and that drew
our attention, and we started looking at that. Then telemetry turned
off, and we didn't realize what had happened to the engine. So when
the engine blew up, in the meanwhile, this propellant tank's being
pressurized, and it overpressurized, you know, so the whole thing
just disintegrated.
The way I realized why it did like it did that I just described, later,
years later, I was out at White Sands Missile Range and I saw how
they tested the Agena with this fuel lead, and the way they tested
that Agena engine is they pointed it downwards. Now, as they pointed
it downwards during the test firing of the engine, you've got gravity
here on the Earth. These two fuel jets are pointed downwards. Now,
this splatter that goes backwards into that oxidizer hole, it's working
against gravity to go uphill. You didn't have any gravity in space.
So when we did a test firing pointing it downwards, we did not uncover
the fact of what was going to happen. I don't think. I don't know.
When I saw that, I just thought, "Well, you know, now I understand
how the fuel got in the chamber." I don't know if that's understood
anywhere or not. Well, anyway, so we lost our first Agena.
Rusnak:
Can you tell me what was going on in the Mission Control Center when
this happened?
Carlton:
Well, the thing lifted off, had a good liftoff, everything was normal,
we're going toward orbit. Gene [Eugene F.] Kranz was on the console.
He was our flight director. Mel Brooks and myself were sitting down
on the console. And we had across the front—I'm sure you've
gone through how the control center was laid out. The flight dynamics
people are keeping the trajectories and so forth. They were there.
The Gemini guys were there waiting. They were going to launch in a
few minutes themselves.
When the thing started having problems, of course, we reported it
to Flight, and what we told him at the time was that we saw the power
loss coming off, and we reported that, and then the instrumentation.
This is just both come within a few seconds of each other.
And incidentally, Bernie Brabant was the electrical guy in the back
room, and this points out the interaction between—I talked about
team work earlier. There's only so much your human brain can assemble.
If you sit down and look at a bunch of systems, you only see one at
a time. If one of them does a little something funny, you begin to
watch it to the exclusion of everything else, and rightly so, because
that's probably where your problem is. And the way we work it is,
you've got a guy in the front room that's sort of looking at everything,
you've got a guy in the back room that's looking at just one subsystem
and another guy looking at another subsystem and another guy looking
at another subsystem. So you've got two sets of eyeballs watching
this thing. You've got two people.
The guy in the back room ought to see it first because he's only looking
at a subset of what's going on. Now, if I carry this thought another
step further, let's go on up to the spacecraft. That poor old astronaut,
he's got all the systems, and he's flying it, and he's got two or
three systems he's got to learn. He's got to understand the booster
while he's sitting on it and riding it. When he gets off the booster,
he may swap the spacecraft. He's got to know the Agena and his Gemini.
So the further up the chain you go, the less the guy's got the ability
to know everything, and the further down the chain you go, the more
detail he can bring to bear on it.
Well, Bernie was looking at the electrical system, and when the electrical
power went, the first thing we saw in telemetry was the electrical
power. We hadn't realized the engine blew up yet. Bernie said, "Bob,
we're losing electrical power," and I think it was about three
data frames. We went back later and played it. It's been so long I've
forgotten, but I think that after the first data frame that hit his
console, he was telling me about it, and then the instrumentation
very quickly thereafter, and then the thing went blank.
After it went blank, we had something called a SMEK [Summary Message
Enable Keyboard]. Has anybody talked about a SMEK? Well, a SMEK was,
you pushed a button, and down in the ground, the bowels of the ground
system, it snapped you a picture of what you were looking at at that
instant. So we would do a playback and SMEK it. In lieu of having
strip charts and all of that brought to the front room, we could just
make four or five SMEKs, and we'd get a pretty good time history of
what was happening.
So the bird went blank and had ground systems down there go back and
replay that data for me, and I SMEKed the heck out of it, you know,
and they sent it up to me in a P [pneumatic] tube, and I had it spread
out on the floor there, and that's when I realized what had happened
to the engine. I told Kranz, I said, "Forget this bird."
In the meanwhile, now, Kranz, he's on the loop, and we had the worldwide
loop up. We had guys on the ships covering the ground sites, these
ground teams, deployed. And Kranz, he's got the ephemeris of what
it should have been had it gone on into orbit without telemetry. He's
making the assumption it made it to orbit and we just lost telemetry.
So he's got them all powered up, all the way around the world, waiting
for this thing to come overhead, you know. What's going to happen
is a bunch of shrapnel fell in the Atlantic. But Kranz would not give
up. [Laughter]
And after I got all the SMEKs spread out on the floor and I saw what
happened to the engine, there was enough there to see the thing blew
up, and didn't know why yet. In fact, it was a long time before we
figured out why. Well, he had them powered up and looking for it.
I told Mel, I said, "Mel, what's he smoking? This bird is gone."
But Kranz wouldn't give up. It wasn't until a full lap had elapsed
and no sign of it until he finally surrendered, and probably Mel took
some of the SMEKs up there and showed him. I don't remember. It's
been too long.
Anyway, on that flight, of course, we were dead in the water. Then
the Gemini guys went on, they scrubbed for the moment. They had no
target to go after. They ended up, if I remember rightly, they had
two Geminis rendezvous with each other.
Then the next flight, well, we were in a big panic to try to find
out what was wrong with the Agena, and it took a while to happen.
So we missed the next flight. The next flight, McDonnell Douglas put
together a little target vehicle [ATDA, Augmented Target Docking Adapter].
What they did is took scrap Gemini parts and just threw it together
right quick to make something that would get up there and make a target
and put a docking adapter on it. Then they sent it up into orbit,
and that was to be their target in lieu of an Agena.
The next flight—let's see, Gemini VI and VII rendezvoused together,
two spacecrafts, so I guess it would have been—no, Gemini VIII.
Yes, VIII would be the next one we flew. Well, somewhere, I forget
which Gemini it was, whether it was VII or VI, but somewhere in there,
in that interim period of time, they had this docking adapter for
target. Tom [Thomas P.] Stafford was the crew member that flew up.
He called it "angry alligator." The shroud on it didn't
come apart. Somehow it stuck. More than that, when it got up, when
they turned on its attitude control system, it was like a dog shaking,
and it just hosed out its propellant. I mean, the tank was just emptying.
I remember I wasn't working on it, but Mel come running in, and said,
"Hey, Bob, come in here and look at this and see if you can figure
out what's wrong with this thing." They had shut it off. They
hadn't rendezvoused with it yet, and they'd shut it off before it
hosed out all its propellant, still had some left to try to hold attitude,
but as it held attitude, it just had the shakes. I never did see the
report of what explained that, whether it was something wrong in the
attitude control. It appeared to me at the time that they had used
a thruster that was oversized. They'd used one of the Gemini thrusters,
and the Gemini weighed a whole lot more than this thing did.
If you've got a mass in orbit and you want to attitude-control it,
if you hit it with a thruster to cause some momentum, the amount of
speed of that momentum that you give it, that rotation you give it,
the bigger the thruster, the faster it goes. Or conversely, the lighter
the vehicle, the faster it goes; the heavier, the slower it goes.
So if you have a thruster that's too big, even though you give it
an absolute minimum impulse, it's still going to go real fast. Well,
the spacecraft has a dead band. When it reaches a certain limit, it
fires the other thruster. So I think what it was doing—I never
did see a report, but I think it was just doing this, you know, fire
thruster, bang, fire thruster, bang. And the thrusters are firing
so quick till, you know, you're just hosing down the propellant. I
think that's what it was. But at any rate, for whatever reason, it
was all academic, because the alligator jaws wouldn't turn loose,
so we couldn't dock with it anyway.
After that, we flew the Agena mission. The Agena finally come to bat.
On that mission, we put the Agena into orbit, it made orbit good,
we now had an ox [oxidizer] lead, went back to ox lead on our fuel,
on our engine. We made orbit. The Agena was looking good. Neil [A.]
Armstrong and Dave [David R.] Scott were in the Gemini, and they launched
and did their rendezvous maneuvers, and they came up to it and they
docked to it.
After they docked, now, in those days, you came in sight—we
didn't have continuous contact like we do today. That was sometime
later that we got continuous contact. But to have continuous contact,
you had to have a way to flow the data from every site back to Houston,
and in those days you had a remote site, and we couldn't flow their
data live continuously to us. What we had to do was, we could send
it like a teletype message. So we had the crews on board. That's why
we had the remote site crews all around. So when you launched, the
thing come overhead, that's a remote site, and the crew that was on
the ground could see it for the duration of their pass.
In low Earth-orbit missions, a pass probably is about eight minutes
on average, no longer than about twelve. Now, the higher you get it,
of course, the further it has to travel in its arc and the longer
it takes it to do it, and the longer you can see it. But when you're
down in the altitudes we were in of over 100 miles, between 100 and
250 miles, well, you were in the matter of passes being seven to twelve
minutes, depending on the altitude you were at.
So when we launched, we could see it continuously as long as it was
in sight of the Cape telemetry, and then it went over Bermuda. Seemed
like we had Bermuda data, and we had continuous data to Texas, I believe,
and maybe California. I've forgotten whether we had continuous from
California or not. But the rest of the world was just little windows
as it flew overhead.
Well, Armstrong and Scott caught it, they docked to it, and as they
came over one of the ground sites, seems like it was over in Australia,
we had a team there and it was doing good, and they were getting ready
to do something, and then they came over one of the ships. We had
a ship out there. When it came over the ship, it was wound up. Now,
what had happened was they had a thruster malfunction in the Gemini,
it turned on. Turned out it was an intermittent on; it wasn't solid,
continuous, all the time. Whenever it would start winding up, the
crew thought it was the Agena. So when they finally got it [stabilized]
enough to where they could, they undocked it from the Agena.
Now, they had an emergency I guess you'd call it a return system of
RCS for attitude control, and they had the regular RCS for on-orbit.
This emergency—this return, or reentry, module was much limited
in volume and in total delta availability. Well, what happened was,
as this thing started spinning up, it was in the main system of the
Gemini, and the thing started winding up on them. Well, Neil Armstrong
was fighting it. He was opposing that failed thruster, and Dave Scott
was over there pulling and pushing circuit breakers. By this time,
I guess, they knew a thruster had failed on. If you could talk to
them, by the way, to see their perspective, that would really be something
to add to—have you been able to interview Armstrong and Scott?
Rusnak:
Not yet. They're both in the works to interview, but we haven't talked
to them yet.
Carlton:
Well, anyway, if you looked at what they were doing as it come overhead,
over that ship, the thruster was on and off and intermittently firing.
On the ground, Gerry Griffin was the systems guy on the ground on
that ship. Now, I've forgotten who the capcom was. Well, the ground
systems—he had all of the spacecraft systems, and furthermore,
those remote sites didn't have much—you know, trying to interpret
the data is tough. You had a limited amount of display. I had needles,
a few needles fundamentally. I'm not familiar with everything they
had on there, but it was very rudimentary kind of a system, and it
was just impossible on the ground to tell what was going on.
When one thruster fails on, the system, on-board system, it causes
the spacecraft to rotate and the on-board system automatically fires
other thrusters to oppose it. So everything's on. Which one of them's
failing and which one of them—you know.
So anyway, this thing went on, and all we could hear was the voice.
They were cutting a teletype message every now and then, but it was
slow and after the fact and minutes before it got to you. Fundamentally,
it was just voice. When they come over that ship, we sat there just
paralyzed with fear. You could hear them, you know. It was winding
up, winding up, winding up, and they were at the point where the crew
was on the verge of unconsciousness, according to the aeromeds [aeromedical
doctors] later. Of course, I didn't know that listening to it. I just
knew it was winding up and getting faster.
What I remember the most vividly was Griffin talk about the RCS propellant,
it's so and so, so and so, so and so, so and so. He was reading the
main, I guess, instead of—and then he said they're on the reentry,
they've activated the reentry. So then they went out of sight.
Now, that was a thriller. Scared. I mean, it sounded like we lost
them. I sat there listening to that, and I thought, "Ah-oh, oh,
man, Gerry, tell them which thruster, tell them which thruster, tell
them which thruster," but he just didn't have the data to do
it. As I sat and listened to that, I said to myself, I said, "I
guarantee you I will have my console arranged in such a way that I
can tell a failed thruster. If this ever happens to us again, I guarantee
you I'll be able to tell them which thruster failed, and I mean now,
right now."
I was paranoid about it from that point on, but I found out—you
know, resolving is one thing, doing something else, it's almost impossible
to do. What makes it so difficult to do is two factors. One is that
the system turns on other thrusters to compensate for the one that's
failed. That mass—you know, the effect to you when you look
at telemetry coming out of the bird, the ground systems can't handle
all of the telemetry at the high data rate it comes out of the bird.
So what they do is, at the remote site, they will slow the sample
rate down and send it to you like, maybe—well, something comes
out of the bird at a thousand times a second, you might see it one
time a second. Well, in that one second, it may have changed configuration
three times. So you can't—and even so, your eye, you know, you
can't think much faster than about two times a second. So it's hard
to do.
The effect was, when it come in sight, you're going to see every light
turned on. That's fundamentally what the net effect of it is. We worried
with it and worried with it, trying to do something in the ground
control system to give us the ability to detect a thruster.
One thing we did was arrange the thrusters. We had these event light
panels. So say you're looking up the tailpipe of the bird, and there's
a thruster here and a thruster here and a thruster here and a thruster
here. Now, if you wanted the bird to turn this way, you'd fire this
thruster and this thruster so they'd be in a balanced pair and it
would cause it to rotate. Or if you wanted it to go the other way,
you'd fire this thruster and this thruster and cause it to rotate.
Now, suppose this thruster fired, inadvertently, turned on. Well,
whether the on-board crew grabs a hand controller to correct it or
whether the automatic system does it, how does it correct for this
torque? It will fire this thruster and this thruster. So I went to
great troubles to align these event lights up so that these two thrusters
were right opposite each other. I visualized here's the spacecraft
and here's the thruster. Now, if these two thrusters are firing against
each other, that should not be. I know one of them has failed on.
So how do I know which one has failed on? The one that's right has
got a mate to him down here, okay? So when I see two fire, I know
I've got a failed thruster, and my next thing is to see which one
it is. Got it, right? No?
Rusnak:
No.
Carlton:
What killed me was the data rate. I thought when I put that on the
console and programmed it to do that, I thought, "Man, I've got
it now. I can find the thruster every time. Instantly I'll tell them
which one it is." So we simulated with it, and oh, it worked
beautifully. I was just tickled pink, you know, "We have got
this whipped."
And the first flight we flew, and we got up there and the astronauts
started their little maneuvers, the whole board turned on. Now, what
happened was these thrusters are only on for a millisecond, but the
ground site with this data rate mess, it turned the bit on for two
seconds. So every time a thruster fired, it turned this bit on, so
my light turned on for two seconds and stayed on for a full two seconds.
Well, I didn't appreciate that they were doing that to me on the remote
site till that first flight come over. So that's what you run into.
Well, we finally got that squared away, though.
Then, the other thing, on the Apollo 11—I'll jump from the Gemini
to 11, but I'll show the application of what took place. On the Apollo,
not only did you have these thrusters that were fine-tuning your attitude,
but the big main engine, the landing engine, could gimbal. So if this
thruster fails, it's going to cause the spacecraft to rotate. Now,
the other thrusters will turn on, granted, but the gimbal of the engine,
when it sees this rotation, that big engine is also controlling the
attitude, so—whup—it'll gimbal the engines. It'll just
totally overpower those thrusters, and it'll hold up a dead band—like
if this is the dead band, the engine will hold it there and the thrusters
won't react till it gets out here.
So the way we discovered this, I hadn't even thought about the dadgummed
engine. I was looking at these event lights and also the RCS usage.
You know, these different thrusters on two different RCS systems,
I can look at which one is using the most gas and kind of deduce that
back into which thruster has failed. I know it's in this system because
it's hosing out gas at a different rate.
The sim [simulation] guys, they threw us a failed thruster problem.
On the sim day that they gave us the failed thruster problem, by then
I had it fixed. I had the event lights. But it just so happened that
the day they threw this at us on the sim day, for some reason the
ground systems was messed up and my event lights were not working
that day. Well, this was the day to really work this over, and they
threw me failure after failure. Oh, that was a nightmare sim day.
They just kept failing thrusters and failing thrusters and failing
thrusters.
Kranz said, "Carlton, what's wrong with you?" [Laughter]
There was nothing I could do. I said, "As long as they fail the
thruster, we're going to crash the mission. I just don't have any
way to detect it." Well, they had their script already made up,
and they cut some of them out.
But about that time, one of the guys in the back room—I don't
remember which one of them it was—said, "Hey, Bob, look
at what the gimbal's been doing." And he got to looking, and
what would happen is, this thruster turns on—whomp—you
know, the spacecraft starts to rotate, the big gimbal, and that gimbal
would go cattywampus off way to one side, torquing against the thruster,
and then it would land. The instant you saw it, you knew, "Well,
why didn't I think of that before?"
Well, that's the way a lot of the flight control things was. You know,
the failure, I said a minute ago, about half the failures, you never
thought about them. Half the solutions you didn't think about ahead
of time. You evolved to them after you thought about it.
Once they realized that the gimbal was going to do that, in the middle
of that sim, those guys back there were working fast and furious,
and before the sim was over, they had this little skinny sheet out
there, you know, we had it, and then we were telling sims, "Throw
them at us. Let's practice this a while," you know. We got to
where we could tell which thruster it was just instantly just from
the gimbal, and never even thought about that before.
Well, come Apollo 11 and we're landing on the Moon, we're two thirds
of the way down the trajectory landing on the Moon, and my event light
popped up, a failed thruster. About the time I saw that, I saw him
take over the hand controller, and then all the thrusters started
blinking. I'm sitting there using my logic, but I went to the gimbal
and looked down there, and the gimbal was just as steady as a rock.
Now, what did that tell me? It says there's no thruster failed. If
there's a thruster failed, that gimbal would be off in the left corner.
There was no thruster failed. It just, that quick [snaps fingers],
had it.
If you go back and listen to those tapes, you will not hear one word
about that thruster. If you listen real carefully, you'll hear me
tell Bill Sturm—he was the guy in the back, he hadn't even noticed
it yet, I said, "Bill, when you notice the thruster, it’s
all right, that's the instrumentation," and we didn't mention
it on the loop. We let them land. They were busy as could be and thought
that we don't want to muck up—if we start talking about that,
that's the wrong thing to do. They had a handful trying to land, and
I don't want them worried about the thruster. So we said to ourselves,
if they have another failure or if they notice that thruster—they
had a little malfunction thing on board that would tell them a failed
thruster. Well, if they notice that thruster, I'll tell them if I
have to, but till then, let's just let it ride.
Then after we landed, we went back and talked about it and were prepared
for it and changed some of the procedures just in case. So that was
application of that to the Apollo.
Now I go back and every time I see the sim guys, I tell them, "Boys,
you saved my life. You saved my life." [Laughter] And they did
a lot of other things like that, the training.
That points out something else I think is an experience learned, a
value learned. In every program, at the front end of the program,
all of these designers and the contractors come forth telling us,
"Oh, we're going to have an automated, on-board automated troubleshooting
system, and it will automatically do the right thing. It'll detect
any failure and turn on the valves, turn off the valves, reconfigure
the system." And I just think to myself, "Well, you're smarter
than I was." I don't care how hard you think, you just can't
think of stuff. Then you compound it by each system is different from
what you've been exposed to. Every one is unique, even though it's
similar system to what used to be, the differences make it act different
than the designs you saw earlier.
And who are the guys doing the programming? A bunch of young engineers
they hired out of college and a few old heads there, and none of them
ever had any exposure to facing the problems in the face. They're
like I was back on the -141. You're designing to performance, and
you just can't do it. Now, every system, and I've watched a lot of
systems evolve through the years, the B-58, we told the Air Force—we've
got this automated ground troubleshooting equipment, and then when
it come along the next generation of birds, we told the Air Force
the same thing, and this same thing, and this same thing.
Every generation gets a little better. The modern-day malfunction
diagnostic systems are tremendous, but they gained a lot of experience
and applied a lot of experience, but I still say, you show me a guy
that just going to blithely assure you he’s going to, you know,
you'll get this bird, and he'll be designed it to have a total automated
diagnostic system on it, and most especially if he's going to have
a totally automated control system on it that does everything automatically,
the real weakness of that is you can't foresee everything that's going
to happen, and you can't foresee the system will work funnier than
you—there'll be something different to what you had analyzed,
something you haven't thought about, and get up and bite you every
time.
Well, back to Gemini, Gemini VIII, when Armstrong came over the next
site, the bird was steady and rolling along, and Dave Scott had found
out which one of the thrusters was on, and he pulled a circuit breaker
on it, disabled it, and they'd hosed out most of the gas in both systems,
both the normal system and the reentry system. I don't remember how
much was left, but it had got down low.
We had a mission rule that said you come home if you break the integrity
of the backup system. So John [D.] Hodge, I believe, was the one that
was on the console then. I can't remember now, but it seems like it
was Hodge that made that decision. I might have been Kranz. But anyway,
they said, "Come home," and Armstrong and Scott argued with
them, "Oh, come on, we know what's wrong. We've got it fixed.
We've got enough gas here to go for a while." They wanted to
keep going, but they brought them in.
After that, the rest of the Agenas worked good, and I guess it would
just be a story of more and more of the same.
There was one other mission that had kind of a uniqueness to it. It
seemed like it was the one that Conrad flew. It might have been the
next to the last one of the Agenas that flew or it might have been
the last one. I've forgotten which flight it was. They did a high
altitude—there's two more things of interest in the Agena Program.
Rusnak:
Well, if we can take a quick break to change out the tape before you
get into that.
Carlton:
Okay.
Carlton:
There are two more things with the Agena that's worth talking about
a moment. One is working with a tether. We tethered the Agena to the
Gemini. I forget how long that tether is long. The other thing we
did was we shot them up into a high orbit, used the Agena to—oh,
there's three things I need to talk about. Let me go back to a totally
different one.
I believe it was the first Agena we flew, the Agena, after they got
through with it and left it, this probably was the one that Armstrong
was docked to and undocked from.
Rusnak:
Yes.
Carlton:
Yes. We did a bunch of burns with the thing. The first burn we did,
we pointed this thing and we did a burn, and the delta-V ought to
took it right there. It didn't take it there; it took it over here.
It was like it was pointed wrong. I couldn't figure out what in the
dickens. So we got to calculating it out, and it skewed off the thrust—the
equivalent thrust—now, the Agena had a big engine, and the engine,
it had a lot of thrust. As I remember, it was 10,000 pounds of thrust,
and the Agena was a pretty light vehicle, so it took off like gangbusters
when you did a burn. The Agena engine bell was kind of sloppy to move
and deliberately so. It was designed to burn with a Gemini hooked
to it. That docking interaction there was a pretty weak link. So you
didn't want it whipping that Gemini out there. You wanted it to be
kind of slow to move.
Well, what happened was the Agena had a little bit of a weight imbalance.
In other words, the weight, it wasn't adjusted so that the center
of gravity [CG] was right along this thrust line. So when the engine
started to burn, and it may have been the engine that's cocked off
a little bit to start with. It drifted off just a little bit. Anyway,
the thrust vector of the engine was misaligned from where the CG really
was instead of down through the center line. So when you first fired
it off, the thrust vector is going outside of the CG, and that causes
a turn. It had an attitude control system on it, but it was little
ten-pound thrusters.
So this big thrust vector of the engine overpowered the attitude control,
and there's no telling which way it would turn. Well, Mel Brooks was
programming the burns, and he said, "Well, the last time, it
just went off—" Now, this is before we understood what
the problem was. All we did was we did a burn here and it pointed
there, and we said, "That's all right. We'll allow for it. The
next burn, we'll play like it's going to be pointed there, and we'll
aim it that way." Well, we done that, and it went some—and
I thought, "Mel, you're going to enter this thing. We never will
live this down if you reenter this thing," but we were shooting
out of plane, but if it had been a retrograde, we could have brought
it in on one of those burns. Anyway, we didn't know till later. It
took us a while to figure out what the problem was. We didn't know
it when we was doing all these burns in orbit, but we learned about
the Agena.
The other thing was the tether. We got up there, and they're going
to do this experiment, gravity gradient. Are you familiar with what
gravity gradient is? Okay. Well, we're going to hook the Agena on
this big long tether, and we'll be going in orbit gravity gradient
with one mass down here and one mass up here and hooked together with
this tether. Well, the problem was, that tether got to giving it this,
you know. They fought that thing and fought that thing.
Finally, somehow or other, it damped out. I don't know what caused
it to damp out, but it was a thrill when we did that tethering operation.
You know, normally, as a flight controller on the ground, you know
what you're doing, you know if something goes wrong how to fix it,
and you know what you want to do next. Now guys, when you're hooked
up to this tether, and it seemed like it was Conrad, he starts talking
about it's doing this, you know, and there's big waves. Brooks asked
me, said, "Bob, what should we do here?" [Laughter] I've
never been so helpless in my life. I've got no idea what to do with
that thing. I had no idea. All I knew to do was unhook from it. That
was all I could tell him. But they finally worked it out, and finally
I think it just damped itself out. I don't know that, if Conrad and
the crew up there got to wiggling and took the dampening out of it
or how they got it out.
But I'll tell you one thing, as a principle of flight control, when
you don't know what you're doing, you're in a world of hurt. That's
just all there is to it. [Laughter] If you don't know what's wrong,
you don't know how to fix it. It's a helpless feeling. That's just
all there is to it. It's a helpless feeling. You always went into
every flight knowing there were a few little things that you felt
uneasy about, "I just hope this don't happen." But you did
have a little inkling. I don't think there was anything we ever went
into, we looked, always tried our best to know everything about the
system and knew a little bit about it. Sure as anything, if it fails
where you don't know what to do about it, there's just no more helpless
feeling in the world.
That bring to mind something else. I think I mentioned this in one
of the papers I was writing, the notes to you. You can also rest assured,
when you're on the console and in a mission and things are going fast
and furious, you're not going to find any help. You can pick up the
phone and call to anybody you want, and the guy, when he realizes—it's
kind of like me with that wavering thing there, you know. When you
realize what's at stake or somebody's life might be at stake, boy,
that makes people docile and quiet.
In other words, the point I'm making is, the poor flight controllers
on the console, especially if it's a time-critical thing where it's
just seconds or minutes, it's on their own shoulders. I mean, it's
either going to get done by those guys or it will not be done at all.
Don't think that there's any outside help. There's not. Now, if you've
got a situation that's got many hours or days, then you can go get
outside consultation. But you can't do it on one of these situations
like during a launch with the Shuttle in ascent. I'll guarantee you
if something goes wrong with a Shuttle in ascent, that it's the team
of flight controllers on the ground is the only people that are going
to have any ability to—it's on their shoulders. If they can
fix it, it'll get fixed. If they can't, that's all of it.
But the other thing that'll happen is, invariably, when all the smoke
settles and the mission's over, and then you understand what happened,
everybody in the world will come in and tell you what a dummy you
were. "Why, anybody can see that," you know. Now that you
can see it, anybody can see that. We used to have a saying, "Hindsight,
that's pretty good stuff."
I can't think of anything else in the Gemini Program that is worth
calling to your attention. I'm sure you've documented the whole program
from end to end. In all of the programs we always had a pot full of
experiments they were doing, and they run together in my mind, and
most of them I don't even remember. They always had a bunch of them,
and in any particular flight we looked at them in great detail trying
to maximize the performance out of them, but after the mission was
over, you had another pot full of them. It changed every mission,
and I don't remember one from another. I have no idea, other than
we took pictures on it. They took a lot of pictures.
That tether was an experiment. I don't recall we ever used it again.
Now, I think in recent years they have done a similar experiment,
maybe by the Italians, and they run a wire down, and I think the purpose
of that experiment was to see if it generated an electric current
from cutting through the fields of force of the magnetic fields. Have
y'all run into that? Did it get any electric force out of that?
Farrell:
I think it got so much, it burned out the wire.
Rusnak:
I know they had a lot of problems with deployment on that. I think
they've tried it twice now.
Carlton:
I don't know. I've lost contact. When I saw them doing that, I thought,
you know, "This is similar to what we did on that tether. I wondering
if they're using the same principle of a gravity gradient to hold
the thing stretched tight," how did they keep it tight. Well,
anyway, it was interesting to me. Other than that, I don't know of
any instance where we ever made use of it again, but it sounded like
a pretty good idea. It also sounded like, if you wound them up, they
would pull them out tight on the tether and it would be a good way
of station-keeping two vehicles and not have to worry about them recontacting
each other and not have to hose out the RCS gas to hold a station-keep.
But as far as I know, we never made any more use of it.
I can't think of anything else in the Agena. Now, go back through
your notes, and the next time, if there's something that you think
I failed to talk about, we should have talked about—
Rusnak:
All right. Yes, I don't want to keep you any more today, since I know
you've got some things to do.
Carlton:
Okay.
[End
of interview]