NASA Johnson Space Center
Oral History Project
Edited Oral History Transcript
Christopher B. Madden
Interviewed by Jennifer Ross-Nazzal
Houston, Texas – 14 April 2014
Today is April 14, 2014. This interview with Chris Madden is being
conducted for the JSC Oral History Project. The interviewer is Jennifer
Ross-Nazzal, assisted by Sandra Johnson. Thanks again for taking time
out of your day to meet with us. As we start today, I thought I’d
ask you about your early ties to NASA, and your memories of the Arc
Jet Facility as a kid.
grew up here in Clear Lake, right next door to NASA, in Seabrook [Texas],
since I was in third grade. One of the stories I thought was interesting—one
of my buddies, his dad was a Shuttle astronaut—this was before
we had the first flight, and my friend Todd would talk about Space
Shuttle. I’m like, “What the heck is that?” Didn’t
even know what it was. “My dad’s going to be flying on
the Space Shuttle.” He was a cool kid, and a couple of times
his dad would bring us to work on a Saturday. His dad would go into
work, he’d bring Todd and myself, and we would come in. He would
go to the astronaut gym, which coincidentally ended up being basically
the same building as one of the labs that we own right now in our
group, the Radiant Heat Test Facility, and then also down the street
a little bit was the Arc Jet.
He used to bring us on site, and he would go to the astronaut gym
and work out. Todd and I would go in there and play a little bit of
racquetball. There’s a little racquetball court in there. We
were, like, 9, 10 years old, I guess. We’d also go out and catch
snakes in the ditch. Todd was a big snake guy. We would be messing
around in the ditch; I didn’t know it at the time, but right
next to the Arc Jet. It was a little bit funny that I ended up coming
to work at NASA and basically using that test facility a lot, and
then ultimately having that under my wings as a branch chief.
That’s funny. Isn’t that the way life works out sometimes?
Who was the astronaut?
Musgrave. The other half of that story is not so much related to Arc
Jet. He would take us to the astronaut gym, and like I said, we would
mess around in there for a little while. About eight years ago, I
was here at work, in a big meeting in Building 1, the top floor, and
they were talking about tearing down Building 260, which is the astronaut
gym number. On the other half of the building is our Radiant Heat
Test Facility; they share a wall. They were talking about tearing
that down, and I’m like, “Hey, we have an engineering
test lab there! We can’t tear it down!”
It was, I think, somewhat of a surprise to people because the numbering
system is a little bit weird. The gym side was 260-A, and the test
side was 260. I think there was confusion, “Is it the whole
building that’s going to go down?” Then, that all got
cleared up, and they ended up tearing down the gym side, which is
260-A, and building a new gym down the street a little bit. [Building]
260 itself survived. Building 260 has a history itself because it
used to house the egress test facilities, so they had an above-ground
pool in there, and they would test getting out of the Apollo capsule
properly. They had a big water tank. They would put in a mock-up of
the capsule, and the astronauts would practice getting out of the
Why don’t you give me a brief overview of your career at JSC,
and how you came to be associated with Building 222.
graduating high school, I went to the University of Texas [Austin].
Partway through my freshman year, my brother said, “Hey, you
ought to try this co-op program.” So I went down to the Co-Op
Office and applied for that, and my recollection is this real simple,
one-page application. What’s your name, what are your grades,
and interests? I got accepted, and so that summer after my freshman
year, I did my first co-op tour here. I ended up doing three more
tours as a co-op, and then, upon graduation, I got a job offer and
came to work in what was, at the time, called the Thermal Branch.
Got to work, first assignment was looking at the Shuttle and the thermal
response you would have if the ET [External Tank] door was cracked
open a little bit. The ET doors, they’re probably about three-foot
doors that are opened during ascent. That’s where the oxygen
and hydrogen fuel lines come in, into the main engines, from the external
tank. Once they get to orbit and they’re done with the tank,
the doors close. So there’s a concern over, “Hey, are
these doors going to latch all the way?” because there’s
an intermediate latching point where the door’s not completely
shut. They’re worried about a failure scenario where the door
is cracked open a little bit. If the door is cracked open, how much
hot gas are we going to get in there? Is there anything inside that’s
going to melt? [If so], we’re going to have a breach. Me and
another guy, Matt Adler [phonetic], did that work. Matt did the heating
side and I did the thermal response side. We determined, yes, you
can have the door open like a half-inch, anything more than that there’d
That was my first assignment, and then for subsequent years, it was
assignments like that, a lot of what-ifs on Shuttle. If there was
problems, what’s going to go wrong, and it was all related to
thermal analysis. We would usually take aerothermodynamic heating
rates and determine what the thermal response would be. I did that
for a few years and found that really interesting. We did a lot of
stuff on Shuttle—what if the tile’s damaged, how much
damage can you stand? Did that, and then about, I guess, ’98
or so, this project came along called X-38. It was ultimately called
X-38, it had a few names before that, but the intent was to build
an in-house Crew Return Vehicle for Station in case we needed to evacuate.
Even back at that time, there was some uncertainty going on with whether
or not we can use Soyuz capsules and would everybody fit if we had
a full crew. NASA decided to try to develop our own Emergency Crew
Return Vehicle. The intent was to do all this in-house, all the engineering,
all the build, all the drawings, and everything else. So I started
working on that project and started as the TPS [Thermal Protection
Then my role grew a little bit to include what we call the composite
aeroshell, which is the skin of the vehicle; the design and build
to that is a composite system, which was fairly new for our engineering
team—to design and build composites. We had to build up the
manufacturing portion all in-house, from the ground up. There was
a lot of work during that. We had a real good team and did that. Later
on, I became lead for all the structures work and the thermal work,
so our team had to design all the structures, the TPS, and the aeroshell.
It included some interesting partnerships. The project partnered with
ESA [European Space Agency] for them to deliver the body flaps, which
was a great engineering integration challenge. ESA also was supplying
the rudders, which were on the fins of the vehicle, a leading edge
on the fins, the nose cap, and the landing gears. We had a lot of
European involvement for some of these major systems that went with
our vehicle. That went on. We did a lot of good work there. The project
built several drop-test vehicles to test parachute systems, control
systems, and everything else. My team was focused on the spaceflight
vehicle, which was intended to fly in Shuttle in the payload bay and
return for a test flight, unmanned. We got that vehicle probably 85-90
percent complete, and by that time, it was one of the classic problems,
I think the cost and the schedules got stretched out. It ended up
After X-38 was canceled, I asked my boss, “Hey, what’s
next?” He says, “Well, why don’t you go ahead and
see if you want to do deputy branch chief?” I said sure, and
so I did that for five or six years, and then my boss ended up moving
on to the NESC [NASA Engineering and Safety Center]— a brilliant
guy, Steve [Steven L.] Rickman. Applied for the branch chief [job],
and I got that job, so I’ve been branch chief for the past five
years. That’s where I’m at, now.
It’s been a tough past few years in terms of the Arc Jet. There
has been several campaigns to close that test facility. The old-timers
have stories, “Years and years ago, teams would come through
and did assessments,” and it’s like, “Hey, we need
to consolidate facilities.” It always turned out, “No,
we still need this one.” A more recent one, in the 2000 timeframe,
was very close to shutting down. We, in fact, ended up reducing the
staff to what we call the bare bones staff, ended up laying off good,
talented, and skilled people. We’re running it bare bones, and
the Columbia disaster [STS-107] occurred.
There was a lot of testing required, number one, to determine the
cause of the accident. We did a lot of testing in terms of the investigation
for Columbia, and then after the Agency decided we’re going
to fly again and started getting serious with that, we did a lot of
testing for the repair systems, and some more failure scenario testing.
The facility was very active for the Columbia investigation and subsequent
return to flight, and it remained very active up until just a few
weeks ago, when we were finally ordered to shut down. The past 10-12
years, we have always had a test article waiting to be tested, 100
percent occupancy, and 100 percent utilization. [We] just always had
a test demand. So it was really tough for my guys and myself, trying
to come to grips with a decision like that, where we have a huge demand.
The engineering community needs a facility like this, yet we’re
being told we must shut down and move everything to Ames [Research
Center, Moffett Field, California]. It was tough to come to grips
Some of the interesting tests on the Columbia investigation, like
I said, we were real involved there. In fact, this was probably a
year into the investigations. Sifting through all the data, the CAIB
[Columbia Accident Investigation Board] started focusing in on a scenario
that we thought might have happened. We set up a test where we put
a wire bundle that was a replicate of the wire bundles in the Shuttle
itself, and we made a little fixture and created a hole in a piece
of aluminum, to replicate the front spar of the wing. We put it in
the Arc Jet, had flow going through that hole and over these wire
bundles. We had continuity on the bundles, and so we can see, with
the hot gas that’s blowing on to the bundles, the rate at which
the signals dropped out. We’re able to correlate that drop-out
rate to the drop-out rate seen in flight. That was a really key test
because it was very eerie in that we think it just replicated what
actually happened inside the wing. From that test, we were able to
back out an assumed hole size on the RCC [Reinforced Carbon-Carbon]
panel, through analysis. Just about that time, within a week or two,
the team that was investigating the foam impact strikes at Southwest
Research Institute [San Antonio, Texas], on full-size RCC panels,
replicated the same hole size that we surmised from the Arc Jet tests.
Everything came together all within a couple of weeks, saying, “We
had a probably 10-12-inch hole in this wing,” and that fed into
the final conclusion for the accident.
You testified in front of the CAIB. Were you talking about the testing
early on, they had questions. They just wanted to get some background
on aerothermodynamics, thermal analysis, TPS, aerodynamics, so myself,
Steve [Steven G.] Labbe, and Joe [Jose M.] Caram just answered a few
questions on general technical questions on re-entry and how that
works, and how TPS systems work.
You also mentioned you guys were doing some work on the repair test.
Can you talk about that a little bit?
so for repair, there was two camps, and boy, these meetings, these
were big meetings with hundreds of people in them. There was always
this tension and friction between these two camps on how to repair
the tile portion of the Shuttle. It was a little bit off-topic, if
you will, because we think what really happened, we had a hole on
the leading edge, which is a carbon system. The tile repair camps,
there was a camp that wanted to put just an overlay. Let’s say
we had a big damage on tile, and they wanted to just screw with auger-type
screws an overlay, a cover that just covered the damage. Then the
other camp wanted to have STA-54, which is a rubber foam-type material;
they were developing a gun to apply that material, to fill up a gouge
with the filler material. There’s a lot of tension, a lot of,
“Hey, this won’t work, that won’t work!” There’s
a lot of difficulty in either one, in executing them. It sounds easy
at first, but how you brace yourself and apply that material was a
little tricky. We did a lot of tests in there. We would carve out
some damage in panels of tile and fill it up with varying levels of
material and different formulations, so the material went through
several different formulations. We did a lot of tests to try and perfect
What did you guys find?
thing that was a complicating factor is they would swell up. They
would heat and swell and bubble up, so now you have effectively a
hump on your aerodynamic surface, so does that induce boundary-layer-transition
earlier downstream? Does it cause localized heating that will further
damage it? We started getting smart enough about the material response
to figure out how much it would swell, so the intent was to under-fill
it by a quarter-inch or so, and allow it to swell, so it doesn’t
stick up as a hump.
Did they end up using it on orbit?
Before we got to the return-to-flight flight [STS-114], we spent about
a year developing test articles that were prefabricated ahead of time,
so that if there was some sort of damage in flight, we can try and
replicate it on the ground on these test articles, test it in Arc
Jet, and see whether or not we would need to repair. If we did repair,
would the repair work? We had all that set up ahead of time. Then,
one of the flights, [STS]-117, there is a bit of a blanket that was
peeled back on the OMS [Orbital Maneuvering System] pod. It really
didn’t look horribly bad. The problem was, on the OMS pod, it’s
a graphite sandwich, and the graphite doesn’t do as well as
aluminum in dissipating heat. If you had the blanket lifted up a little
bit, you’re going to get some hot gases in there and heat up
that graphite epoxy skin, and it wouldn’t dissipate that heat
as well as something like skin-string or aluminum. The teams, they
couldn’t really clear it analytically. This was during the mission.
We didn’t have a test article set up ahead of time like we intended.
We can’t build the whole vehicle ahead of time. We had a guy
come in and try to replicate that area, which is tile that terminates
into a blanket that bucks up to it. We decided, “Instead of
testing to see if it’s going to be bad, let’s go ahead
and test and see if repair’s going to work.”
We built a test article and then damaged it like it was in flight,
peeled back the blanket, and then the MOD [Mission Operations Directorate]
guys came up with a repair technique. What they came up with was to
use actual surgical staples that they have in a dock kit, so they
took in that little staple gun. We went out there and stapled the
blanket back together, as they would during an EVA [Extravehicular
Activity]. We put that in the facility, tested it, and showed that
the repair worked. That was interesting because it was an arduous
process. We were up a couple of nights in a row, trying to get the
tests off, and finally get it off right before one of the morning
MMTs [Mission Management Team meetings], where they had to decide
whether or not they’re going to do the EVA or not. We came back
from testing and says, “Hey, yes, it’s going to work.”
Nancy [J.] Currie ran over to the MMT and said, “The repair
is going to work, let’s do it,” and they decided to do
it. Danny [John D.] Olivas went out there during the EVA and repaired
it, just like his astronaut buddy did on the ground. Came back, and
it looked great.
It was an interesting test because we had video of the test, and you
can see the staples. We also had these pins with a ring on it, and
you can see them glowing, but they held together, and it worked great.
The very next mission, there was a deep damage in the tile—it
was all the way down to what we called the filler bar, which is the
very bottom pad between the gaps of the tile. Through the inspection
system, they were able to get what we called the point cloud; it was
just a set of data, X, Y, Z coordinates for the surface of the damaged
area. We were able to take that data and go to a CNC [Computer Numerical
Control] machine on one of our test articles, hog out, and replicate
that damage just like it was in flight. That test was the corollary
to the previous test: here, we’re testing, “Hey, is this
damage going to be a problem?” We don’t really want to
repair it, but the teams were getting ready to do the repair. We recreated
the damage, went and tested it, and we probably over-tested it.
We chose a boundary-layer transition. You have to guess on when the
boundary layer is going to transition. The turbulence is a whole new
field, a whole separate field of science on trying to predict when
boundary layers go from laminar to turbulent. It’s important
for reentry vehicles, because the turbulent heating is much higher
than the laminar. You have the hot energy much closer to the surface,
gradients are higher, and it just pumps more heat in. It’s a
higher heating rate on the surface. We tested it, and like I said,
probably over-tested it because it formed what we call a wormhole
in the test article. Once you start forming any sort of little cavity,
what happens is what we call the view factor to space is reduced in
that cavity. It’s a propagation problem, so inside of that cavity
gets a little bit hotter and it gets deeper, as it gets deeper, that
view factor to space gets lower and lower, and so it exasperates itself
and it grows. We’ve got this wormhole where we basically melted
out some tiles a good three or four inches into the downstream tile.
It looked horrible. We’re like, “Crap, what are we going
to do? We’re going to have to repair.” After thinking
about it a little bit more, we’re looking at the aluminum temperatures
on the back side. Aluminum really didn’t get very hot; it just
got a little bit hotter than it normally would if it had a virgin
tile. What’s good about aluminum is if you have local damage,
that extra heat you get in to that aluminum will actually conduct
out. It’ll almost cool itself via conduction in the aluminum
skin itself. What we decided was, “Hey, the aluminum was great.
Sure, it’s ugly, but we’re not going to have a breach,
and it should be good.” The program made a tough decision to
go ahead and come back in without a repair. We came back, and it did
not have the wormhole. Of course, the subsurface aluminum looked great.
There was no problems there, so it was another real-time test we did
that helped us fly the Shuttle.
It sounds like Columbia really changed the way the Arc Jet operated.
don’t know about changing it—it’s certainly a ton
of things to do. Like I said, we were just really busy for years and
years and years and remained so. One thing it did do that we didn’t
like was it didn’t leave us any time to develop better ways
to test. My guys got some good ideas on better ways to do this. You’ve
got to understand, this technology was developed on the fly, back
in the mid-sixties, so we can help the space program itself. They
didn’t have the tools we have today. They put stuff together,
turned it on, and see if it’d work, and kept trying until it
did, just trial and error. They’re getting to a point where,
“Yes, this is working. We’re getting this hot gas. We’re
able to float over a test article, and the chemistry’s somewhat,
we think, similar.” There were some improvements made in the
seventies on different types of heaters; there’s three or four
different heater designs out there, but we think there’s probably
better ways to do it.
We think we have ways to replicate the re-entry environment from Mars,
which is very difficult. Coming back from Low-Earth Orbit, a very
high-energy event, it’s very underrated in terms of its danger
and its potential for disaster. We saw [it] on Columbia. Coming back
from the Moon it’s even tougher, because you got to realize
that all that kinetic energy has to be dissipated in the form of heat.
Your landing velocity is basically, for argument’s sake, zero.
All that kinetic energy, flying at mach 24, 26,000 feet per second,
½(mv2), gets converted into heat through the laws of thermodynamics.
Low-Earth Orbit is tough, coming back from the Moon is tougher, and
then coming back from Mars, you call it an order of magnitude more
difficult because of the energies involved.
What happens when you come back from Mars is—and a little bit
for the Moon, but for Mars, especially—those high velocities,
it’s really not convection-type of heating around the spacecraft;
most of the heating is a form of radiation coming off the shock itself.
The shock gets so hot that it’s radiating like the Sun and just
is a big heat lamp right in front of your vehicle, the whole time
you’re flying through until you dissipate all your energy. These
Arc Jet facilities test, and they test convector plates; it’s
hot gases flowing to form a convection, whereas coming back from Mars,
it’s mostly going to be radiation. We have ideas in using the
high-powered lasers to replicate that radiant heat at the same time
as the convective heat and try and develop ways to simulate coming
from Mars, which really, there’s not much capability right now
in testing the Mars return environment. We regret, and it was unfortunate
we didn’t have time to develop these techniques because we were
so busy testing. It was always people basically pounding on the table,
“Hey, I need to get my test data; I need to get my tests,”
and we didn’t have the time, [during the] past 10-12 years,
to do any of that development. We, as an Agency, need to have that
Were you primarily working Shuttle flights at that point, or were
you also working on different programs and projects, looking at different
so much material development. The flagship test for that facility
back in the sixties and seventies and eighties was material development.
It’s great as a materials tester. For Shuttle after Columbia,
we’re testing systems. We’re testing repair systems, we’re
testing gaps, we’re testing different configurations. More of
what I would call system tests as opposed to material tests.
That was the other thing, we didn’t have a lot of time to do
material development, nor test it. Ames was doing some of that. There
was progress made within the Agency, but here at the JSC facility
we got away from material development tests. The facility’s
good for material testing. When I say “material testing,”
it’s, “Will this material perform well as a thermal protection
material? Will it survive? Will it maintain its shape? Will it insulate
and protect whatever structure is underneath?” It’s well
suited for that because for the most part, it recreates the different
forms of energy flowing around the vehicle.
There’s thermal energy, of course, in the flow field, but there’s
also chemical energy, in the fact that the flow gases are so hot that
the atomic molecules in air—nitrogen and oxygen—those
molecules split into atomic forms of atomic oxygen and atomic nitrogen.
The atomic oxygen, it’s especially important because oxygen,
as you know, likes to react with everything. Even at room temperature,
burning of a flame, that’s oxidation of the wood. Oxygen wants
to be involved in that reaction. At higher temperatures, it wants
to react with even more things, and when it’s atomic, it really
wants to react. When it reacts with something, it’s going to
do damage to it, almost by definition. It’s important that for
re-entry simulation, that you do at least a reasonable job in replicating
the chemical environment as well—that is, the atomic oxygen.
Arc jets are good in that because it gets the gases hot enough where
the enthalpy is high enough where the oxygen gets disassociated. You
can see if you’re going to get this atomic oxygen attack on
Can you explain, for the reader who may not be aware, how an Arc Jet
works? You’ve given some indication about gas and thermal and
radiant heat and things, but can you give a summary?
got to think about your spacecraft. You’re flying through the
upper atmosphere at a very high velocity, it’s hypersonic, Mach
24 is usually what you’re coming in at for Low-Earth Orbit,
and you develop a shock in front of it. If you can just picture a
molecule in front of the shock, it’s at the ambient temperature
up in the upper atmosphere; it’s actually pretty cold, it’s
probably –100 degrees, and it very rapidly gets accelerated
or decelerated, depending on your vantage point, through this shock.
The energy level goes up, so if you think about it as decelerating,
it has a cool temperature and a huge velocity, and it suddenly decelerates
through the shock, so that has to be converted into heat. The temperature
of that molecule rises very high. If you were to look at just a straight
MCCP [phonetic] on the temperature, it’d be tremendous, it’d
be like 10,000 degrees Fahrenheit, the temperature of the Sun.
After you get so hot, it doesn’t make sense to talk about temperature.
Temperature’s not a good parameter to talk about because temperature
really is the average vibrational energy of a molecule, and heck,
it’s not even a molecule anymore. A lot of the energy is tied
up into splitting those molecules into atoms. The word “temperature”
doesn’t apply anymore. Of course, you have this real high-energy
gas that’s very hot, and it’s also very chemically active.
It’s split into these atomic species, like I said. In Arc Jet,
we want to replicate that, but we don’t do it hypersonically.
Instead, we take cold air, cold nitrogen and cold oxygen, insert it
into a heater, and actually pre-heat the gas in our heater. Then the
gas is accelerated into the test chamber, and it’ll cross a
supersonic shock, probably Mach 8 or so, rather than Mach 24. The
sin, if you will, in testing on the ground is you have to pre-heat
the gas and cross a mild shock, versus flight, where you have cold
gas crossing a strong shock. Ultimately, behind the shock, where the
gas is flowing around the test article or the vehicle, we hope that
it’s similar environment when we test on the ground. We’ve
done a lot of diagnostics to see how similar it is.
How do you heat the gas? We insert the oxygen and nitrogen into this
long column, and it’s actually a column of copper segments that
are insulated electrically between each little segment. In one end,
there’s an electrode, the cathode, and the other end is an anode.
The anode actually has a hole in it to allow the gases to flow through
it. The gases get injected into this heater, and we strike an arc
from the cathode to the anode with a very high voltage and current.
It’s where the Arc Heater or Arc Jet comes from. There’s
an electrical arc on the inside that heats up and excites the air
as it’s inserted into the heater, so that’s how it gets
preheated. Then it flows into the test chamber, crosses the shock,
like I said, and flows around the test article to provide your test.
It takes a lot to do that, though. We need a lot of power. Our facility’s
rated at 10 megawatt. We’ve tested up even higher, up to 12-13
megawatt. The energy densities are tremendous; focusing all the energy
into tiny three-or-four-inch diameter test specimens, we need a lot
of power; we need a lot of cooling water.
What happens is that arc, it’s heating up the gas, but you’re
heating up the heater itself and you’re heating up the walls
at the test chamber. All those are copper and have to be cooled with
water-cooling loops. Part of the replication is getting the pressure
right, so most of the heating for re-entry spacecraft are pretty high
in the atmosphere, and the density’s fairly low, the pressure’s
fairly low. Our stagnation pressure, during peak heating, it’s
100 PSF [Pounds Per Square Foot] or so. In order to get that low pressure,
we evacuate the test chamber with a vacuum system, keeping in mind
we’re constantly adding gases to it. We constantly have to remove
those gases, number one, to accelerate the flow, number two, to keep
the pressure down and get them out of there. You can’t do it
with a roughing pump or a turbo pump or anything else. The only way
we’ve found to do it is with a steam injector system.
We need a bunch of steam, so we have a dedicated boiler in the facility,
out back. That high-enthalpy steam is routed to these ejectors, which
are really simply little horns inside our diffuser system, which is
just a big tunnel attached to our test chamber. You blow the steam
in through these horns and through motive forces and Venturi effects
and momentum, it just sucks the air with it. You have to do it four
different times, because each time you only get a certain amount of
pressure drop. We have four stages, so the steam is injected down
this tube. One hundred feet later, you got another steam ejector,
and it happens four times. Now, you’ve got to start cooling
that stuff too, and you have to condense the steam. We have an industrial
system out back that condenses that steam, cools our diffuser, cools
everything else that needs to be cooled, and it goes through a big
cooling tower. We got a pond, as well.
The cooling water on the Arc Heater side has to be de-ionized and
very pure in order to remove the electrical conductivity. Very pure
water isn’t electrically conductive. The only reason why it’s
electrically conductive in a rainstorm [is] because it’s got
dirt and ions and everything else that would conduct the electricity.
If it’s pure water, it’s a good insulator, so we’ve
got to keep that water pure, and it’s on a closed loop. That’s
one of the complicating factors, we don’t want electrical conduction
from the heater to ground somewhere else because the arc will attach
to that, and there’d be problems.
We had a real good video system. One of the key products, if you will,
is video. Steve [Steven V.] Del Papa invented a real nice video system,
where he took basically still cameras and just took pictures, 15 times
a second, like you would on a cartoon. It’s real simple in terms
of cost and set-up, but it results in tremendous amounts of data because
you have full resolution picture of your test article, 15 times a
second. You add those up, and then you start getting into lots and
lots of data. Our solution to that was just to brute force it at the
time. Hard drives got to be real cheap, so we just bought terabyte
hard drives and kept filling them up. It made for a real nice high-def
finish, and a video system on the cheap.
You took us on a tour of the facility, and it looks like there have
been changes made since the building was first designed in 1967.
The guys that have been around, like Jim [James] Milhoan, even before
’67, they did some testing with Arc Jets out in 260-something.
It’s where they actually have the key shop, now, for all the
keys on site. They make them in this little building. That [was where]
they had the first Arc Jet, and it actually was much smaller. It exhausted
into the high bay, the room itself. We got some pictures from the
old times. There’s guys standing right next to it, who are turning
it on—for short durations, presumably. That was back before—
Before OSHA [Occupational Safety and Health Administration]?
Testing sounds pretty complicated. How long does it take you to prepare,
and how many people are involved in this whole effort?
of the simpler tests where you’re just taking a puck without
much instrumentation—the instrumentation is what is [complicated],
in terms of the hours we need [to prepare]—but if it’s
a real simple puck, if you will, on just a carbon holder, the guys,
they got to put it in there and they got to pack insulation around
it, so that just takes a few hours. We like to take pictures before
of every test article, so we do photographs. We log them in, try to
keep track of every test article we’ve had. The facility itself
needs to be in good shape. For any test, simple or not, the facility
has to work. There’s a lot of maintenance because it is high-energy
density, so any time you have a high-energy or high-energy density
spacecraft, airplane, car—compare an Indy car versus a regular
car. That’s why they fail so much. They’re operating at
the edge of the envelope. It’s very high-energy, high horsepower;
they have a high failure rate because of the energy, really.
This takes a lot of maintenance. Most of the systems are single-string,
so if one of the transformers isn’t working, we don’t
have electrical power; if one of the injectors isn’t working,
we don’t have a vacuum; if the boiler’s out, we don’t
have vacuum. If one of the water cooling pumps is out, we don’t
test. There’s a lot of maintenance in terms of repairing pumps
and stuff that go out, and preventative maintenance. That’s
why it’s tough to run these facilities. There’s not a
ton of them around, and with everything being single-string it makes
the maintenance part tough. We’ve had periods of time where
we just couldn’t test for months at a time, and weeks at a time,
days at a time. [That] comes up periodically, and it drives the customers
crazy, because they’re waiting for the test. We’re like,
“Hey, we’re down, we cannot test.”
Maintenance is a big one and then preparing the test article. Our
test directors will meet with the customer and figure out how they
want to test. “Do you want us to focus on the heat rate or pressure
or enthalpy?” That’s the parameters that they toss back
and forth. “Do you want a shear-type test where it’s more
of a shear flow, like in a wedge, or do you want a stagnation test?”
There’s engineering work upfront, designing the test itself.
Like I said, preparing the model, preparing the facility, and then
installing the model into the arms or into the chamber in the test
cell. That’s one thing—we always had great plans to improve
our way of getting specimens in and out of the test chamber, because
right now we’ve got to do a lot of hands-on work inside the
chamber itself. If there’s a guy in there working, we can’t
test, of course. If we had a system where he can be doing his integration
outside we can be testing. Then if we get the system rolling, that
test article or that assembly gets bolted in, and then he’s
working on another one while they’re testing that one. We never
really had a chance to perfect a system like that. It’d take
some time and money to do.
It sounds like there’s a lot of work prior to a test. The test
is probably a very small effort.
You can spend a day or two preparing for a simple test, and then the
test itself is usually just a few minutes long. Shuttle was an interesting
spacecraft because it’s got a large wing area, so high lift
to drag ratio. That’s what made it successful. It’s able
to keep its lift up and fly in the upper atmosphere, modulate the
energy disposition, and spread it over a longer period of time, so
that the heat rates weren’t so high. What happens with more
of a capsule-type shape, like Apollo or MPCV [Multi-Purpose Crew Vehicle],
is it doesn’t have as much lift, and it’s diving into
the atmosphere quicker and hotter, so you have much higher heat rates
on a capsule like that. It’s a shorter duration, but the rates
are higher, so it’s a more peaky profile on heat rate. For Shuttle,
we’d do 10, 12, 15-minute tests because that’s how long
the heat pulse lasted.
In fact, there were some RCC tests for mass loss. I’ll explain
that in a second, but there is some tests that were hours and hours
long, all in one test. If you were talking about a one-time, single-use
system, on a capsule, the test is just a couple of minutes long. Those
RCC mass loss tests, I was mentioning the oxygen in the flow field
wants to combine with anything, so for a system like reinforced carbon-carbon,
like the leading edge of the Shuttle, it’s made out of carbon,
and that oxygen wants to chew it up. They had to build a protective
coating with silicon carbide to protect the carbon underneath because
if you didn’t have that, it would just go away in seconds from
the oxidation. That coating isn’t perfect—it’s got
a bunch of tiny holes in it—so over a period of time, there
is actually some oxidation of the carbon underneath the protective
coating on those RCC parts.
Don [Donald M.] Curry went through a big test program to quantify
how much mass loss you get and then correlate that to how much strength
you lose over a period of time after you’ve flown so many missions.
From all that testing, they were able to develop, for each panel,
how many missions it can last before it has to be replaced. Some of
them, some of the hotter panels, they would only—I’m going
to get my numbers wrong—they’re only good for 10 missions
or so before they had to be replaced, whereas some of the cooler panels
can go 100 missions without having enough mass loss to be a problem.
You had a good point—usually, it’s a few days to set up
for a test campaign, and then each test is a couple of minutes long.
We had a system set up, and Ames has this as well, where we can test
multiple test articles in one run. The hard part is firing up the
Arc Jet itself, so we try to take advantage of that and test more
than one specimen in each Arc on, or each run.
How many people are working when there is a run, and you’ve
got that control center in the building?
have a little control room. Nowadays, there’s a power operator.
He’s basically in charge of the power to the heater and dialing
in the current and the voltage that’s required. The test conductor,
actually, he’s the one throwing switches, turning valves on
and off, inserting the test article, and putting all the commands
into the control system. We have a quality engineer, usually, that’s
witnessing the test.
Then we’ll have a test director, who’s the leader of the
whole test. He gives all the calls, so he’s like the Flight
Director. He’s saying, “Hey, go ahead and insert the arm,”
and then the test conductor presses the buttons to do the things.
He’s calling the shots. He’s also looking at the data
coming from the test article and from the facility. We monitor pressures,
we monitor enthalpies, temperatures. Most of the time, the test requestor
or test article expert’s sitting right next to him, looking
at the data, saying, “Hey, I don’t like what’s going
on, let’s pull it out,” or, “Hey, leave it in there,
it’s supposed to look like that.” We also have a data
person that’s running all the data systems and making sure the
data gets recorded properly. There’s a guy in the boiler house
operating the boiler. He’s on the headset, but he’s doing
things to the boiler and making sure it’s producing enough steam.
When you were working on the X-38 and working on its thermal system,
were you doing testing out at the Arc Jet?
Now, we didn’t have a huge campaign because we’re using
systems which we had a competence in. The Europeans tested some of
their materials there, for the body flap and our rudder, then we had
some tests of the gaps, and some of our discontinuities.
That brought up another question. Space Station is such an international
effort, do the international partners have a chance, or have they
had a chance to test at the Arc Jet Facility, besides ESA for the
X-38? Do they ever bring stuff over?
very often. There’s a similar facility in Germany that ESA owns,
and then Italy built a really nice Arc Jet. They’ve run into
tough times, and I think it’s associated with the economy and
the politics in Italy. They haven't been running for a couple of years,
but they had a real nice complex. I went there one time. They’re
able to test, theoretically, to high heat rates, high pressures, and
pretty nice test envelope. Like I said, they’ve been down for
a while. I think a lot of the Europeans tested there and in Germany,
as opposed to dealing with us and our ITAR [International Traffic
in Arms Regulations] policies.
You mentioned that there weren’t that many Arc Jet facilities,
but there are other Arc Jet facilities. How is JSC’s facility
different from this one in Italy and Germany, Ames, I think there
was one at Langley [Research Center, Hampton, Virginia] at one point?
There’s one in Russia I know very little about. In the U.S.—our
facility we had here at Johnson, and then Ames has a complex where
they have, I think, four test cells now, with different types of heaters
and set-ups. They’ve got a nice operation. Langley has a small
Arc Jet. It’s actually pretty cool because one or two guys run
it. It’s just set up in a laboratory somewhere, so it’s
easy to get in and out of there and fairly cheap because it’s
nice and simple. It’s just that the sample size gets small once
you’re talking about smaller facilities. It’s good for
preliminary evaluation, but for systems, if you’re going to
test the gaps in a tile system or a TPS system or test repairs, you
just can’t do it. Test article size is the main thing, and even
material tests, once you talk about small sizes, you get a lot of
3-D effects that you just can’t get rid of because it’s
so small. Since heat shields are usually bigger, the 3-D effects aren’t
very great. Most of the heat flow is one-dimensional, so you want
to simulate that in your test. Let’s see, Boeing had a good
one. In St. Louis [Missouri], they had their own facility, and it’s
gone back and forth. It’s hard to keep up with them on how active
they are. I think they’ve pretty much gotten away from that
operation. In fact, one of our last tests we did was for Boeing.
There’s a big, classic difference between JSC and Ames. We’re
certainly doing some things that are a little bit different. We developed
a capability to test CO2. This is if you wanted to simulate the environment
during the Mars entry, which can be done. It’s been done certainly
without a CO2 Arc Jet in terms of developing a heat shield for Mars.
It’s been, as everybody knows, many success stories. Once you
start getting to larger class missions, like a manned mission, we
feel you’re going to need to test in a CO2 environment to get
closer to replicating a Mars entry. We developed techniques to use
lasers to interrogate the flow field, and actually determine the velocity
of the flow, the chemical component, the thermodynamic component,
and the kinetic energy component. It just goes back to what I was
telling you about the chemistry, how similar is this flow field to
the flow field you get in flight. You end up going through that critical
thinking and that assessment and those analyses.
Actually, in partnership with Ames, [we] developed that technique
to interrogate the flow with lasers. We also have some unique capabilities,
in we can test low pressure, low heat rate, so different areas. If
you were to plot the flight envelope for say, MPCV [Multi-Purpose
Crew Vehicle], it’s not always high heat and what you might
call mid pressure. There’s also low pressure/high heat, there’s
high pressure/low heat, there’s different corners. We’re
able to do some work to get these unique corners of the envelope.
Those were some of the, “Hey, Ames, if y’all can do these
things, then we’re not so unique,” right? They’re
tasked with developing these different techniques so that the Agency
would retain the capability. People will tell you that Ames facilities
are much higher power. Their power system is rated to a higher power,
but we’re able to hit some of these high heat rate, high pressures.
Not quite as high as them, but we’re pretty competitive with
them in terms of the high-energy, high-pressure test points.
Their heater is a little bit different in its design, that was the
other big thing. They use compressed air as opposed to we manufacture
our air, in that we insert nitrogen and oxygen separately. What that
allows us to do, is it protects our cathodes, our electrodes, so we
don’t insert the oxygen close to the electrodes. Otherwise,
they would burn up, so we shield them with the nitrogen. Since Ames
is injecting air, they needed to shield their electrodes with argon,
which is an inert gas, but there’s always been an argument,
“How much does it affect the results?” That’s one
reason they took a lot of our heater parts. We gave them a heater
a year and a half ago, I think, so they can have the capability to
test with simulated air, rather than compressed air, and get rid of
What role has the Arc Jet played in testing of the MPCV Orion capsule?
been huge. You got to go back to Apollo for that lunar return. They
developed a material called AVCO, which is an ablative system, the
TPS system, used on Apollo. After Apollo, all the focus was on Shuttle,
of course, and developing a reusable system, which at this time was
a great innovation, in that you don’t have to replace the thermal
protection system or the heat shield every flight. There was a lot
of energy spent in developing these reusable systems, and it was a
great success story. This is before my time, of course, but they had
to drop ablative systems. That’s where NASA’s going—we’re
building a Shuttle. We’re going back and forth 100 times. They
developed these reusable systems, dropped the ablative systems. The
company that made it put the formula away and started working on something
Actually, a little bit before MPCV came along, some of our guys recognized,
“Hey, we can’t lose this capability forever. What are
we going to do if we want to build a capsule and explore outer space?”
They were able to convince [NASA] Headquarters [Washington, DC] to
form this program called ADP [Advanced Development Project], really
for the sole purpose of retaining the skills and retaining the capability
in the country to build ablative systems. They put a call out to industry
and had people respond with their best ablators. Since the Apollo
days, [the company] who used to make AVCO—I think the company
was Avco—traded hands a few times and I think it even traded
since the ADP days to Textron. There was also PICA, which is Phenolic
Impregnated Carbon Ablator.
The ADP did a lot of good tests and a lot of good analysis to develop
these materials and get them back. MPCV started firing it up and had
to pick a system, and initially, they base-lined PICA. Then there
was problems discovered with building PICA on a large scale because
it’s built in blocks, if you will, so now you got to dice it
up. The MPCV program went through a re-down-select, and actually changed
the system to Avcoat. Textron had to fire up their production floor
to build it on a larger scale, and they ultimately did. The guys got
EFT [Exploration Flight Test]-1, the heat shield built. It’s
ready to go. That was a long process to get that capability back up
in the United States. It’s still a bit tenuous because now Textron
has got to wait around a little bit before EM [Exploration Mission]-1,
and they just can’t have guys sitting around. It’s a tricky
thing, building these spaceships one at a time, spread apart. A huge
program like Shuttle is so big, it’s there to keep everybody
active all the time, whereas a smaller MPCV, it’s a little bit
The Arc Jet did a lot of work for developing the Avcoat, doing tests
on PICA. We have to do a lot of testing to get the properties. It
went through different formulations. To get the properties and everything
else so that we can do analytical predictions on how it’s going
to behave, those predictions are impossible without having Arc Jet
testing. There’s a lot of testing, just straight on the material
itself, determining its properties, that we need to do our ablation
analysis. The other interesting part of the Avcoat story is when Textron
went to go make it for the first time, during the ADP, we tested it
and it did horribly. It just crumbled apart. They had the recipe back
from the Apollo days; they followed it. It’s just one of those
weird things; it’s like cooking: the recipe says to do this
but you really need to use a cast-iron skillet as opposed to an aluminum
skillet. It makes a difference; it’s just not written in the
recipe, or there might even be something in there that you throw in
and it’s not written down. They did a really good job, but they
struggled a little bit in recreating that material because it’s
just not as simple as putting parts A and B together and mixing it
up. There’s a lot of nuances that they had to re-perfect to
get it to work right.
How funny. I can imagine. I would also think that materials changed
just slightly over time.
and there’s materials that get outlawed. They did have to make
some substitutes due to, I forget what laws, but some of the materials
you just can’t use anymore in the industry.
Do you guys keep track of how many tests you guys have done over the
years at the Arc Jet facility?
We probably ought to take the time one day to go count them. We test
about 200-300 a year, has been our clip recently. I think one year
we did 400-and-something.
How many people work at the facility?
How many people worked at the facility?
not super clean, but on order of 20. For a while, we were running
two shifts during Return to Flight. Before Columbia, we’re down
to bare bones, and then we had to build back up. That process wasn’t
immediate—it was a good three, four years before we really felt
a lot of confidence, and we’re blowing and going and had some
good inertia and some continuity. For our first few years, we just
clunked along; things would break. Our PM [Project Management] plan
wasn’t very good, and we’re just basically inefficient.
It took a long time to get into a mode where we’re really humming
along nicely. Probably two years ago, we’re doing great. Just
real good reliability, we’d be turning that Arc on any time
we wanted, and morale was really high. Guys were doing real good.
The shut-down process killed that. That was another thing that was
disappointing. It’s like, “Man, we got this place running
how we always wanted, it’s really doing the best it’s
ever done, and here we are, shutting it down.” That was tough
I can imagine. What are your memories of that last test? Were you
I missed the very last test. I was with my father, he’s been
sick, so I missed the last one. Matt and some of the long-time customers
came and thanked the team for their dedication. I had written a little
note that I had Ron read for me, but I wasn’t able to be there.
Then, that afternoon, they tested a test for the Safety Office, testing
a system that if you do have a breach in your heat shield and you’re
able to turn on some sort of fluid or compressed gas to pressurize
the compartment between the pressure vessel and the heat shield—there’s
two different pieces of structure, usually, in the capsule like that.
There’s a pressure vessel where the crew is, and then outside
that is a heat shield. If you’re able to pressurize that, you
can actually make air—let’s just say it’s nitrogen—if
the pressure’s great enough, it would actually flow out of the
breach, so you have cold gas flowing out rather than hot gas coming
in to your spacecraft. You wouldn’t need to repair it—you’ve
just got this air flowing out. No hot air is coming in, and it’s
entirely feasible because, like I said, the heating usually is at
a high altitude and the pressure’s pretty low. You really don’t
need that much mass flow or pressure to get the flow to go out. They
built a little rig to test the system like that, and that was the
last test we did. It worked out great, evidently.
A bittersweet moment.
Would you talk a little bit about the camaraderie out there at the
I always liked going out there because my seat wasn’t there.
Just a real good atmosphere, everybody’s super friendly and
always willing to do whatever it took to get things to happen. They
share food all the time and they have a Christmas dinner every year,
so it’s a really good group.
I saw you guys have a logo or a patch, when we were out there, doing
some of the taping.
I became deputy branch chief, that was one of my assignments, to be
lab manager. I got—it was Lockheed, at the time—a Lockheed
guy to design us a patch, so we made a little patch and went out and
bought some shirts. It ended up being a pretty cool patch.
Would you tell us about the patch?
had the name on the outside, like a mission patch, a little icon for
a shockwave, and a little lightning bolt arc heating it up. It said,
“JSC Arc Jet” on it.
Earlier, you had mentioned one of the tests you did was for Boeing.
Was that for their commercial space venture?
Can you talk about that?
country’s developing four re-entry spacecraft right now. We
have MCPV, and then Boeing, Sierra Nevada, and SpaceX, vying to be
the next Low-Earth Orbit guys. Right now, there’s four, and
those guys need to test. That’s one thing I’ve been trying
to point out. Boeing approached us to do testing for their vehicle.
They’re going through different materials in their selection,
and they have an in-house material they’re trying to see if
they are confident enough in. They approached us to do some testing,
and we signed the Space Act. We have a similar one with Sierra Nevada
to do radiant testing. Boeing was real anxious to get in. In fact,
we delayed, if you will, our closing so that we can meet that commitment
because we had already signed the Space Act. It was basically the
last set of tests we did before that one for the Safety Office.
Did you do any testing for SpaceX?
did not. They’ve tested a little bit at Ames, but we’ve
never tested for SpaceX. Their heat shields are doing good, so you
got to hand it to them.
Have you done any work for any universities, perhaps, or other Centers
out at the Arc Jet?
we’ve done work for other Centers. We did a little bit of things
for the Air Force, Air Force Research Lab. I’m trying to think.
I can’t recall too much else. Our bread and butter has been
the big programs: Apollo, Shuttle, X-38, MPCV. The other interesting
thing is, Shuttle—we were testing up until two weeks before
the last mission. Arc Jet testing isn’t just for developing
the material. We found that we tested every year for the Shuttle in
its existence, through development, through qualification, certification,
first flight, last flight. Through the whole life of the program,
the Shuttle, at least, required Arc Jet testing. It’s a very
big player, the whole time.
You’ve talked several times about the CO2 landing for Mars.
Why’d did you come up with that concept? What drove you to come
up with the idea?
just thought, “We can do it,” and the guys did it in their
spare time, on the side. Manned mission to Mars and big missions to
Mars has been talked back and forth forever. Sometimes, our Agency
says, “Hey, don’t worry about it; don’t talk about
it,” the other time it says, “Hey, that’s what you
should be thinking.” We just felt if we attained the capability,
we would probably have a customer some day to use it.
You’ve shared that with Ames, since they’ll be the only
Arc Jet Facility?
haven’t developed it yet, but they intend to, by the end of
the fiscal year, develop that capability.
Tell me a little bit about the deactivation process that the Arc Jet
Facility has undergone since the decision was made to close the facility.
process has been long, long in terms of actually closing. There’s
basically an order given, “You need to shut down,” and
our Center Director at the time, [Michael L.] Coats, wrote back a
letter, said, “Okay, we’re going to shut down, but we’ve
got to do these things first. Ames needs to develop the capabilities
that we have that they don’t have, so the Agency doesn’t
lose those capabilities.” We had a list of tests we needed to
do, and Ames had to develop these capabilities. At first, each of
them were projected to be six months away, and then we both kept marching
in time. They really marched together, coincidentally. So it wasn’t
until probably a year after that, [that] we originally intended to
close, before we finally finished up what we needed to finish up.
Ames has come real close to finishing their capabilities, so we did
our last test. That shutdown process has been basically: drain the
systems, de-energize everything, safe it out, and walk away.
We got a set of contractors running it because nobody’s giving
us any money to do any sort of real tearing it apart. Our guys did
the last test, we spent two weeks turning off things and de-energizing
it, then Ames came out last week and pulled all the Arc Heater parts
that they felt they could use, packed them up, and shipped them out
to Ames. That’s where we’re at now. We got a building
full of Arc Jetty type pieces of equipment and pumps and valves and
tools and copper tubing and all sorts of crazy things, that we’re
going to have to figure out a way to disposition. The shutdown was
simple; it was turn everything off and leave.
It’s slated to be demolished sometime this year?
I hadn’t realized it was going to sit for a while. That seems
like a shame, to sit.
I wanted to ask sort of an, I don’t know, emotional question:
what do you think that the Arc Jet Facility has meant to you, since
you worked there for so many years on different projects?
means a lot to me, and it means a lot to the guys because they want
to do a good job. We want to build thermal protection systems that
are going to work, and we lost a ship during the re-entry. It’s
very frustrating for my guys and for me, because all we want to do
is do a good job. We want to do it right. You got to have Arc Jets
to do it right. You cannot analytically predict how a material, if
you don’t know anything about it, is going to behave in an environment
like a re-entry environment. There’s no codes that do that.
We have codes that will predict a known material, but that’s
after a long test campaign in an Arc Jet. The analogy I give, it’s
like trying to do a stress analysis or a structural analysis without
the stiffness matrix—which is what the material is—and
without the strength. You can’t do the analysis. We can’t
do this predictive analysis without testing. Testing is all there
We’re real disappointed and just didn’t understand it.
It’s been a struggle because there’s a lot of love for
the facility. We think it’s important. That’s why there’s
been a lot of emotion in that. We’re sitting here, saying, “Hey,
you trusted us to be the engineers for these systems and we’re
telling you we need this facility,” and [we’re] largely
ignored. We’re like, “What are we going to do?”
Their answer is, “We’ll test everything at Ames.”
It’s been argued that that’s feasible, but you are going
to miss some throughput. More importantly, you’re going to miss—it
wouldn’t be fair to call it healthy competition all the time,
but just a second set of eyes, the diversity of having another group
do this. You have a risk of a monoculture and not being challenged,
in terms of the way to do things, with the single facility. You also
have a reliability risk. Like I said, take a race car versus regular
car, take a re-entry spacecraft versus a bicycle. The energy density
in the facility is tremendous, and the reliability is poor. They can
lose their boiler, they can lose their steam injector system, and
then the TPS testing is gone, or it’ll be suspended for a long
time. That’s where a lot of emotion has come in.
What do you think has been the significance of the Arc Jet facility
here at JSC?
think it’s been a key player in all of our spaceships that we’ve
produced. It’s got to be up there.
Do you think there’s anything else that you would like to talk
about, about the Arc Jet Facility? I think we’ve talked about
most of the material that I wanted to discuss.
think that’s it.
I thank you for coming in today and sharing your thoughts with me.
[End of interview]
to JSC Oral History Website