Masters of improv

Photo: Reidar Hahn, Fermilab

World-class detective Angus MacGyver of the hit 1980s television show MacGyver could jury-rig almost anything with duct tape and a pocket knife. High-energy physics labs demand as much and more from technicians and engineers, relying on their creativity and intelligence to navigate technical quagmires. And when a problem demands it, they deliver—engineering tiny cameras mounted on bocce balls that snake through 10,000 feet of steel piping; rigging a 13-ton cement block to bash deformed brass into shape; or aiming a high-powered laser around corners to unblock water lines. Unlike MacGyver's fixes—such as the fuse he repaired with a chewing-gum wrapper—some of these devices last.

An improvised grinder sanded welds along the long, straight sections of 10,000 feet of pipe at Fermilab. The sander within the rotating silver cylinder cleaned each weld. Photo: Fred Ullrich, Fermilab

Leon Lederman, the Nobel Prize-winning former director of Fermilab, is a legendary lab MacGyver. He used a pocket knife, tape, and items on anyone's grocery list to confirm that interactions involving the weak force do not show perfect mirror symmetry, or parity, as scientists had long assumed. Just as a watch hand always sweeps clockwise, nuclei of atoms eject electrons in a preferred direction as they decay, rather than spraying them randomly. The technical term for this is "parity violation."

Intrigued by the experiments of Madame Chien-Shiung Wu, Lederman called his friend, Richard Garwin, to propose an experiment that would detect parity violation in the decay of the pi meson particle. That evening in January 1957, Lederman and Garwin raced to Columbia's Nevis laboratory and immediately began rearranging a graduate student's experiment into one they could use. "It was 6 p.m. on a Friday, and without explanation, we took the student's experiment apart," Lederman later recalled in an interview. "He started crying, as he should have."

The men knew they were onto something big. "We had an idea and we wanted to make it work as quickly as we could—we didn't look at niceties," Lederman said. And, indeed, niceties were overlooked. A coffee can supported a wooden cutting board, on which rested a Lucite cylinder cut from an orange juice bottle. A can of Coca-Cola propped up a device for counting electron emissions, and Scotch tape held it all together.

"Without the Swiss Army Knife, we would've been hopeless," Lederman said. "That was our primary tool."

Their first attempt, at 2 a.m., showed parity violation the instant before the Lucite cylinder—wrapped with wires to generate the magnetic field—melted.

"We had the effect, but it went away when the instrument broke," Lederman said. "We spent hours and hours fixing and rearranging the experiment. In due course, we got the thing going, we got the effect back, and it was an enormous effect. By six o'clock in the morning, we were able to call people and tell them that the laws of parity violate mirror symmetry," confirming the results of experiments led by Wu at Columbia University the month before.

Another giant figure in physics, founding Fermilab director Robert Wilson, is the hero of a widely circulated tale.

Ernie Malamud, a physicist at Fermilab, remembers working with Wilson during his graduate studies at Cornell. The pair wanted to use helium gas, often used to fill balloons, to locate a leak in the glass vacuum chamber; but they discovered the hose from the helium supply wouldn't reach the area where they perceived the leak to be. Wilson filled his mouth with helium from the hose, ran to the tank and blew on a gasket to find the leak. He turned to Malamud and grinned.

"This typifies to me everything great about the man: his decisiveness, his knowledge, his originality, and a 'can do' attitude," Malamud says.
 

Changing times

With today's expensive and complex experiments, there's less room for spontaneity. A solution often involves a team of MacGyvers, long hours of strategic planning, and lots of execution time.

Safety is of the utmost importance, and today's problem-solvers carefully review their unconventional fixes to make sure they aren't hazardous.

"High-energy physics is becoming a very industrialized science with projects lasting for years, so elegant solutions are possible," says Dmitri Denisov, spokesperson for Fermilab's DZero experiment. "It's strongly inadvisable to do anything unapproved in government-funded labs. Yes, some fun of doing science is gone. But taking into account the size and complexity of the experiments, this is the right way to proceed."

Technicians and engineers continue the tradition today, albeit more carefully, finding clever solutions to problems of ever-increasing complexity. Duane Plant, senior operations specialist in the Accelerator Division at Fermilab, is one such MacGyver.

"One of the things that Duane is known for is figuring out that something's going to be a problem before anyone else does," says Patrick Hurh, who works in the division's Mechanical Support Department. "He's invaluable for that. He's a really inventive, creative scientist."

His specialty: designing and building, along with partner Todd Johnson, tiny cameras for tricky fixes. "In my office, I've got 10 different cameras to look at things," Plant says. His cameras go where humans can't: one inched along thousands of feet of pipe to survey microbe corrosion, making way for a string of bocce balls and a cylindrical weld grinder to sand and clean offending welds. Plant is loath to take credit for his ingenuity, which over the years has included identifying a snow plow as the cause of disturbances in the Tevatron beam and using a toy dart to open the insides of some magnets for inspection.

"You kludge things together using what you have lying around," he says. "At the lunch table, we can fix anything. Our lunch table has come up with more solutions than we could on our own."
 

Kludges that last

Some kludge jobs are used only once; others, such as a brass-bashing technique used at Stanford Linear Accelerator center, can be recycled.

In 2002, Jim Krebs, chief mechanical engineer for SLAC's BaBar detector, was helping install one-ton brass plates in the detector when he and his team realized that one of the plates was too bowed to fit. Krebs came up with an inelegant but clever solution.

"I remembered visiting a steel factory in Gary, Indiana, years before, where they flattened bowed steel plates with concrete," he says. "I watched them do it for a while and it was pretty crude."

Nevertheless, Krebs thought the technique would work perfectly for his deformed brass. He supported the plate with beams and shims, and used a crane to lower a 13-ton concrete block onto the bent portion. One hour and a few tries later, the brass plate was flat and the crew slipped it into position. When the opposite problem occurred in 2004, Krebs and others used the same method on a flat plate that needed bending.

At Fermilab, a string of balls navigated tight corners within the pipes to clean welds out of the weld grinder's reach. Photo: Fred Ullrich, Fermilab

Some inspired techniques become permanent fixtures. And some work so well they spread beyond their original homes—such as the aspirin tablets used to detect water leaks in the radio frequency cavities of accelerators.

These RF cavities, which are used to boost the particle energy in accelerators, contain cooling systems that are prone to leakage over time. It's important to detect small leaks before they become big enough to damage equipment. But detecting teaspoons of water in a cavity that could hold many gallons is difficult.

Each cavity has two holes in the bottom to drain accumulated water. To detect a leak, technicians cap the outsides of the holes with aspirin tablets, which are held in place by spring-loaded switches. When water seeps through a hole and dissolves the aspirin, the switch clicks, indicating a leak. In most cases, leaks are found before any damage is done.

"This was started so many years ago that the names of the original technicians and engineers who invented the treatment are lost in history," Plant says. "But the practice lives on, and I understand other labs have used this idea."
 

Success and satisfaction

Legendary stories like Lederman's and Wilson's may be few, but engineering triumphs abound. Satisfaction often comes not with formal recognition, but with the success of an unconventional solution—such as Doug Glenzinski's epoxy-blasting lasers, which restored an overheating portion of Fermilab's then-newly installed CDF detector.

"Everything we did was at the very brink of possible," says Glenzinski, who worked with a small team on the project in 2001. "At every stage, we would think, 'Oh, God. How are we going to do this?' Each subsequent stage seemed more difficult. It was very challenging, but in the end it was fun because we succeeded."

First, the team wriggled custom-ordered bore scopes—long, thin fiber-optic cables commonly used in surgery—into the thin cooling lines of the silicon detector to identify areas where epoxy had leaked out of joints and blocked water flow. The next challenge was to clear the epoxy globs without damaging the aluminum cooling lines. The team snaked a flexible laser fiber into the tubes and attached a prism at the end so they could aim the laser around corners. Reflections viewed on an oscilloscope allowed them to distinguish the epoxy clots from the aluminum tube before blasting the epoxy away. The whole process took two years of intermittent work, but the lines have worked perfectly ever since.

The fix was so seamless that the former blockages are undetectable today.

"The new people who have joined the CDF collaboration didn't know about the epoxy blockage until we mentioned it later," Glenzinski says. "I think that's a good definition of success."
 

Click here to download the pdf version of this article.