One Friday night in March, accelerator operator Alyssa Miller was chatting with fellow staff members in the rec center at the US Department of Energy’s Fermi National Accelerator Laboratory right after she finished her shift. As they talked, several of her colleagues’ cell phones started buzzing. Scrambling to check their messages, they realized the local utility company had abruptly stopped providing the accelerator complex with electricity. The lights were still on, thanks to back-up power, but the complex requires too much energy to keep running by generator alone.
The Fermilab Accelerator Complex, a DOE Office of Science user facility, comprises seven particle accelerators and storage rings and hosts more than 2000 scientists a year. Fermilab’s Booster provides a low-energy beam of neutrinos for the MicroBooNE experiment. The Main Injector provides the world’s highest intensity neutrino beams for the MINOS, MINERvA and NOvA experiments. In the future it will send neutrinos to the Deep Underground Neutrino Experiment detectors in South Dakota. It provides beams of muons to the Muon g-2 experiment and will send them to the Mu2e experiment as well. It also sends particles to the SeaQuest fixed-target experiment, as well as a facility for testing detector technologies.
“It’s an enormously complicated chain,” says Mike Lindgren, head of Fermilab’s accelerator division.
On that Friday, the chain was broken. The facility’s accelerator operators and technical specialists ran to their stations.
“Accelerators like to be turned off nicely,” says Miller, a former operator who is now an engineering physicist at Fermilab. “When they’re shut off all at once, that leaves things in kind of a bad state.”
As power returned, the operators and technical specialists combined forces to retune the giant machine. It was an all-hands-on-deck effort. Shift leaders called workers in early from their days off. The entire team pushed hard for nearly 24 hours to get the entire complex running again.
While this was a memorable incident, it was all part of the job. For accelerator operators at the DOE’s national laboratories, there’s no such thing as an average day.
A day in the life
More than 3000 people work on or with accelerators at DOE’s 17 national labs. About 300 of them are operators.
Because the accelerators generally run 24 hours a day, seven days a week, operators work a shift schedule, changing between the day, night, and overnight (or “owl”) shifts.
An operator’s shift starts with a briefing from the crew chief about what happened since the team was last at work. After the meeting, operators settle down in the main control room. The walls there are covered with screens showing multi-color lines, graphs, and maps.
The amount of incoming information can be overwhelming. During Miller’s first few months at Fermilab, she says, “I kept thinking to myself, ‘Oh my goodness, how am I going to learn all of this?’ Now I sit down and I feel comfortable at my console.”
Operators use the data to judge how well the accelerator is delivering beams to the scientists’ experiments. They constantly check the many console displays to ensure everything is running smoothly. If the beam is getting “loose” and shedding too many particles, they tweak the magnets to tighten it. Depending on the lab and researchers’ needs, they may also adjust the magnets to change the beam’s parameters such as its energy, size and shape.
The first line of defense
Operators are also accelerators’ first responders, identifying and fixing problems as they arise. From electrical ground faults that bring down power supplies to imperfections in the vacuum around the particle beam, operators must recognize and deal with a whole host of issues.
“Operators are the first line of defense for any problems at the accelerator,” says Fermilab operator KelliAnn Rubrecht.
Operators have to be familiar with a multitude of topics, from charged particle dynamics to cryogenic systems maintenance. “We have to know a little bit of everything,” Rubrecht says. “That’s one of the things I really like about this job.”
Each facility has unique needs. They often require one-of-a-kind equipment and devices that can’t be purchased. “We take a lot of pride in building things ourselves,” Miller says. “It requires ingenuity for sure.”
Sometimes, operators can solve an issue at their computers with a twist of a knob. Other times, they face physical problems that require hands-on inspection. If they don’t have the expertise to repair the problem themselves, they call in people who specialize in particular sections of the accelerator. “Once you know what the problem is, you find the right people to fix it,” says Mary Convery, Deputy Division Head of Accelerator Systems at Fermilab.
Descending into the accelerator tunnel (always in groups of at least two for safety), operators move into a strange, underground world. The walls are thick concrete; the ceilings are low; water that runs through the cooling pipes humidifies the air.
For operators, it can be a welcome change of pace from gazing at computer screens.
“There’s so much going on behind the scenes that you can’t get a true appreciation for until you go down there,” Miller says.
The learning curve
Because most operators have no prior experience running accelerators, there’s extensive on-the-job training. It takes up to two years for a rookie operator to cover it all.
To help get new and aspiring operators up to speed, eight DOE labs partner with two universities to run the US Particle Accelerator School. Twice a year, the school offers intensive sessions for lab staff, university students, and those in industry who want to learn more about accelerators.
“We have instructors that are leaders in their field, and they bring a lot of state-of-the-art material,” says Steve Lund, a professor at Michigan State University and the head of the Particle Accelerator School. “Students are learning from people who are established pillars of the community.”
The school offers classes on topics available nowhere else. The universities hosting the sessions provide students with academic credits.
Running a particle accelerator requires a niche set of skills. But the challenge of acquiring them is worth the chance to use them, Miller says.
“If you want to get your hands dirty literally and figuratively and you want to learn a lot and you want to be a little confused and frustrated the whole time, operations is for you,” she says. “It’s an experience like no other.”
