Last month NASA launched the James Webb Space Telescope, the most sophisticated space telescope of all time.
Webb is the successor to the Hubble Space Telescope launched in 1990. A joint effort of NASA, the European Space Agency, and the Canadian Space Agency, the new telescope is packed with a range of new instruments, such as sensors capable of detecting infrared light that will enable the telescope to see far further into space than its predecessors.
“Webb's not just a better version of the Hubble, it's a new thing,” says Erik Tollerud, Assistant Astronomer at the Space Telescope Science Institute (STScI), the organization that operates Webb. “It took 25 years to complete and will be the engine for discovery for the future of astronomy.”
Once it reaches its final destination, approximately 1 million miles from Earth, Webb will put all of this new technology to use on its lofty goals, including detecting the very first stars and galaxies ever formed, studying the evolution of celestial bodies, and maybe even surfacing clues regarding the origins of life.
As important as the telescope is, the science it will enable will happen back on Earth. “Astronomy is a software-based field,” says Tollerud. “We spend our time at a computer writing and running code to analyze the images and data collected from telescopes and other instruments.”
Open source software provides the backbone for astronomical science. The Python ecosystem is particularly important to astronomers, who rely heavily on mathematical packages like NumPy and matplotlib for their work. Perhaps the most central tool in the modern astronomer’s workflow is Astropy, a collection of specialized Python tools built and maintained by and for the astronomical community. More than 400 people have contributed to Astropy, including astronomers and other scientists, software engineers, and infrastructure specialists.
The project consists of a core package that contains several subpackages that handle the complex work that would need to be done manually in other scientific computing platforms.
For example, the astropy.coordinates subpackage is used for representing and converting between coordinates for different celestial bodies such as stars and planets. There are many different ways to represent coordinates. Some are two-dimensional and are meant to represent the sky as it appears from Earth. Others are three-dimensional, taking distance into account. Different systems use different reference points as well. “The coordinates subpackage makes it easy to convert between different systems using standard methods without requiring users to get into the messy details,” says Perry Greenfield, a consulting computer scientist at STScI and co-founder of Astropy.
Another key subpackage is astropy.fits, a library for reading and writing FITS files, the standard file format for astronomical data. “One of the most important things that Astropy does is that it makes data more interoperable,” says Kelle Cruz, an astrophysicist at Hunter College. “That makes it much easier to share data between teams.”
Origins
In the days before Astropy, the most widely used software for astronomers was Image Reduction and Analysis Facility (IRAF), developed by the US federal government laboratory called National Optical Astronomy Observatories (NOAO). IRAF is still heavily used by astronomers today, but NOAO stopped supporting the software in 2012, so astronomers began looking for alternatives. Many of these academics converged on the Python ecosystem. “We started using Python at STScI in the 90s,” Greenfield says. “Lots of people shared packages on a mailing list for Python in astronomy, but there wasn’t really a coordinated effort until Astropy”.
As Python caught-on in the community, many astronomers shared their own custom Python packages. But many packages did more or less the same things and it was hard for newcomers to know which packages were best, and which ones were still maintained. “Eventually we realized that there was a lot of duplication of effort,” Tollerud says. “We decided to work together instead of each having our own packages.”
Greenfield, Tollerud, and others began organizing contributors. “I set up a wiki where people could sign-up to be involved,” says Thomas Robitaille, another co-founder of the project. “It helped advance things from just talking to actually working on projects.” Thus, Astropy was born.
Cruz says many people in the astronomy community were skeptical at first. “People worried that a grassroots community initiative wouldn't be reliable enough to gain widespread adoption within the field,” Cruz says. “There was also a lot of resistance among professors who didn't want to have to learn Python in order to teach it to their students.”
Cruz, who runs a popular astronomy blog called AstroBetter, hadn't done any open source before, but she was following the project's development on the Python astronomy mailing list and wanted to help Astropy succeed. “Through the blog, I had a pretty good finger on the pulse of what junior astronomers were thinking,” she says. “They were really struggling with the transition away from IRAF. Migrating to open source Python tools instead of being at the whims of commercial software developers made sense to me.”
She worried, however, that the project's maintainers, who were focused on code development, didn’t have the bandwidth to focus on non-technical issues, like how to promote adoption of Astropy to a resistant user base or how to raise funds to sustain development. “I got involved to represent the community,” she says.
One of her first projects was a series of Astropy tutorials that she created with Adrian Price-Whelan. “We would just get together at coffee shops and work,” she says. “Basically, Adrian was teaching me Python. I didn't know the language so I kept asking naive questions about how to do different things, and we'd create tutorials based on my questions.”
Astropy steadily saw adoption, which Cruz credits in part to the Python language itself. "The language's syntax is so clear that if you can program in any language, you can read and understand Python code," she says. "That meant that professors didn't have to learn Python, even if their students used it."
Cruz also helped motivate Astropy's maintainers to seek out funding. “When I started out, there were no funding opportunities that made sense for us,” she says.” Cruz and other maintainers had multiple conversations with various program managers, helping the project win grants from the Moore Foundation and NASA.
New Frontiers
Astropy has been around for a decade now, but it's always evolving. To work with the data that will be collected by Webb, several new features were added in the last few years, particularly around visualization and spectroscopy.
Spectroscopy is the study of how light interacts with matter. You can see how this would be important for studying distant stars and other celestial bodies, as the only hints of their existence and behavior is often traces of light not visible to the human eye. The new spectroscopy features will be useful both for identifying new bodies in space and learning more known ones. For example, it might be possible to detect entirely new galaxies, while also discovering new properties of existing galaxies. “You can use spectroscopy to learn many things about an object, including their age or metallicity,” explains Camilla Pacifici, an instrument scientist at STScI who helped coordinate the development of these new Astropy features.
That work continues today. "We use funding not only to support new work, but also for bug fixes, incremental improvement, things that might not look exciting but that are important to the day to day work of so many scientists."
NASA did decide in 2020 that maintenance of software is an allowable expense. "That was a major victory for Astropy," Cruz says. “It took working with politicians to change the law.”
The project's biggest challenges now involve how to use those funds to pay developers while also nurturing a volunteer community. “We don't want people to think that they shouldn't contribute to the project just because there are people being paid to work on it now,” Tollerud says. “We want the community to keep guiding development.”
