1. Introduction
It has long been understood that astrophysical data offers an important opportunity for the development of Artificial Intelligence (AI) and Data Science. Beginning with the Sloan Digital Sky Survey (York et al. Reference York, Adelman, Anderson, Anderson, Annis, Bahcall, Bakken, Barkhouser, Bastian and Berman2000, SDSS), the pathfinder for today’s large synoptic surveys, astrophysicists at academic institutions began collaborating with computer scientists from both academia and industry. The collaboration between Johns Hopkins’ Prof. Alex Szalay and Microsoft’s Jim Gray sets the prime example. The appeal for the industry side is that, in Gray’s words, “astronomers’ […] data is worthless” (Szalay Reference Szalay2008). This worthlessness refers to the lack of concern for ‘personally identifiable information’ of galaxies; the minimal harm that would be done to a star if an applied algorithm were biased, which would be limited to incorrect scientific conclusions that would surely be overwritten by better analysis or more data (or both);Footnote 1 and the limited financial investments, compared to industry standards, that make our data not marketable in a strict sense. Furthermore, for the most part, the astrophysical community has embraced open-access practices and made freely available the vast and ever-increasing amount of data we collect (Stephens et al. Reference Stephens, Lee, Faghri, Campbell, Zhai, Efron, Iyer, Schatz and Sinha2015), and this data is of significant complexity, which makes it ideal for AI development.
Artificial neural network (ANN)-based solutions to astrophysical problems started appearing in the literature as early as the 1980s (right in the middle of the “second AI winter” Toosi et al. Reference Toosi, Bottino, Saboury, Siegel and Rahmim2021), with applications to the very same problems we try to solve today with ANNs and AI. These include the spectral classification of stars (von Hippel et al. 1994), morphological classification of galaxies (Storrie-Lombardi et al. 1992; Rappaport and Anderson Reference Rappaport and Anderson1988), and even survey design and operation (Adorf Reference Adorf1995). Review articles, books, and Ph.D. theses on the topic appeared as early as 1988 (Rosenthal Reference Rosenthal, Murtagh, Heck and Benvenuti1988; Heck and Murtagh Reference Heck and Murtagh1989; Naim Reference Naim1995), yet these solutions only stopped being “novelties” in the mid-2010s, with annual publications reaching double digits in 2016 and triple digits in 2019.Footnote 2
In optical astronomy, this rapid proliferation of machine learning and artificial intelligence solutions is powered by the convergence of two events: the era of AI accessibility and popularization, and the start of the Vera C. Rubin Observatory Legacy Survey of Space and Time (Ivezić et al. 2019, LSST). The promise of LSST to deliver a dataset 10x larger than any prior survey has pushed the astrophysical community to ramp up the development of innovative, often machine-learning and ANN-based solutions (Bianco et al. Reference Bianco, Banerji, Blum, Bochanski, Brandt, Burchat, Gizis, Ivezic, Kahn, Keaton, Kaviraj, Loredo, Mandelbaum, Marshall, McGehee, Schafer, Schwamb, Sokoloski, Strauss, Street, Trilling and Verma2019). Indeed, the size of current and upcoming datasets from astrophysical surveys (Law et al. Reference Law, Corbett, Galliher, Gonzalez, Vasquez, Walters, Machia, Ratzloff, Ackley, Bizon, Clemens, Cox, Eikenberry, Howard, Glazier, Mann, Quimby, Reichart and Trilling2022; Bellm et al. Reference Bellm, Kulkarni, Graham, Dekany, Smith, Riddle, Masci, Helou, Prince and Adams2019; Ivezić et al. 2019) is such that traditional computation is impractical at best and impossible in many cases. We may therefore be witnessing a new paradigm shift, from data-intensive (Hey et al. Reference Hey, Tansley and Tolle2009) to AI-driven science. The Rubin Observatory is planning to start the LSST in late 2025, and commissioning and Science Validation activities have been running since late 2024 (NSF-DOE Vera C. Rubin Observatory 2025; Guy et al. Reference Guy, Bechtol, Bellm, Blum, Dubois-Felsmann, Graham, Ivezić, Lupton, Marshall, Slater, Strauss and Observatory2025). At the start of LSST, Rubin will begin ramping up its real-time delivery of time-domain data products, referred to as ‘alerts’, leading to an expected 10 million alerts per night by the end of the first year (Graham et al. Reference Graham, Bellm and Guy2019). I will hereafter focus on selected challenges and opportunities we face at the interface of astrophysics and AI at the dawn of the LSST era, with an emphasis on time-domain astronomy.
2. The size and complexity of astronomical datasets
Astrophysical datasets pose unique challenges. Each astronomy regime, such as radio, optical, and high-energy, faces distinct problems. Data from different optical astronomy surveys present different challenges as well. For the upcoming LSST, the main challenge is the staggering size of the information-rich images that will flow at a rapid rate from the LSST Department of Energy (DOE) camera: 20 terabytes (TB) of data in nearly 1,000 images each night, each containing 3.2 billion pixels. While other astrophysics observing modalities may produce even larger datasets (e.g., the upcoming Square Kilometer Array, Possenti Reference Possenti2025), the information content in those data streams is typically relatively low, enabling edge computing and the disposal of most of the raw data early on in the pipeline.
For LSST, every pixel will be saved and used to produce data products, such as deep image stacks, catalogs of objects, and light curves, and will be made accessible to the astrophysical community for further analysis (via the Rubin Science Platform, O’Mullane et al. Reference O’Mullane, Economou, Huang, Speck, Chiang, Graham, Allbery, Banek, Sick and Thornton2021). Astrophysical images are often complex due to the presence of individual overlapping sources, a complex background from unresolved sources and dust, and instrumental noise. To exacerbate the problem, the interesting information often sits at the limit of the signal-to-noise ratio: faint sources, the edges of galaxies, which are critical for inferring morphological characteristics, and fine details of shapes that are critical for understanding covariances, such as those imprinted by gravitational fields in weak lensing.
Aimed at transforming multiple areas of astrophysics (Ivezić et al. 2019; Collaboration et al. 2009; Hambleton et al. Reference Hambleton, Bianco, Street, Bell, Buckley, Graham, Hernitschek, Lund, Mason, Pepper, Prsa, Rabus, Raiteri, Szabo, Szkody, Andreoni, Antoniucci, Balmaverde, Bellm, Bonito, Bono, Botticella, Brocato, Bricman, Cappellaro, Carnerero, Chornock, Clarke, Cowperthwaite, Cucchiara, D’Ammando, Dage, Dall’Ora, Davenport, de Martino, de Somma, Criscienzo, Stefano, Drout, Fabrizio, Fiorentino, Gandhi, Garofalo, Giannini, Gomboc, Greggio, Hartigan, Hundertmark, Johnson, Johnson, Jurkic, Khakpash, Leccia, Li, Magurno, Malanchev, Marconi, Margutti, Marinoni, Mauron, Molinaro, Moller, Moniez, Muraveva, Musella, Ngeow, Pastorello, Petrecca, Piranomonte, Ragosta, Reguitti, Righi, Ripepi, Sandoval, Stassun, Stroh, Terreran, Trimble, Tsapras, van Velzen, Venuti and Vink2022), LSST will collect nearly 20 TB of data each night and deliver 10 million alert packets to the entire world each night (indeed, the alerts are publicly available worldwide, Blum and the Rubin Operations Team 2020). This sets the stage for bigger, and in some cases entirely new, data science challenges. I will offer as examples two areas that experienced massive and rapid development in the past decade, but for which there is still room for innovation: the automation of the discovery pipeline, and the encoding of multi-band light curves.
2.1 Automation in data collection and survey design
The traditional data collection for time-domain astrophysical data where astronomers discovered time-evolving phenomena in the sky and applied for telescope time to follow them up taking spectra and additional photometry, had already significantly changed to accommodate data streams like PTF’s (Bellm et al. Reference Bellm, Kulkarni, Graham, Dekany, Smith, Riddle, Masci, Helou, Prince and Adams2019). At the same time, we came to understand that there exists a wealth of faster-evolving transients than supernovae in the optical sky (Drout et al. Reference Drout, Chornock, Soderberg and Edward Sanders2014), evolving on timescales of hours and days, and that the early evolution of transients is particularly informative (Kochanek Reference Kochanek2019). This led to the development of queue observing infrastructure at telescopes, target-of-opportunity programs that are remotely triggered, and so on. But even this is not sufficient in the LSST era. Everything needs automation in the LSST era: 106 alerts distributed at a rate of 1,000 every 30 seconds command automated classification, automated distribution of observing requests, and automated data collection. The time-domain ecosystem has profoundly changed in the lead-up to LSST: Broker Alert systems (Möller et al. Reference Möller, Peloton, Ishida, Arnault, Bachelet, Blaineau, Boutigny, Chauhan, Gangler, Hernandez, Hrivnac, Leoni, Leroy, Moniez, Pateyron, Ramparison, Turpin, Ansari, Tarek Allam, Biswas, Boucaud, Bregeon, Campagne, Johann Cohen-Tanugi, Dornic, Fouchez, Godet, Gris, Karpov, Nebot Gomez-Moran, Neveu, Plaszczynski, Savchenko and Webb2020; Narayan et al. Reference Narayan, Zaidi, Soraisam, Wang, Lochner, Matheson, Saha, Yang, Zhao and Kececioglu2018; Smith Reference Smith2019; Förster et al. Reference Förster, Cabrera-Vives, Castillo-Navarrete, Estévez, Sánchez-Sáez, Arredond, Bauer, Carrasco-Davis, Catelan and Elorrieta2021; Nordin et al. Reference Nordin, Brinnel, van Santen, Bulla, Feindt, Franckowiak, Fremling, Gal-Yam, Giomi and Kowalski2019) are a new entry in the astrophysics discovery chain, first conceptualized in 2008 (Borne Reference Borne2008) and practically realized in the mid-2010s. Target of Opportunity Managers (e.g., Street et al. Reference Street, Bowman, Saunders and Boroson2018), and the automation of the largest class of telescopes via The Astrophysical Events Observatories Network (AEON Street et al. Reference Street, Adamson, Blakeslee, Blum, Bolton, Boroson, Bowman, Briceno, Elias and Gomez2020) are all infrastructural elements that were ideated and produced with LSST in sight.
A milestone in this process was met in 2024 with the first entirely automated identification-to-follow-up-to-classification-to-publication (Rehemtulla et al. Reference Rehemtulla, Miller, Fremling, Perley, Qin, Sollerman, Mahabal, Neill, Laz and Coughlin2023) of a ZTF transient (Rehemtulla et al. Reference Rehemtulla, Miller, Laz, Coughlin, Fremling, Perley, Qin, Sollerman, Mahabal and Laher2024). This work demonstrates that the technical aspects of the problem are tractable, but the sociological aspect–i.e., increasing community trust in automated and ANN-driven decisions–still requires much attention. If we can build trust in an automated AI system, we may, in the future, enable not only automated follow-up, but also automated survey design and optimization decisions. For LSST, this entails deciding the sequence of images that LSST will collect for 10 years across the > 18,000 square-degree footprint, plus special regions, across six filters, all while adapting in real time to evolving conditions. Currently, the LSST survey optimization problem is solved by tapping into the community expertise to provide metrics that measure science success from simulated visit databases, and empowering a committee of community members, the Survey Cadence Optimization Committee, to aggregate and optimize based on community recommendations (Bianco et al. Reference Bianco and Ivezić2021). While this process was extremely successful in planning the 10-year LSST survey strategy (Rubin’s Survey Cadence Optimization Committee et al. Reference Committee, Bianco, Lynne Jones, Anguita, Bauer, Edwards, Jha, Mandelbaum, Miller, Olsen, Slater, Smartt, Strader, Street, Volk and Yoachim2025), the effectiveness of this method, with human intervention in all steps, in generating sensible and effective real-time operation adjustments, remains to be demonstrated.
2.2 Data Encoding challenges in astronomical photometric surveys
A key challenge in turning LSST time-domain data into science is the irregular and sparse sampling of its observing cadence. Broadly speaking, the current plan for the cadence is to collect, over a staggering > 18,000 square degrees of sky, ‘visit-pairs’: two images in different filters at a median time separation of 28 minutes. Roughly 4% of the fields will receive a third visit in either of the two filters used in the visit-pair within 24 hours. This enables the detection of rapid transients and their rough characterization (Bianco et al. Reference Bianco, Graham, Drout, Andreoni, Biswas, Cowperthwaite and Narayan2018; Bellm et al. Reference Bellm, Burke, Coughlin, Andreoni, Raiteri and Bonito2022). Most fields will be revisited at 2-5 day separations, depending on the filter and the sky region, while five 10-square-degree fields are pursued at a much higher cadence (the Deep Drilling Fields Rubin’s Survey Cadence Optimization Committee et al. Reference Committee, Bianco, Lynne Jones, Anguita, Bauer, Edwards, Jha, Mandelbaum, Miller, Olsen, Slater, Smartt, Strader, Street, Volk and Yoachim2025) and other regions (e.g. the Galactic and Ecliptic Planes) follow specialized cadences. In alternating years from Years 2 through 9, half of the sky will receive more visits than the other half to enable higher-cadence observations (the so-called rolling cadence). Observations over this entire footprint will be sustained for all 10 years of the LSST survey. Details aside, this leads to persistent, but rather sparse and irregular light curves, which pose a significant challenge for AI. The following list of typical light curve characteristics makes them, simply put, terrible tensors:
-
• Variable sizes of data vectors;
-
• Uneven sampling;
-
• Different sampling at different wavelengths;
-
• Phase gaps can be months long over ∼ 1 year observing cycles;
-
• Multiple relevant time scales are present in the transients and variable stars evolution;
-
• Aleatory and epistemic heteroscedastic uncertainties are simultaneously present and significant.
The current state of the art proposes Gaussian Processes (GPs) as a solution to interpolate sparse light curves (Aigrain and Foreman-Mackey Reference Aigrain and Foreman-Mackey2023). GPs are a specific kind of stochastic process that, by setting a priori the shape of a kernel representing the covariance of data points, allows for the generation of a family of functions that are consistent with the observations and informed by the kernel. The distribution of values at each point in time and wavelength space, in the case of a GP applied to a multicolor light curve, based on the ensemble of functions predicted by the GP, can be treated as the uncertainty in the prediction. Thus, GPs enable interpolation with quantified observed (aleatory) and model (epistemic) uncertainties. But it must be noted that (1) the same kernel (with tunable parameters) is assumed to be valid for all transients, regardless of their nature and of the physics that determines the changes in time or wavelength, and in most cases the same kernel functional form is assumed for the time and wavelength axes; (2) the parameters of the kernel can either be fit to each object individually (Boone Reference Boone2019; Qu et al. Reference Qu, Sako, Möller and Doux2021), which carries the risk of overfitting, or optimized based on a subset of objects (Khakpash et al. Reference Khakpash, Bianco, Modjaz, Fortino, Gagliano and Larison2024) which risks having unusual phenomena–including exciting anomalies that may reveal new physics (Li et al. 2021; Giles and Walkowicz Reference Giles and Walkowicz2019; Aleo et al. 2024; Villar et al. 2021; Ishida et al. Reference Ishida, Kornilov, Malanchev, Pruzhinskaya, Volnova, Korolev, Mondon and Sreejith2021; Romao et al. Reference Romao, Croon and Godines2025; Soraisam et al. Reference Soraisam, Saha, Matheson, Lee and Narayan2020)–to be fit with incorrect parameters and thus perhaps entirely missed, with unbound scientific loss.
A possible solution is offered by Neural Processes (Garnelo et al. Reference Garnelo, Schwarz, Rosenbaum, Viola, Rezende, Ali Eslami and Whye Teh2018b; Garnelo et al. Reference Garnelo, Rosenbaum, Maddison, Ramalho, Saxton, Shanahan, Teh, Rezende and Ali Eslami2018a), a form of stochastic processes where the GP kernel is replaced by a learnable model, an ANN. These are beginning to appear in astrophysics, but perhaps they are being overlooked as a solution to the sparsity problem (Čvorović-Hajdinjak et al. 2022; Park et al. Reference Park, Villar, Li, Jiang, Ho, Lin, Marshall and Roodman2021).
Recently, attention-based AI models have allowed researchers to incorporate more naturally unevenly sampled data by inputting timestamps processed through a positional encoder (Pan et al. Reference Pan, Ting and Yu2024; Donoso-Oliva et al. Reference Donoso-Oliva, Becker, Protopapas and Cabrera-Vives2023; Pimentel et al. Reference Pimentel, Estevez and Förster2022; Moreno-Cartagena et al. 2023). However, it should be noted that these models are significantly more memory-intensive than alternative NN architectures.
Still, while once encoded for ANN consumption, light curves can be turned into tensors, most NN architectures do not easily support the incorporation of uncertainties, which are critical for physical inference. While methodologically thorough approaches, such as Bayesian Neural Networks (NNs), have been applied to the classification and characterization of transients (Möller and de Boissière Reference Möller and de Boissière2019), they are typically very computationally intensive to train and require large datasets. More exotic solutions have been tried, with good results. For example, Qu et al. Reference Qu, Sako, Möller and Doux2021 uses 2D arrays of multiwavelength time evolution and its uncertainty, as generated by GP interpolation of sparse multiband light curves, as two channels, equivalent to image channels, as input to a convolutional network. While effective for classification, the success of this method rests on empirical findings, not on a theoretical backbone: flux arrays and uncertainty arrays are not the same as two color channels in an image. This problem remains fertile for AI methodological innovation, and the limited progress to this day may be due to the fact that uncertainties are a key feature of the physical world, but do not dominate real-world images, and thus the problem is of less commercial interest. Nonetheless, innovation in this area may lead to critical advances in fields like medicine, which also relies on noisy imaging and inference at the limit of the signal-to-noise ratio (Pesapane et al. Reference Pesapane, Codari and Sardanelli2018).
3. Ethical concerns in the practice and education of AI in astronomy
When we train Ph.D. students in academia, we provide them with sophisticated technical knowledge that they can leverage and apply in their future careers, wherever they may lead (Mulvey 2020). While there is evidence that the majority of students who enroll in Astronomy and Astrophysics Doctoral programs begin with the intention to continue on an academic track (Young and Petrescu Reference Young and Petrescu2025), it is a well-documented fact (Cooray et al. Reference Cooray, Abate, Häußler, Trump and Williams2015) that the reality of the job market is not consistent with this aspiration. In the 2023-2024 Academic Year, the Rumor Mill, a popular forum where available jobs are advertised and discussed, listed roughly 50 positions at the assistant professor level. In the same year, U.S. institutions graduated 201 Astronomy Ph.D.sFootnote 3. Nearly 10x more students earn Ph.D.s in Physics and Astronomy, although the fraction of those who focus in Astronomy is harder to estimate. Thus, this 4-fold difference is a gross underestimate of the problem. As academic institutions face increasing funding and enrollment challenges, and even explicit threats in the U.S., the fraction of students who will actually remain in academia is likely to shrink in future years.
But the expertise in the creation and adaptation of AI models acquired in their work in Astrophysics exposes them to the risk of creating tools that may do immense societal harm (Yampolskiy 2016; Verma Reference Verma2019; Buolamwini Reference Buolamwini2024; Benjamin Reference Benjamin2023). This is nothing new: the ‘dual-use’ dilemma (Douglas Reference Douglas2014) has been discussed at length in philosophy and is a core pillar of the ethical framework of the physical and natural sciences (Miller and Selgelid 2007). But with 18% of Physics and Astronomy Ph.D. students reporting that they “develop AI tools” as part of their graduate research,Footnote 4 and with the dangers that AI poses to society (see, for example, Gadbail et al. Reference Gadbail, Verma, Pokle, Zade, Waghale and Jumbde2025; Bender et al. Reference Bender, Gebru, McMillan-Major and Shmitchell2021), this concern is renewed and amplified.
Training and informing our students in the ethics of AI is a moral imperative and a societal responsibility. The “worthlessness” of astronomical data, the fact that their use does not pose a social risk, should not imply we can be careless in the application of AI to astronomical data, but rather that we can use the astronomy sandbox to learn how to responsibly address issues of bias, among others, to develop innovative solutions which may in the end benefit society.
Another important aspect of AI ethicsFootnote 5 is the responsible use of computational resources (Patterson et al. Reference Patterson, Gonzalez, Le, Liang, Munguia, Rothchild, So, Texier and Dean2021; Bender et al. Reference Bender, Gebru, McMillan-Major and Shmitchell2021). As we progressively increase our adoption of ANNs and AI in astrophysics, we are contributing to the carbon footprint of AI. It is well documented (e.g. Adam et al. Reference Adam, Bell, Bullard, Figueroa, Gaylord, Gbadegesin, Goodland, McCurdy, Mills and Shrader-Frechette2001; Bullard 1993) that the populations that will (and already are) most and sooner suffer the impact of climate change are under-resourced communities, which are also, precisely because of the lack of resources, less likely and able to participate in the astrophysics discovery process (White Reference White2019) and while there are concrete efforts to offset this imbalance and make astronomy more inclusive, they have so far only scratched the surface (Norman et al. Reference Norman, Sacco and Russell2024; Bianco Reference Bianco2023), and the fate of these activities, at least in the U.S. is highly uncertain. Can we be responsible users of AI? We have the opportunity to lead by example, starting with tracking the carbon footprint of our projects. Fortunately, several tools exist to make this easy: for example, the Machine Learning Emissions CalculatorFootnote 6 allows users to estimate the kg of CO2 equivalent emissions of training ML models, and the EPA Greenhouse Gas Equivalencies Calculator converts emissions into concrete and relatable units, such as miles driven, or offset costs, such as acres of U.S. forests needed to sequester the emissions, or tons of waste to be recycled, to provide intuitive and impactful estimates. Tracking and disclosing does not solve the problem, but it’s a step toward encouraging responsible use. Of course, the carbon footprint of astrophysical inference remains minuscule compared to that of industry, but we have an opportunity to establish practices and train responsible AI experts who may then bring their norms to industry, and lead to impactful change.
Foundation models (Awais et al. Reference Awais, Naseer, Khan, Anwer, Cholakkal, Shah, Yang and Shahbaz Khan2023; Smith et al. Reference Smith, Roberts, Angeloudi and Huertas-Company2024; Pan et al. [n. + .1667emd.]) may offer an opportunity to ameliorate this problem: while extremely resource-intensive to train, once trained, they are easily adaptable to different tasks by fine-tuning on small datasets. The footprint of training one or a few foundation models for astronomy would likely be smaller than the footprint of the models currently built by the 18% of Physics and Astronomy grad students, plus post-docs and researchers, building and training AI models from scratch. However, it is concerning that instead of unifying behind the effort of building a true foundation model for astronomy, we are seeing in the literature the blooming of many self-supervised multitask models, that are, however, restricted and only suitable for some narrow classes of objects (e.g. Zhang et al. Reference Zhang, Helfer, Gagliano, Mishra-Sharma and Ashley Villar2024; Zuo et al. Reference Zuo, Tao, Huang, Kang, Chen, Cui, Pan, Kong, Tang, Han, Mu, Xu, Fan, Xue, Luo and Liu2025). Again, this may be mostly a sociological, rather than technical problem: if we converge to build one or a few truly foundation models for astronomy, which is a massive and expensive endeavor, who will support the effort? How do we ensure that the likely large group of scholars needed to do this will get sufficient credit if this model enables broad and far-reaching scientific results that other authors will publish? This opportunity may backfire if we end up with a variety of expensive multipurpose self-supervised models that are, after all, only suitable for a restricted number of tasks and only embraced by a small user base of the same researchers who built them and their extended network.
4. Conclusions
The era of synoptic surveys, ushered by the SDSS (York et al. Reference York, Adelman, Anderson, Anderson, Annis, Bahcall, Bakken, Barkhouser, Bastian and Berman2000) and soon to culminate on the LSST (Ivezić et al. 2019), provided opportunities and challenges for the integration of AI in astrophysics sec:intro. Many problems have seen rapid proliferation of AI solutions, like the automation of the astronomical discovery chain sec:automation, others still offer critical opportunities for development sec:encoding. Development of AI solutions for astrophysics faces both technical challenges, due to the unique characteristics of astrophysical data, and sociological challenges. While a significant fraction of the new generations of astrophysicists have integrated the development of AI in their work sec:ethics, trust in AI solutions still needs to be built across large swaths of the astronomy workforce. The most important opportunity we have tho, is to be thoughtful and conscientious AI practitioners, set the example, and shape our communities to realize the promise of AI, without contributing to growing its threat.
I acknowledge the use of DeepSeek (https://www.deepseek.com/) to correct grammar and syntax of my own work. I submitted the text with the prompt “fix grammar and improve syntax but do not make any major changes to the following text”. Some changed were accepted, others rejected.
Open access
