© OB/JVL

You may use or have heard of, or use, ChatGPT -- the Generative Pre-trained Transformer. Such transformer models have recently become very successful language processing. At heart they are sequence-to-sequence translators, taking the words you typed, and predicting what most likely follows. We think this technique is also interesting for learning relationships between astrophysical time sequences -- for example, predicting what a pulsar will do next, or forecasting how the lightcurve of a tidal disruption event will evolve. In a paper that appeared on arxiv last week we pioneer this new method on the most promising first case, the two phases of Gamma-Ray Bursts (GRBs). We choose GRBs over other transients because of a unique trait: in a GRB, a single event produces two distinct time variable phenomena: the prompt emission and the afterglow. In the paper we investigate if a transformer model can predict the afterglow flux from the prompt emission. Our aim is to, if successful, in the future, convert this predictive scheme to the actual physics link. We find that the transformer model can successfully predict different phases of the afterglows in some instances. An example of this is shown in the image, for GRB 090618. The prompt emission (yellow) would be followed by data around the black line, according to the existing, baseline model. But the new transformer predicts the green line -- and the red data points follow this quite well. With this we show transformers are a promising new astrophysical analysis technique. For the GRB test case, the method does a little better on average than the old model, but it has trouble with gaps in the data, and sometimes just goes off the rails (the "exposure bias" problem) -- so we also discuss the required future improvements.