{ "cells": [ { "cell_type": "markdown", "id": "a13b8653", "metadata": {}, "source": [ "# Finding X-ray observations for a sample of SNe 30 days either side of discovery" ] }, { "cell_type": "markdown", "id": "2453f437", "metadata": {}, "source": [ "This case study searches for X-ray observations (from XMM, Chandra, and Swift; but it is applicable to all DAXA missions) of Type 1a Supernovae locations, within a time window around each SNe's discovery date. This process is applicable to any search for an observation of a specific place at a specific time (or a sample of places at a set of different times)." ] }, { "cell_type": "markdown", "id": "5e2a28b1", "metadata": {}, "source": [ "## Import Statements" ] }, { "cell_type": "code", "execution_count": 1, "id": "aa9f16e8", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "from astropy.coordinates import SkyCoord\n", "from datetime import datetime, timedelta\n", "\n", "from daxa.mission import XMMPointed, Chandra, Swift\n", "from daxa.archive import Archive" ] }, { "cell_type": "markdown", "id": "fa824356", "metadata": {}, "source": [ "## Other Tutorials" ] }, { "cell_type": "markdown", "id": "cc8b77fc", "metadata": {}, "source": [ "These case studies are meant to be highly specific examples of how you might acquire data for a particular science case, they do not provide general instruction on how to use DAXA missions or archives. We instead direct you to:\n", "\n", "* [Using DAXA missions](../missions.html) - Here we explain what DAXA mission classes are and how to use them to select only the data you need.\n", "* [Creating a DAXA archive](../archives.html) - This explains how to create an archive, load an existing archive, and the various properties and features of DAXA archives.\n", "* [Processing telescope data](../../../tutorials.process.html) - The processing tutorials for different missions are presented here, though there may not yet be processing support for all missions.\n", "\n", "Reading through these should give you a good understanding of how DAXA can be used to acquire, organise, and process multi-mission X-ray datasets for your specific use case." ] }, { "cell_type": "markdown", "id": "d39a0cd0", "metadata": {}, "source": [ "## Sample" ] }, { "cell_type": "markdown", "id": "30377d82", "metadata": {}, "source": [ "We read in a demonstrative sample, which consists of 1000 randomly selected Type 1a Supernovae from the transient name server (TNS); there are many columns, but the only ones we require are position (Right-Ascension and Declination) and the discovery date (which we will search around):" ] }, { "cell_type": "code", "execution_count": 2, "id": "78c35ab9", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
objidname_prefixnameradeclinationredshifttypeidtypereporting_groupidreporting_group...source_groupdiscoverydatediscoverymagdiscmagfilterfilterreporterstime_receivedinternal_namescreationdatelastmodified
0133295SN2023ock232.47413066.0750110.0393.0SN Ia48.0ZTF...ZTF2023-07-26 08:16:47.00019.2400110.0gC. Fremling (Caltech) on behalf of the Zwicky ...2023-07-28 07:24:04ZTF23aauqmys, ATLAS23pvm2023-07-28 07:24:052023-08-06 05:49:07
174545SN2021dj208.77722754.3049020.0703.0SN Ia48.0ZTF...ZTF2021-01-01 11:24:00.00019.2500111.0rC. Fremling (Caltech) on behalf of the Zwicky ...2021-01-03 22:01:11ZTF21aaabucr, ATLAS21aza, PS21xg2021-01-03 22:01:132021-02-07 11:04:35
256011SN2020enj155.2377950.5737640.1043.0SN Ia18.0ATLAS...ATLAS2020-03-16 10:59:31.20019.235072.0orangeJ. Tonry, L. Denneau, A. Heinze, H. Weiland, H...2020-03-16 18:48:30ATLAS20hza, PS20agg, ZTF20aaubotx2020-03-16 18:48:322020-03-16 18:48:32
374416SN2021E62.97842816.8030080.0503.0SN Ia74.0ALeRCE...ZTF2021-01-01 03:34:56.99719.9476110.0gF. Forster, F.E. Bauer, A. Munoz-Arancibia, G....2021-01-01 16:43:48ZTF21aaaalaf, ATLAS21ash2021-01-01 16:43:532021-01-13 00:38:47
449744SN2019xck97.07010523.6085910.0353.0SN Ia74.0ALeRCE...ZTF2019-12-19 09:51:42.00019.3648110.0gF. Forster, F.E. Bauer, G. Pignata, J. Arredon...2019-12-19 16:00:58ZTF19aczeomw, ATLAS19bdqm2019-12-19 16:01:092019-12-19 16:01:09
\n", "

5 rows × 21 columns

\n", "
" ], "text/plain": [ " objid name_prefix name ra declination redshift typeid \\\n", "0 133295 SN 2023ock 232.474130 66.075011 0.039 3.0 \n", "1 74545 SN 2021dj 208.777227 54.304902 0.070 3.0 \n", "2 56011 SN 2020enj 155.237795 0.573764 0.104 3.0 \n", "3 74416 SN 2021E 62.978428 16.803008 0.050 3.0 \n", "4 49744 SN 2019xck 97.070105 23.608591 0.035 3.0 \n", "\n", " type reporting_groupid reporting_group ... source_group \\\n", "0 SN Ia 48.0 ZTF ... ZTF \n", "1 SN Ia 48.0 ZTF ... ZTF \n", "2 SN Ia 18.0 ATLAS ... ATLAS \n", "3 SN Ia 74.0 ALeRCE ... ZTF \n", "4 SN Ia 74.0 ALeRCE ... ZTF \n", "\n", " discoverydate discoverymag discmagfilter filter \\\n", "0 2023-07-26 08:16:47.000 19.2400 110.0 g \n", "1 2021-01-01 11:24:00.000 19.2500 111.0 r \n", "2 2020-03-16 10:59:31.200 19.2350 72.0 orange \n", "3 2021-01-01 03:34:56.997 19.9476 110.0 g \n", "4 2019-12-19 09:51:42.000 19.3648 110.0 g \n", "\n", " reporters time_received \\\n", "0 C. Fremling (Caltech) on behalf of the Zwicky ... 2023-07-28 07:24:04 \n", "1 C. Fremling (Caltech) on behalf of the Zwicky ... 2021-01-03 22:01:11 \n", "2 J. Tonry, L. Denneau, A. Heinze, H. Weiland, H... 2020-03-16 18:48:30 \n", "3 F. Forster, F.E. Bauer, A. Munoz-Arancibia, G.... 2021-01-01 16:43:48 \n", "4 F. Forster, F.E. Bauer, G. Pignata, J. Arredon... 2019-12-19 16:00:58 \n", "\n", " internal_names creationdate lastmodified \n", "0 ZTF23aauqmys, ATLAS23pvm 2023-07-28 07:24:05 2023-08-06 05:49:07 \n", "1 ZTF21aaabucr, ATLAS21aza, PS21xg 2021-01-03 22:01:13 2021-02-07 11:04:35 \n", "2 ATLAS20hza, PS20agg, ZTF20aaubotx 2020-03-16 18:48:32 2020-03-16 18:48:32 \n", "3 ZTF21aaaalaf, ATLAS21ash 2021-01-01 16:43:53 2021-01-13 00:38:47 \n", "4 ZTF19aczeomw, ATLAS19bdqm 2019-12-19 16:01:09 2019-12-19 16:01:09 \n", "\n", "[5 rows x 21 columns]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "samp = pd.read_csv(\"samp_files/sn1a_samp.csv\")\n", "samp.head(5)" ] }, { "cell_type": "markdown", "id": "3991c55c", "metadata": {}, "source": [ "We read out the coordinates into an astropy coordinate object, and set up the time windows we will be searching for each SNe - for this demonstration we will search for X-ray observations 30 days either side of the disovery date:" ] }, { "cell_type": "code", "execution_count": 3, "id": "083984d3", "metadata": {}, "outputs": [], "source": [ "pos = SkyCoord(samp['ra'].values, samp['declination'].values, unit='deg')\n", "start_times = np.array([datetime.strptime(dd, \"%Y-%m-%d %H:%M:%S.%f\") - timedelta(days=30) \n", " for dd in samp['discoverydate'].values])\n", "end_times = np.array([datetime.strptime(dd, \"%Y-%m-%d %H:%M:%S.%f\") + timedelta(days=30) \n", " for dd in samp['discoverydate'].values])" ] }, { "cell_type": "markdown", "id": "cda7ed3b", "metadata": {}, "source": [ "## Defining missions" ] }, { "cell_type": "markdown", "id": "1db78091", "metadata": {}, "source": [ "We create instances of the XMM, Chandra, and Swift missions in order to search their archives - Swift is the most likely to have many matching observations, as it acts as a transient follow-up telescope, but XMM and Chandra are workhorses and may have some observations that we might want to explore: " ] }, { "cell_type": "code", "execution_count": 4, "id": "032436b4", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/dt237/code/DAXA/daxa/mission/xmm.py:83: UserWarning: 140 of the 17697 observations located for this mission have been removed due to NaN RA or Dec values\n", " self._fetch_obs_info()\n", "/Users/dt237/code/DAXA/daxa/mission/swift.py:101: UserWarning: 598 of the 353616 observations located for Swift have been removed due to all instrument exposures being zero.\n", " self._fetch_obs_info()\n", "/Users/dt237/code/DAXA/daxa/mission/swift.py:101: UserWarning: 17 of the 353616 observations located for Swift have been removed due to all chosen instrument (XRT, BAT) exposures being zero.\n", " self._fetch_obs_info()\n" ] } ], "source": [ "xm = XMMPointed()\n", "ch = Chandra()\n", "sw = Swift()" ] }, { "cell_type": "markdown", "id": "c7dda794", "metadata": {}, "source": [ "## Searching for observations" ] }, { "cell_type": "markdown", "id": "df78c594", "metadata": {}, "source": [ "We will make use of the DAXA filtering method that allows us to search for observations of a particular coordinate, within a particular time frame, for a whole sample. It is called the same way for the three missions we are using, we pass the positions, start times, and end times, and the keyword arguments have the following meanings:\n", "\n", "* **return_obs_info** - If True, a dataframe is returned from the method to allow the user to link specific ObsIDs to particular entries in our original sample table. The dataframe contains a 'pos_ind' column, which contains indexes corresponding to the input positions (i.e. the 4th entry of pos would have index 3), it also contains ObsIDs matched to that coordinate and time window.\n", "* **over_run** - If True, observations that start or end outside of the specified time window are accepted. If False, they are not." ] }, { "cell_type": "code", "execution_count": 5, "id": "2e622e5a", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/dt237/code/DAXA/daxa/mission/base.py:1389: UserWarning: Every value in the filter array is False, meaning that no observations remain.\n", " self.filter_array = np.full(self.filter_array.shape, False)\n", "/Users/dt237/code/DAXA/daxa/mission/base.py:1075: UserWarning: Chandra FoV are difficult to define, as they can be strongly dependant on observation mode; as such take these as very approximate.\n", " fov = self.fov\n", "/Users/dt237/code/DAXA/daxa/mission/base.py:1389: UserWarning: Every value in the filter array is False, meaning that no observations remain.\n", " self.filter_array = np.full(self.filter_array.shape, False)\n", "/Users/dt237/code/DAXA/daxa/mission/base.py:97: UserWarning: There are multiple chosen instruments XRT, BAT for swift with different FoVs, but they observe simultaneously. As such the search distance has been set to the largest FoV of the chosen instruments.\n", " any_ret = change_func(*args, **kwargs)\n", "/Users/dt237/code/DAXA/daxa/mission/base.py:1389: UserWarning: Every value in the filter array is False, meaning that no observations remain.\n", " self.filter_array = np.full(self.filter_array.shape, False)\n" ] } ], "source": [ "xm_assoc = xm.filter_on_positions_at_time(pos, start_times, end_times, return_obs_info=True, over_run=True)\n", "ch_assoc = ch.filter_on_positions_at_time(pos, start_times, end_times, return_obs_info=True, over_run=True)\n", "sw_assoc = sw.filter_on_positions_at_time(pos, start_times, end_times, return_obs_info=True, over_run=True)" ] }, { "cell_type": "markdown", "id": "4a42ebd5", "metadata": {}, "source": [ "## Identified observations" ] }, { "cell_type": "markdown", "id": "27f08768", "metadata": {}, "source": [ "We will now use the returns from the filtering methods to highlight the observations which have been identified as fulfilling our criteria:" ] }, { "cell_type": "code", "execution_count": 6, "id": "fe604767", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
pos_indpos_rapos_decObsIDssn_name
30414158.4278124908335339.4906107168647608240301012018hus
\n", "
" ], "text/plain": [ " pos_ind pos_ra pos_dec ObsIDs sn_name\n", "30 414 158.42781249083353 39.49061071686476 0824030101 2018hus" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "xm_assoc['sn_name'] = samp.loc[xm_assoc['pos_ind'].values.astype(int), 'name'].values\n", "xm_assoc" ] }, { "cell_type": "code", "execution_count": 7, "id": "e15d4bb5", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
pos_indpos_rapos_decObsIDssn_name
1045219.74128078451.0827058066216972020hyi
41142234.55379239.732811225282019rmq
44160161.26947012.31901835224942020kxf
114429186.4951766617.23543298389237712021ita
166664206.84352096926.38468774227023,27026,27021,27022,27024,27029,27020,278062023hrk
174706222.2402518.326689226592021min
196783193.821752.897311235632021pkz
219894222.90508657618.9261358766269622023jgq
\n", "
" ], "text/plain": [ " pos_ind pos_ra pos_dec \\\n", "10 45 219.741280784 51.0827058066 \n", "41 142 234.553792 39.732811 \n", "44 160 161.2694701 2.31901835 \n", "114 429 186.495176661 7.23543298389 \n", "166 664 206.843520969 26.384687742 \n", "174 706 222.24025 18.326689 \n", "196 783 193.82175 2.897311 \n", "219 894 222.905086576 18.9261358766 \n", "\n", " ObsIDs sn_name \n", "10 21697 2020hyi \n", "41 22528 2019rmq \n", "44 22494 2020kxf \n", "114 23771 2021ita \n", "166 27023,27026,27021,27022,27024,27029,27020,27806 2023hrk \n", "174 22659 2021min \n", "196 23563 2021pkz \n", "219 26962 2023jgq " ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ch_assoc['sn_name'] = samp.loc[ch_assoc['pos_ind'].values.astype(int), 'name'].values\n", "ch_assoc" ] }, { "cell_type": "code", "execution_count": 8, "id": "a35e30ef", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
pos_indpos_rapos_decObsIDssn_name
11208.7772250634608354.3049020501095600013608061,00013608056,00013608059,0001360805...2021dj
22155.237801715446780.573759293699524300095611001,000750410082020enj
770.814294178066041116.1457103807638100014427002,00014427004,00014427001,0001442700...2021rhu
1010250.6030678210881678.91498537452249000951790012019kyz
1717219.498588390004249.38854056545892500013556003,00013556005,00013556002,000135560042020lil
..................
873955267.948046596122244.9042097837244500074924063,00074924061,000757540122022ucs
879961295.4186780442415-21.2626856035394300016144004,00016144003,00016144001,000161440022023mvl
88797039.0107199416329143.47205948460794600010346005,00010346003,00010346008,0001034601...2017hjy
905990347.387924718551615.6592726737687700013713001,00013713003,00013713002,0001371300...2020szr
909995355.9468761067174751.24149371216688600081310002,00035031184,00035031190,0003503118...2020rsi
\n", "

119 rows × 5 columns

\n", "
" ], "text/plain": [ " pos_ind pos_ra pos_dec \\\n", "1 1 208.77722506346083 54.30490205010956 \n", "2 2 155.23780171544678 0.5737592936995243 \n", "7 7 0.8142941780660411 16.14571038076381 \n", "10 10 250.60306782108816 78.91498537452249 \n", "17 17 219.49858839000424 9.388540565458925 \n", ".. ... ... ... \n", "873 955 267.9480465961222 44.90420978372445 \n", "879 961 295.4186780442415 -21.26268560353943 \n", "887 970 39.01071994163291 43.472059484607946 \n", "905 990 347.3879247185516 15.65927267376877 \n", "909 995 355.94687610671747 51.241493712166886 \n", "\n", " ObsIDs sn_name \n", "1 00013608061,00013608056,00013608059,0001360805... 2021dj \n", "2 00095611001,00075041008 2020enj \n", "7 00014427002,00014427004,00014427001,0001442700... 2021rhu \n", "10 00095179001 2019kyz \n", "17 00013556003,00013556005,00013556002,00013556004 2020lil \n", ".. ... ... \n", "873 00074924063,00074924061,00075754012 2022ucs \n", "879 00016144004,00016144003,00016144001,00016144002 2023mvl \n", "887 00010346005,00010346003,00010346008,0001034601... 2017hjy \n", "905 00013713001,00013713003,00013713002,0001371300... 2020szr \n", "909 00081310002,00035031184,00035031190,0003503118... 2020rsi \n", "\n", "[119 rows x 5 columns]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sw_assoc['sn_name'] = samp.loc[sw_assoc['pos_ind'].values.astype(int), 'name'].values\n", "sw_assoc" ] }, { "cell_type": "markdown", "id": "4e9d5b50", "metadata": {}, "source": [ "## Defining an archive" ] }, { "cell_type": "markdown", "id": "a21ed8fc", "metadata": {}, "source": [ "The filtered missions can then be used to define an archive containing the selected data:" ] }, { "cell_type": "code", "execution_count": 9, "id": "9b77df38", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Downloading XMM-Newton Pointed data: 100%|██████████████████████████████████████| 1/1 [00:22<00:00, 22.25s/it]\n", "Downloading Chandra data: 100%|███████████████████████████████████████████████| 15/15 [00:38<00:00, 2.56s/it]\n", "Downloading Swift data: 100%|███████████████████████████████████████████████| 454/454 [07:33<00:00, 1.00it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "-----------------------------------------------------\n", "Number of missions - 3\n", "Total number of observations - 470\n", "Beginning of earliest observation - 2016-02-21 03:25:58\n", "End of latest observation - 2024-01-05 07:08:51\n", "\n", "-- XMM-Newton Pointed --\n", " Internal DAXA name - xmm_pointed\n", " Chosen instruments - M1, M2, PN\n", " Number of observations - 1\n", " Fully Processed - False\n", "\n", "-- Chandra --\n", " Internal DAXA name - chandra\n", " Chosen instruments - ACIS-I, ACIS-S, HRC-I, HRC-S\n", " Number of observations - 15\n", " Fully Processed - False\n", "\n", "-- Swift --\n", " Internal DAXA name - swift\n", " Chosen instruments - XRT, BAT\n", " Number of observations - 454\n", " Fully Processed - False\n", "-----------------------------------------------------\n", "\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n" ] } ], "source": [ "arch = Archive('sne_search', [xm, ch, sw])\n", "arch.info()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.0" } }, "nbformat": 4, "nbformat_minor": 5 }