Observations of Messier 51 from many telescope archives

This case study searches for X-ray observations of the Messier 51 galaxy, taken by a variety of X-ray telescopes. We chose this galaxy because it is active (and thus likely X-ray bright), and nearby - this makes it a priority target for many X-ray observatories. This process could equally be applied to any named object (or object at a specific set of coordinates).

Import Statements

[1]:

from warnings import warn

from astropy.coordinates import SkyCoord

from daxa.archive import Archive
from daxa.archive.assemble import assemble_archive_from_positions
from daxa.exceptions import NoObsAfterFilterError
from daxa.mission import XMMPointed, Chandra, ASCA, Suzaku, ROSATPointed, ROSATAllSky, \
    MISS_INDEX
from daxa.mission.tools import multi_mission_filter_on_positions

Other Tutorials

These case studies are meant to be highly specific examples of how you might acquire data for a particular science case, they do not provide general instruction on how to use DAXA missions or archives. We instead direct you to:

  • Using DAXA missions - Here we explain what DAXA mission classes are and how to use them to select only the data you need.

  • Creating a DAXA archive - This explains how to create an archive, load an existing archive, and the various properties and features of DAXA archives.

  • Processing telescope data - The processing tutorials for different missions are presented here, though there may not yet be processing support for all missions.

Reading through these should give you a good understanding of how DAXA can be used to acquire, organise, and process multi-mission X-ray datasets for your specific use case.

One-line solution

Though we provide individual functions that wrap the various steps required to collect all data at a given position for various telescope archives, we also include a one-line solution which executes these steps; the assemble_archive_from_positions() function. This function will by default initialise all available missions within DAXA, and then calls each mission’s filter_on_positions() method. The return is a list of mission objects that have had the positional filtering applied, note that if no observations are returned after filtering then this mission is excluded.

In this tutorial we assume knowledge of the BaseMission.filter_on_positions() method, if you are unfamiliar please consult the relevant tutorial.

In this case study, the goal is to collect all observations of M51 from XMM, Chandra, ASCA, Suzaku, ROSAT Pointed, and ROSAT All-Sky missions (note that other missions are supported by DAXA). We present here how to achieve this with the one line solution.

Firstly, in order to see how to format the input to the missions argument of assemble_archive_from_positions(), we can look at the MISS_INDEX dictionary:

[6]:

MISS_INDEX

[6]:

{'xmm_pointed': daxa.mission.xmm.XMMPointed,
 'nustar_pointed': daxa.mission.nustar.NuSTARPointed,
 'erosita_calpv': daxa.mission.erosita.eROSITACalPV,
 'erosita_all_sky_de_dr1': daxa.mission.erosita.eRASS1DE,
 'chandra': daxa.mission.chandra.Chandra,
 'rosat_all_sky': daxa.mission.rosat.ROSATAllSky,
 'rosat_pointed': daxa.mission.rosat.ROSATPointed,
 'swift': daxa.mission.swift.Swift,
 'suzaku': daxa.mission.suzaku.Suzaku,
 'asca': daxa.mission.asca.ASCA,
 'integral_pointed': daxa.mission.integral.INTEGRALPointed}

We can therefore use the following list to input into the missions argument of the assemble_archive_from_positions() function:

[2]:

missions = ['xmm_pointed', 'chandra', 'asca', 'suzaku', 'rosat_pointed', 'rosat_all_sky']

We then input the position of M51 in the SkyCoord format:

[3]:

position = SkyCoord(202.47, 47.2, unit='deg')

So we can finally run the assemble_archive_from_positions() function with our inputs (note that the missions argument is optional, and by default all missions within DAXA would be searched):

[6]:

# M51 is the name of the archive
arch = assemble_archive_from_positions('M51', positions=position, missions=missions,
                                       download_products=False)

/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/xmm.py:83: UserWarning: 140 of the 18123 observations located for this mission have been removed due to NaN RA or Dec values
  self._fetch_obs_info()
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/base.py:1373: UserWarning: Chandra FoV are difficult to define, as they can be strongly dependant on observation mode; as such take these as very approximate.
  fov = self.fov
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/asca.py:125: UserWarning: 5 of the 3079 observations located for ASCA have been removed due to all instrument exposures being zero.
  self._fetch_obs_info()
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/base.py:98: UserWarning: There are multiple chosen instruments SIS0, SIS1, GIS2, GIS3 for asca with different FoVs, but they observe simultaneously. As such the search distance has been set to the largest FoV of the chosen instruments.
  any_ret = change_func(*args, **kwargs)
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/suzaku.py:109: UserWarning: 14 of the 3055 observations located for Suzaku have been removed due to all instrument exposures being zero.
  self._fetch_obs_info()
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/suzaku.py:297: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '['EGS' 'GS' 'EGS' ... 'GCL' 'GCL' 'EGS']' has dtype incompatible with int16, please explicitly cast to a compatible dtype first.
  rel_suzaku.loc[type_recog, 'target_category'] = rel_suzaku.loc[type_recog, 'target_category'].apply(
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/archive/assemble.py:61: UserWarning: No observations found after the filter for suzaku, will not be included in the output dictionary.
  miss_list = multi_mission_filter_on_positions(positions, search_distance, missions)
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/archive/base.py:196: UserWarning: Proprietary data have been selected, but no credentials provided; as such the proprietary data have been excluded from download and further processing.
  mission.download(download_products=download_products[mission.name])
Downloading XMM-Newton Pointed data: 100%|██████████| 16/16 [1:30:41<00:00, 340.11s/it]
Downloading Chandra data: 100%|██████████| 28/28 [59:09<00:00, 126.78s/it]
Downloading ASCA data: 100%|██████████| 2/2 [09:06<00:00, 273.26s/it]
Downloading ROSAT Pointed data: 100%|██████████| 6/6 [00:59<00:00,  9.90s/it]
Downloading RASS data: 100%|██████████| 2/2 [00:52<00:00, 26.14s/it]

[8]:

arch.info()


-----------------------------------------------------
Version - 0.0.0
Number of missions - 5
Total number of observations - 54
Beginning of earliest observation - 1990-07-11 00:00:00
End of latest observation - 2022-01-08 17:51:21

-- XMM-Newton Pointed --
   Internal DAXA name - xmm_pointed
   Chosen instruments - M1, M2, PN
   Number of observations - 16
   Fully Processed - False

-- Chandra --
   Internal DAXA name - chandra
   Chosen instruments - ACIS, HRC
   Number of observations - 28
   Fully Processed - False

-- ASCA --
   Internal DAXA name - asca
   Chosen instruments - SIS0, SIS1, GIS2, GIS3
   Number of observations - 2
   Fully Processed - False

-- ROSAT Pointed --
   Internal DAXA name - rosat_pointed
   Chosen instruments - PSPCB, PSPCC, HRI
   Number of observations - 6
   Fully Processed - False

-- RASS --
   Internal DAXA name - rosat_all_sky
   Chosen instruments - PSPC
   Number of observations - 2
   Fully Processed - False
-----------------------------------------------------

To perform the search without creating an Archive object, we can use the multi_mission_filter_on_positions() function:

[4]:

multi_mission_filter_on_positions(position, missions=missions)

/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/xmm.py:83: UserWarning: 140 of the 18126 observations located for this mission have been removed due to NaN RA or Dec values
  self._fetch_obs_info()
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/base.py:1373: UserWarning: Chandra FoV are difficult to define, as they can be strongly dependant on observation mode; as such take these as very approximate.
  fov = self.fov
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/asca.py:125: UserWarning: 5 of the 3079 observations located for ASCA have been removed due to all instrument exposures being zero.
  self._fetch_obs_info()
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/base.py:108: UserWarning: There are multiple chosen instruments SIS0, SIS1, GIS2, GIS3 for asca with different FoVs, but they observe simultaneously. As such the search distance has been set to the largest FoV of the chosen instruments.
  any_ret = change_func(*args, **kwargs)
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/suzaku.py:109: UserWarning: 14 of the 3055 observations located for Suzaku have been removed due to all instrument exposures being zero.
  self._fetch_obs_info()
/Users/jp735/Desktop/DAXA_dev/DAXA/daxa/mission/suzaku.py:297: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '['EGS' 'GS' 'EGS' ... 'GCL' 'GCL' 'EGS']' has dtype incompatible with int16, please explicitly cast to a compatible dtype first.
  rel_suzaku.loc[type_recog, 'target_category'] = rel_suzaku.loc[type_recog, 'target_category'].apply(
/var/folders/86/x534hjnd60nfqyng9m_xwb740000gr/T/ipykernel_46523/339423454.py:1: UserWarning: No observations found after the filter for suzaku, will not be included in the output dictionary.
  multi_mission_filter_on_positions(position, missions=missions)

[4]:

[<daxa.mission.xmm.XMMPointed at 0x122b0dbe0>,
 <daxa.mission.chandra.Chandra at 0x123d757f0>,
 <daxa.mission.asca.ASCA at 0x123d77770>,
 <daxa.mission.rosat.ROSATPointed at 0x123d76270>,
 <daxa.mission.rosat.ROSATAllSky at 0x123d77620>]

Breaking Down the Steps

To better understand what the assemble_archive_from_positions() function is doing, we show the individual steps that have taken place.

Defining Missions

We create instances of the XMM, Chandra, ASCA, Suzaku, ROSAT Pointed, and ROSAT All-Sky missions in order to search their archives - other missions are supported by DAXA (and can be found in the missions tutorial), but these are a subset likely to have observations of M51:

[2]:

xm = XMMPointed()
ch = Chandra()
asc = ASCA()
su = Suzaku()
rp = ROSATPointed()
ra = ROSATAllSky()

/Users/dt237/code/DAXA/daxa/mission/xmm.py:83: UserWarning: 140 of the 17697 observations located for this mission have been removed due to NaN RA or Dec values
  self._fetch_obs_info()
/Users/dt237/code/DAXA/daxa/mission/asca.py:91: UserWarning: 5 of the 3079 observations located for ASCA have been removed due to all instrument exposures being zero.
  self._fetch_obs_info()
/Users/dt237/code/DAXA/daxa/mission/suzaku.py:96: UserWarning: 14 of the 3055 observations located for Suzaku have been removed due to all instrument exposures being zero.
  self._fetch_obs_info()

Searching for observations

In this instance we use the filter_on_name method to search for observations of M51 - this will use the Sesame name resolver to look-up the coordinates for the object. Alternatively, we could use the filter_on_positions method and supply the coordinate ourselves:

[3]:

xm.filter_on_name("M51")
ch.filter_on_name("M51")
asc.filter_on_name("M51")
rp.filter_on_name("M51")
ra.filter_on_name("M51")

/Users/dt237/code/DAXA/daxa/mission/base.py:1075: UserWarning: Chandra FoV are difficult to define, as they can be strongly dependant on observation mode; as such take these as very approximate.
  fov = self.fov
/Users/dt237/code/DAXA/daxa/mission/base.py:97: UserWarning: There are multiple chosen instruments SIS0, SIS1, GIS2, GIS3 for asca with different FoVs, but they observe simultaneously. As such the search distance has been set to the largest FoV of the chosen instruments.
  any_ret = change_func(*args, **kwargs)

We have deliberately separated the Suzaku search, as we are aware that it will not find any matching data - as such we’ll use this to highlight that the standard Python exception-catching statements can be used to stop a failure to find data derailing your script (for instance you might wish to iterate through a list of missions and have a try-except statement like this:

[4]:

try:
    su.filter_on_name("M51")
except NoObsAfterFilterError as err:
    warn(err.message, stacklevel=2)

/opt/anaconda3/envs/daxa_dev/lib/python3.9/site-packages/IPython/core/interactiveshell.py:3433: UserWarning: The positional search has returned no Suzaku observations.
  exec(code_obj, self.user_global_ns, self.user_ns)

Example of observations identified from filtering

We can use the filtered_obs_info property to retrieve the information table describing the accepted observations:

[5]:

rp.filtered_obs_info

[5]:
ra dec ObsID science_usable start end duration instrument with_filter target_category target_name proc_rev fits_type
368 202.47 47.2 RH600601N00 True 1994-06-18 13:11:33.000000 1994-06-24 08:04:24.000001 0 days 10:05:23 HRI N NGS M 51 2 RDF 3_4
917 202.47 47.2 RP600158N00 True 1991-11-28 16:07:59.999998 1991-12-13 17:50:20.999999 0 days 06:39:16 PSPCB N NGS N5194/N5195 2 RDF 3_4
2904 202.47 47.2 RH600062A03 True 1994-05-22 05:24:01.000002 1994-05-23 07:16:07.000000 0 days 02:36:32 HRI N NGS M51 2 RDF 3_4
3096 202.47 47.2 RH600062A01 True 1992-05-22 23:44:46.000000 1992-06-05 21:50:42.999999 0 days 02:27:11 HRI N NGS M51 2 RFITS V3.
3174 202.47 47.2 RH600062A00 True 1991-12-07 09:46:57.000003 1992-01-10 05:10:12.999996 0 days 02:22:21 HRI N NGS M51 2 RFITS V3.
3303 202.47 47.2 RH601115N00 True 1997-12-26 23:37:19.000001 1997-12-30 07:46:24.000001 0 days 02:15:04 HRI N NGS 2 RDF 4_2

Defining an Archive

The filtered missions can then be used to define an archive containing the selected data:

[6]:

arch = Archive('M51', [xm, ch, asc, rp, ra])
arch.info()

Downloading XMM-Newton Pointed data: 100%|████████████████████████████████████| 16/16 [07:37<00:00, 28.60s/it]
Downloading Chandra data: 100%|███████████████████████████████████████████████| 28/28 [02:57<00:00,  6.32s/it]
Downloading ASCA data: 100%|████████████████████████████████████████████████████| 2/2 [01:00<00:00, 30.23s/it]
Downloading ROSAT Pointed data: 100%|███████████████████████████████████████████| 6/6 [00:11<00:00,  1.93s/it]
Downloading RASS data: 100%|████████████████████████████████████████████████████| 2/2 [00:09<00:00,  4.55s/it]


-----------------------------------------------------
Number of missions - 5
Total number of observations - 54
Beginning of earliest observation - 1990-07-11 00:00:00
End of latest observation - 2022-01-08 17:51:21

-- XMM-Newton Pointed --
   Internal DAXA name - xmm_pointed
   Chosen instruments - M1, M2, PN
   Number of observations - 16
   Fully Processed - False

-- Chandra --
   Internal DAXA name - chandra
   Chosen instruments - ACIS-I, ACIS-S, HRC-I, HRC-S
   Number of observations - 28
   Fully Processed - False

-- ASCA --
   Internal DAXA name - asca
   Chosen instruments - SIS0, SIS1, GIS2, GIS3
   Number of observations - 2
   Fully Processed - False

-- ROSAT Pointed --
   Internal DAXA name - rosat_pointed
   Chosen instruments - PSPCB, PSPCC, HRI
   Number of observations - 6
   Fully Processed - False

-- RASS --
   Internal DAXA name - rosat_all_sky
   Chosen instruments - PSPC
   Number of observations - 2
   Fully Processed - False
-----------------------------------------------------