Creating and interacting with a DAXA Archive

This tutorial will explain the basic concepts behind the second type of important class in DAXA, the Archive class (with mission classes being the first type, see the missions tutorial). DAXA Archives are what manage the datasets that we download from various missions, enabling easy access and greatly simplifying processing/reduction - they allow you to stop thinking about all the files and settings that any large dataset entails.

We will cover the following:

  • Setting up an Archive from scratch, using filtered DAXA missions.

  • Loading an existing Archive from disk.

  • The properties of an Archive.

  • Accessing processing logs and success information (though we do not cover processing in this part of the documentation).

Import Statements

[1]:
from daxa.mission import XMMPointed, Chandra, eRASS1DE, ROSATPointed
from daxa.archive import Archive

import os

What is a DAXA archive?

DAXA Archives take a set of filtered missions, make sure that their data are downloaded, and enable easy access and organisation of all data files and processing functions. Key functionality includes:

  • Storing the logs and errors of all processing steps (if run).

  • Allowing for their easy retrieval.

  • Managing the myriad files produced during the processing.

  • Keeping track of which processes failed for which data, ensuring that any further processing only runs on data that have successfully passed through the earlier processes.

Archives can also be loaded back into DAXA at a later date, so that the processing logs of data that has since been found to be problematic can be easily inspected, or indeed so that processing steps can be re-run with different settings; this also allows for archives to be updated, if more data become available.

Creating a new archive

Here we will demonstrate how to set up a new DAXA Archive from scratch - this information can be combined with the the missions tutorial and the case studies to create an archive from any dataset you might be using.

Step 1 - Set up and filter missions

The first thing we have to do is to select the observations that we wish to include in the archive (and indeed the missions that we wish to include). The missions all have different characteristics, so your choice of which to include will be heavily dependent on your science case.

Here we will create an archive of XMM, Chandra, eROSITA All-Sky DR1, and ROSAT pointed observations of a famous galaxy cluster (though the archive would behave the same if it held data for a large sample of objects).

First of all, we define instances of the mission classes that we wish to include:

[2]:
xm = XMMPointed()
ch = Chandra()
er = eRASS1DE()
rp = ROSATPointed()
/Users/dt237/code/DAXA/daxa/mission/xmm.py:83: UserWarning: 140 of the 17701 observations located for this mission have been removed due to NaN RA or Dec values
  self._fetch_obs_info()

Then we filter them to only include observations of our cluster:

[3]:
xm.filter_on_name("A3667")
ch.filter_on_name("A3667")
er.filter_on_name("A3667")
rp.filter_on_name("A3667")
/Users/dt237/code/DAXA/daxa/mission/base.py:1335: UserWarning: Chandra FoV are difficult to define, as they can be strongly dependant on observation mode; as such take these as very approximate.
  fov = self.fov

We then download the available data (though the declaration of an Archive would also trigger this, we do it this way because we wish to download pre-generated products for Chandra and ROSAT pointed observations):

[4]:
xm.download()
ch.download(download_products=True)
er.download()
rp.download(download_products=True)
Downloading XMM-Newton Pointed data: 100%|██████████████████████████████████████| 8/8 [02:18<00:00, 17.33s/it]
Downloading Chandra data: 100%|███████████████████████████████████████████████| 12/12 [02:58<00:00, 14.86s/it]
Downloading eRASS DE:1 data: 100%|██████████████████████████████████████████████| 1/1 [01:18<00:00, 78.21s/it]
Downloading ROSAT Pointed data: 100%|███████████████████████████████████████████| 3/3 [00:17<00:00,  5.77s/it]

Step 2 - Setting up an Archive object

Now we create the actual DAXA Archive instance - we can pass the following arguments:

  • archive_name - The name to be given to the archive (used to load it back in at a later date, if necessary). The only input required

  • missions - The filtered missions that we have already created (can be left as None if loading in an existing archive).

  • clobber - Will overwrite an existing archive if the passed archive_name has already been used. Default is False.

  • download_products - If the missions have not already had downloads triggered, this controls whether pre-processed products should be downloaded or not. Default is True, a dictionary with mission names as keys and True/False as values can be passed to provide more nuanced control.

  • use_preprocessed - Whether pre-processed data (for those missions that provide it) should be automatically imported into the archive processed data structure. Default is False, a dictionary with mission names as keys and True/False as values can be passed to provide more nuanced control.

We demonstrate how to set up an archive using the pre-processed data that is downloadable from the Chandra, eRASS1DE, and ROSAT-Pointed online datasets, as well as the raw data available from the XMM online dataset:

[5]:
arch = Archive("A3667", [xm, ch, er, rp], clobber=True,
               use_preprocessed={'xmm_pointed': False, 'chandra': True,
                                 'erosita_all_sky_de_dr1': True,
                                 'rosat_pointed': True})
Including pre-processed Chandra data in the archive: 100%|████████████████████| 12/12 [00:00<00:00, 24.73it/s]
Including pre-processed eRASS DE:1 data in the archive: 100%|███████████████████| 1/1 [00:00<00:00,  1.14it/s]
Including pre-processed ROSAT Pointed data in the archive: 100%|████████████████| 3/3 [00:00<00:00, 16.83it/s]

Now we’ve declared it, we can use the info() method to get a summary of its current status, including the amount of data available:

[6]:
arch.info()

-----------------------------------------------------
Number of missions - 4
Total number of observations - 24
Beginning of earliest observation - 1992-04-14 18:55:38.000003
End of latest observation - 2020-04-20 12:23:50

-- XMM-Newton Pointed --
   Internal DAXA name - xmm_pointed
   Chosen instruments - M1, M2, PN
   Number of observations - 8
   Fully Processed - False

-- Chandra --
   Internal DAXA name - chandra
   Chosen instruments - ACIS-I, ACIS-S, HRC-I, HRC-S
   Number of observations - 12
   Fully Processed - True

-- eRASS DE:1 --
   Internal DAXA name - erosita_all_sky_de_dr1
   Chosen instruments - TM1, TM2, TM3, TM4, TM5, TM6, TM7
   Number of observations - 1
   Fully Processed - True

-- ROSAT Pointed --
   Internal DAXA name - rosat_pointed
   Chosen instruments - PSPCB, PSPCC, HRI
   Number of observations - 3
   Fully Processed - True
-----------------------------------------------------

Step 3 - Processing the Archive

We’re not actually going to cover how to process things here, as each telescope tends to have its own backend software with a unique way of doing things; they each have their own processing tutorials, which will demonstrate both a one-line processing method, and how to control the reduction in more detail. Any processing method will take the archive object as an argument, and act on the data stored within it.

So instead we include this step here to highlight that the next logical step after the creation of a new archive is to run processing and reduction routines, if raw data have been downloaded. The successful completion of this step will leave you with an archive of data that you can easily manage, access, and use for your scientific analyses.

If you elected to download existing products (most missions support this), then only one processing step is necessary - this reorganises the downloaded data so that it is compatible with DAXA storage and file naming conventions. It will have run automatically on declaration

Step 4 - Using the data

Once an archive has been constructed, whether of raw data to be processed locally, pre-processed data products, or a combination of both, the end result will be a set of X-ray data that can be used for scientific analysis.

The storage and organisation of the archive’s data is entirely consistent between different telescopes; the archive’s processed data (the ‘processed_data’ directory within the overall archive path, which is identified using the archive_path property) is organised like this:

  • Mission Name - For instance, ‘xmm_pointed’ or ‘chandra’

    • ObsID - Each ObsID of a mission gets a sub-directory.

      • events - Where event lists (uncleaned, cleaned, and final) are stored.

      • images - Where all images and exposure maps are stored.

      • background - Where any background maps or models are stored.

      • cleaning - Where any by-products of the cleaning processes are stored.

      • logs - Where logs from all processing steps are stored.

To provide an example, we show you the contents of an eRASS1DE directory that contains pre-processed data:

[7]:
demo_pth = arch.get_current_data_path('erosita_all_sky_de_dr1', '304147')

os.listdir(demo_pth)
[7]:
['background', 'images', 'logs', 'events', 'cleaning']

Now the contents of the images sub-directory:

[8]:
os.listdir(os.path.join(demo_pth, 'images'))
[8]:
['obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en0.2_0.5keV-image.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en0.2_2.3keV-image.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en0.6_2.3keV-image.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en2.3_5.0keV-image.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en0.2_0.6keV-image.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en0.5_1.0keV-expmap.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en0.2_2.3keV-expmap.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en1.0_2.0keV-image.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en1.0_2.0keV-expmap.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en2.3_5.0keV-expmap.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en0.2_0.6keV-expmap.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en0.5_1.0keV-image.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en0.6_2.3keV-expmap.fits',
 'obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-en0.2_0.5keV-expmap.fits']

As well as the events sub-directory:

[9]:
os.listdir(os.path.join(demo_pth, 'events'))
[9]:
['obsid304147-instTM1_TM2_TM3_TM4_TM5_TM6_TM7-subexpNone-finalevents.fits']

The common format of the filenames used by DAXA should be fairly evident by this point; it extends across all data products produced by DAXA and is shared across all the different supported missions. Some elements may differ between data products, but the overall structure is the same.

Each name contains summary information that allow it to be uniquely identified, both programatically and by a user; different pieces of information start with an identifier (for instance obsid to denote where the relevant ObsID begins), and different pieces are separated by a ‘-‘. If there are multiple bits of information connected to a particular category then they are separated by ‘_’ (see ‘inst’ part of eROSITA file names above).

The following can be contained in a filename:

  • ObsID - Denoted by ‘obsid’, this will be present in the name of every data file.

  • Instrument - Denoted by ‘inst’, this should be present in the name of every data file.

  • Sub-exposure ID - Denoted by ‘subexp’; some missions support multiple sub-exposures per ObsID instrument, in which case the sub-exposure ID will be included here. It is also possible for ‘None’ to be the value if there are no sub-exposures, or ‘ALL’ if the file is a combination of sub-exposures.

  • Lower and Upper Energy - Denoted by ‘en’, this will be of the form en{lower}_{upper}{unit} (en0.2_10.0keV for instance). This is for cases where the product is energy bound. It is also possible for the value to be ‘None’ if the ‘en’ identifier is included in the filename but no energy limits were applied.

  • The type of product - This will always be at the end of the filename, and indicates what the file contains. For instance ‘finalevents’ would represent the final event file for use in an analysis, ‘image’ would be an image, etc.

Note on saving Archives

Archive instances can be saved and loaded back in (as you’ll see in the next section). This can be triggered manually by running the save() method, but this shouldn’t be generally necessary - this is because the archive is automatically saved upon first setup, and after every processing step.

Loading an existing archive

As we have intimated, previously created archives can be loaded back in to memory in exactly the same state as when they were saved. We will demonstrate this here with an archive we prepared earlier - it has had XMM processing applied, which will allow us to demonstrate the logging and management functionality.

Reloading an archive has a number of possible applications:

  • Access to archive data management functions - e.g. locating specific data files, identifying what observations are available.

  • Checking processing logs - e.g. finding errors or warnings in the processing of data that has since been identified as problematic.

  • Updating the archive - either adding another mission, or using the archive to check for new data matching your original mission filtering operations (these are stored in the mission saves, so can be re-run automatically).

All you need to do is set up an Archive instance and pass the name of an existing archive - this assumes your code is running in the same directory as it was originally, as Archives are stored in ‘daxa_output’ (if the DAXA configuration file hasn’t been altered). The configuration can also be altered so that all DAXA outputs are stored in an absolute path, in which case defining an Archive object with the name of an existing dataset would work from any directory).

Loading in an archive (note that you don’t need to pass any missions, loading the archive back in will also reinstate the missions as they were when the Archive was last saved):

[10]:
prev_arch = Archive("PHL1811_made_earlier")
/Users/dt237/code/DAXA/daxa/mission/xmm.py:83: UserWarning: 140 of the 17701 observations located for this mission have been removed due to NaN RA or Dec values
  self._fetch_obs_info()

Once again, we will run the info() method, but note that for this archive the XMM-Newton Pointed mission is marked as ‘fully processed’:

[11]:
prev_arch.info()

-----------------------------------------------------
Number of missions - 4
Total number of observations - 10
Beginning of earliest observation - 1990-10-31 00:00:00
End of latest observation - 2015-11-30 04:11:08.184002

-- XMM-Newton Pointed --
   Internal DAXA name - xmm_pointed
   Chosen instruments - M1, M2, PN
   Number of observations - 5
   Fully Processed - False

-- Chandra --
   Internal DAXA name - chandra
   Chosen instruments - ACIS-I, ACIS-S, HRC-I, HRC-S
   Number of observations - 3
   Fully Processed - False

-- NuSTAR Pointed --
   Internal DAXA name - nustar_pointed
   Chosen instruments - FPMA, FPMB
   Number of observations - 1
   Fully Processed - False

-- RASS --
   Internal DAXA name - rosat_all_sky
   Chosen instruments - PSPC
   Number of observations - 1
   Fully Processed - False
-----------------------------------------------------

We note that it is possible to declare an Archive with a previously used name and overwrite it - you just have to pass clobber=True when you declare the Archive instance. We print the docstring of the Archive class here for reference:

[12]:
print(Archive.__doc__)

    The Archive class, which is to be used to consolidate and provide some interface with a set
    of mission's data. Archives can be passed to processing and cleaning functions in DAXA, and also
    contain convenience functions for accessing summaries of the available data.

    :param str archive_name: The name to be given to this archive - it will be used for storage
        and identification. If an existing archive with this name exists it will be read in, unless clobber=True.
    :param List[BaseMission]/BaseMission missions: The mission, or missions, which are to be included
        in this archive - any setup processes (i.e. the filtering of data to be acquired) should be
        performed prior to creating an archive. The default value is None, but this should be set for any new
        archives, it can only be left as None if an existing archive is being read back in.
    :param bool clobber: If an archive named 'archive_name' already exists, then setting clobber to True
        will cause it to be deleted and overwritten.
    :param bool/dict download_products: Controls whether pre-processed products should be downloaded for missions
        that offer it (assuming downloading was not triggered when the missions were declared). Default is
        True, but False may also be passed, as may a dictionary of DAXA mission names with True/False values.
    :param bool/dict use_preprocessed: Whether pre-processed data products should be used rather than re-processing
        locally with DAXA. If True then what pre-processed data products are available will be automatically
        re-organised into the DAXA processed data structure during the setup of this archive. If False (the default)
        then this will not automatically be applied. Just as with 'download_products', a dictionary may be passed for
        more nuanced control, with mission names as keys and True/False as values.

Accessing component missions

The missions that were used to create an archive can be retrieved, giving access to their information tables - note that you cannot just use the filtering methods of a mission to change the data in the archive; altering the observations in an archive requires using the archive update() method.

To retrieve a mission you can either address the archive with the DAXA internal name of the mission, or get the whole list using the missions property:

[13]:
prev_arch['xmm_pointed']
[13]:
<daxa.mission.xmm.XMMPointed at 0x7f77aafeedf0>
[14]:
prev_arch.missions
[14]:
[<daxa.mission.xmm.XMMPointed at 0x7f77aafeedf0>,
 <daxa.mission.chandra.Chandra at 0x7f77aad6f220>,
 <daxa.mission.nustar.NuSTARPointed at 0x7f7778575fd0>,
 <daxa.mission.rosat.ROSATAllSky at 0x7f77985d8790>]
[15]:
prev_arch['xmm_pointed'].filtered_obs_info
[15]:
ra dec ObsID start science_usable duration proprietary_end_date revolution proprietary_usable end
922 157.746300 31.048890 0102041001 2000-12-07 04:57:14 True 0 days 01:30:06 2002-04-06 00:00:00 182 True 2000-12-07 06:27:20
1044 67.555425 -61.350560 0105261001 2000-09-27 06:56:36 True 0 days 04:16:54 2002-08-25 00:00:00 147 True 2000-09-27 11:13:30
3802 328.756200 -9.373528 0204310101 2004-11-01 09:06:42 True 0 days 09:08:39 2005-12-01 00:00:00 897 True 2004-11-01 18:15:21
6014 234.896250 -83.593060 0502671101 2008-04-01 17:24:48 True 0 days 05:25:20 2009-05-29 00:00:00 1522 True 2008-04-01 22:50:08
12006 328.756250 -9.373333 0761910201 2015-11-29 09:38:07 True 0 days 16:30:00 2016-12-11 23:00:00 2925 True 2015-11-30 02:08:07

Archive general properties

Here we run through the general properties of the archive class, summarising their meaning and content.

Name

The archive_name class returns the name that was given to the archive on creation - this cannot be changed.

[16]:
prev_arch.archive_name
[16]:
'PHL1811_made_earlier'

Archive Path

This property (archive_path) returns the absolute path to the top level of this archive’s storage directory:

[17]:
prev_arch.archive_path
[17]:
'/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/'

Mission Names

In addition to the missions property discussed earlier, we include a mission_names property which lists the internal names of the mission classes associated with the archive:

[18]:
prev_arch.mission_names
[18]:
['xmm_pointed', 'chandra', 'nustar_pointed', 'rosat_all_sky']

Data management Archive functions

This section introduces the built-in methods that allow you to manage the archive and its data:

Get path to data

One of the most useful methods of the archive class is get_current_data_path(), which allows you to programmatically retrieve the current path to a particular ObsID’s data in the archive storage structure (this takes into account the final_process_success flag - remember that entirely failed data are moved to a ‘failed_data’ directory.

All we need to do is pass the mission name and the ObsID, and the top-level data path will be returned. Here we show two examples, one for an ObsID with a True final success flag, and one with a False final success flag:

[28]:
prev_arch.get_current_data_path('xmm_pointed', '0204310101')
[28]:
'/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0204310101/'
[29]:
prev_arch.get_current_data_path('xmm_pointed', '0102041001')
[29]:
'/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/failed_data/xmm_pointed/0102041001/'

Get failed processes

Often we will wish to know exactly which data failed a particular processing step - this could be inferred from the process_success property, but we also provide a convenient get method (get_failed_processes()) that will retrieve the unique identifer of each piece of data that failed a specified processing step:

[30]:
prev_arch.get_failed_processes('espfilt')
[30]:
{'xmm_pointed': ['0502671101PNS003']}

Get logs

The get_process_logs() method provides a more convenient way of accessing specific logs stored in the process_logs property. It allows for the retrieval of logs based on several criteria, with the only requirement being the passing of a process name.

Beyond that you can specify the mission name, ObsID, and instrument to retrieve logs for - as for some processes there are sub-exposures of a given instrument, giving this information can still result in multiple logs being returned. You can also pass lists of mission name, ObsID, and instrument and retrieve sets of logs that way.

It is also possible to specify an exact unique identifier (0502671101M2S002 for instance).

We demonstrate fetching the espfilt logs for a single PN instrument of 0502671101:

[31]:
prev_arch.get_process_logs('espfilt', obs_id='0502671101', inst='PN')
[31]:
{'xmm_pointed': {'0502671101PNS003': "espfilt:- Executing (routine): espfilt eventfile=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits withoot=yes ootfile=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-ootevents.fits method=histogram withsmoothing=yes smooth=51 withbinning=yes binsize=60 ratio=1.2 withlongnames=yes elow=2500 ehigh=8500 rangescale=15 allowsigma=3 limits='0.1 6.5' keepinterfiles=no  -w 1 -V 4\nespfilt:- espfilt (espfilt-4.3)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:07.000\nespfilt:-  ESPFILT: Processing eventlist: /Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits\nespfilt:-  *FOV IMAGE* = pnS003-fovim-2500-8500.fits\nevselect:- Executing (routine): evselect table=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits filteredset=filtered.fits withfilteredset=no keepfilteroutput=no flagcolumn=EVFLAG flagbit=-1 destruct=yes dssblock='' expression=true filtertype=expression cleandss=no updateexposure=yes filterexposure=yes writedss=yes blockstocopy='' attributestocopy='' energycolumn=PHA zcolumn=WEIGHT zerrorcolumn=EWEIGHT withzerrorcolumn=no withzcolumn=no ignorelegallimits=no imageset=pnS003-fovim-2500-8500.fits xcolumn=DETX ycolumn=DETY ximagebinsize=1 yimagebinsize=1 squarepixels=no ximagesize=600 yimagesize=600 imagebinning=imageSize ximagemin=1 ximagemax=640 withxranges=no yimagemin=1 yimagemax=640 withyranges=no imagedatatype=Real64 withimagedatatype=no raimagecenter=0 decimagecenter=0 withcelestialcenter=no withimageset=yes spectrumset=spectrum.fits spectralbinsize=5 specchannelmin=0 specchannelmax=11999 withspecranges=no nonStandardSpec=no withspectrumset=no rateset=rate.fits timecolumn=TIME timebinsize=1 timemin=0 timemax=1000 withtimeranges=no maketimecolumn=no makeratecolumn=no withrateset=no histogramset=histo.fits histogramcolumn=TIME histogrambinsize=1 histogrammin=0 histogrammax=1000 withhistoranges=no withhistogramset=no  -w 1 -V 4\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:08.000\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] ended:    2024-04-24T19:20:09.000\nespfilt:-  Running evselect to create *FOV LIGHTCURVE*\nespfilt:-  *FOV LIGHTCURVE * = pnS003-fovlc-2500-8500.fits\nevselect:- Executing (routine): evselect table=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits:EVENTS filteredset=filtered.fits withfilteredset=yes keepfilteroutput=no flagcolumn=EVFLAG flagbit=-1 destruct=yes dssblock='' expression='(PATTERN<=4)&&(PI in [2500:8500])&&((FLAG & 0xfb0000)==0)' filtertype=expression cleandss=no updateexposure=yes filterexposure=yes writedss=yes blockstocopy='' attributestocopy='' energycolumn=PHA zcolumn=WEIGHT zerrorcolumn=EWEIGHT withzerrorcolumn=no withzcolumn=no ignorelegallimits=no imageset=image.fits xcolumn=RAWX ycolumn=RAWY ximagebinsize=1 yimagebinsize=1 squarepixels=no ximagesize=600 yimagesize=600 imagebinning=imageSize ximagemin=1 ximagemax=640 withxranges=no yimagemin=1 yimagemax=640 withyranges=no imagedatatype=Real64 withimagedatatype=no raimagecenter=0 decimagecenter=0 withcelestialcenter=no withimageset=no spectrumset=spectrum.fits spectralbinsize=5 specchannelmin=0 specchannelmax=11999 withspecranges=no nonStandardSpec=no withspectrumset=no rateset=pnS003-fovlc-2500-8500.fits timecolumn=TIME timebinsize=1 timemin=0 timemax=1000 withtimeranges=no maketimecolumn=yes makeratecolumn=yes withrateset=yes histogramset=histo.fits histogramcolumn=TIME histogrambinsize=1 histogrammin=0 histogrammax=1000 withhistoranges=no withhistogramset=no  -w 1 -V 4\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:09.000\nevselect:- selected 883492 rows from the input table.\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] ended:    2024-04-24T19:20:12.000\nespfilt:-  Running evselect to create *CORNER EVENTLIST*\nespfilt:-  *CORNER EVENTLIST * = pnS003-corev-2500-8500.fits\nevselect:- Executing (routine): evselect table=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits:EVENTS filteredset=pnS003-corev-2500-8500.fits withfilteredset=yes keepfilteroutput=yes flagcolumn=EVFLAG flagbit=-1 destruct=yes dssblock='' expression='(PATTERN<=4)&&(#XMMEA_EP)&&!((DETX,DETY) in circle(-2200,-1100,18080))' filtertype=expression cleandss=no updateexposure=yes filterexposure=yes writedss=yes blockstocopy='' attributestocopy='' energycolumn=PHA zcolumn=WEIGHT zerrorcolumn=EWEIGHT withzerrorcolumn=no withzcolumn=no ignorelegallimits=no imageset=image.fits xcolumn=RAWX ycolumn=RAWY ximagebinsize=1 yimagebinsize=1 squarepixels=no ximagesize=600 yimagesize=600 imagebinning=imageSize ximagemin=1 ximagemax=640 withxranges=no yimagemin=1 yimagemax=640 withyranges=no imagedatatype=Real64 withimagedatatype=no raimagecenter=0 decimagecenter=0 withcelestialcenter=no withimageset=no spectrumset=spectrum.fits spectralbinsize=5 specchannelmin=0 specchannelmax=11999 withspecranges=no nonStandardSpec=no withspectrumset=no rateset=rate.fits timecolumn=TIME timebinsize=1 timemin=0 timemax=1000 withtimeranges=no maketimecolumn=no makeratecolumn=no withrateset=no histogramset=histo.fits histogramcolumn=TIME histogrambinsize=1 histogrammin=0 histogrammax=1000 withhistoranges=no withhistogramset=no  -w 1 -V 4\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:13.000\nevselect:- selected 115052 rows from the input table.\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] ended:    2024-04-24T19:20:15.000\nespfilt:-  Running evselect to create *CORNER IMAGE*\nespfilt:-  *CORNER IMAGE * = pnS003-corim-2500-8500.fits\nevselect:- Executing (routine): evselect table=pnS003-corev-2500-8500.fits filteredset=filtered.fits withfilteredset=no keepfilteroutput=no flagcolumn=EVFLAG flagbit=-1 destruct=yes dssblock='' expression=true filtertype=expression cleandss=no updateexposure=yes filterexposure=yes writedss=yes blockstocopy='' attributestocopy='' energycolumn=PHA zcolumn=WEIGHT zerrorcolumn=EWEIGHT withzerrorcolumn=no withzcolumn=no ignorelegallimits=no imageset=pnS003-corim-2500-8500.fits xcolumn=DETX ycolumn=DETY ximagebinsize=1 yimagebinsize=1 squarepixels=no ximagesize=600 yimagesize=600 imagebinning=imageSize ximagemin=1 ximagemax=640 withxranges=no yimagemin=1 yimagemax=640 withyranges=no imagedatatype=Real64 withimagedatatype=no raimagecenter=0 decimagecenter=0 withcelestialcenter=no withimageset=yes spectrumset=spectrum.fits spectralbinsize=5 specchannelmin=0 specchannelmax=11999 withspecranges=no nonStandardSpec=no withspectrumset=no rateset=rate.fits timecolumn=TIME timebinsize=1 timemin=0 timemax=1000 withtimeranges=no maketimecolumn=no makeratecolumn=no withrateset=no histogramset=histo.fits histogramcolumn=TIME histogrambinsize=1 histogrammin=0 histogrammax=1000 withhistoranges=no withhistogramset=no  -w 1 -V 4\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:15.000\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] ended:    2024-04-24T19:20:16.000\nespfilt:-  Running evselect to create *CORNER LIGHTCURVE*\nespfilt:-  *CORNER LIGHTCURVE * = pnS003-corlc-2500-8500.fits\nevselect:- Executing (routine): evselect table=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits:EVENTS filteredset=filtered.fits withfilteredset=yes keepfilteroutput=no flagcolumn=EVFLAG flagbit=-1 destruct=yes dssblock='' expression='(PATTERN<=4)&&(PI in [2500:8500])&&(#XMMEA_EP)&&!((DETX,DETY) in circle(-2200,-1100,18080))' filtertype=expression cleandss=no updateexposure=yes filterexposure=yes writedss=yes blockstocopy='' attributestocopy='' energycolumn=PHA zcolumn=WEIGHT zerrorcolumn=EWEIGHT withzerrorcolumn=no withzcolumn=no ignorelegallimits=no imageset=image.fits xcolumn=RAWX ycolumn=RAWY ximagebinsize=1 yimagebinsize=1 squarepixels=no ximagesize=600 yimagesize=600 imagebinning=imageSize ximagemin=1 ximagemax=640 withxranges=no yimagemin=1 yimagemax=640 withyranges=no imagedatatype=Real64 withimagedatatype=no raimagecenter=0 decimagecenter=0 withcelestialcenter=no withimageset=no spectrumset=spectrum.fits spectralbinsize=5 specchannelmin=0 specchannelmax=11999 withspecranges=no nonStandardSpec=no withspectrumset=no rateset=pnS003-corlc-2500-8500.fits timecolumn=TIME timebinsize=1 timemin=0 timemax=1000 withtimeranges=no maketimecolumn=yes makeratecolumn=yes withrateset=yes histogramset=histo.fits histogramcolumn=TIME histogrambinsize=1 histogrammin=0 histogrammax=1000 withhistoranges=no withhistogramset=no  -w 1 -V 4\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:16.000\nevselect:- selected 43048 rows from the input table.\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] ended:    2024-04-24T19:20:18.000\nespfilt:-  ESPFILT: Processing using the HISTOGRAM method.\nespfilt:-  CLEAN_LC: There are        17659  rows in the FOV LC\nespfilt:-  Starting Multidimensional minimzation of fit_gauss (amoeba)\nespfilt:- plim(1:2) =   0.1000   6.5000\nespfilt:- **** sortarray max bins    =   663 histo binwidth =    0.0196\nespfilt:- **** sortarray actual bins (nverls) =   658\nespfilt:- min LC value=    0.000 max LC value =  144.255 limits(2) =    6.500\n"}}

Get raw errors

There is also an equivalent method, get_process_raw_error_logs() that can fetch raw error logs. It behaves exactly the same as get_process_logs(), see the last section for information on the possible arguments:

[32]:
prev_arch.get_process_raw_error_logs('espfilt', obs_id='0502671101', inst='PN')
[32]:
{'xmm_pointed': {'0502671101PNS003': '** evselect: warning (NoWCS), No WCS information available for image column DETX.\n** evselect: warning (SummaryOfWarnings), \n   warning NoWCS silently occurred 1 times\n** evselect: warning (NoWCS), No WCS information available for image column DETX.\n** evselect: warning (SummaryOfWarnings), \n   warning NoWCS silently occurred 1 times\n** espfilt: error (noCounts), All histo counts are zero! Check your FOV Lightcurve!\nmv: rename pnS003-gti-2500-8500.fits to /Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/cleaning/obsid0502671101-instPN-subexpS003-en2500_8500keV-gti.fits: No such file or directory\nmv: rename pnS003-allevc-2500-8500.fits to /Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/cleaning/PNS003-allevc-2500-8500.fits: No such file or directory\nmv: rename pnS003-hist-2500-8500.qdp to /Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/cleaning/obsid0502671101-instPN-subexpS003-en2500_8500keV-hist.qdp: No such file or directory\n'}}

Get failed logs

There is one final method that allows for log retrieval - get_failed_logs(). This only takes a process name as an argument, and will retrieve the logs and raw error logs for every piece of data that failed the specified processing step. It returns them as a tuple, with the first entry being the dictionary of logs and the second being the dictionary of raw errors:

[33]:
logs, errors = prev_arch.get_failed_logs('espfilt')

We only show 0502671101M2S002:

[34]:
logs['xmm_pointed']['0502671101PNS003']
[34]:
"espfilt:- Executing (routine): espfilt eventfile=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits withoot=yes ootfile=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-ootevents.fits method=histogram withsmoothing=yes smooth=51 withbinning=yes binsize=60 ratio=1.2 withlongnames=yes elow=2500 ehigh=8500 rangescale=15 allowsigma=3 limits='0.1 6.5' keepinterfiles=no  -w 1 -V 4\nespfilt:- espfilt (espfilt-4.3)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:07.000\nespfilt:-  ESPFILT: Processing eventlist: /Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits\nespfilt:-  *FOV IMAGE* = pnS003-fovim-2500-8500.fits\nevselect:- Executing (routine): evselect table=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits filteredset=filtered.fits withfilteredset=no keepfilteroutput=no flagcolumn=EVFLAG flagbit=-1 destruct=yes dssblock='' expression=true filtertype=expression cleandss=no updateexposure=yes filterexposure=yes writedss=yes blockstocopy='' attributestocopy='' energycolumn=PHA zcolumn=WEIGHT zerrorcolumn=EWEIGHT withzerrorcolumn=no withzcolumn=no ignorelegallimits=no imageset=pnS003-fovim-2500-8500.fits xcolumn=DETX ycolumn=DETY ximagebinsize=1 yimagebinsize=1 squarepixels=no ximagesize=600 yimagesize=600 imagebinning=imageSize ximagemin=1 ximagemax=640 withxranges=no yimagemin=1 yimagemax=640 withyranges=no imagedatatype=Real64 withimagedatatype=no raimagecenter=0 decimagecenter=0 withcelestialcenter=no withimageset=yes spectrumset=spectrum.fits spectralbinsize=5 specchannelmin=0 specchannelmax=11999 withspecranges=no nonStandardSpec=no withspectrumset=no rateset=rate.fits timecolumn=TIME timebinsize=1 timemin=0 timemax=1000 withtimeranges=no maketimecolumn=no makeratecolumn=no withrateset=no histogramset=histo.fits histogramcolumn=TIME histogrambinsize=1 histogrammin=0 histogrammax=1000 withhistoranges=no withhistogramset=no  -w 1 -V 4\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:08.000\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] ended:    2024-04-24T19:20:09.000\nespfilt:-  Running evselect to create *FOV LIGHTCURVE*\nespfilt:-  *FOV LIGHTCURVE * = pnS003-fovlc-2500-8500.fits\nevselect:- Executing (routine): evselect table=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits:EVENTS filteredset=filtered.fits withfilteredset=yes keepfilteroutput=no flagcolumn=EVFLAG flagbit=-1 destruct=yes dssblock='' expression='(PATTERN<=4)&&(PI in [2500:8500])&&((FLAG & 0xfb0000)==0)' filtertype=expression cleandss=no updateexposure=yes filterexposure=yes writedss=yes blockstocopy='' attributestocopy='' energycolumn=PHA zcolumn=WEIGHT zerrorcolumn=EWEIGHT withzerrorcolumn=no withzcolumn=no ignorelegallimits=no imageset=image.fits xcolumn=RAWX ycolumn=RAWY ximagebinsize=1 yimagebinsize=1 squarepixels=no ximagesize=600 yimagesize=600 imagebinning=imageSize ximagemin=1 ximagemax=640 withxranges=no yimagemin=1 yimagemax=640 withyranges=no imagedatatype=Real64 withimagedatatype=no raimagecenter=0 decimagecenter=0 withcelestialcenter=no withimageset=no spectrumset=spectrum.fits spectralbinsize=5 specchannelmin=0 specchannelmax=11999 withspecranges=no nonStandardSpec=no withspectrumset=no rateset=pnS003-fovlc-2500-8500.fits timecolumn=TIME timebinsize=1 timemin=0 timemax=1000 withtimeranges=no maketimecolumn=yes makeratecolumn=yes withrateset=yes histogramset=histo.fits histogramcolumn=TIME histogrambinsize=1 histogrammin=0 histogrammax=1000 withhistoranges=no withhistogramset=no  -w 1 -V 4\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:09.000\nevselect:- selected 883492 rows from the input table.\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] ended:    2024-04-24T19:20:12.000\nespfilt:-  Running evselect to create *CORNER EVENTLIST*\nespfilt:-  *CORNER EVENTLIST * = pnS003-corev-2500-8500.fits\nevselect:- Executing (routine): evselect table=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits:EVENTS filteredset=pnS003-corev-2500-8500.fits withfilteredset=yes keepfilteroutput=yes flagcolumn=EVFLAG flagbit=-1 destruct=yes dssblock='' expression='(PATTERN<=4)&&(#XMMEA_EP)&&!((DETX,DETY) in circle(-2200,-1100,18080))' filtertype=expression cleandss=no updateexposure=yes filterexposure=yes writedss=yes blockstocopy='' attributestocopy='' energycolumn=PHA zcolumn=WEIGHT zerrorcolumn=EWEIGHT withzerrorcolumn=no withzcolumn=no ignorelegallimits=no imageset=image.fits xcolumn=RAWX ycolumn=RAWY ximagebinsize=1 yimagebinsize=1 squarepixels=no ximagesize=600 yimagesize=600 imagebinning=imageSize ximagemin=1 ximagemax=640 withxranges=no yimagemin=1 yimagemax=640 withyranges=no imagedatatype=Real64 withimagedatatype=no raimagecenter=0 decimagecenter=0 withcelestialcenter=no withimageset=no spectrumset=spectrum.fits spectralbinsize=5 specchannelmin=0 specchannelmax=11999 withspecranges=no nonStandardSpec=no withspectrumset=no rateset=rate.fits timecolumn=TIME timebinsize=1 timemin=0 timemax=1000 withtimeranges=no maketimecolumn=no makeratecolumn=no withrateset=no histogramset=histo.fits histogramcolumn=TIME histogrambinsize=1 histogrammin=0 histogrammax=1000 withhistoranges=no withhistogramset=no  -w 1 -V 4\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:13.000\nevselect:- selected 115052 rows from the input table.\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] ended:    2024-04-24T19:20:15.000\nespfilt:-  Running evselect to create *CORNER IMAGE*\nespfilt:-  *CORNER IMAGE * = pnS003-corim-2500-8500.fits\nevselect:- Executing (routine): evselect table=pnS003-corev-2500-8500.fits filteredset=filtered.fits withfilteredset=no keepfilteroutput=no flagcolumn=EVFLAG flagbit=-1 destruct=yes dssblock='' expression=true filtertype=expression cleandss=no updateexposure=yes filterexposure=yes writedss=yes blockstocopy='' attributestocopy='' energycolumn=PHA zcolumn=WEIGHT zerrorcolumn=EWEIGHT withzerrorcolumn=no withzcolumn=no ignorelegallimits=no imageset=pnS003-corim-2500-8500.fits xcolumn=DETX ycolumn=DETY ximagebinsize=1 yimagebinsize=1 squarepixels=no ximagesize=600 yimagesize=600 imagebinning=imageSize ximagemin=1 ximagemax=640 withxranges=no yimagemin=1 yimagemax=640 withyranges=no imagedatatype=Real64 withimagedatatype=no raimagecenter=0 decimagecenter=0 withcelestialcenter=no withimageset=yes spectrumset=spectrum.fits spectralbinsize=5 specchannelmin=0 specchannelmax=11999 withspecranges=no nonStandardSpec=no withspectrumset=no rateset=rate.fits timecolumn=TIME timebinsize=1 timemin=0 timemax=1000 withtimeranges=no maketimecolumn=no makeratecolumn=no withrateset=no histogramset=histo.fits histogramcolumn=TIME histogrambinsize=1 histogrammin=0 histogrammax=1000 withhistoranges=no withhistogramset=no  -w 1 -V 4\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:15.000\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] ended:    2024-04-24T19:20:16.000\nespfilt:-  Running evselect to create *CORNER LIGHTCURVE*\nespfilt:-  *CORNER LIGHTCURVE * = pnS003-corlc-2500-8500.fits\nevselect:- Executing (routine): evselect table=/Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/events/obsid0502671101-instPN-subexpS003-events.fits:EVENTS filteredset=filtered.fits withfilteredset=yes keepfilteroutput=no flagcolumn=EVFLAG flagbit=-1 destruct=yes dssblock='' expression='(PATTERN<=4)&&(PI in [2500:8500])&&(#XMMEA_EP)&&!((DETX,DETY) in circle(-2200,-1100,18080))' filtertype=expression cleandss=no updateexposure=yes filterexposure=yes writedss=yes blockstocopy='' attributestocopy='' energycolumn=PHA zcolumn=WEIGHT zerrorcolumn=EWEIGHT withzerrorcolumn=no withzcolumn=no ignorelegallimits=no imageset=image.fits xcolumn=RAWX ycolumn=RAWY ximagebinsize=1 yimagebinsize=1 squarepixels=no ximagesize=600 yimagesize=600 imagebinning=imageSize ximagemin=1 ximagemax=640 withxranges=no yimagemin=1 yimagemax=640 withyranges=no imagedatatype=Real64 withimagedatatype=no raimagecenter=0 decimagecenter=0 withcelestialcenter=no withimageset=no spectrumset=spectrum.fits spectralbinsize=5 specchannelmin=0 specchannelmax=11999 withspecranges=no nonStandardSpec=no withspectrumset=no rateset=pnS003-corlc-2500-8500.fits timecolumn=TIME timebinsize=1 timemin=0 timemax=1000 withtimeranges=no maketimecolumn=yes makeratecolumn=yes withrateset=yes histogramset=histo.fits histogramcolumn=TIME histogrambinsize=1 histogrammin=0 histogrammax=1000 withhistoranges=no withhistogramset=no  -w 1 -V 4\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] started:  2024-04-24T19:20:16.000\nevselect:- selected 43048 rows from the input table.\nevselect:- evselect (evselect-3.71.1)  [xmmsas_20211130_0941-20.0.0] ended:    2024-04-24T19:20:18.000\nespfilt:-  ESPFILT: Processing using the HISTOGRAM method.\nespfilt:-  CLEAN_LC: There are        17659  rows in the FOV LC\nespfilt:-  Starting Multidimensional minimzation of fit_gauss (amoeba)\nespfilt:- plim(1:2) =   0.1000   6.5000\nespfilt:- **** sortarray max bins    =   663 histo binwidth =    0.0196\nespfilt:- **** sortarray actual bins (nverls) =   658\nespfilt:- min LC value=    0.000 max LC value =  144.255 limits(2) =    6.500\n"
[35]:
errors
[35]:
{'xmm_pointed': {'0502671101PNS003': '** evselect: warning (NoWCS), No WCS information available for image column DETX.\n** evselect: warning (SummaryOfWarnings), \n   warning NoWCS silently occurred 1 times\n** evselect: warning (NoWCS), No WCS information available for image column DETX.\n** evselect: warning (SummaryOfWarnings), \n   warning NoWCS silently occurred 1 times\n** espfilt: error (noCounts), All histo counts are zero! Check your FOV Lightcurve!\nmv: rename pnS003-gti-2500-8500.fits to /Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/cleaning/obsid0502671101-instPN-subexpS003-en2500_8500keV-gti.fits: No such file or directory\nmv: rename pnS003-allevc-2500-8500.fits to /Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/cleaning/PNS003-allevc-2500-8500.fits: No such file or directory\nmv: rename pnS003-hist-2500-8500.qdp to /Users/dt237/code/DAXA/docs/source/notebooks/tutorials/daxa_output/archives/PHL1811_made_earlier/processed_data/xmm_pointed/0502671101/cleaning/obsid0502671101-instPN-subexpS003-en2500_8500keV-hist.qdp: No such file or directory\n'}}

Adding region files to the Archive (optional)

Region files are created by running source detection algorithms on images generated from X-ray observations, and DAXA does not yet have the ability to generate them itself. However, they are a crucial part of many analyses, and as such a crucial part of data archives.

If you have created region files (in the DS9 format), you can add them to the archive so that they are included in the storage structure. This process verifies that the passed region files are in RA-Dec coordinates (rather than defined in the pixel coordinates of the image they were detected in), by requiring that images or WCS information be passed in those circumstances.

  • {‘mission_name’: {‘ObsID’: ‘path to regions’}}

  • {‘mission_name’: {‘ObsID’: [list of region objects]}}

  • {‘mission_name’: {‘ObsID’: {‘region’: ‘path to regions’}}}

  • {‘mission_name’: {‘ObsID’: {‘region’: [list of region objects]}}}

  • {‘mission_name’: {‘ObsID’: {‘region’: …, ‘wcs_src’: ‘path to image’}}}

  • {‘mission_name’: {‘ObsID’: {‘region’: …, ‘wcs_src’: XGA Image}}}

  • {‘mission_name’: {‘ObsID’: {‘region’: …, ‘wcs_src’: Astropy WCS object}}}

For instance, we have used a tool to generate source regions for the observations we have processed, and want to add those regions to the archive. As the region files are in pixel coordinates we pass the paths to the images we used (unnecessary if regions are already in RA-Dec coordinates):

[37]:
# Listing the region files in the test directory
reg_files = os.listdir('region_files')

# Setting up the structure of the dictionary we will pass to the archive at the end of this
reg_paths = {'xmm_pointed': {}}
# Iterating through the ObsIDs in the XMMPointed mission
for oi in prev_arch['xmm_pointed'].filtered_obs_ids:
    # Checking to see which have a corresponding region file
    if any([oi in rf for rf in reg_files]):
        # Generating the path to the image we need for pixel to RA-Dec conversion
        im_pth = prev_arch.get_current_data_path('xmm_pointed', oi) + \
        'images/obsid{}-instM2-subexpALL-en0.5_2.0keV-image.fits'.format(oi)
        # Setting up the entry in the final dictionary, with the path to the regions and the image
        reg_paths['xmm_pointed'][oi] = {'regions': 'region_files/{}.reg'.format(oi), 'wcs_src': im_pth}

# Adding the regions to the archive
prev_arch.source_regions = reg_paths
/var/folders/td/gw9qkx6d3szb1nkt_cfvcbzm000vzl/T/ipykernel_30819/3922313599.py:17: UserWarning: The source_regions property already had an entry for 0105261001 under xmm_pointed, this has been overwritten!
  prev_arch.source_regions = reg_paths
/var/folders/td/gw9qkx6d3szb1nkt_cfvcbzm000vzl/T/ipykernel_30819/3922313599.py:17: UserWarning: The source_regions property already had an entry for 0204310101 under xmm_pointed, this has been overwritten!
  prev_arch.source_regions = reg_paths
/var/folders/td/gw9qkx6d3szb1nkt_cfvcbzm000vzl/T/ipykernel_30819/3922313599.py:17: UserWarning: The source_regions property already had an entry for 0502671101 under xmm_pointed, this has been overwritten!
  prev_arch.source_regions = reg_paths
/var/folders/td/gw9qkx6d3szb1nkt_cfvcbzm000vzl/T/ipykernel_30819/3922313599.py:17: UserWarning: The source_regions property already had an entry for 0761910201 under xmm_pointed, this has been overwritten!
  prev_arch.source_regions = reg_paths

Once added to the archive, you can also retrieve the regions through a property (we did not discuss it earlier in this tutorial) - source_regions. It returns them as Python ‘regions’ module objects:

[38]:
prev_arch.source_regions
[38]:
{'xmm_pointed': {'0105261001': [<EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.59133516, -61.53306334)>, width=0.007545166292795352 deg, height=0.007545166292795352 deg, angle=186.39335083127384 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.178534, -61.37451667)>, width=0.008274114359963867 deg, height=0.008274114359963867 deg, angle=466.9975179351541 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.46879016, -61.36577221)>, width=0.00850673285572214 deg, height=0.00850673285572214 deg, angle=140.94986422162475 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (68.01317832, -61.35516589)>, width=0.007519977428013009 deg, height=0.007519977428013009 deg, angle=261.2582496754984 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (68.00486199, -61.33624502)>, width=0.013147992908181612 deg, height=0.013147992908181612 deg, angle=265.95154602021546 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.41445681, -61.28706975)>, width=0.0075120802732052015 deg, height=0.0075120802732052015 deg, angle=419.76253296921374 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.52454168, -61.26549895)>, width=0.007888040300578339 deg, height=0.007888040300578339 deg, angle=368.13995128822404 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.38459977, -61.24728435)>, width=0.007642156673586847 deg, height=0.007642156673586847 deg, angle=405.1377277122432 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.66282973, -61.13583214)>, width=0.007484864530890935 deg, height=0.007484864530890935 deg, angle=342.51362589811765 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.67081204, -61.41366132)>, width=0.007664217192863797 deg, height=0.007664217192863797 deg, angle=214.13162270745238 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.44099122, -61.2872128)>, width=0.008544191122124041 deg, height=0.008544191122124041 deg, angle=413.6812532986161 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.47859205, -61.17991392)>, width=0.008220698389583923 deg, height=0.008220698389583923 deg, angle=372.0472646434019 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.62366961, -61.57349813)>, width=0.007591811484076978 deg, height=0.007591811484076978 deg, angle=188.7899825817821 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.46197943, -61.31728401)>, width=0.009600351102606488 deg, height=0.009600351102606488 deg, angle=441.75783953763363 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.109084, -61.3151228)>, width=0.007726056638463273 deg, height=0.007726056638463273 deg, angle=448.4264497362917 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.19833167, -61.30776102)>, width=0.00700383303154554 deg, height=0.00700383303154554 deg, angle=445.20514853614804 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.30056555, -61.23205705)>, width=0.007642141740875588 deg, height=0.007642141740875588 deg, angle=412.2134543327273 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.63170002, -61.15814151)>, width=0.009628376655476337 deg, height=0.009628376655476337 deg, angle=345.1369371937843 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.63227828, -61.15627305)>, width=0.009228444139956988 deg, height=0.009228444139956988 deg, angle=345.2044298700188 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.67485433, -61.37499077)>, width=0.0027905209722769216 deg, height=0.0016111095912574162 deg, angle=44.883076244857286 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.69462508, -61.33509251)>, width=0.03179339749398203 deg, height=0.02240286728485413 deg, angle=44.63583059287267 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.92311013, -61.3253819)>, width=0.009572126121424652 deg, height=0.0028367896810407556 deg, angle=-257.05172906999195 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.57937638, -61.39856243)>, width=0.06787196199238821 deg, height=0.02245098652897439 deg, angle=-19.414157271481155 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.85008628, -61.42796549)>, width=0.1397191347246724 deg, height=0.04780714778773332 deg, angle=-237.4726619426717 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (67.42445028, -61.17772595)>, width=0.030515763924281118 deg, height=0.01992090470829036 deg, angle=4.480105303940038 deg)>],
  '0204310101': [<EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.68906505, -9.57605556)>, width=0.007494624973566207 deg, height=0.007494624973566207 deg, angle=514.3266958726614 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.68959451, -9.56149785)>, width=0.006759131697903771 deg, height=0.006759131697903771 deg, angle=512.5796094746042 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.88756277, -9.52098468)>, width=0.007452465671294455 deg, height=0.007452465671294455 deg, angle=579.5203052526585 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.80541496, -9.51795062)>, width=0.008375859232780183 deg, height=0.008375859232780183 deg, angle=551.5787100398195 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.72559081, -9.51892627)>, width=0.007444719230222005 deg, height=0.007444719230222005 deg, angle=517.5127356369848 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.55371487, -9.49046251)>, width=0.0076420963731746 deg, height=0.0076420963731746 deg, angle=474.0587823060275 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.56596717, -9.4880535)>, width=0.007737029310810679 deg, height=0.007737029310810679 deg, angle=474.7287829609348 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.82139591, -9.47361547)>, width=0.007737085424898903 deg, height=0.007737085424898903 deg, angle=566.8961022892662 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.67183058, -9.44703777)>, width=0.0071365428053444206 deg, height=0.0071365428053444206 deg, angle=477.7355081764938 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.74299637, -9.44486753)>, width=0.007846231696543896 deg, height=0.007846231696543896 deg, angle=506.2988954235226 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.76086754, -9.41124584)>, width=0.008050640320955097 deg, height=0.008050640320955097 deg, angle=497.7129792302257 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.8464958, -9.40473667)>, width=0.007737097719562047 deg, height=0.007737097719562047 deg, angle=258.67400652493257 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.80730171, -9.39688777)>, width=0.007737102356468667 deg, height=0.007737102356468667 deg, angle=258.914402158858 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.70381145, -9.38479759)>, width=0.00724999380145057 deg, height=0.00724999380145057 deg, angle=444.90628957212795 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.83026421, -9.38399593)>, width=0.007865255978998582 deg, height=0.007865255978998582 deg, angle=278.38565516990224 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.8120376, -9.38092408)>, width=0.007338448567576843 deg, height=0.007338448567576843 deg, angle=287.9436293684878 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.67039651, -9.37572647)>, width=0.007646681844692583 deg, height=0.007646681844692583 deg, angle=441.6907421859443 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.75681337, -9.37342961)>, width=0.007919729453573972 deg, height=0.007919729453573972 deg, angle=410.70964078256986 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.70810058, -9.37211098)>, width=0.00966665832004527 deg, height=0.00966665832004527 deg, angle=434.593589673473 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.72034748, -9.37150905)>, width=0.009437367340938366 deg, height=0.009437367340938366 deg, angle=431.04359123537307 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.89561695, -9.37075174)>, width=0.00773387258312214 deg, height=0.00773387258312214 deg, angle=280.1779909796026 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.67993575, -9.36243761)>, width=0.00854418282413368 deg, height=0.00854418282413368 deg, angle=433.5141855894659 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.89314428, -9.3557064)>, width=0.00828300911798855 deg, height=0.00828300911798855 deg, angle=287.59639446126425 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.57318109, -9.35301779)>, width=0.007552153247472566 deg, height=0.007552153247472566 deg, angle=439.34160578877743 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.78984701, -9.34039916)>, width=0.00908259926340609 deg, height=0.00908259926340609 deg, angle=348.28825039618556 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.80239825, -9.33163815)>, width=0.007737094243088075 deg, height=0.007737094243088075 deg, angle=338.9892023029214 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.57572829, -9.31540772)>, width=0.0076731460209169145 deg, height=0.0076731460209169145 deg, angle=429.2991901378363 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.83465169, -9.31090206)>, width=0.007815190929166629 deg, height=0.007815190929166629 deg, angle=325.7609392262642 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.62179047, -9.30706564)>, width=0.00744031732502786 deg, height=0.00744031732502786 deg, angle=421.5136736975769 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.93982357, -9.30419317)>, width=0.006817785524070047 deg, height=0.006817785524070047 deg, angle=298.78277735769643 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.5824604, -9.2981542)>, width=0.007498418709017829 deg, height=0.007498418709017829 deg, angle=424.3720915847924 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.61992022, -9.27890238)>, width=0.007409588481195356 deg, height=0.007409588481195356 deg, angle=414.4219788807195 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.73873297, -9.24004532)>, width=0.006809609883438864 deg, height=0.006809609883438864 deg, angle=374.75959111728724 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.84875786, -9.2334555)>, width=0.007624181442133438 deg, height=0.007624181442133438 deg, angle=336.4766373980855 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.70002037, -9.22635466)>, width=0.007312596562708717 deg, height=0.007312596562708717 deg, angle=385.32690819492217 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.73053419, -9.21549118)>, width=0.007092895516765346 deg, height=0.007092895516765346 deg, angle=375.2559289849012 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.66945038, -9.21538118)>, width=0.007507169766322179 deg, height=0.007507169766322179 deg, angle=391.60689657572266 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.92113086, -9.51503501)>, width=0.00920233061969241 deg, height=0.00920233061969241 deg, angle=588.5428276309194 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.76718805, -9.46938864)>, width=0.008479874183753903 deg, height=0.008479874183753903 deg, angle=531.4119478557901 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.97067115, -9.44025128)>, width=0.007134318556827573 deg, height=0.007134318556827573 deg, angle=255.47074364670326 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.60991084, -9.35741364)>, width=0.007591343401669003 deg, height=0.007591343401669003 deg, angle=438.52061221514987 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.58434174, -9.35144598)>, width=0.007715982802198556 deg, height=0.007715982802198556 deg, angle=438.29078458752 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.75127407, -9.32831492)>, width=0.007877356155183543 deg, height=0.007877356155183543 deg, angle=383.5305446771204 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.83988442, -9.24448306)>, width=0.0074608492348664825 deg, height=0.0074608492348664825 deg, angle=337.7900853561976 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.67936205, -9.22831308)>, width=0.007642103323825737 deg, height=0.007642103323825737 deg, angle=391.13527919824 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.62917513, -9.21923338)>, width=0.007642082016264236 deg, height=0.007642082016264236 deg, angle=400.7084686717307 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.79871532, -9.16670201)>, width=0.012440372645364971 deg, height=0.012440372645364971 deg, angle=355.0788032786662 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.84029561, -9.54883107)>, width=0.007667803827869568 deg, height=0.007667803827869568 deg, angle=560.96001825782 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.8978813, -9.37772185)>, width=0.007862583932634504 deg, height=0.007862583932634504 deg, angle=276.65062148184717 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.89105993, -9.54884994)>, width=0.011787833479783082 deg, height=0.011787833479783082 deg, angle=575.0197345874648 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.97061681, -9.43944536)>, width=0.006871217861284668 deg, height=0.006871217861284668 deg, angle=255.69675250947418 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.74727506, -9.53826212)>, width=0.004185759438683856 deg, height=0.0034176512547450105 deg, angle=0.00519723029853467 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.65521757, -9.25972201)>, width=0.005085643146284783 deg, height=0.004979068953513777 deg, angle=-89.97978092689743 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.74532898, -9.1401906)>, width=0.008910391116877896 deg, height=0.005099599293395697 deg, angle=-4.3431752110019035 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.84740316, -9.12401742)>, width=0.010836210345668586 deg, height=0.004752678895310275 deg, angle=22.76255845772804 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.66155125, -9.25468904)>, width=0.007461134604631991 deg, height=0.006089759412612027 deg, angle=-243.26581444855768 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.87250749, -9.17123746)>, width=0.015571927865370996 deg, height=0.012841259287035708 deg, angle=-5.447628043271801 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.76800879, -9.58453019)>, width=0.04900630168181078 deg, height=0.0418518412810454 deg, angle=14.492613927564252 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.73703671, -9.42752247)>, width=0.02737187371185334 deg, height=0.025547049908862033 deg, angle=-234.62113208461741 deg)>],
  '0502671101': [<EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (235.90154368, -83.80147385)>, width=0.00660837290419497 deg, height=0.00660837290419497 deg, angle=202.4868703807935 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (234.49541433, -83.68304058)>, width=0.007100641472299129 deg, height=0.007100641472299129 deg, angle=157.52414219789802 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (233.74507431, -83.51250489)>, width=0.00714719294578614 deg, height=0.00714719294578614 deg, angle=430.5717571135679 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (233.94244606, -83.35916674)>, width=0.007460790554446995 deg, height=0.007460790554446995 deg, angle=390.6036279032619 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (234.20203946, -83.63612767)>, width=0.011149504631671156 deg, height=0.011149504631671156 deg, angle=491.55066920006726 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (235.09173966, -83.3819441)>, width=0.007336634582089766 deg, height=0.007336634582089766 deg, angle=354.7415736534235 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (235.27920679, -83.62139769)>, width=0.009230085090930708 deg, height=0.009230085090930708 deg, angle=212.69728753428527 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (236.23212762, -83.57686191)>, width=0.007570885109952949 deg, height=0.007570885109952949 deg, angle=263.42136755600364 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (232.78254537, -83.64354873)>, width=0.007287453342756499 deg, height=0.007287453342756499 deg, angle=471.22423270980056 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (236.73121362, -83.53946232)>, width=0.028088980637951882 deg, height=0.015714807252284024 deg, angle=65.80502135791009 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (234.90593367, -83.5888805)>, width=0.060431397862485636 deg, height=0.05796591495634242 deg, angle=8.671039046651588 deg)>],
  '0761910201': [<EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.68937419, -9.56078069)>, width=0.007605173317941238 deg, height=0.007605173317941238 deg, angle=512.5237110475313 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.77253784, -9.53257425)>, width=0.007225087744081948 deg, height=0.007225087744081948 deg, angle=537.2291426911661 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.77991627, -9.52102755)>, width=0.007733317091148604 deg, height=0.007733317091148604 deg, angle=540.1609388825366 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.88736665, -9.52040684)>, width=0.007642123949260695 deg, height=0.007642123949260695 deg, angle=579.1564080819821 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.80523125, -9.51758771)>, width=0.007509971224357759 deg, height=0.007509971224357759 deg, angle=551.2198086846786 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.72564001, -9.51836509)>, width=0.007901393639954415 deg, height=0.007901393639954415 deg, angle=517.526794183721 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.92044482, -9.51495767)>, width=0.009964105840928699 deg, height=0.009964105840928699 deg, angle=587.989011277084 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.67642934, -9.51242001)>, width=0.0069114309234041505 deg, height=0.0069114309234041505 deg, angle=500.3228230353246 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.56531187, -9.4881104)>, width=0.007708270005386793 deg, height=0.007708270005386793 deg, angle=474.9739522459599 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.55365585, -9.48913623)>, width=0.0072669366616291056 deg, height=0.0072669366616291056 deg, angle=474.05785398408636 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.76595056, -9.47897744)>, width=0.007737084485437304 deg, height=0.007737084485437304 deg, angle=531.4412174213251 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.82192403, -9.47346034)>, width=0.007405325670107783 deg, height=0.007405325670107783 deg, angle=566.5750852172926 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.67191667, -9.44653819)>, width=0.008188997983536301 deg, height=0.008188997983536301 deg, angle=478.0965592859854 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.74315096, -9.44429672)>, width=0.007494703478737374 deg, height=0.007494703478737374 deg, angle=506.57593722899287 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.98165457, -9.43094931)>, width=0.008713355304176376 deg, height=0.008713355304176376 deg, angle=258.3220246762259 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.7608298, -9.41117408)>, width=0.008338247600854148 deg, height=0.008338247600854148 deg, angle=499.03105288380016 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.70626465, -9.40879626)>, width=0.0077696729844267025 deg, height=0.0077696729844267025 deg, angle=464.653955323828 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.84643592, -9.40421552)>, width=0.007794735186659312 deg, height=0.007794735186659312 deg, angle=257.75908603777935 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.80710602, -9.39717254)>, width=0.0077371023425100985 deg, height=0.0077371023425100985 deg, angle=255.05350280846199 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (329.00979931, -9.38984914)>, width=0.007642116597171322 deg, height=0.007642116597171322 deg, angle=269.96043273636946 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.70467401, -9.38436828)>, width=0.006738410167270855 deg, height=0.006738410167270855 deg, angle=445.7222148366446 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.83070165, -9.38367648)>, width=0.007874826013197438 deg, height=0.007874826013197438 deg, angle=277.03565312549773 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.81200351, -9.38025547)>, width=0.007045742026326322 deg, height=0.007045742026326322 deg, angle=286.77870377092853 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.67005046, -9.37491259)>, width=0.008233181457226636 deg, height=0.008233181457226636 deg, angle=442.1198637685323 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.75649891, -9.37322249)>, width=0.007933552857170845 deg, height=0.007933552857170845 deg, angle=413.71675948051967 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.70796622, -9.37173101)>, width=0.008681480850895254 deg, height=0.008681480850895254 deg, angle=435.5766777103424 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.8957851, -9.37103292)>, width=0.007538171566229098 deg, height=0.007538171566229098 deg, angle=279.32203458597206 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.72058903, -9.37144816)>, width=0.009346645864457591 deg, height=0.009346645864457591 deg, angle=432.3986182648724 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.68086324, -9.36191652)>, width=0.007842746784627016 deg, height=0.007842746784627016 deg, angle=433.9790996303439 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.89339805, -9.35562868)>, width=0.007894676140839866 deg, height=0.007894676140839866 deg, angle=286.9454240409902 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.61028928, -9.35544602)>, width=0.0076466605615973645 deg, height=0.0076466605615973645 deg, angle=438.3786140008967 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.57316187, -9.35288901)>, width=0.0074928486894210375 deg, height=0.0074928486894210375 deg, angle=439.74467148930773 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (329.01076915, -9.35201347)>, width=0.009124096971773004 deg, height=0.009124096971773004 deg, angle=279.37627450237886 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.78943218, -9.33982115)>, width=0.009362956833258915 deg, height=0.009362956833258915 deg, angle=348.97538633888524 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.85669407, -9.32949866)>, width=0.008796769907473507 deg, height=0.008796769907473507 deg, angle=308.4194123975086 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.7503647, -9.32688732)>, width=0.007820733673050688 deg, height=0.007820733673050688 deg, angle=384.55676029198094 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.57621029, -9.31464728)>, width=0.007424424945689914 deg, height=0.007424424945689914 deg, angle=429.485174123743 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.83474633, -9.31090046)>, width=0.007501871457599387 deg, height=0.007501871457599387 deg, angle=325.4059932279165 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.79077339, -9.31158361)>, width=0.00797343822778213 deg, height=0.00797343822778213 deg, angle=351.94516751782294 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.93981651, -9.3040177)>, width=0.007445535674519624 deg, height=0.007445535674519624 deg, angle=298.46985071857654 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.58249999, -9.29807621)>, width=0.0071450812701841235 deg, height=0.0071450812701841235 deg, angle=424.7641479315843 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.6189924, -9.27921549)>, width=0.008684419291152426 deg, height=0.008684419291152426 deg, angle=415.0951940564478 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.64197997, -9.23477804)>, width=0.0074608267523772415 deg, height=0.0074608267523772415 deg, angle=401.22044355499946 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.84886845, -9.23328715)>, width=0.008116936777280776 deg, height=0.008116936777280776 deg, angle=336.38868916703535 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.67951663, -9.22868351)>, width=0.007729466376465991 deg, height=0.007729466376465991 deg, angle=391.50031933343274 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.70024959, -9.22699216)>, width=0.0076718807023934054 deg, height=0.0076718807023934054 deg, angle=385.6729366953833 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.66815329, -9.21530287)>, width=0.007642091503343634 deg, height=0.007642091503343634 deg, angle=392.2191732974662 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.6767295, -9.19053487)>, width=0.007045646840351006 deg, height=0.007045646840351006 deg, angle=386.9967740636772 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.58244675, -9.4627311)>, width=0.0073499475291128385 deg, height=0.0073499475291128385 deg, angle=470.57915661676134 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.77709921, -9.39113194)>, width=0.007642176657240857 deg, height=0.007642176657240857 deg, angle=476.75139849239014 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.68704345, -9.52917156)>, width=0.009760532586818882 deg, height=0.009760532586818882 deg, angle=506.7890913118479 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.70787811, -9.50168701)>, width=0.01039442263290166 deg, height=0.01039442263290166 deg, angle=507.7046920804661 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.76719065, -9.46949572)>, width=0.008898874358419878 deg, height=0.008898874358419878 deg, angle=531.2970150964355 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.96751421, -9.44376535)>, width=0.0074926116789422875 deg, height=0.0074926116789422875 deg, angle=253.76133173052233 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.87544149, -9.42650864)>, width=0.008883965127052801 deg, height=0.008883965127052801 deg, angle=608.831353701835 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.87027321, -9.41502843)>, width=0.009208357013779252 deg, height=0.009208357013779252 deg, angle=254.30719692583813 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.63355, -9.2672484)>, width=0.007570125451236267 deg, height=0.007570125451236267 deg, angle=409.62181893037854 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.8818919, -9.13298913)>, width=0.007606100056132776 deg, height=0.007606100056132776 deg, angle=338.51230130245455 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.8430574, -9.32774321)>, width=0.00931608840720637 deg, height=0.00931608840720637 deg, angle=314.75663725655045 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.6213323, -9.3083263)>, width=0.007494668592118892 deg, height=0.007494668592118892 deg, angle=422.44481229159595 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.58389853, -9.40170782)>, width=0.007762796464037069 deg, height=0.007762796464037069 deg, angle=453.54391975536015 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.943744, -9.49940368)>, width=0.007266861168549896 deg, height=0.007266861168549896 deg, angle=595.9032099359723 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.65523524, -9.25937826)>, width=0.0069789185302487015 deg, height=0.006771908415006877 deg, angle=38.18458091605515 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.74495384, -9.13987952)>, width=0.010799658012802556 deg, height=0.005065284194131101 deg, angle=-3.41899691429815 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.8472948, -9.12389621)>, width=0.011241188923289154 deg, height=0.005825221156390184 deg, angle=22.931945913121808 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.66737543, -9.43260352)>, width=0.022303894441709094 deg, height=0.011907153729425595 deg, angle=51.78980020234835 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.57712017, -9.51571182)>, width=0.014725846448283647 deg, height=0.01185632766887667 deg, angle=69.67002567183334 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.75689362, -9.57390549)>, width=0.06516474075786996 deg, height=0.03133683329802987 deg, angle=-12.004004917278055 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.7342496, -9.42368759)>, width=0.03320522012875693 deg, height=0.028047573474162493 deg, angle=54.06498950775162 deg)>,
   <EllipseSkyRegion(<SkyCoord (ICRS): (ra, dec) in deg
       (328.80555656, -9.33028177)>, width=0.017124914642677427 deg, height=0.014535991053605816 deg, angle=-44.42214439125614 deg)>]},
 'chandra': {},
 'nustar_pointed': {},
 'rosat_all_sky': {}}