To better understand this part, I recommend reading this paper first
The 2-8 keV cosmic X-ray background spectrum as observed with XMM-Newton (De Luca & Molendi 2004)
. In this chapter, I am going to introduce the scripts to check for soft proton flares from the official
XMM-Newton website and by me.
Here are what I use for this tutorial:
Source: MCXC J0106.8+0103 and A0267 (flaring source)
Observation ID: 0762870601 and 0084230401
SAS Version: 19.0.0
Python version: 3
csh is used
Everything you need in this tutorial is in:
residual_sp.zip
.
The products are for MCXC J0106.8+0103 only!
You work in the main folder residual_sp, there are several subfolders:
1. src - the source code is found here.
2. InputFiles - the folder in which the good ccd lists for mos1(goodccdlist_mos1.dat) and mos2(goodccdlist_mos2.dat) are put. This is created in
Anomalous CCD Check
.
3. products - the files created in this tutorial. They are put in the main folder if you create them successfully.
If you have already finished the last chapter and keep everything well. Then, you just need to copy the scripts in src to your current src folder.
Here are what you need in the main folder residual_sp. You should have them already if you follow the steps in the previous chapters. I am not uploading them since they are too big:
1. the "clean" files: mos1S001-clean.fits, mos2S002-clean.fits and pnS003-clean.fits
2. the corner files: mos1S001-corn.fits, mos2S002-corn.fits and pnS003-corn.fits. You can produce them from the "clean" files if you don't have them. Just check the commands in
FOV Spectrum and Image
.
3. The combined exposure: comb-exp-im-400-2300-mos1.fits, comb-exp-im-400-2300-mos2.fits and comb-exp-im-400-2300-pn.fits. Again they are the products from
FOV Spectrum and Image
.
4. The folders full_spectrum_mos1, full_spectrum_mos2 and full_spectrum_pn. They are again created in
FOV Spectrum and Image
.
5. mos1S001-obj-image-sky.fits,mos2S002-obj-image-sky.fits and pnS003-obj-image-sky.fits. These are images in sky coordinate created in
Filtering for Soft Protons
. If you don't have them, just look up the commands in that chapter or produce an image in sky coordinate yourself. Then, remember to modify the code in SB() of src/residual_sp_commands.py. These images are used for coordinate conversion using ecoordconv.
We already check for soft proton flares in
Filtering for Soft Protons
. Let me remind you, we extract the lightcurves and plot a histogram of the count rate to filter out events of high counts, right? See Fig. 1 below for MCXC J0106.8+0103. This source is not contaminated by soft proton flares and indeed it is not.
Fig.1 Diagnostic plots of mos-filter for the source MCXC J0106.8+0103. This source is not contaminated by soft proton flares.
Wait, it is not enough. Soft proton flares with slow time variability and low amplitudes cannot be detected in such a plot. Here is one example (A0267):
Fig.2 Diagnostic plots of mos-filter and pn-filter for the source A0267. Top: mos1, middle:mos2, botton:pn. Note that pn observations started an hour later than mos.
This source is actually very contaminated for mos1 and mos2. According to
De Luca & Molendi 2004
, to check for soft-proton flares, compare the IN and OUT FOV surface brightness in the high energies (e.g. 8-12 keV). A ratio > 1.3 is considered very contaminated and may not be suitable for data analysis. In the high energy regime, only the quiescent particle background, unresolved X-ray sources and soft protons flares are found for areas far away from the source (most cosmic X-ray background are of low energies, more details in
Spectral fit
). For the OUT FOV (i.e. corner), there is only quiescent particle background which is of non-cosmic origin. Thus, a comparison of the IN and OUT FOV would tell whether soft proton flares exist. If the flares are strong enough, they manifest themselves in te image. In the left panel of Fig.3, I created an image of photon counts in 6-12 keV of mos2 but I found the flares not noticeable (ratio=1.8), so I created another image using the whole observation time. Now in the right panel, the soft proton flares are obvious compared to the corner(ratio=3.8). You may want to click on the image to enlarge it to see clearly.
Fig.3 Left: Photon counts image of mos2 of A0267 after mos-filter in 6-12 keV. Soft proton flares exist but not apparent compared to the corner. Right: the same criteria using dataset before mos-filter. Soft proton flares are prominent now.
(In case you want to try yourself, here is how the image including corners are extracted:
evselect table=mos2S002-clean.fits:EVENTS withfilteredset=yes expression='(PATTERN<=12) && region(mos2S002-bkg_region-det.fits)&&(PI in [6000:12000])' filtertype=expression imagebinning='imageSize' imagedatatype='Int32' imageset=mos2S002-obj-im-6-12keV.fits squarepixels=yes ignorelegallimits=yes withxranges=yes withyranges=yes xcolumn='DETX' ximagesize=780 ximagemax=19500 ximagemin=-19499 ycolumn='DETY' yimagesize=780 yimagemax=19500 yimagemin=-19499 updateexposure=yes filterexposure=yes
For the image including the full observation time, go back to
Filtering for Soft Protons
, look up the part of gti files, then make mos2S002-clean.fits again.)
Official Script
To check for residual proton flares, there is a ready-made script available at
XMM-Newton
Here is the idea of the script:
1. Extract the IN FOV spectrum for areas not including (0-10) arcmin centered on the ccd center for energies of your choice.
Here are the centers used in DET coordinate:
mos1: 191.5,-345.5
mos2: 191.5,-345.5
pn: -2203.5,-1167.5
2. Extract the OUT FOV spectrum in the same energy range.
3. surface brightness = (number of counts)/area/exposure
For the number of counts and exposure, they can be extracted from the spectrum directly. For the area, you cannot get the exact area without doing backscal, which takes a long time to do. Here what the script does is to provide a fixed area ratio of IN to OUT area. The problem with this is if there are missing ccds (the case of mos1), the ratio is over estimated. The area ratio is changed in this case. Also, for pn, the result is always "very contaminated".
My Script
Here is the basic idea of my script.
1. Create a whole FOV image in 8-12 keV (or any other range) in sky coordinate.
Take mos2 as an example:
evselect table=mos2S002-clean.fits:EVENTS filtertype=expression expression='(PATTERN<=12)&&(FLAG == 0) && region(mos2S002-bkg_region-sky.fits)&&(PI in [8000:12000]) ' filtertype=expression imagebinning='imageSize' imagedatatype='Int32' imageset=mos2S002-obj-im-8-12keV.fits squarepixels=yes ignorelegallimits=yes withxranges=yes withyranges=yes xcolumn='X' ximagesize=900 ximagemax=48400 ximagemin=3401 ycolumn='Y' yimagesize=900 yimagemax=48400 yimagemin=3401 updateexposure=yes filterexposure=yes
2. Calculate the surface brigtness in 10-12 arcmin from the ccd center using surface brightness = (number of counts)/area/exposure. Here don't use the vignetting-corrected exposure, just extract the exposure from the header. As stated in the paper, the flares are focussed by the mirrors, they shouldn't be affected by vignetting effect. If you use the vignetting-corrected exposure, you find every source is contaminated. The area is easily calculated using the exposure map by just summing up the number of pixels having a non-zero value. Use the combined exposure here because it has been corrected for cheese. You can create an exposure map in different energy ranges but the actual range is not importnat as they are almost identical whatever range you use. What is more, we just want the area, not the real value of the exposure. So, just use the one you have.
In the script, I convert the ccd center from DET coordinate to IMAGE coordinate since the image extracted is in sky coordinate (if in DET coordinate, you cannot use the exposure map since it is also in sky coordinate). Here are the ccd centers I use. I just get them by eyes:
mos1: 112,199
mos2: 155.661,285.833
pn: 285.8,719.722
3. Create a spectrum of the corner in the same energy range:
evselect table=mos2S002-corn.fits filtertype=expression expression='((CCDNR==2)||(CCDNR==3)||(CCDNR==4)||(CCDNR==6)||(CCDNR==7))&&(PI in [8000:12000])' spectrumset='mos2S002-corn-8-12keV.pi' energycolumn='PI' spectralbinsize=5
In the script, I filter out all the bad ccd chips by reading the text files(goodccdlist_mos1.dat and goodccdlist_mos2.dat) in InputFiles, which I create
previously
. Indeed it is not necessary since the anomalous part of a ccd chip only lies in the low energy range. After the spectrum is created, the area is retrived from the corner ccd chip spectrum (e.g. mos2S002-2oc.pi for chip 2) in full_spectrum_mos2 using fkeyprint, again the folder is what I create in the
previous chapter
. The total corner area is just the sum of the ccd chips used. Finally the surface brightess is again: number of counts)/area/exposure. Here instead of extracting an image of the corner I extract a spectrum because I don't know how to calculate the area in the case. The exposure map(or cheese) doesn't include the corner. Also, for the same expression, you can extract both the spectrum and image, the number of events you have in both files are slightly different, and so are the area. The reason is probably due to the different pixel size (0.05 arcsec for spectrum and 2.5 arcsec for image). So to be consistent, I extract the counts and area using the same source.
4. Finally the script returns you the ratio (surface brightness of 10-12 arcmin)/(surface brightness of the corner)
As for pn, you don't have to consider the oot events. I tried both cases, including and excluding oot events and the results are the same.
Now let's run the script:
setenv SAS_CCF ccf.cif
python bin/residual_sp_script.py
Here is what you see on the screen (result of MCXC J0106.8+0103):
mos1 ratio=0.88. It is not contaminated
mos2 ratio=0.98. It is not contaminated
pn ratio=1.00. It is not contaminated
Note:
ratio < 1.15 = not contaminated
1.15 < ratio < 1.3 = slightly contaminated
1.3 < ratio < 1.5 = very contaminated
ratio > 1.5 = extremely contaminated
Remeber the demarcation line is 1.3 as suggested by the paper.
Now we know whether the data has residual proton flares. We can move on the check for
Solar Wind Charge-Exchange (SWCX)
.
Online Materials:
XMM-NEWTON Solar Wind Charge Exchange/Soft Proton Tool
This website can produce the diagnostic plots(Fig.1 and Fig.2) online.
Fig.1 Diagnostic plots of mos-filter for the source MCXC J0106.8+0103. This source is not contaminated by soft proton flares.
Wait, it is not enough. Soft proton flares with slow time variability and low amplitudes cannot be detected in such a plot. Here is one example (A0267):
Fig.2 Diagnostic plots of mos-filter and pn-filter for the source A0267. Top: mos1, middle:mos2, botton:pn. Note that pn observations started an hour later than mos.
This source is actually very contaminated for mos1 and mos2. According to
De Luca & Molendi 2004
, to check for soft-proton flares, compare the IN and OUT FOV surface brightness in the high energies (e.g. 8-12 keV). A ratio > 1.3 is considered very contaminated and may not be suitable for data analysis. In the high energy regime, only the quiescent particle background, unresolved X-ray sources and soft protons flares are found for areas far away from the source (most cosmic X-ray background are of low energies, more details in
Spectral fit
). For the OUT FOV (i.e. corner), there is only quiescent particle background which is of non-cosmic origin. Thus, a comparison of the IN and OUT FOV would tell whether soft proton flares exist. If the flares are strong enough, they manifest themselves in te image. In the left panel of Fig.3, I created an image of photon counts in 6-12 keV of mos2 but I found the flares not noticeable (ratio=1.8), so I created another image using the whole observation time. Now in the right panel, the soft proton flares are obvious compared to the corner(ratio=3.8). You may want to click on the image to enlarge it to see clearly.
Fig.3 Left: Photon counts image of mos2 of A0267 after mos-filter in 6-12 keV. Soft proton flares exist but not apparent compared to the corner. Right: the same criteria using dataset before mos-filter. Soft proton flares are prominent now.
(In case you want to try yourself, here is how the image including corners are extracted: