The PyMca Application and Toolkit V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific Computing Conference Slide: 1
The ESRF: Just an X-Ray Source Slide: 2
(Partial) Synchrotron Chart Slide: 3
X-Ray Fluorescence Basics Energy of emitted X rays is element dependent Slide: 4
Visualization via Multi-Channel Analyzers (MCA) Slide: 5
PyMca? PyMca is set of software tools on its way to become a reference in XRF For the end users and the specific developers Set of programs and widgets for XRF analysis Spectrum modeling Quantification ROI imaging Fit imaging via batch processing For the general developer Set of python modules Data visualization Peak search Function fitting Imaging of 1D data V.A. Solé, E. Papillon, M. Cotte, Ph. Walter, J. Susini, Spectrochimica Acta B 62 (2007) 63-68 Slide: 6
Generalities - First release in 2004 - Source code SVN repository available at sourceforge since 2006 - Official releases aim to provide ready-to-use binaries to end users - External dependencies on numpy, PyQt4 and PyQwt5 - If installed, Matplotlib is used to generate high quality output - Provides high level functional widgets that can be embedded in your own application Slide: 7
XRF Spectrum Analysis Typical procedure: 1. Calibration 2. Peak identification 3. Peak area extraction Region of interest (ROI) Deconvolution (FIT) 4. Quantification Documentation at http://pymca.sourceforge.net/documentation.html Slide: 8
1. Calibration Detailed documentation at http://pymca.sourceforge.net/documentation.html Slide: 9
2. Identification Theoretical database in ASCII format IUPAC notation whenever possible PyMca proposes, the user decides Slide: 10
3.1 Peak area via ROI Slide: 11
3.2 Peak area via fitting Slide: 12
3.2 Peak area via fitting Slide: 13
Fit configuration Dialog (I) Slide: 14
Fit Configuration Dialog (II) Slide: 15
Emission Lines Allowed transitions Implemented transitions Any transition defined KShellRates.dat LShellRates.dat MShellRates.dat Slide: 16
4. Quantification (I) Parallel beam approximation Slide: 17
4. Quantification (II) Nominal concentration 500 ppm Slide: 18
XRF Analysis Integration in other Applications Integration in mxcube (ESRF) Integration elsewhere Slide: 19
Is it easy to embed? For the previous examples, basically one just needs 4 lines of code: from PyMca import McaAdvancedFit fitwindow = McaAdvancedFit.McaAdvancedFit() fitwindow.setdata(x, y) fitwindow.show() It can be used from ipython just starting it as ipython q4thread Slide: 20
Advanced Fit Batch processing Select the input files Select the fit configuration Select the output directory Select the output options Start Slide: 21
Output Images in ASCII and ESRF format Easy to import in other programs Individual peak contributions in ASCII Use your own plotting program Fully automated HTML report Browse your results! Slide: 22
XRF Imaging One spectrum for each pixel + A sum spectrum for the whole image Slide: 23
Pb Ca S Sb We wanted to know what pigments were used in this section of the painting Copyright C2RMF Slide: 24
Based on the batch generated element distribution maps Slide: 25
and their correlations as shown by the program Sulfur and antimony correlated Lead and antimony not correlated we were able to determine the possible presence of stibnite grains (Sb 2 S 3 ) embedded in a lead containing matrix. M. Cotte, E. Welcomme, V.A. Solé, M. Salomé, M. Menu, Ph. Walter, J. Susini, Anal. Chem. 79 (2007) 6988-6994 Slide: 26
Generic Fitting Module Levenberg-Marquardt algorithm with constraints on fitting parameters It accepts user defined functions The simplest form of user function: Slide: 27
Automatic Peak Search Routines Slide: 28
1D Stack ROI Imaging In this example: Stack = 101x200x2000 numpy array 20200 spectra of 2000 channels Pixel[i, j] = numpy.sum(stack[i, j, :]) Pixel[i, j] = numpy.sum(stack[i, j, ch0:ch1]) Slide: 29
1D Stack ROI Imaging We can generate new images by moving the cursors or defining new ROIs in the table Slide: 30
Imaging and Principal Component Analysis Subtract the average spectrum from each spectrum of the dataset and arrange it as NpixelsxNchannels Calculate the covariance = Data.T * Data Get the eigenvectors of the covariance Add the average spectrum to each row and get the projections of that dataset on each eigenvector (Npixels scalars for each eigenvector) Reshape the projections in the original map shape to obtain the Eigenimages Covariance method, other ways in http:// en.wikipedia.org/wiki/principal_components_analysis Slide: 31
Eigenimages and Eigenvectors Slide: 32
Eigenimages and Eigenvectors Slide: 33
Eigenimages and Eigenvectors Slide: 34
Eigenimages and Eigenvectors Slide: 35
Eigenimages and Eigenvectors Slide: 36
Eigenimages and Eigenvectors Slide: 37
Getting the actual information We can select a set of pixels on any of the displayed images and display the cumulative spectrum associated to those pixels. Here we can see the average spectrum associated to the hotter pixels of the Eigenimage 02 (in red) compared to the average spectrum of the map (in black). Slide: 38
We could have easily missed the presence of one element if we would have just analyzed the sum spectrum via ROIs. Slide: 39
What have we done? We have used principal components analysis to know what sample regions were worth to take a closer look. Not bad when you have a lot of data This data treatment is totally generic and applicable to other methods of analysis Slide: 40
Current Developments HDF5 Support - Collaboration with D. Dale from Cornell High Energy Synchrotron Source (CHESS) - HDF5 expected to become the file format of most European synchrotrons sources - Additional dependency on h5py http://h5py.alfven.org - Partial support already available in current SVN sources Slide: 41
Current Developments 3D and 4D -The goal is to provide ways to visualize and interact with multiparametric data -Already available at the ESRF through an additional PyMca set of modules -Currently evaluating the pros and cons of using MayaVi Slide: 42
Slide: 43
Slide: 44
Conclusion PyMca is a program as well as an XRF analysis toolkit - Open source and distributed under the conditions of the GPLv2+ - Can be used as a fitting and visualization tool (for up to 4-dimensional data) - Provides high level widgets based on PyQt that can be used independently or integrated into your application - Allows you to specify a physically meaningful model which can quantitatively determine element concentrations from energy dispersive X-ray spectra - Is implemented at many synchrotron facilities and labs around the world - Active development is funded by the ESRF Slide: 45
Acknowledgements My ESRF colleagues (mainly software group and microscopy beamlines ID21 and ID22) Ph. Walter and M. Cotte (Centre de Recherche et de Restauration des Musées de France) Darren Dale (Cornell High Energy Synchrotron Source) The PyMca users, for their enthusiasm and their encouragements Slide: 46