SAS Data reduction and analysis Need to “do it right”… Jan Ilavsky [email protected]

  1

Talking about:

SAS data

è

Results (numbers!)

§  Star%ng  with  SAS  data  :  typically  area  detector  images  (pinhole  SAXS,  SANS)  but   also  step  scans  (USAXS/USANS)  and  other  data  formats.     §  Processing  steps:   –  Correc%ons,  normaliza%on,  masking,  trimming,  binning  =  reduc&on     –  Calibra%on  to  absolute  intensity  scale     –  Model  selec%on   •  Type  of  science   •  Informa%on  sought   •  Informa%on  available  (other  techniques)  

–  Model  fiOng     •  Method  selec%on   •  Analysis  package  selec%on    

–  Model  verifica%on  and  check  of  “uniqueness”    

§  Publica%on  :  How  to  present  the  data,  what  to  present?  How  to  es%mate   uncertain%es  of  the  resul%ng  values?    

Data reduction

-0.1

-0.2

-0.3

AgBehenate_341.hdf5

Convert  this     Into  this  

-0.2

AgBehenate_626.hdf5

-1

'r_AgBehenate_626.hdf5'

250

-0.8

200

-1

qx [A ]

qx [A ]

-0.6

-0.6

150

100

50

λ

0.0

-0.4

150

0.1

-0.8

100

0.2

-0.2

50

0

0

sin θ

-0.4

To…

Q=

4π

-1

qy [A ]

5

-1.0

-1.0

4

10

3

450

400

10

-1.2

-1.2

350

Intensity

300

10

MaxInt=1.031e+06 2

10

0.2

0.4

0.6

0.8

0.2

0.1

1.0

-1

Q vector [A ]

4  

0.0

-0.1

-1

qy [A ]

1.2

-0.2

-0.3

Alternatively, may be want to create visualization

5  

Following appropriate data reduction and calibration procedure Number of different approaches, often specific to the used area detector & instrument design Instrumental background 2D image

Measured 2D image

Mask

Correct

Mask

Detector background 2D image

Data2D = (Sa2D – Dark2D) – C * (Bckg2D – Dark2D) C ~ sample transmission, measurement times, incoming intensity etc. 6

Calibration factor

Reduce 2D to 1D

Geometrical parameters wavelength, distance etc.

Calibrate

Other corrections?

2-D (area) detectors §  § 

§  § 

§ 

Most common for both desktop & synchrotron based instruments Many different types available –  Image plate –  CCD –  Wire detectors… Each different dynamic range, dark current, offset, readout speed, pixel size, pixel bleeding, …. Require: –  Corrections •  Flat-fielding (pixel sensitivity) •  Dark field subtraction (readout offset and dark noise) •  Unwarping (pixel positions) •  …. –  Masking (beam stop, bad detector areas, shadows of instrumental parts…) Needed corrections vary detector from detector (e.g., MarCCD has dark field subtraction and unwarping built in the data collection software)

7

Some of the tools to convert 2D data to 1D data § 

§ 

§  § 

Ideally – tools should be provided with instrument –  Like ESRF (software is mostly specific for their data) •  http://www.sztucki.de/SAXSutilities/ (Michael Sztucki, processing of SAXS data) •  http://www.esrf.eu/computing/scientific/SAXS/ (Peter Boesecke, manipulation of 2D data) Fit2D - http://www.esrf.fr/computing/scientific/FIT2D/ free, in use for very long time (= debugged), large user base, _very_ capable –  However, not very user friendly and cumbersome for data analysis of large number of data sets – need to learn how to write scripts. –  Ideal for processing large sets of samples (scripting). –  Available for many platforms Datasqueeze - http://www.datasqueezesoftware.com/, $100/$50 for user license, Windows/Linux/MacOS. Nika – Igor Pro (6.0, Mac & Windows) based package (http://usaxs.xor.aps.anl.gov/) – free but need Igor Pro license (http://www.wavemetrics.com/), $550/$395 for user license. –  Igor Pro scripts are open source and can be modified by anyone. – Open source

8

“Nika” – for SAXS, WAXS, GISAXS/ GIWAXS, even SANS §  Display & average 2D image(s) –  Circular average (SAXS/WAXS) –  Sector average (SAXS/WAXS) –  Arbitrary line/circle/ellipse average (SAXS/WAXS, GISAXS/GIWAXS) §  Design mask, Create flood field §  Load & average 2D image(s) and convert them to “lineouts” –  Use dark field/empty field –  Calibrate, correct for thickness –  Correct with various combinations of parameters •  Transmission •  I0, exposure time –  Lookup these parameters using user designed Igor function §  Graph & export resulting line-outs (ASCII data), make movies… §  Easily integrates with Irena package

9

Nika example

10  

Data presentation & analysis

IRENA PLOTTING TOOL : SAXS DATA COLLECTED USING AMPIX CELL DURING FIRST THREE DISCHARGE-CHARGE BATTERY CYCLES Primary  larger   phase  

Second   smaller  phase   1st  cycle   3rd  cycle   2nd  cycle   1st  cycle   0   Average Radius [Å] 15  

Relative Abundance

Reaction Progress

3rd     2nd    cycle   st 1     cycle   cycle  

3rd  cycle   2nd  cycle   1st  cycle  

2nd  cycle   3rd  cycle  

Reaction Progress

Data analysis …. Know what are you doing!

Small-angle scattering (~F(q))

Small-angle diffraction (~S(q))

(dilute limit)

Same size & shape

Size &/or shape variations

Same type particles

Multiple types of particles

13

Model selection §  Monodisperse  dilute  systems   –  Simple  values  (Rg)   –  Form  factor  fiOng  (lots  of  form  factors  available)   –  Shape  reconstruc%on  

§  Monodisperse  concentrated  systems     –  Form  factor  +  Structure  factor  fiOng   –  Structure  factor  extrac%on  

§  Polydisperse  systems   –  –  –  –  –  –  – 

Unified  fit  (Rg  +  Porod)  or  Guinier-­‐Porod     Specific/analy%cal  models  (Debye-­‐Bueche,  Ciccariello-­‐BenedeO,  Treubner-­‐Strey,…)   Size  distribu%on   Size  distribu%on  +  Structure  factors   Fractal  models   Diffrac%on  peaks   Combina%on  of  any  above  

Example list of Form factors listing (per Jan Skov Pedersen presentation) Homogeneous  rigid  par&cles:    

1. Homogeneous sphere 2. Spherical shell 3. Spherical concentric shells 4. Particles consisting of spherical subunits 5. Ellipsoid of revolution 6. Tri-axial ellipsoid 7. Cube and rectangular parallelepipedons 8. Truncated octahedra 9. Faceted Sphere 9x Lens 10. Cube with terraces 11. Cylinder 12. Cylinder with elliptical cross section 13. Cylinder with hemi-spherical end-caps 13x Cylinder with ‘half lens’ end caps 14. Toroid 15. Infinitely thin rod 16. Infinitely thin circular disk 17. Fractal aggregates

Polymer  models   18. Flexible polymers with Gaussian statistics 19. Polydisperse flexible polymers with Gaussian statistics 20. Flexible ring polymers with Gaussian statistics 21. Flexible self-avoiding polymers 22. Polydisperse flexible self-avoiding polymers 23. Semi-flexible polymers without self-avoidance 24. Semi-flexible polymers with self-avoidance 24x Polyelectrolyte Semi-flexible polymers with self-avoidance 25. Star polymer with Gaussian statistics 26. Polydisperse star polymer with Gaussian statistics 27. Regular star-burst polymer (dendrimer) with Gaussian statistics 28. Polycondensates of Af monomers 29. Polycondensates of ABf monomers 30. Polycondensates of ABC monomers 31. Regular comb polymer with Gaussian statistics 32. Arbitrarily branched polymers with Gaussian statistics And many others…  

Example of really complicated system §  USAXS  data  span  ~3.5   decades   §  Matches  microstructure   length  scales   §  Underlying  SAXS  is   modeled  by  Unified  fit   §  Small  angle  diffrac%on   peaks  can  be  iden%fied   and  quan%fied   §  The  fit  has  ~  18   parameters  –  and  is   stable    

16  

Analysis tools selection

SASview, SASFit,…

§  SASview,  hgp://www.sasview.org   –  Result  of  NSF  project  (ended  2011),  currently  supported  development  by  NIST,  UMD,   ISIS,  ILL.     –  1D  and  2D  fiOng  of  models   –  Python  +  C/C++   –  Free  +  open  source  

§  SASFit,  hgps://kur.web.psi.ch/sans1/SANSSoj/sasfit.html   –  –  –  –   

Joachim  Kohlbrecher  &  Ingo  Bressler,  PSI    1D  fiOng  of  models   C  +  Tcl/tk   Free  +  open  source  

GSAS – II (pyGSAS) §  §  §  §  §  § 

Open  source   Well  made  and  documented   APS  authors  (Bob  von  Dreele,  Brian  Toby)   Python   Free  +  open  source   hgps://subversion.xor.aps.anl.gov/trac/pyGSAS  

§  Can  do  data  reduc%on  (2D  -­‐>  1D)   §  Calibra%on,  etc.   §  SAXS  tools  available  :   –  Size  distribu%on  (MaxEnt)   –  Unified  fit   –  Modeling  –  close  to  Modeling  II  of  Irena  

§  Tested  among  Nika,  Irena  and  GSAS-­‐II  

“Irena” data analysis package (for Igor Pro) §  § 

Version 1 written ~2000. Combines number of tools to one suit : –  Import & export data (ASCII) –  Modify & manipulate (subtract/divide/scale…) –  Graph SAS data (save graphs, graph styles, some basic fitting, export graphics) –  Model data using various models: •  •  •  •  • 

§  § 

Size distribution (dilute limit) using Maximum entropy, TNNLS, or regularization Direct modeling with fitting (with selected structure factors) Unified Fit model (Rg/Power law slopes) Fractals Debye-Bueche (gels)

–  X-ray and neutron reflectivity tool (simple systems for up to 8 layers and no relationships between the layers) –  Other tools: •  Calculate contrast (X-ray & neutron) incl. anomalous effects {Cromer-Liberman} •  Desmear data for slit smeared instruments (USAXS, uses Lake method) •  Etc… Free and open source - link from http://usaxs.xray.aps.anl.gov/ Manual has about 180 pages, please READ IT.

21

Need for user friendly tools… §  Experience  as  beamline  scien%st:   –  –  –  – 

SAXS/USAXS  is  included  ojen  as  “also”  to  do  in  students  work   Limited  training  during  course  work  (assumed  “known  technique”)   Major  problem  in  publica%on  ac%vity  –  lack  of  analysis  tools   Lot  of  garbage  analysis  published  due  to  use  of  outdated/bad  tools  

§  Irena/Nika  wrigen  during  last  10  -­‐  12  years   –  Had  to  do  it  (nothing  beger  available)   –  Open  source  code  is  important  (users  DO  check  the  code)   –  Helps  community,  helps  me,  helps  everyone  

§  Collabora%ng  &  helping  the  community  is  helpful  and  sa%sfying   §  Cannot  provide  infinite  support  –  never  paid  to  develop  code!   §  Therefore:  manual,  handouts,  movies,  lectures  on  Denver  X-­‐ray  conference   workshops,  “Beyond  Rg”  small-­‐angle  scagering  short  course…    

22  

What is Igor Pro? §  Sojware  package  (now  version  6.3x)  from  Wavemetrics  Inc.   (www.wavemetrics.com)   –  –  –  –  – 

Available  for  both  Mac  (OSX)  and  PC  plasorms   Same  scrip%ng  code  runs  on  both  plasorms   Scrip%ng  code  is  simple  text  file  =  open  source   Extendable  with  C-­‐code  (xop,  plasorm  specific)   High-­‐level  programming  language  with  very  ligle  programming  experience  needed   to  write  code   •  •  •  •  • 

Data  management,  import/export   GUI  capabili%es,  same  code  for  both  plasorms     Highly  op%mized  library  of  mathema%cal  opera%ons  (build-­‐in  Numerical  recipes  and  more)   Publica%on  quality  graphs   Notebooks,…  

–  Excellent  support:  ac%ve  user  community,  responsive  company  

§  Previous  data  analysis  sojware  already  available  for  this  plasorm  –  IPNS  tools,   NIST  SANS  data  reduc%on  and  analysis  sojware,  Mogofit  (reflec%vity),  etc.    

Unified fit method From simple systems to hierarchical structures §  Represent “populations” or “levels” of structures in the sample by Rg (and pre-factor) & Power law slope (with pre-factor) –  See references to Greg Beaucage work ( http://www.eng.uc.edu/~gbeaucag/BeaucageResearchGroup.html )

§  Structure factor “interferences” (~Hard sphere model) §  Very generic, very little knowledge about internal structure needed §  But only limited information is obtained. –  Based on microstructure model can get details •  Fractals •  Size distributions (e.g., parameters for assumed log normal size distribution) •  Various shapes (form factors)…

§  Great tool for first look at the sample, sometimes the only tool really useful §  Fails for very narrow size distributions §  NOTE: moves “data analysis” from analysis of 1D data to analysis of discreet Unified parameters: Rg, P, B, G, …

Greg  Beaucage,  J.  Appl.  Cryst  28(1995),  717  -­‐  728   hIp://www.eng.uc.edu/~gbeaucag/BeaucageResearchGroup.html  

31

Unified fit

32

Size distribution – maximum entropy, regularization, or TNNLS/IPG § 

§ 

§ 

§ 

Size distribution –  Volume distribution –  Number distribution How much volume -or- number of scatterers - is between R – dr & R + dr where 2*dr is width of the bin in radii (diameter) Total volume of particles –ornumber of particles = area under the curve (between R1 and R2) In SAS often convenient to have log distribution of radii bins! n  Number  of  available  par%cle  shapes  (F(Q))  including  user  defined  F(Q)  func%on   n  Fast,  easy  –  but  all  scagerers  have  to  be  same  shape  &  contrast   n  Uniqueness  is  achieved  by  use  of  the  Maximum  entropy  method,  TNNLS/IPG,  or   Regulariza%on  

33

SAS modeling n  Modeling  II   –  Up  to  10  input  data  sets    (Q-­‐Int-­‐error)   –  10  models,  each  can  be:   –  Size  distribu%on   •  With  its  own  contrast   •  Form  factor  (~10  F(Q)  available)   •  Structure  factor  (~5  available)   •  Gauss/Log-­‐normal/LSW/power  law  distribuDons   –  or  Unified  level   –  or  Diffrac%on  peak     –  or  Fractal  (Surface  or  Mass)   –  Least  square  or  Gene%c  op%miza%on  fiOng  of  parameters   –  Size  distribu%on  parameters   –  Form  &  Structure  factor  parameters   –  Useful  for  really  complicated  systems  –  do  NOT  use  if  simpler  models  (Unified   fit,  Size  distribu%on)  are  appropriate   34

Results uncertainty estimation – real challenge §  Intensity  uncertain%es   –  Real  challenge  to  obtain  correctly   –  Some  data  reduc%on  programs  ignore  them  altogether  (Fit2D)   –  Some  data  reduc%on  programs  “fudge”  them  since  detector  technology  is  not  know   (Nika)   –  …    

§  Calculated  model  results  uncertain%es:   –  For  op%miza%on  methods  values  from  the  rou%nes  are  rarely  useful  (never  really).   –  Following  Irena  tools  have  Uncertainty  analysis  es%mator:   •  Modeling  II   •  Size  distribu%on   •  Unified  fit  

  Irena  Uncertainty  analysis  can  be  used  to  es%mate  uncertain%es  of  the  resul%ng   values  due  to:   §  Intensity  uncertain%es  –  but  what  is  the  value  of  these?     §  Inter-­‐rela%onships  of  parameters  in  the  model  

Conclusions… §  Data  reduc%on   –  Data  reduc%on  packages  should  be  provided  by  manufacturers  of  the  devices  or   beamline/instrument  staff   –  General  purpose  tools  availability  is  limited,  but  op%ons  exist  (Fit2D,  Datasqueeze,   Nika).  Typical  penalty  for  using  those  is  more  complicated  use,  but  on  the  other   hand  user  has  more  controls  

§  Data  analysis   –  –  –  – 

Much  less  available  from  manufacturers   Free  tools  available  (ATSAS,  SASview,  SASFit,  NIST  package,  Irena…)   Complementary  capabili%es   Limited  support  as  no  dedicated  %me  is  provided  to  authors  

§  How  to  use  these  free  tools:   –  –  –  – 

Read  manual,  view  movies   Learn  theory   Contact  author   Cite  author’s  papers!  It  is  their  only  “payback”  for  their  work  J  

36