Particle characterization of transport in global gyrokinetic calculations of ion channel turbulence in tokamak plasmas J. N. Leboeuf, JNL Scientific, Casa Grande, AZ 85222, USA B. A. Carreras, BACV Solutions, Oak Ridge, TN 37831, USA V. K. Decyk, UCLA, Los Angeles, CA 90095, USA D. Newman, U. Alaska, Fairbanks, AK 99775, USA R. Sanchez, ORNL, Oak Ridge, TN 37831, USA

Poster Presentation ICNSP 2007 Austin, Texas, USA, October 10-12, 2007

Abstract We are in the process of characterizing transport in gyrokinetic calculations of ion channel turbulence in tokamak plasmas with the three-dimensional global toroidal nonlinear parallel particle-in-cell UCAN code. In particular, we have extended the particle manager in UCLA's own PLIB library of massively parallel particle and field managing MPI routines to automatically handle tracking/tracing of the same active simulation particles through space and time and especially multiple processors. The particle data thus tracked and stored comprise the complete set of positions and velocities for each tracked particle at each chosen instant of time (typically every 100th time step). These particle data have been analyzed with tools previously applied to passive marker particles in fluid turbulence simulations which are specifically aimed at revealing the non-diffusive aspects of particle and heat transport. The transport characteristics from UCAN calculations without and with zonal flows self-consistently generated from the fluctuations allowed to evolve will be presented.

UCAN Code Description -Three-dimensional (3D) -Nonlinear -Toroidal -Delta-f -Global -Cartesian geometry (Circular particle boundary in poloidal plane) -Particle-In-Cell (PIC) -Zero Beta -Electrostatic -Adiabatic electrons -Gyrokinetic particle ions -Massively parallel using UCLA’s own PLIB library of parallel MPI particle and field management routines [1D (z) domain decomposition] -F90 modernized following UCLA’s UPIC Framework -Check-pointed and portable R. D. Sydora, V. K. Decyk, and J. M. Dawson, Fluctuation-induced heat transport results from a large global 3D toroidal particle simulation model. Plasma Physics & Controlled Fusion 38, A281-94 (1996); J. N. Leboeuf, J. M. Dawson, V. K. Decyk, M. W. Kissick, T. L. Rhodes, and R. D. Sydora, Effect of externally imposed and self-generated flows on turbulence and magnetohydrodynamic activity in tokamak plasmas. Physics of Plasmas 7, 1795-801 (2000).

UCAN Parallel Performance Measured on NERSC's IBM-SP3 Linear Scaling

200

GFlops

150

100

50

0

0

500

1000

1500

2000

Number of Processors Performance scaling limited by 1D domain decomposition in z-direction

Time Per Particle Per Time Step

UCAN Ports and Performance 10-5

UCAN-Track Timings 64 Processors 128 x 128 x 64 Grid Points 256 x 256 x 64 Particles Time per Particle per Step

10-6

10-7 NERSC CRAY T3E Mcurie

NERSC ARSC IBM SP-3 IBM SP-4 Seaborg Iceberg

ARSC Opteron Cluster Midnite

NERSC Opteron Cluster Jacquard

UCAN Validation and Verification: Multi-Code

UCAN

Linear

UCAN

Nonlinear

CYCLONE:Comparisons and physics basis of tokamak transport models and turbulence simulations, A. M. Dimits et al., Phys. Plasmas 7, 969 (2000)  

UCAN Validation and Verification: Linear Theory 2.5 10-3

2.5 10-3 Simulation Growth Rate

Simulation Growth Rate 2 10-3

1.5 10-3

1.5 10-3

γ/ωci

γ/ω

ci

2 10-3

1 10-3 y = m1 + m2*M0^ 0.5 Value Error m1 -0.0012958 3.4757e-05 m2 0.016557 0.00024665 Chisq 3.6785e-09 NA R 0.99933 NA

-4

5 10

0 100 0

0.01

0.02

0.03

0.04

(Δ/LTi)

Growth rate in flat density limit:

γ = -0.0012958 + 0.016557x R= 0.99933

1 10-3

5 10-4

0.05

0 100 0.06 0.08

0.1

0.12 0.14 0.16 0.18

(Δ/LTi)1/2

0.2

0.22

Typical UCAN Results System size: Nx= 128; Ny=128; Nz=64 (R/a=5; a/ρi=1/ρ*~90) Particle number: Nox=256; Noy=256;Noz=64 Electrostatic Energy (A. U.)

10-1 10-2 10-3 10-4 10-5 10-6

Zonal Flows Off Zonal Flows On

10-7 0

2 104

4 104

6 104

8 104

1 105 1.2 105

ω t ci

Zonal flows dramatically reduce saturation level

Typical UCAN Results: Zonal Flows On Linear phase

Nonlinear saturated phase

Note characteristic zonal flow-tilted fingers

Note much chopped-up eddies

UCLA’s Matt House TORUSMAKER used to map fluctuations onto torus

Dynamic Particle Tracking in UCAN Actual gyrokinetic simulation particles tracked, not passive markers To do so UCAN needs: -New variables npick = fraction of particles to be tracked. ntags = total number of particles to be tracked. ntagsmax = maximum number of tracked particles per processor. itrack = number of components to be tracked. -New arrays tpart = tracked particle data. spart = ordered tracked particle data. ntagsp = number of tracked particles on processor

Dynamic Particle Tracking in UCAN -New subroutines • intrack Places a global id number (integer from 1 to ntags) into word 18 of array part for tracked particles, places 0 into word 18 for untracked particles. Assumes initial uniform distribution. • copy_track Copies components 1:7 and 11 of part for tagged particles into components 1:8 of tpart. Component 3 of tpart is then moved to component 9 and component 3 is replaced by a fictitious z co-ordinate calculated from word 18 of part, which maps the tag number 1:ntags to z co-ordinate 0:ncz. ntagsp is set to the number of tagged particles found on this processor. The fictitious z co-ordinate in copy_track was created so that subroutine pmove in PLIB could be used without modification.

Dynamic Particle Tracking in UCAN -New subroutines •order_track Orders the particles in tpart on the processor. First converts the fictitious z co-ordinate to an integer global tag, then finds the minimum tag number of each processor. Particles in tpart are then copied in the correct order into the array spart. Word 3 in spart is then restored to its proper place from word 9 and word 9 is set to the tag number as a sanity check. (Tag numbers should now be in global order). - Existing subroutines in PLIB • pmove Particle manager moves tagged particles in tpart to the processor where they were born, based on word 3 of tpart. Updates ntagsp. Particles in tpart are now ordered in processor groups, but not necessarily within a group.

Dynamic Particle Tracking in UCAN -Existing subroutines in PLIB •wrdata Writes out ordered, tagged particles in spart on node 0 in processor order number. - New post-processing module •track.rw Reads in ordered, tagged particles in spart on node 0 in processor order number at each time dump. Writes out [x , y, z, vx, vy, vz, v-parallel, µ] for each tracked particle at each time dump.

Tracer/tagged particle motion
Tracer/tagged particles have been used extensively in particle and fluid simulations.
If one assumes an underlying diffusive paradigm, tracer/tagged particle motion information can be easily recasted into effective diffusivities, conductivities, etc.
But one can also use tracer/tagged particle motion to validate the diffusive approach. It has been found that this information can make apparent a lack of characteristic transport scales, reveal the existence of underlying scale-free transport dynamical mechanisms and identify regimes of applicability for fractional transport paradigms.
Finally, the comparison of tracer/tagged particle motion can also be used for validation & verification purposes among codes built on different numerical/physical principles which run in similar regimes beyond the usual comparison of saturation levels and effective transport coefficients.

Basic diffusive motion concepts Diffusive flux is proportional to local gradient:

Γ = −D∇n

D from measurements/microscopic theory/simulations: Characteristic length and time scales MUST exist!

mean-free path, eddy size, etc

D ≃ ∆r2 /2∆t collison time, eddy turnover time, etc

Γ

If characteristic scales do not exist, transport is non-diffusive. The diffusive paradigm may not be the most adequate to capture the underlying dynamics.

Sub, super and diffusive transport Transport dynamics can be much richer than simple diffusion:

∂n ∂ 2n =D 2 ∂t ∂x

2

1 H< 2

H=1

Subdiffusion


2 1/2 ∆x ∝ t1/2

α

Ballistic

1

1