Computational methods for characterizing single neuron function under natural condition
Saturday, June 24, 2006 Michael CK Wu UC Berkeley, Biophysics Graduate Group
MCK Wu, SV David & JL Gallant (2006) Complete Functional Characterization of Sensory Neurons by System Identification, Ann. Rev. of Neurosci. Vol. 29 Biophysics, UC Berkeley
~1~
6/24/2006
M. C-K Wu
Outline
The problem
The experiment
The data
The challenges
Method & solutions
Assessment of solutions & results
Applications
Biophysics, UC Berkeley
~2~
6/24/2006
M. C-K Wu
Outline
The problem
The experiment
The data
The challenges
Method & solutions
Assessment of solutions & results
Applications
Biophysics, UC Berkeley
~3~
6/24/2006
M. C-K Wu
The brain is modular
Biophysics, UC Berkeley
~4~
6/24/2006
M. C-K Wu
Visual areas and circuits
Biophysics, UC Berkeley
~5~
6/24/2006
M. C-K Wu
How to characterize single neuron function?
Biophysics, UC Berkeley
~6~
6/24/2006
M. C-K Wu
How to characterize single neuron function?
Visual stimulus: Movie
Biophysics, UC Berkeley
~7~
6/24/2006
M. C-K Wu
Spike counts
How to characterize single neuron function?
Visual stimulus: Movie
Biophysics, UC Berkeley
Neural response
~8~
6/24/2006
M. C-K Wu
Spike counts
How to characterize single neuron function?
Visual stimulus: Movie
Input
Neural response
Black Box
Spike counts (# of impulse per video frame)
Sequence of natural images
Biophysics, UC Berkeley
Output
~9~
6/24/2006
M. C-K Wu
Spike counts
How to characterize single neuron function?
Visual stimulus: Movie
Input
Sequence of natural images
Biophysics, UC Berkeley
Neural response
Black Box
f : \n → \+ Stimulus-Response Mapping Function
~ 10 ~
Output
Spike counts (# of impulse per video frame) 6/24/2006
M. C-K Wu
Outline
The problem
The experiment
The data
The challenges
Method & solutions
Assessment of solutions & results
Applications
Biophysics, UC Berkeley
~ 11 ~
6/24/2006
M. C-K Wu
How do we get our data?
Awake & Behaving Subject
Biophysics, UC Berkeley
~ 12 ~
6/24/2006
M. C-K Wu
How do we get our data?
Awake & Behaving Subject
Biophysics, UC Berkeley
~ 13 ~
6/24/2006
M. C-K Wu
How do we get our data?
Visual Stimulus x1, x2, x 3, …
Biophysics, UC Berkeley
Awake & Behaving Subject
~ 14 ~
Extracellular Single unit Recording Neural Response y 1 , y2 , y3 , …
6/24/2006
M. C-K Wu
How do we get our data?
Visual Stimulus x1, x2, x 3, …
Awake & Behaving Subject
Extracellular Single unit Recording Neural Response y 1 , y2 , y3 , …
Fixation Target
Biophysics, UC Berkeley
~ 15 ~
6/24/2006
M. C-K Wu
How do we get our data?
Awake & Behaving Subject
Visual Stimulus x1, x2, x 3, …
~0.5o
Extracellular Single unit Recording Neural Response y 1 , y2 , y3 , …
Fixation Target
Find where the neuron is looking? Biophysics, UC Berkeley
~ 16 ~
6/24/2006
M. C-K Wu
How to achieve natural condition in laboratory?
Traditional visual stimuli (simple & low dimensional).
noise
grating
bars
We use naturalistic stimulus (image sequences, movies).
natural scenes
Biophysics, UC Berkeley
~ 17 ~
6/24/2006
M. C-K Wu
Natural vision includes eye movements
Biophysics, UC Berkeley
~ 18 ~
6/24/2006
M. C-K Wu
Simulated free viewing
Biophysics, UC Berkeley
~ 19 ~
6/24/2006
M. C-K Wu
Outline
The problem
The experiment
The data
The challenges
Method & solutions
Assessment of solutions & results
Applications
Biophysics, UC Berkeley
~ 20 ~
6/24/2006
M. C-K Wu
An image is a point in Rn
x ∈ \n
x Biophysics, UC Berkeley
~ 21 ~
6/24/2006
M. C-K Wu
An image is a point in Rn
x ∈ \n
...
x Biophysics, UC Berkeley
~ 22 ~
6/24/2006
M. C-K Wu
An image is a point in Rn x1 x2 x3 x4
...
x= x∈\
n
...
xj xj+1 xj+2 xj+3
... ...
xn-3
xn-2 xn-1 xn
x Biophysics, UC Berkeley
~ 23 ~
6/24/2006
M. C-K Wu
An image is a point in Rn x1 x2 x3 x4
...
x= x∈\
n
...
xj xj+1 xj+2 xj+3
x ∈ \n
.
... ...
xn-3
xn-2 xn-1 xn
x Biophysics, UC Berkeley
~ 24 ~
6/24/2006
M. C-K Wu
The problem in another perspective
X = [ x1 ," , x N ]
T
s.t. xi ∈ \ n
Biophysics, UC Berkeley
~ 25 ~
6/24/2006
M. C-K Wu
The problem in another perspective f X = [ x1 ," , x N ]
T
s.t. xi ∈ \
Biophysics, UC Berkeley
~ 26 ~
n
Y = [ y1 ," , y N ]
T
s.t. yi ∈ \ +
6/24/2006
M. C-K Wu
The problem in another perspective f Y = [ y1 ," , y N ]
T
X = [ x1 ," , x N ]
T
s.t. xi ∈ \
s.t. yi ∈ \ +
n
\+
\n Biophysics, UC Berkeley
~ 27 ~
6/24/2006
M. C-K Wu
The problem in another perspective f Y = [ y1 ," , y N ]
T
X = [ x1 ," , x N ]
T
s.t. xi ∈ \
s.t. yi ∈ \ +
n
\+
\n Biophysics, UC Berkeley
~ 28 ~
6/24/2006
M. C-K Wu
The problem in another perspective f Y = [ y1 ," , y N ]
T
X = [ x1 ," , x N ]
T
s.t. xi ∈ \
s.t. yi ∈ \ +
n
\+
\n Biophysics, UC Berkeley
~ 29 ~
6/24/2006
M. C-K Wu
Outline
The problem
The experiment
The data
The challenges
Method & solutions
Assessment of solutions & results
Applications
Biophysics, UC Berkeley
~ 30 ~
6/24/2006
M. C-K Wu
Why is this hard? Input
Black Box
Output
High Dimensionality: Input have very high dimensionality.
Biophysics, UC Berkeley
~ 31 ~
6/24/2006
M. C-K Wu
Why is this hard? Input
Black Box
High Dimensionality: Input have very high dimensionality.
Sparse sampling: There is not enough samples to cover the vast input space densely.
Biophysics, UC Berkeley
~ 32 ~
Output
6/24/2006
M. C-K Wu
Why is this hard? Input
Black Box
High Dimensionality: Input have very high dimensionality.
Sparse sampling: There is not enough samples to cover the vast input space densely.
Unknown Input Statistics: Natural scene statistics are not well understood.
Biophysics, UC Berkeley
~ 33 ~
Output
6/24/2006
M. C-K Wu
Why is this hard? Input
Black Box
High Dimensionality: Input have very high dimensionality.
Sparse sampling: There is not enough samples to cover the vast input space densely.
Unknown Input Statistics: Natural scene statistics are not well understood.
Biophysics, UC Berkeley
~ 34 ~
Output
Non-Gaussian: Response are Poisson or Negative Binomial (over dispersed).
6/24/2006
M. C-K Wu
Why is this hard? Input
Black Box
High Dimensionality: Input have very high dimensionality.
Sparse sampling: There is not enough samples to cover the vast input space densely.
Output
Non-Gaussian: Response are Poisson or Negative Binomial (over dispersed).
High Noise: Neurophysiology data are very noisy, and we can’t tell the signal from noise.
Unknown Input Statistics: Natural scene statistics are not well understood.
Biophysics, UC Berkeley
~ 35 ~
6/24/2006
M. C-K Wu
Why is this hard? Input
Black Box
High Dimensionality: Input have very high dimensionality.
Output
Non-Gaussian: Response are Poisson or Negative Binomial (over dispersed).
Sparse sampling: There is not enough samples to cover the vast input space densely.
High Noise: Neurophysiology data are very noisy, and we can’t tell the signal from noise.
Unknown Input Statistics: Natural scene statistics are not well understood.
Large Scale Problem: Sample size ~ 8000 to 50,000.
Biophysics, UC Berkeley
~ 36 ~
6/24/2006
M. C-K Wu
Why is this hard? Input
Black Box
High Dimensionality: Input have very high dimensionality.
Output
Non-Gaussian: Response are Poisson or Negative Binomial (over dispersed).
Sparse sampling: There is not enough samples to cover the vast input space densely.
High Noise: Neurophysiology data are very noisy, and we can’t tell the signal from noise.
Unknown Input Statistics: Natural scene statistics are not well understood.
Large Scale Problem: Sample size ~ 8000 to 50,000.
The black box is an unknown highly nonlinear and stochastic function. Biophysics, UC Berkeley
~ 37 ~
6/24/2006
M. C-K Wu
Outline
The problem
The experiment
The data
The challenges
Method & solutions
Assessment of solutions & results
Applications
Biophysics, UC Berkeley
~ 38 ~
6/24/2006
M. C-K Wu
How do we estimate f ?
Goal: Given data = { stimulus xi & response yi }, i=1,2,…N, infer a function f (x) = y. T T Notations: X = [ x1 ," , x N ] and Y = [ y1 ," , y N ]
Biophysics, UC Berkeley
~ 39 ~
6/24/2006
M. C-K Wu
How do we estimate f ?
Goal: Given data = { stimulus xi & response yi }, i=1,2,…N, infer a function f (x) = y. T T Notations: X = [ x1 ," , x N ] and Y = [ y1 ," , y N ] Assumption 1: samples in data are independent. Assumption 2: joint probability P ( X, Y, f ) exist.
Biophysics, UC Berkeley
~ 40 ~
6/24/2006
M. C-K Wu
How do we estimate f ?
Goal: Given data = { stimulus xi & response yi }, i=1,2,…N, infer a function f (x) = y. T T Notations: X = [ x1 ," , x N ] and Y = [ y1 ," , y N ] Assumption 1: samples in data are independent. Assumption 2: joint probability P ( X, Y, f ) exist. Find the most probable [maximum a posteriori (MAP)] estimate of f given the data.
f * = arg max P ( f | X, Y ) = arg max ∏ i =1 p ( yi | xi , f ) p ( f ) N
Likelihood Biophysics, UC Berkeley
~ 41 ~
Prior 6/24/2006
M. C-K Wu
How do we estimate f ?
F = Space of all functions { f }
Biophysics, UC Berkeley
~ 42 ~
6/24/2006
M. C-K Wu
How do we estimate f ?
F = Space of all functions { f }
F is usually too large. There are many functions that can fit the data with zero error. Overfit to noise Predict poorly
f(x)
x Biophysics, UC Berkeley
~ 43 ~
6/24/2006
M. C-K Wu
How do we estimate f ?
F = Space of all functions { f } Space of parameters { θ } M = Model class = { fθ }
f ← θ = arg max P ( θ | X, Y ) = arg max ∏ i =1 p ( yi | fθ ( xi ) ) p ( θ ) *
N
*
Biophysics, UC Berkeley
~ 44 ~
6/24/2006
M. C-K Wu
How do we estimate f ?
F = Space of all functions { f }
Noise Distribution & Data = {X, Y} Space of parameters { θ }
M = Model class = { fθ }
Likelihood p( Y | fθ(X) )
.θ
ML
.f
ML
f ← θ = arg max P ( θ | X, Y ) = arg max ∏ i =1 p ( yi | fθ ( xi ) ) p ( θ ) *
N
*
Biophysics, UC Berkeley
~ 45 ~
6/24/2006
M. C-K Wu
How do we estimate f ?
F = Space of all functions { f }
Noise Distribution & Data = {X, Y} Space of parameters { θ }
M = Model class = { fθ } Prior p( fθ )
.
f*
Likelihood p( Y | fθ(X) )
.θ Prior p( θ )
.f
ML
ML
θ*
.
f ← θ = arg max P ( θ | X, Y ) = arg max ∏ i =1 p ( yi | fθ ( xi ) ) p ( θ ) *
N
*
Biophysics, UC Berkeley
~ 46 ~
6/24/2006
M. C-K Wu
Outline
The problem
The experiment
The data
The challenges
Method & solutions
Assessment of solutions & results
Applications
Biophysics, UC Berkeley
~ 47 ~
6/24/2006
M. C-K Wu
How do we assess the model?
Prediction on a unseen validation data set.
Biophysics, UC Berkeley
~ 48 ~
6/24/2006
M. C-K Wu
How do we assess the model?
Prediction on a unseen validation data set. x1
y1
x2
y2
.
.
.
.
.
.
xN
yN
xN+1
yN+1
xN+2
yN+2
.
.
.
.
xN+M
yN+M
Biophysics, UC Berkeley
~ 49 ~
6/24/2006
M. C-K Wu
How do we assess the model?
Y=
=X
y1
x2
y2
.
.
.
.
.
.
xN
yN
x1
y1
x2
y2
.
.
.
.
xM
yM
Biophysics, UC Berkeley
~ 50 ~
6/24/2006
Validation (~10%) resp.
Validation (~10%) stim.
x1
Estimation (~90%) resp.
Prediction on a unseen validation data set. Estimation (~90%) stim.
M. C-K Wu
How do we assess the model?
. . .
y1 y2
x2
xN
Validation (~10%) stim.
Y=
=X
.
θ = arg max P ( θ | X , Y ) *
.
N
.
= arg max ∏ p ( yi | f θ ( x i ) ) p ( θ )
yN
i =1
x1
y1
x2
y2
.
.
.
.
xM
yM
Biophysics, UC Berkeley
~ 51 ~
6/24/2006
Validation (~10%) resp.
x1
Estimation (~90%) resp.
Prediction on a unseen validation data set. Estimation (~90%) stim.
M. C-K Wu
How do we assess the model?
. . .
y1 y2
x2
xN
Validation (~10%) stim.
Y=
=X
.
θ = arg max P ( θ | X , Y ) *
.
N
.
= arg max ∏ p ( yi | f θ ( x i ) ) p ( θ )
yN
i =1
x1
y1
x2
y2
.
fθ* ( x )
.
.
.
xM
yM
Biophysics, UC Berkeley
~ 52 ~
6/24/2006
Validation (~10%) resp.
x1
Estimation (~90%) resp.
Prediction on a unseen validation data set. Estimation (~90%) stim.
M. C-K Wu
How do we assess the model?
. . .
y1 y2
x2
xN
Validation (~10%) stim.
Y=
=X
.
θ = arg max P ( θ | X , Y ) *
.
N
.
= arg max ∏ p ( yi | f θ ( x i ) ) p ( θ )
yN
i =1
x1
ŷ1
y1
x2
ŷ2
y2
.
.
.
.
.
xM
ŷM
yM
.
Biophysics, UC Berkeley
fθ* ( x )
~ 53 ~
6/24/2006
Validation (~10%) resp.
x1
Estimation (~90%) resp.
Prediction on a unseen validation data set. Estimation (~90%) stim.
M. C-K Wu
How do we assess the model?
. . .
y1 y2
x2
xN
Validation (~10%) stim.
Y=
=X
.
θ = arg max P ( θ | X , Y ) *
.
N
.
= arg max ∏ p ( yi | f θ ( x i ) ) p ( θ )
yN
i =1
\+
x1
ŷ1
x2
ŷ2
y2
.
.
.
.
.
xM
ŷM
yM
.
Biophysics, UC Berkeley
fθ* ( x )
y1
\n ~ 54 ~
6/24/2006
Validation (~10%) resp.
x1
Estimation (~90%) resp.
Prediction on a unseen validation data set. Estimation (~90%) stim.
M. C-K Wu
How do we assess the model?
. . .
θ = arg max P ( θ | X , Y )
.
N
.
= arg max ∏ p ( yi | f θ ( x i ) ) p ( θ )
yN
i =1
ŷ1
x2
ŷ2
fθ* ( x )
.
.
.
xM
ŷM
Biophysics, UC Berkeley
.
*
x1 .
y1 y2
x2
xN
Validation (~10%) stim.
Y=
=X
y1 y2
rn(Ŷ,Y) Normalized correlation coefficient % explanable variance
~ 55 ~
. . yM
6/24/2006
Validation (~10%) resp.
x1
Estimation (~90%) resp.
Prediction on a unseen validation data set. Estimation (~90%) stim.
M. C-K Wu
Change our assumptions to improve prediction
F = Space of all functions { f }
Noise Distribution & Data = {X, Y} Space of parameters { θ }
M = Model class = { fθ } Prior p( fθ )
.
f*
Likelihood p( Y | fθ(X) )
.θ Prior p( θ )
.f
ML
ML
θ*
.
f ← θ = arg max P ( θ | X, Y ) = arg max ∏ i =1 p ( yi | fθ ( xi ) ) p ( θ ) *
N
*
Biophysics, UC Berkeley
~ 56 ~
6/24/2006
M. C-K Wu
What have people tried for f(x) ? N * θ = arg max ∏ i =1 p ( yi | fθ ( xi ) ) p ( θ ) Likelihood Statistical Gaussian Poisson Binomial Biophysical Integrate and fire model
Model Class Linear models f θ ( x ) = xT β Second order models fθ ( x ) = xT β + xT Bx Linearized models T fθ ( x ) = L ( x ) β Neural network models fθ ( x ) = a + w T tanh ( b + UT x ) Kernel regression models fθ ( x ) = Φ ( x ) , β
Biophysics, UC Berkeley
~ 57 ~
Prior Flat Gaussian Priors Spherical Independence Stimulus covariance Subspace priors Biophysical Smooth Sparse
6/24/2006
M. C-K Wu
How good are we doing?
V2 V4 33% V1 24% 41%
Biophysics, UC Berkeley
~ 58 ~
6/24/2006
M. C-K Wu
Future works
Biophysics, UC Berkeley
~ 59 ~
6/24/2006
M. C-K Wu
Outline
The problem
The experiment
The data
The challenges
Method & solutions
Assessment of solutions & results
Applications
Biophysics, UC Berkeley
~ 60 ~
6/24/2006
M. C-K Wu
Some potential applications
Biophysics, UC Berkeley
~ 61 ~
6/24/2006
M. C-K Wu
Some potential applications
Face recognition & biometrics
Terrorist target in database Can a computer recognize that these different faces are actually the same person?
Biophysics, UC Berkeley
~ 62 ~
6/24/2006
M. C-K Wu
Some potential applications
Face recognition & biometrics
Satellite image analysis Are there any tanks or airplanes?
Are there military installations?
Biophysics, UC Berkeley
~ 63 ~
6/24/2006
M. C-K Wu
Some potential applications
Face recognition & biometrics
Satellite image analysis
Non-annotated image content search
Biophysics, UC Berkeley
~ 64 ~
6/24/2006
M. C-K Wu
Some potential applications
Face recognition & biometrics
Satellite image analysis
Non-annotated image content search
Unmanned self navigating vehicles
Biophysics, UC Berkeley
~ 65 ~
6/24/2006
M. C-K Wu
Some potential applications
Face recognition & biometrics
Satellite image analysis
Non-annotated image content search
Unmanned self navigating vehicles
Medical imaging diagnosis
Biophysics, UC Berkeley
~ 66 ~
6/24/2006
M. C-K Wu
Some potential applications
Face recognition & biometrics
Satellite image analysis
Non-annotated image content search
Unmanned self navigating vehicles
Medical imaging diagnosis
Artificial vision for blindness
Biophysics, UC Berkeley
~ 67 ~
6/24/2006
M. C-K Wu
Reference MCK Wu, SV David & JL Gallant (2006) Complete Functional Characterization of Sensory Neurons by System Identification, Annual Review of Neuroscience Vol. 29 Biophysics, UC Berkeley
~ 68 ~
6/24/2006
M. C-K Wu
Acknowledgement Jack Gallant & The Gallant Lab (a.k.a. The G-Force)
NSF IGERT, DOE CSGF & Krell Institute
Biophysics, UC Berkeley
~ 69 ~
6/24/2006
M. C-K Wu
Biophysics, UC Berkeley
~ 70 ~
6/24/2006
M. C-K Wu
![[Computational Numerical Methods for Engineers]](https://kipdf.com/img/300x300/computational-numerical-methods-for-engineers_5b01b0808ead0e7c5d8b45a7.jpg)
