James J. DiCarlo MD, PhD Curriculum vitae updated November, 2013! !
Contact information Department of Brain and Cognitive Sciences and McGovern Institute for Brain Research Massachusetts Institute of Technology 43 Vassar St. Bldg. 46-6161 Cambridge. MA 02139 Office: (617) 452-2045 fax: (617) 253-0104 e-mail:
[email protected] www: http://dicarlolab.mit.edu
Degrees! !
1998! 1998! 1990
Ph.D. Biomedical Engineering, Johns Hopkins University, Baltimore, MD M.D., Johns Hopkins University School of Medicine, Baltimore, MD B.S.E. with Highest Distinction in Biomedical Engineering, Northwestern University, Evanston, IL
Employment Present appointment 2012-Present! ! ! !
Head, Department of Brain and Cognitive Sciences Professor of Neuroscience Investigator, McGovern Institute for Brain Research Massachusetts Institute of Technology, Cambridge, MA
Previous appointments 2007-2012! ! ! 2002-2007! ! ! 1998 - 2002!
!
!
!
1998! !
! !
Associate Professor of Neuroscience (tenured 2009) McGovern Institute for Brain Research Department of Brain and Cognitive Sciences Massachusetts Institute of Technology, Cambridge, MA Assistant Professor of Neuroscience, McGovern Institute for Brain Research Department of Brain and Cognitive Sciences Massachusetts Institute of Technology, Cambridge, MA Research Associate, Howard Hughes Medical Institute and Division of Neuroscience, Baylor College of Medicine, Houston, TX Laboratory of Dr. John H.R. Maunsell Postdoctoral Fellow, Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD Laboratory of Dr. Kenneth O. Johnson
Other research appointments 1992-1998 !
Research Assistant, Department of Biomedical Engineering and
Page 1 of 18
1991! ! 1987-1990! 1987-1989!
Krieger Mind/Brain Institute, Johns Hopkins University Research Assistant, Department of Psychology, Johns Hopkins University Research Assistant, Department of Psychology, Northwestern University Research Intern, National Aeronautics and Space Agency, Cleveland, OH
External positions held Section Co-Editor, “Sensation and Perception”, The Cognitive Neurosciences textbook (2011present) General Co-Chair, Computational and Systems Neuroscience (COSYNE) (2011-present) Program Committee Co-Chair, Computational and Systems Neuroscience (COSYNE) (2010-2011) Program Committee, Computational and Systems Neuroscience (COSYNE) (2008-2011) Program Planning Committee, Society for Neuroscience (2007-2010) Technical Advisory Board, Numenta, Inc., Menlo Park, CA (2008-2011) Consultant, The PreTesting Company, Inc, Tenafly, NJ (2008)
!
Membership:
! ! ! ! !
Society for Neuroscience (1994-present) American Physiological Society American Association for the Advancement of Science Associate Member, Canadian Institute for Advanced Research (CIFAR), Neural Computation and ! Adaptive Perception (2010-present)
Honors and awards McKnight Scholar Award in Neuroscience, McKnight Foundation, 2006-2009! Surdna Research Foundation Award, MIT, 2005! ! MIT School of Science Prize for Excellence in Undergraduate Teaching, 2005 Pew Scholar in the Biomedical Sciences, 2002-2006! Alfred P. Sloan Research Fellow, 2002 Martin and Carol Macht Young Investigator Research Prize, Johns Hopkins University, 1998 National Institutes of Health Medical Scientist Training Program Award, Johns Hopkins University, 1990-1998 ! Honors in Biomedical Engineering, Northwestern University, 1990
Student and postdoctoral supervision !
Sponsored undergraduates in research (UROP) David Van Aken (MIT class of 2003), fall 2002 Nadja Oertelt (MIT class of 2007), 2003-2004, 2007 Jonathan Karr (MIT class of 2006), spring 2006 Prashant Dilwali (MIT class of 2008), spring 2006 Julia Green (Brown class of 2008), summer 2006 Michelle Fogerson (MIT class of 2007), spring 2005-2006 Imran Hendley (MIT class of 2007), summer 2006 Sabrina Tsang (MIT class of 2008), summer 2007 - 2008 Laura Mariano (UCONN class of 2008) AMGEN Scholar, Summer 2007 Rebecca Rothkopf (Wellesley class of 2009), summer 2008 Radhika Palkar (Univ. of California at Irvine class of 2008), MSRP Student, Summer 2008 Isaac Buenrostro (MIT class of 2011), fall 2008 Ethan Solomon (MIT class of 2012), fall 2009-spring 2012
Page 2 of 18
Edith Reshef (MIT class of 2011), spring 2010-fall 2011 Darren Seibert (U of Houston class of 2012), summer 2011 Cesar Echavarria (MIT class of 2012), Fall 2011-fall 2012
Masters students thesis supervised Vuong, Yihvan, ME Biological Engineering, MIT, 2003-2004 ! Current position: Materials Engineer, US Department of Defense, Washington, DC (USA) Oreper, Daniel, ME Electrical Engineering and Computer Science, MIT, 2004-2006 ! Current position: Senior Software Engineer, BAE Systems, Burlington, MA (USA) Pinto, Nicolas, ME Computer Science, UTBM, France 2006-2007 ! Current position: Postdoctoral fellow, MIT / Harvard (USA). Radwan, Basma, ME Biomedical Engineering, Boston University, 2007-2008 ! Current position: PhD candidate, New York University, New York, NY (USA). !
Visiting Masters students supervised Pagan, Marino, ME Control Engineering, University of Pisa, Italy, 2008 ! Current position: PhD candidate, University of Pennsylvania, Philadelphia, PA (USA). Corda, Benoit, ME Computer Science, University of Technology of Compiégne, France ! Current position: Second year PhD student at New York University, New York, NY (USA). Doukhan, David, École Pour lʼInformatique et les Techniques Avancées, France ! Current position: PhD student at LIMSI / CNRS (France) Mirza-Mohammadi, Mehdi. MS Artificial Intelligence, Universitat Politècnica de Catalunya, Spain ! Current position: Trainee at Idiap Research Institute, Martigny, (Switzerland) Bendale, Abjijit. MS Computer Science, University of Colorado, 2009 ! Current position: PhD Candidate, Media Lab, MIT (USA) Barhomi, Youseff. MS Mathematics, Vision and Learning, École Polytechnique. France ! Current position: Research Associate, Laboratory of Dr. Thomas Serre, Brown University (USA). Moghimi Pantea. ME Biomedical Engineering, Chalmers University of Tech., Gothenburg, Sweden, 2011 Current position: PhD Candidate, U of Minnesota (USA) Zhang, Xiyaun. MS Mathematics, Vision and Intelligence, École Normale Superieur de Cachan, France, 2011
!
Ph.D. students supervised (primary advisor role) Cox, David. The role of visual experience in the tolerance of neuronal object representations in monkeys and humans. Supervised 2002-2007 (PhD granted 2007 Dept. of Brain and Cog Sciences, MIT). Current position: Rowland Junior Fellow (independent laboratory), The Rowland Institute at Harvard, Harvard University, Cambridge, MA
Page 3 of 18
Li, Nuo. The construction of invariant neuronal object representations in the primate ventral stream. ! Supervised 2005-2010 (PhD granted 2010 Dept. of Brain and Cog Sciences, MIT). Current position: Postdoctoral Fellow, Janelia Farm HHMI, Laboratory of Dr. Karl Svoboda. Pinto, Nicolas. High-throughput exploration of bio-inspired visual object recognition algorithms. ! Supervised 2006-2010 (PhD granted 2010 Dept. of Brain and Cog Sciences, MIT). Current position: Postdoctoral Associate, Laboratories of Dr. David Cox (Harvard Rowland Institute), and Dr. James DiCarlo (MIT). Aparicio, Paul (PhD candidate 2003-2013), Dept. of Brain and Cognitive Sciences, MIT. Functional organization of object-selectivity in monkey temporal lobe. Supervised 2005-2013. Next position: Postdoctoral Associate, NIH, Laboratory of Dr. Bruce Cummings. Hong, Ha (PhD candidate 2009-in progress), Health Sciences Technology Program, MIT. ! The performance of the ventral visual stream in real-world visual object recognition. ! Supervised 2009-present Ardila, Diego (PhD candidate 2012-in progress), Dept. of Brain and Cognitive Sciences, MIT. Supervised 2013-present Rajalingham, Rishi (PhD candidate 2012-in progress), Dept. of Brain and Cognitive Sciences, MIT. Supervised 2013-present Seibert, Darren (PhD candidate 2012-in progress), Dept. of Brain and Cognitive Sciences, MIT. Supervised 2013-present.
Postdoctoral researchers supervised (primary supervisor role) Hung, Chou (Ph.D.) 2002-2006 ! Read-out and write-in of neuronal object representations in non-human primates. ! ! Current position: Assistant Professor, Institute of Neuroscience, National Yang Ming University, Taipei, Taiwan ! Position as of July 2012: Research Assistant Professor, Department of Neuroscience, Georgetown University. Op de Beeck, Hans (Ph.D., Human Frontiers Long-term Postdoctoral Fellow Award) 2003-2006 ! Functional organization of neuronal object representations in monkeys and humans; Effects of visual experience. ! Current position: Tenured Associate Professor, Laboratory of Experimental Psychology, University of Leuven, Leuven, Belgium Zoccolan, Davide (Ph.D., Human Frontiers Long-term Postdoctoral Fellow Award) 2003-2008 ! Selectivity and tolerance of neuronal populations underlying object recognition in clutter. ! Current position: Assistant Professor, International School for Advanced Studies (SISSA), Trieste, Italy Papanastassiou, Alexander (M.D.) 2005-2007 ! Spiking- and fMRI-determined spatial organization of object representations in monkeys. ! Current position: Assistant Professor, Department of Neurosurgery, University of Texas Health Science Center, San Antonio,Texas Rust, Nicole (Ph.D., NIH NRSA Postdoctoral Award) 2006-2009
Page 4 of 18
! !
Transformation of visual representations along the ventral visual processing stream. Current position: Assistant Professor, Department of Psychology, University of Pennsylvania
Majaj, Najib (Ph.D., New York University) 2007-present ! The role of learning in building invariant neuronal object representation and supporting perception. ! Current position: Senior Research Scientist, Center for Neural Science, New York University Issa, Elias (Ph.D., Johns Hopkins University; NIH NRSA Postdoctoral Award) 2008-present ! Neural properties of fMRI identified face, body, and object selective regions in IT cortex Afraz, Seyed Reza (Arash) (Ph.D., Harvard University) 2009-present ! Manipulation of the neural responses in IT cortex through light-sensitive channels. Yamins, Daniel (Ph.D., Harvard University) 2010-present ! High-throughput exploration of bio-inspired object recognition algorithms. Cadieu, Charles (Ph.D., University of California, Berkeley) 2011-present ! Understanding the neural basis of visual face processing Jia, Xiaoxuan (Ph.D., Albert Einstein College of Medicine) 2012-present ! Unsupervised learning of object representation in primate temporal lobe !
Teaching experience MIT 9.02 Systems Neuroscience Laboratory (undergraduate neurophysiology laboratory) Department of Brain and Cognitive Sciences, MIT Semesters taught: spring 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013 Role: Lead instructor (along with one or two co-instructors), course design, organization, execution and administration Approximately 10 hours of lecture and 60 hours of direct laboratory teaching per semester. MIT 9.720 Neural Basis of Object Recognition in Monkeys and Humans (graduate course) Department of Brain and Cognitive Sciences, MIT Semesters taught: spring 2005, fall 2006, spring 2008, fall 2009 Role: Co-instructor (of two), course design, organization, execution and administration Approximately 10 hours of lecture and 30 hours of shared teaching per semester. MIT Matlab (undergraduate IAP course) Department of Brain and Cognitive Sciences, MIT Semesters: IAP 2008, 2009 MIT 9.95 Research Topics in Neuroscience (undergraduate IAP course) Department of Brain and Cognitive Sciences, MIT Semesters taught: IAP 2004, IAP 2005, IAP 2006, IAP 2007, !AP 2009, IAP 2010, IAP 2011 Role: Lecturer, Approximately 3 hours of direct lecture teaching per semester. MIT Responsible Conduct in Science Department of Brain and Cognitive Sciences, MIT Semesters taught: IAP 2004, IAP 2005, IAP 2006, IAP 2007, !AP 2009, IAP 2010 Role: Guest instructor on ethics of animal research, Approximately 3 hours of student instruction. Computational Neuroscience of Vision (graduate / postdoctoral course) Page 5 of 18
Cold Spring Harbor Laboratories Summer Courses, Cold Spring, NY Semesters taught: summer 2004 Role: Co-instructor, Approximately 4 hours of direct lecture teaching. Methods in Computational Neuroscience (graduate / postdoctoral course) Marine Biological Laboratory at Woods Hole, MA Semesters taught: summer 2008, 2009, 2010 Role: Guest lecturer, Approximately 2 hours of direct lecture teaching. BU CN730 Models of Visual Perception (graduate) Department of Cognitive and Neural Systems, Boston University Semesters taught: spring 2007 Role: Guest lecturer, Approximately 3 hours of direct lecture teaching. Neural Networks (undergraduate course) Department of Biomedical Engineering, Johns Hopkins University Semesters taught: 1995 Role: Teaching assistant Computational models of the Neuron (undergraduate course) Department of Biomedical Engineering, Johns Hopkins University Semesters taught: 1994 Role: Teaching assistant Human Histology (medical student course) School of Medicine, Johns Hopkins University Semesters taught: 1994 Role: Teaching assistant
Service Internal service: Departmental service (Dept. of Brain and Cognitive Sciences, BCS) Department Head (2012-present) Chair, BCS Council (2012-present) BCS Education Committee (2009-present) Principal Investigator for NEI-funded BCS Core Vision Processes Grant (2010-present). Primary supervisor of BCS Electronics Fabrication and Repair Shop (2004-present) Primary supervisor of BCS Machine Shop (2010-present) McGovern / Martinos Imaging Center user committee, McGovern Institute for Brain Research, MIT (2005-present). BCS Graduate Admission Committee (2009-2012) BCS Research Rotation Coordinator (2009-2012) MIT Faculty search committees: !McGovern Institute for Brain Research (2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012); !Dept. of Brain and Cognitive Sciences (2006, 2007, 2008, 2009, 2011, 2012)
Page 6 of 18
Institute-wide service Head, Dept. of Brian and Cognitive Sciences (2012-present) Science Council, School of Science (2012-present) MIT Committee on Curricula (CoC) (2011-2012) MIT Pre-health Undergraduate Student Advisor (2009-2012) Participation in MIT commencement 2003, 2011, 2012 Ph.D. student committees (MIT, outside of primary PhD mentorship role, above) Liu, Jia. Dept. of Brain and Cognitive Sciences, MIT, 2002-2003 (PhD 2003) Wu, Wan-Chen. Dept. of Mechanical Engineering, MIT, 2003-2006 (PhD 2006) Serre, Thomas . Dept. of Brain and Cognitive Sciences, MIT, 2004-2006 (PhD 2006) Balas, Benjamin. Dept. of Brain and Cognitive Sciences, MIT, 2006-2007 (PhD 2007) Feingold, Joseph. Dept. of Brain and Cognitive Sciences, MIT, 2002-present Haushofer, Johannes. Dept. of Neurobiology, Harvard, 2004-2007 (PhD 2007) Schwarzlose, Rebecca. Dept. of Brain and Cognitive Sciences, MIT, 2005-2007 (PhD 2008) Cronin, Beau. Dept. of Brain and Cognitive Sciences, MIT, 2007-2008 (PhD 2008) Tan, Cheston. Dept. of Brain and Cognitive Sciences, MIT, 2009-present. Ghadooshahy, Azriel. Dept. of Brain and Cognitive Sciences, MIT, 2011-present. Kornblith, Simon. Dept. of Brain and Cognitive Sciences, MIT, 2011-present. Lynch, Galen. Dept. of Brain and Cognitive Sciences, MIT, 2012-present. Rikhye, Rajeev. Dept. of Brain and Cognitive Sciences, MIT, 2013-present. MIT undergraduate advisees (past and present) Kim, JinSuk (class of 2005), Won, Annie (class of 2005), Bobrow, Laurel (class of 2006), Golji, Javad (class of 2006), Liang, Joy (class of 2006), Dohlman, Thomas (class of 2007), Motola-Barnes, Rebecca (class of 2007), Evrony, Gilad (class 2007), Garcia, Adrian (class of 2007), Nakano, Lisa (class of 2008), Wentz, Christian (class of 2008), Chandawarker, Akash (class of 2009), Pollard, Courtney (class of 2009), Thornton, Elliot (class of 2009), Pointer, Kelli (class of 2009), Hatch, Mary (class of 2008), Greenman, Susan (class of 2011), DeBoer, Caroline (class of 2011), Dere, Kathryn (class of 2013), Feather, Jenelle (class of 2013), Kim, Heejung (class of 2013), Nora Darago (2014), Jiahao (Andy) Liang (2014), Priyanka Gaur (2016)
External Service: Society for Neuroscience, Annual Meeting Program Planning Committee (2007-2010) Computational and Systems Neuroscience (COSYNE), Annual Meeting Program Committee (2008-2010), Program Committee Co-Chair (2010-2011), General Co-Chair (2011-present) Reviewer for Biological Cybernetics, Cerebral Cortex, Computation and Systems Neuroscience (COSYNE), Current Biology, Journal of Cognitive Neuroscience, Journal of Neurophysiology, Journal of Neuroscience, Journal of Neuroscience Methods, Learning and Memory, Nature, Nature Neuroscience, Neural Information Processing Systems (NIPS), Neuron, Pattern Recognition Letters, Public Library of Science (PLOS), Visual Neuroscience, Science Study section reviewer: NIH Sensorimotor Integration (SMI) study section, Ad hoc member; NSF Ad hoc reviewer; NIH Mechanisms of Sensory, Perceptual and Cognitive Neuroscience (SPC) Study Section, Ad hoc member
Page 7 of 18
Ph.D. student committees (outside of MIT) Maimon, Gaby. Dept. of Neurobiology, Harvard Medical School, 2004-2005 (PhD 2005) Mruczek, Ryan. Dept. of Neuroscience, Brown University, 2006-2007 (PhD 2007) Cury, Kevin. Dept. of Neurobiology, Harvard Medical School (PhD 2011) Ni, Amy. Dept. of Neurobiology, Harvard Medical School (PhD candidate) Science Fair Judge: The Driscoll School Science Fair (Grades K-8), March 2011, 2012.
Publications Refereed papers ( * indicates papers arising from a supervised PhD thesis ) *Yamins DL, Hong H, Cadieu C, and DiCarlo JJ. Hierarchical Modular Optimization of Convolutional Networks Achieves Representations Similar to Macaque IT and Human Ventral Stream. Neural Information Processing Systems (2013). Issa EB, Papanastassiou AM, DiCarlo JJ. Large-scale, high-resolution neurophysiological maps underlying FMRI of macaque temporal lobe. Journal of Neuroscience 33(38): 15207-19 (2013). Baldassi C, Alemi-Neissi A, Pagan M, DiCarlo JJ, Zecchina R, Zoccolan D. Shape similarity, better than semantic membership, accounts for the structure of visual object representations in a population of monkey inferotemporal neurons. PLoS Computational Biology 9(8): e1003167 (2013). Rust N and DiCarlo JJ. Balanced Increases in Selectivity and Tolerance Produce Constant Sparseness along the Ventral Visual Stream. Journal of Neuroscience 32(30): 10170-10182 (2012). Issa EB, DiCarlo JJ. Precedence of the eye region in neural processing of faces. Journal of Neuroscience 32(47: 16666-82 (2012). * Li N, DiCarlo JJ. Neuronal learning of invariant object representation in the ventral visual stream is not dependent on reward. Journal of Neuroscience 32(19): 6611-20 (2012). DiCarlo JJ, Zoccolan DD, and Rust N. How does the ventral visual stream solve object recognition? Refereed Perspective in Neuron 73(3): 415-34 (2012). Majaj N, Hong H, Solomon E, and DiCarlo JJ. A unified neuronal population code fully explains human object recognition. Accepted for oral presentation (top 3% of papers); Computation and Systems Neuroscience (COSYNE), Salt Lake City, UT (2012). * Pinto N, Barhomi Y, Cox DD, and DiCarlo JJ. Comparing State-of-the-Art Visual Features on Invariant Object Recognition Tasks. IEEE Workshop on Applications of Computer Vision, Kona, HI (2011). Rust N and DiCarlo JJ. Selectivity and tolerance ("invariance") both increase as visual information propagates from cortical area V4 to IT. Journal of Neuroscience 30: 12978 - 12995 (2010). * Li N and DiCarlo JJ. Unsupervised Natural Visual Experience Rapidly Reshapes Size-Invariant Object Representation in Inferior Temporal Cortex. Neuron 67(6): 1062 - 1075 (2010). * Pinto N, Doukan D, DiCarlo JJ, and Cox DD. A high-throughput screening approach to discovering good forms of visual representation. PLoS Computational Biology 5(11): e1000579 (2009). * Li N, Cox DD, Zoccolan D, DiCarlo JJ. What response properties do individual neurons need to Page 8 of 18
underlie position and clutter “invariant” object recognition? J Neurophysiology: 102: 360-376 (2009). * Zoccolan D, Oertelt N, DiCarlo JJ, and Cox DD. Rodent model for the study of invariant object recognition, PNAS 106 (21):8748-53 (2009) * Pinto N, DiCarlo JJ, and Cox DD. How far can you get with a modern face recognition test set using only simple features? IEEE Computer Vision and Pattern Recognition (2009) * Cox DD, Papanastassiou A, Oreper D, Andken B, and DiCarlo JJ. High-resolution three-dimensional microelectrode brain mapping using stereo microfocal x-ray imaging, Journal of Neurophysiology 100: 2966-2976 (2008) Op de Beeck H, DiCarlo JJ, Goense J, Grill-Spector K, Papanastassiou A, Tanifuji M, and Tsao D. Finescale spatial organization of face and object selectivity in the temporal lobe: Do fMRI, optical imaging, and electrophysiology agree? Journal of Neuroscience 28: 11796-11801 (2008). * Pinto N, DiCarlo JJ, and Cox DD. Establishing Good Benchmarks and baselines for Face Recognition. Proceedings of the European Conference on Computer Vision (ECCV) (2008). * Li N and DiCarlo JJ. Unsupervised natural experience rapidly alters invariant object representation in visual cortex Science, 321:1502-07 (2008). * Cox DD and DiCarlo JJ. Does learned shape selectivity in inferior temporal cortex automatically generalize across retinal position? Journal of Neuroscience, 28: 10045-55 (2008). * Pinto N, Cox DD, DiCarlo JJ. Why is real-world object recognition hard? PLoS Computational Biology, 4(1): e27 (2008). Op de Beeck H, Deutsch J, Vanduffel W, Kanwisher N, DiCarlo JJ. A stable topography of selectivity for unfamiliar shape classes in monkey inferior temporal cortex. Cerebral Cortex, 18: 1676-94 (2008). Zoccolan D, Kouh M, Poggio T and DiCarlo JJ. Trade-off between shape selectivity and tolerance to identity-preserving transformations in monkey inferotemporal cortex. Journal of Neuroscience, 27: 12292-307 (2007). * DiCarlo JJ and Cox DD. Untangling invariant object recognition. Trends in Cognitive Neuroscience 11: 333-341 (2007). Op de Beeck H, Baker C, DiCarlo JJ and Kanwisher N. Discrimination training alters object representations in human extrastriate cortex. Journal of Neuroscience 26: 13025-36 (2006). Kreiman GK, Hung CP, Kraskov A, Quian Quiroga R, Poggio TA, DiCarlo JJ. Object selectivity of local field potentials and spikes in the macaque inferior temporal cortex. Neuron 49: 433-445 (2006). Hung CP, Kreiman GK, Poggio T, and DiCarlo JJ. Fast read-out of object identity from macaque inferior temporal cortex. Science 310: 863-866 (2005). * Zoccolan, D, Cox DD, DiCarlo JJ. Multiple objects response normalization in monkey inferotemporal cortex. Journal of Neuroscience 36: 8150-64 (2005). * Cox DD, Meier P, Oertelt N, and DiCarlo JJ. “Breaking” position invariant object recognition. Nature Neuroscience 8:1145-1147 (2005). DiCarlo JJ and Maunsell JHR. Using neuronal latency to determine sensory-motor processing pathways Page 9 of 18
in reaction time tasks. Journal of Neurophysiology 5: 2974-86 (2005). DiCarlo JJ and Maunsell JHR. Anterior inferotemporal neurons of monkeys engaged in object recognition can be highly sensitive to object retinal position. Journal of Neurophysiology 89: 3264-3278 (2003). DiCarlo JJ and Maunsell JHR. Form representation in monkey inferotemporal cortex is virtually unaltered by free viewing. Nature Neuroscience 3: 814-821 (2000). DiCarlo JJ and Johnson KO. Changes in stimulus scanning direction reveal the spatial and temporal receptive field structure of neurons in primary somatosensory cortical area 3b of the alert monkey. Journal of Neuroscience 20: 495-510 (2000). DiCarlo JJ and Johnson KO. Velocity invariance of receptive field structure in somatosensory cortical area 3b of the alert monkey. Journal of Neuroscience 19: 401-419 (1999). DiCarlo JJ, Johnson KO, Hsiao SS. Structure of receptive fields in area 3b of primary somatosensory cortex in the alert monkey. Journal of Neuroscience 18: 2626-2645 (1998). DiCarlo JJ, Lane JW, Hsiao SS, and Johnson KO. Marking microelectrode penetrations with fluorescent dyes. Journal of Neuroscience Methods 64: 75-81 (1996). Schmajuk NA and DiCarlo JJ. Stimulus configuration, classical conditioning and hippocampal function. Psychological Review 99: 268-305 (1992). Schmajuk NA and DiCarlo JJ. A neural network approach to hippocampal function in classical conditioning. Behavioral Neuroscience 105: 125-153 (1990).
Non-refereed publications DiCarlo JJ. Do we have a strategy for understanding how the visual system accomplishes object recognition? Object Categorization: Computer and Human Vision Perspectives, Dickenson A, Leonardis A, Schiele B, and Tarr MJ (Eds.), Cambridge University Press (2010) DiCarlo JJ. Making faces in the brain (News & Views). Nature 442: 644 (2006). Kourtzi Z and DiCarlo JJ. Learning and neural plasticity in visual object recognition. Current Opinion in Neurobiology 16: 152-8 (2006). Hung CP, Kreiman GK, Poggio TA, DiCarlo JJ. Ultra-fast object recognition from few spikes, MIT AI Memo 2005-022 (2005). Kreiman GK, Hung CP, Poggio TA, DiCarlo JJ. Selectivity of local field potentials in macaque inferior temporal cortex, MIT AI Memo 2004-020 (2004). DiCarlo JJ and Johnson KO. Receptive field structure in cortical area 3b of the alert monkey. Behavioral Brain Research 135: 167-178 (2002). Hsiao SS, Johnson KO, Twombly IA, DiCarlo JJ. Form processing and attention effects in somatosensory cortex. Somesthesis and the Neurobiology of the Somatosensory Cortex, Birkhauser, O. Franzen, R. Johansson, and L. Terenius (Eds.), Birkhauser Verlag Basel, Switzerland (1996). Schmajuk NA and DiCarlo JJ. Neural dynamics of hippocampal modulation of classical conditioning.
Page 10 of 18
Neural Network Models of Conditioning and Action, M. Commons, S. Grossberg, and J.E.R. Staddon (Eds.), Lawrence Erlbaum Assoc., Hillsdale, NJ, (1991). Schmajuk NA and DiCarlo JJ. A hippocampal theory of schizophrenia. Behavioral and Brain Sciences 14: 47-49 (1991).
Abstracts Afraz A, Boyden ES, DiCarlo JJ. Optogenetic suppression of “face neurons” reveals their causal role in face discrimination behavior. Society for Neuroscience Annual Meeting, San Diego, CA (Nov 2013) Cadieu, C., Issa, EB and DiCarlo, JJ. Understanding the neural basis of face processing in functionally defined area PL. COSYNE, Salt Lake City, UT (2013). Yamins DL, Hong H, Solomon E, and DiCarlo JJ. Ventral stream models that solve hard object recognition tasks naturally exhibit neural consistency. COSYNE, Salt Lake City, UT (2013). Reshef E., Afraz A., DiCarlo, JJ. Varying object identity while maintaining the continuity of its movement breaks position invariant perception. Vision Science Society Annual Meeting (2012) Issa, EB and DiCarlo, JJ. Neuronal responses in fMRI-targeted face-selective regions in posterior inferotemporal cortex. Society for Neuroscience Annual Meeting, Washington, DC (Nov. 2011) Aparicio A, Issa EB and DiCarlo, JJ. Is the monkey middle face patch a module for face detection? Society for Neuroscience Annual Meeting, Washington, DC (Nov. 2011) Pagan A, Alemi-Neissi A, Baldassi C, Zecchina R, DiCarlo JJ, Zoccolan D. From luminance to semantics: how natural objects are represented in monkey inferotemporal cortex. COSYNE, Salt Lake City, UT (2011). Aparicio, P., Issa EB, and DiCarlo, JJ. What is the middle face patch? Society for Neuroscience Annual Meeting, San Diego, CA (Nov. 2010) Pinto N, Majaj NJ, Barhomi Y, Solomon EA, Cox DD, DiCarlo JJ. Human versus machine: comparing visual object recognition systems on a level playing field. Learning Workshop, Snowbird, UT (2010). ! Pinto N, DiCarlo JJ, Cox DD. A High-Throughput Screening Approach to Biologically-Inspired Object Recognition. Learning Workshop, Snowbird, UT (2010). Pinto N, Majaj NJ, Barhomi Y, Solomon EA, Cox DD, DiCarlo JJ. Human versus machine: comparing visual object recognition systems on a level playing field. COSYNE, Salt Lake City, UT (2010). Li N, DiCarlo JJ. Does the visual system use natural experience to construct size invariant object representations? COSYNE, Salt Lake City, UT, (2010). Pinto N, Cox DD, DiCarlo JJ. Unlocking Brain-Inspired Computer Vision. GPU@BU, Boston University, MA (2009). Pinto N, Cox DD, DiCarlo JJ. The Visual Cortex and GPUs. GPU Computing for Biomedical Research, MGH Boston, MA (2009). Pinto N, Cox DD, DiCarlo JJ. Unlocking Biologically-Inspired Computer Vision: a High-Throughput
Page 11 of 18
Approach. NVIDIA GPU Technology Conference, San Jose, CA (2009). Li N, and DiCarlo JJ. The size invariance of neuronal object representations can be reshaped by temporally contiguous visual experience Society for Neuroscience Annual Meeting, Chicago, IL (Oct. 2009) Rust N and DiCarlo JJ. Balanced increases in selectivity and invariance produce constant sparseness across the ventral visual pathway, Vision Science Society Annual Meeting, (May. 2009) Papanastassiou A, Op de Beeck H, Andken B and DiCarlo JJ. A systematic exploration of the relationship of fMRI signals and neuronal activity in the primate temporal lobe, Society for Neuroscience Annual Meeting (mini-symposium), Washington, DC (Nov. 2008) Majaj N, Li N and DiCarlo JJ. Inferior temporal cortex robustly signals encounters with new objects, but is not an online representation of the visual world, Society for Neuroscience Annual Meeting, Washington, DC (Nov. 2008) Rust N and DiCarlo JJ. Increases in selectivity are offset by increases in tolerance ("invariance") to maintain sparseness across the ventral visual pathway, Society for Neuroscience Annual Meeting, Washington, DC (Nov. 2008) Li N, and DiCarlo JJ. Unsupervised natural experience rapidly alters invariant object representation in visual cortex, Society for Neuroscience Annual Meeting, Washington, DC (Nov. 2008) Rust N, and DiCarlo JJ. Concurrent increases in selectivity and tolerance produce constant sparseness across the ventral visual stream. COSYNE, Salt Lake City, Utah (Feb. 2008). Li N, and DiCarlo JJ. Natural experience drives online learning of tolerant object representations in visual cortex. COSYNE, Salt Lake City, Utah (Feb. 2008). Cox DD*, Pinto N*, Doukhan D, Corda B and DiCarlo JJ. A high-throughput screening approach to discovering good forms of visual representation. COSYNE, Salt Lake City, Utah (Feb. 2008). Pinto N*, Cox DD*, Corda B, Doukhan D and DiCarlo JJ. Why is real-world object recognition hard?: Establishing honest benchmarks and baselines for object recognition. COSYNE, Salt Lake City, Utah (Feb. 2008). Zoccolan D, Cox D, Oertelt N, Radwan B, Tsang S and DiCarlo JJ. Is the rodent a valuable model system for studying invariant object recognition? COSYNE, Salt Lake City, Utah (Feb. 2008). Li N, Cox DD, Zoccolan D, and DiCarlo JJ. Flexible and robust object recognition in inferior temporal cortex supported by neurons with limited position and clutter tolerance. Society for Neuroscience, Atlanta, GA, Oct. (2006). Zoccolan D, Kouh M, Poggio T and DiCarlo JJ. Trade-off between shape selectivity and tolerance to identity-preserving transformations in monkey inferotemporal cortex. Gordon Conference: Sensation and the Natural Environment, Bozeman, MT, Aug. (2006). Op de Beeck H, Deutsch J, Vanduffel W, Kanwisher N, DiCarlo JJ. A large-scale shape map in monkey inferior temporal cortex. Society for Neuroscience, Atlanta, GA, Oct. (2006). Cox DD and DiCarlo JJ. Is the “binding problem” a problem in inferiotemporal cortex? Society for Neuroscience, Washington, DC, Nov. (2005).
Page 12 of 18
Zoccolan D, Cox DD and DiCarlo JJ. Multiple object response normalization in monkey inferotemporal cortex. Society for Neuroscience, Washington, DC, Nov. (2005). Hung CP, Kreiman GK, Quiroga R, Kraskov A, Poggio T, and DiCarlo JJ. Using ‘read-out’ of object identity to understand object coding in the macaque anterior inferior temporal cortex. Computational and Systems Neuroscience (COSYNE), Salt Lake City, UT, March (2005). Cox DD and DiCarlo JJ. The effect of visual experience on the position tolerance of primate object representations. Society for Neuroscience, San Diego, CA, Nov. (2004). Kreiman GK, Hung CP, Poggio TA, and DiCarlo JJ. Object recognition by selective spike and LFP data in macaque inferior temporal cortex. Society for Neuroscience, San Diego, CA, Nov. (2004). DiCarlo JJ and Maunsell JHR. Mapping functional neuronal processing chains underlying sensory-motor tasks in the primate. Gordon Research Conference: Sensory coding and the natural environment, Oxford, UK, August (2004). DiCarlo JJ and Maunsell JHR. Using reaction time tasks to map sensory-motor chains in the monkey. Society for Neuroscience, Orlando, FL, Nov. (2002). DiCarlo JJ and Maunsell JHR. Inferotemporal representations underlying object recognition in the free viewing monkey. Society for Neuroscience, New Orleans, LA, Nov. (2000). DiCarlo JJ and Johnson KO. Form processing in area 3b. International Symposium on Brain Mechanisms of Tactile Perception, Stockholm, Sweden, Oct. (1999). DiCarlo JJ, Hsiao SS, and Johnson KO. Spatial and temporal properties of neural receptive fields in area 3b of the awake monkey. Society for Neuroscience, New Orleans, LA, Nov. (1997). Twombly IA, DiCarlo JJ, Hsiao SS and Johnson KO. Linear and non-linear processing of tactile spatial form in area 3b of the awake macaque. Society for Neuroscience, Washington, D.C., Nov. (1996). DiCarlo JJ, Twombly IA, Hsiao SS and Johnson KO. Laminar differences in spatiotemporal receptive field structure of neurons in area 3b of the awake macaque. Society for Neuroscience, Washington, D.C., Nov. (1996). Hsiao SS, DiCarlo JJ and Johnson KO. Interlaminar processing of tactile spatial form in area 3b of the somatosensory system. Biomedical Engineering Society, Boston, Oct. (1995). DiCarlo JJ, Hsiao SS and Johnson KO. Transformation of tactile spatial form within a cortical column in area 3b of the macaque. Society for Neuroscience, Miami, FL, Nov. (1994). Schmajuk NA and DiCarlo JJ. The short-term memory regulation hypothesis of hippocampal function. Midwestern Psychology Association, Chicago, IL, May, (1990). Schmajuk NA and DiCarlo JJ. Neural dynamics of hippocampal modulation of classical conditioning. 12th Symposium on Models of Behavior: Neural Network Models of Conditioning and Action, Cambridge, MA, June, (1989).
Submitted publications Majaj N, Hong H, Solomon E, and DiCarlo JJ. A unified neuronal population code fully explains human object recognition (submitted) Page 13 of 18
Yamins D, Hong H, Solomon E, Seibert D, and DiCarlo JJ. Key Features of Higher Visual Cortex Emerge in Behaviorally Optimized Neural Networks (submitted)
Invited presentations and lectures 1. Stanford University, Department of Neurobiology, Palo Alto, CA (1997) 2. Baylor College of Medicine, Division of Neuroscience, Houston, TX (1997) 3. Johns Hopkins University, Department of Biomedical Engineering, Baltimore, MD (1997) 4. Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences, Cambridge, MA (2001) 5. University of California at Davis, Center for Neuroscience, Davis, CA (2001) 6. University of California at Santa Barbara, Institute for Theoretical Physics, Santa Barbara, CA (2001) 7. McGovern Institute 1st Annual Retreat, M.I.T., Falmouth, MA (2002) 8. Harvard University, Department of Psychology, Cambridge, MA (2002) 9. Harvard Medical School, Department of Neurobiology, Boston, MA (2002) 10. Pew Scholars and Fellows Annual Meeting, Bahamas (2002) 11. Johns Hopkins University, Krieger Mind/Brain Institute, Baltimore, MD (2003) 12. Conte Center Annual Meeting, Detection and Recognition of Objects in Visual Cortex, Cambridge, MA (2004) 13. Computational and Systems Neuroscience annual meeting (COSYNE), Salt Lake City, UT (2005) 14. Conte Center Annual Meeting, , Detection and Recognition of Objects in Visual Cortex, Cambridge, MA (2005) 15. Stanford University, Neuroscience Institute, Palo Alto, CA (2005) 16. Massachusetts General Hospital Martinos Imaging Center, Charlestown, MA (2005) 17. University of Washington, Dept. of Physiology and Biophysics, Seattle, WA (2006) 18. Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences and CSAIL, Cambridge, MA (2006) 19. Pew Scholars and Fellows Annual Meeting, Costa Rica (2006). 20. Harvard University, Department of Psychology, Cambridge, MA (2006) 21. DARPA NeoVision workshop, Washington, DC (2006) 22. Gordon Research Conference: Sensory coding and the natural environment, Big Sky, MO (2006) 23. University of California at San Diego, Dept. of Neuroscience, San Diego, CA (2006) 24. California Institute of Technology, Pasadena, CA (2006) 25. Smith-Kettlewell Eye Institute, San Francisco, CA (2007) 26. University of California at San Francisco, Dept. of Neuroscience, San Francisco, CA (2007) 27. Computational and Systems Neuroscience annual meeting (COSYNE), Salt Lake City, UT (2007) 28. Cold Spring Harbor Laboratory Invited Lecture, Cold Spring Harbor, NY (2007) 29. Functional Requirements of Visual Theory Group Meeting, Montana State University, MT (2007) 30. European Brain and Behavior Society Annual Meeting, Trieste, Italy (2007)
Page 14 of 18
31. International Conference on Computer Vision (ICCV), Rio de Janeiro, Brazil (2007) 32. Harvard Medical School, Department of Neurobiology, Boston, MA (2007) 33. Columbia University, New York, NY (2008) 34. University of California at Los Angeles, CA (2008) 35. University of Southern California, CA (2008) 36. National Institutes of Health, Washington, DC (2008) 37. Cognitive Neuroscience Society (CNS) Annual Meeting, San Francisco, CA (2008) 38. Principles of Biological Computation workshop, Santa Fe Institute, Santa Fe, NM (2008) 39. Perceptual Expertise Network (PEN) workshop, Banff, Canada (2008) 40. Japan Annual Neuroscience Meeting, Tokyo, Japan (2008) 41. RIKEN Brain Science Institute, Wako, Japan (2008) 42. National Institute for Physiological Sciences, Okazaki, Japan (2008) 43. Harvard University, Brigham and Womenʼs, Cambridge, MA (2008) 44. Workshop of Learning and Dynamics in Vision, Glion, Switzerland (2008) 45. 26th Army Science Conference, Orlando, FL (2008) 46. Yale University, Swartz Computational Systems Series, New Haven, CT (Jan. 2009) 47. University of Rochester, Center for Visual Science, Rochester, NY (Jan. 2009) 48. Center for Nonlinear Studies Colloquium, Los Alamos National Laboratory Los Alamos, NM (March 2009) 49. New York University, Center for Neural Science, New York, NY (March 2009) 50. The Thirteenth International Conference on Cognitive and Neural Systems (ICCNS), Boston University,Boston, MA (May 2009) 51. McKnight Endowment Fund Annual Neuroscience Conference, Aspen, CO (June 2009) 52. Annual Meeting of the Sloan-Swartz Centers for Theoretical Neurobiology, Harvard, Cambride, MA (July 2009) 53. Frankfurt Institute for Advanced Studies, Frankfurt, Germany (Oct. 2009) 54. International Conference on Computer Vision Systems (ICVS), Keynote speaker (Oct. 2009) 55. University of California at San Diego (UCSD), Cognitive Science colloquium, San Diego, CA (Feb. 2010) 56. Computational ans Systems Neuroscience (COSYNE) Annual Meeting invited speaker, Salt Lake City, UT (March 2010). 57. Stanford University, Center for Mind, Brain and Computation Minisymposium, Palo Alto, CA (March 2010). 58. University of Texas at Austin, Workshop of Natural Environments, Tasks, and Intelligence, Austin, TX (March 2010). 59. Boston University, Department of Psychology, Boston, MA (April 2010). 60. Woods Hole MBL, Woods Hole, MA (August 2010). 61. McGill Univeristy, Montreal Canada (Oct. 2010) 62. Columbia University, New York, NY (Oct. 2010) 63. Vanderbilt University, Nashville, TN (Nov. 2010)
Page 15 of 18
64. KU Leuven, Leuven, Belgium (Dec. 2010) 65. Scene Understanding Symposium (SUnS), MIT, Cambridge, MA (Jan 2011) 66. Boston University Cognitive and Neural Systems Conference, Boston, MA (May 2011) 67. DARPA NeoVision2 workshop, Washington, DC (May 2011) 68. Dolby Resarch Laboratories, San Franscisco, CA (June 2011) 69. Dartmouth College, Workshop in Neural Computation, Burlington, VT (August 2011) 70. Frontiers in Computer Vision Workshop, Campbridge, MA (August 2011) 71. Champalimaud Inaugural Neuroscience Symposium, Lisbon, Portugal (Sept. 2011) 72. Workshop of Learning and Plasticity, International Mathematics Meeting Center (CIRM), Marseille, France (Nov. 2011) 73. International Conference on Computer Vision (ICCV), Keynote speaker, Barcelona, Spain (Nov. 2011) 74. Johns Hopkins University, Ken Johnson Memorial Speaker (Nov, 2011) 75. Harvard Medical School, Dept. of Neurobiology Systems Group (March, 2012) 76. MIT Museum public lecture, Cambridge, MA (March, 2012) 77. VisoNYC (Greater New York City vision scientists), Columbia/NYU/Suny College of Optometry (March, 2012) 78. Canonical Neural Computation, Florence, Italy (May, 2012) 79. Johns Hopkins University, Center for Lnaguage and Speech Processing (July, 2012) 80. University of Pennsylvania, Dept. of Psychology (July 2012) 81. Princeton University (Nov. 2012) 82. Collaborative Research in Computational Neuroscience Meeting, MIT (June, 2013) 83. Assembly and Function of Neural Circuits Meeting, Ascona, Switzerland (Sept. 2013)
Research contracts and grants 5-P30-EY002621-35, DiCarlo (PI) 09/01/13-07/30/18 Granting agency: National Eye Institute, National Institutes of Health (NIH) Core -- Vision Processes (Support of Instrument, electronics, and imaging core for multiple NEI laboratories at MIT) Total amount for funding period: $576,935 Role: PI 1-R21-EY023053-01, DiCarlo (PI) 2/1/13-1/31/15 Granting agency: National Eye Institute, National Institutes of Health (NIH) Time delimited neural silencing to dissect the basis of visual object perception Total amount for funding period: $211,600 Role: PI 2-R01-EY014970-06, DiCarlo (PI) 09/01/09-8/30/14 Granting agency: National Eye Institute, National Institutes of Health (NIH) Construction of invariant shape selectivity in the ventral visual stream (Renewal) Total amount for funding period: $1,946,148
Page 16 of 18
Role: PI IIS-0964269 DiCarlo (PI) 09/01/10-08/31/13 Granting agency: National Science Foundation High-Throughput Discovery as a Gateway to Understanding Biological VIsion Total amount for funding period: $410,000 Role: co-PI HR0011-10-C-0032 (Evolved machines) Granting agency: DARPA Neovision2 Next Generation Visual Object Recognition Total amount for funding period: $817,112 Role: PI of sub-contract
03/1/10-12/18/11
1-R01-EY014970-01, DiCarlo (PI) 1/1/04-3/31/09 Granting agency: National Eye Institute, National Institutes of Health (NIH) Visual object processing in the inferotemporal cortex Total amount for funding period: $1,557,997 Role: PI McKnight Scholar Award, DiCarlo (PI) 7/1/06-7/1/09 Granting agency: The McKnight Foundation Neuronal mechanisms underlying object recognition during natural viewing Total amount for funding period: $225,000 Role: PI 2893sc, DiCarlo (PI) 7/1/02-6/30/06 Granting agency: The Pew Charitable Trusts - University of California Understanding the Neuronal Invariances that Underlie Visual Object Recognition Total amount for funding period: $240,000 Role: PI 1-P20-MH66239-01, Poggio (PI), DiCarlo (co-PI) 7/1/03-7/31/07 Granting agency: National Institutes of Mental Health (NIMH), NIH Detection and Recognition Of Objects in Visual Cortex Total amount for project period $480,000 Role: Co-PI N00014-02-1-0915, Poggio (PI) DiCarlo (co-PI) 8/1/02-4/30/06 Granting agency: Office of Naval Research (ONR) and Defense Advanced Research Projects Agency (DARPA) A Cortical Interface for Analysis and Synthesis of Visual Recognition Total costs for this project period $730,000 Role: Co-PI BR-4155, DiCarlo (PI) Granting agency: Alfred P. Sloan Foundation Studies of Brain Mechanisms that Underlie Recognition Total costs for this project period $40,000 Role: PI
9/16/02-9/15/04
Research summary My research goal is a computational understanding of how the brain accomplishes visual object recognition -- a task that brings our best computers to their knees, but that our own brain accomplishes effortlessly. We are focussed on understanding how transformations carried out by a series of brain Page 17 of 18
processing stages -- called the primate ventral visual stream -- are effortlessly able to extract object identity in the face of large variability in object position, scale, and pose. My collaborators and I have shown that populations of neurons at the highest visual processing stage (IT) rapidly (
