EFTA02693161

NemoImage 47 (2009) 1691-1700 Contents lists available at ScienceDirect Neurolmage journal homepage: www.elseviet.com!locatelynimg Decoding center-out hand velocity from MEG signals during visuomotor adaptation Trent J. Bradberry Feng Rang '.1, Jose L Contreras-Vidal &schen Department of &Geminating Unarnity of Ala04md. College Park. MD 20742. USA Graduate Program in Neuroscience and Cognitive Science. University of Marylamt College Park. MD 20742. USA Department of Kinesiology. University of Maryforut College Park. MD 20742. USA ARTICLE INFO Ankle history: Received 4 March 2009 Revised 5 May 2009 Accepted 8 June 2009 Available online 16 June 2009 Keywords: Magnemencephalography Hand movement decoding Conical network Visual rotation Visuomotor adaptation Brain-computer interface Introduction ABSTRACT During reaching or drawing, the primate cortex carries information about the current and upcoming position of the hand. Researchers have decoded hand position. velocity. and acceleration during center-out reaching or drawing tasks from neural recordings acquired invasively at the microscale and mesoscale levels. Here we report that we can continuously decode information about hand velocity at the macroscale level from magnetoencephalography (MEG) data acquired from the scalp during a center-out drawing task with an imposed hand-cursor rotation. The grand mean (ri = 5) correlation coefficients (CCs) between measured and decoded velocity profiles were 0.48. 0.40. 0.38. and 0.28 for the horizontal dimension of movement and 0.32. 0.49. 0.56, and 023 for the vertical dimension of movement where the order of the CCs indicates pre- exposure. early-exposure. late-exposure, and post-exposure to the hand-cursor rotation. By projecting the sensor contributions to decoding onto whole-head scalp maps. we found that a macroscale sensorimotor network carries information about detailed hand velocity and that contributions from sensors over central and parietal scalp areas change due to adaptation to the rotated environment. Moreover, a 3-D linear estimation of distributed current sources using standardized low-resolution brain electromagnetic tomography (sLORETA) permitted a more detailed investigation into the cortical network that encodes for hand velocity in each of the adaptation phases. Beneficial implications of these findings include a non- invasive methodology to examine the neural correlates of behavior on a macroscale with high temporal resolution and the potential to provide continuous, complex control of a non-invasive neuromotor prosthesis for movement-impaired individuals. In the last several decades, great strides have been made in revealing how the primate cortex may encode the current and upcoming position of the hand in space during reaching or drawing (Scott 2008). In addition to contributing to the body of neuroscientific knowledge, these discoveries have begun to beneficially impact society. Greater elucidation of the neural code for hand movement has served as an impetus to the development of brain-controlled prostheses for the movement-impaired population. Prior to the advent of brain-controlled prostheses, several seminal discoveries laid a foundation with arguably the most momentous discovery being that of a population vector code for the direction of hand movement in three-dimensions (Georgopoulos et al.. 1986: Kettner et al.. 1988). At the beginning of this century. researchers launched the field of brain- controlled neuromotor prostheses with the application of the population vector algorithm as well as other methods to extract • Corresponding author. E-mad address: r rem Neared edu (TJ. Bradbern4 ' Present address: Department SCognalive Sciences:University of Calikania. California 92697. USA. 10534119/g - see front matter O 2009 Elsevier Inc. All rights reserved. doi:10 MI6/ iwuroimage.2009.06 023 0 2009 Elsevier Int. All rights reserved. control signals related to hand movement from neural data (Schwartz et al.. 2001). Researchers have demonstrated the ability to decode hand kinematics at the microscale from neuronal signals acquired with microwires or microelectrode arrays seated into small patches of sensorimotor cortical tissue and to use this information to drive a cursor or robotic arm (Wessberg et al.. 2000; Serruya et al.. 2002: Taylor et al.. 2002: Hochberg et al.. 2006: Santhanam et at. 2006: Truccolo et al., 2008: Velliste et al.. 2008: Mulliken et al., 2008). Other intracranial studies have analyzed neural data at the mesoscale with coarser spatial resolution but wider spatial extent from local field potential (LFP) recordings. For example, hand movement direction and two-dimensional trajectories have been decoded from LFPs (Mehring el al.. 2003. 2004; Leuthardt et al.. 2004; Rickert et at. 2005: Scherberger et al., 2.005; Schalk et al., 2007: Pistohl et al., 2008; Sanchez et al., 2008). In the late 1990s, pioneering work on the macroscale began to relate scalp potentials acquired non-invasively to hand movement (Kelso et at, 1998: 0•Suilleabhain et al., 1999). Some recent non- invasive studies have demonstrated the presence of a macroscale network that carries the neural code for detailed hand movement. For instance, hand movement direction has been decoded from electro- encephalography ( EEG) and MEG data (I lam mon et al.. 2008; Walden E FTA_R 1_02036240 EFTA02693161 1692 TJ. Bradbury er at / Numlinage 47 (2009) 1691-1700 et al.. 2008). and hand position and velocity have been decoded from MEG data collected during continuous joystick and trackball move- ments (Georgopoulos et al., 2005: Jerbi et al. 2007). However, with the exception of Hammon et al.. these non-invasive studies have constrained subjects to small finger and wrist movements as opposed to multi-joint drawing or reaching movements. Also, most impor- tantly. the tasks employed for non-invasive decoding of hand position and velocity have not incorporated discrete center-out movements. To examine our hypothesis that hand kinematics of natural. multi- joint, center-out movements are decodable from non-invasive neural signals, we aimed to continuously decode hand velocity from MEG data collected during a two-dimensional drawing task. Currently only invasive studies have continuously decoded hand velocity during discrete center-out movements. Since MEG coupled with our decod- ing method facilitates the ability to examine sensor involvement on a macroscale with high temporal resolution, we also sought to create snapshots of sensor importance in a network covering multiple brain regions across time during adaptation to a hand-cursor rotation. Furthermore, we aimed to examine the importance of estimated current sources in the network using sLORETA to determine whether they corroborated non-decoding visuomotor adaptation studies that employed other imaging modalities like EEG (Contreras-Vidal and Kerick, 2004), positron emission tomography (PET) (Inoue et al.. 2000: Ghilardi et al.. 2000; Krakauer et al.. 2004). and functional magnetic resonance imaging (fMRI) (Graydon et al.. 2005; Seidler et al., 2006). Materials and methods Experimental procedure and data collection The Institutional Review Board of the University of Maryland at College Park approved the following experimental procedure. After giving informed consent, five healthy, right-handed subjects drew center-out lines with an optic pen on a glass panel positioned in front of them while they lay supine with their heads in an MEG recording dewar located inside a magnetically shielded room in the Kanazawa Institute of Technology (KIT)-Maryland MEG laboratory at the University of Maryland (Fig. IA). Cushions were positioned in the dewar and under the right elbow to minimize movement of the head and upper limb respectively. The distance between the glass panel and each subject's head was adjusted for comfort (approxi- mately 35 cm from nose tip to the center of the panel). A black curtain occluded the subjects' vision of their hands while visual feedback was provided on a screen located in front of them that displayed the position of the pen tip as a cursor. Subjects were instructed to position the pen tip in a circle (0.5 cm diameter) located in the middle of the screen, wait for one of four circle targets (03 cm diameter) to appear in the corner of the screen at 45. 135. 225. or 315°. wait for the target to change color, and then draw a straight line to the target as fast as possible. The inter-trial delay was randomized between 2 and 2.5 s. Working space dimensions were a 10/ 10 cm virtual square. After 40 trials ( pre- exposure), the cursor was rotated 60' counterclockwise (exposure). The exposure phase consisted of 240 trials with the early-exposure phase composed of the first 40 trials and the late-exposure phase composed of the last 40 trials. After the exposure phase, the original orientation of the cursor was restored, and 20 more trials were collected and labeled as the post-exposure phase. The number of trials analyzed in the pre-exposure phase was reduced from 40 to 36 because the behavioral performance during several initial trials of some subjects was poor due to lack of familiarization with the task To maintain consistency, the number of trials analyzed in the early- and late-exposure phases was also reduced from 40 to 36. A video camera sampled the movement of the pen tip at 60 Hz. and whole-head MEG data were acquired from 157 channels at a sampling rate of 1 kHz. The MEG system used coaxial type first-order gradiometers with a magnetic field resolution of 4 ft/Hz" or 0.8 (ft/cm)/ Hzu2 in the white noise region. On-line. electronic circuits band-pass and notch-filtered the MEG data from 1-100 Hz and 60 Hz respectively. Adaptation confirmation To quantitatively confirm the occurrence of adaptation. the mean initial directional error (IDE) was calculated across subjects for each phase of the task. A vector from the center location of the screen (home) to the position of the pen at 80 ms after the pen completely left the center circle determined the initial direction of the planned movement trajectory. The IDE was calculated as the angular difference between this vector and a vector extending from the home location to the target. Four separate t-tests were performed between the IDE in pre-exposure and zero. IDE in pre-exposure and early-exposure. IDE in pre-exposure and late-exposure. and IDE in pre-exposure and post- exposure. Signal pre-processing Data from each MEG sensor were first standardized according to Eq. ( I ): S„[t] = salt] gn for all n from I to N ( 1) Slk where S„Iti and s„ItI are respectively the standardized and measured magnetic field strength of sensor n at time r, s, and SD„ are the mean and standard deviation of s„ respectively. and N is the number of sensors. The kinematic data were resampled from 60 Hz to 1 kHz by using a polyphase filter with a factor of 5/3. For computational efficiency. the MEG and kinematic data were then decimated from kHz to 100 Hz by applying a low-pass anti-aliasing filter with a cutoff frequency of 40 Hz and then downsampling. The best decoding results were obtained when both the MEG and kinematic data were subsequently filtered with a zero-phase. fourth-order, low-pass Butterworth filter with a cutoff frequency of 15 Hz. The data for each phase of the task were pre-processed separately. Decoding model In the subsequent analyses. we only considered hand velocity based on our previous work that revealed better decoding of hand velocity than hand position from MEG signals (Brad berry et al.. 2008). To continuously decode hand velocity from the MEG signals, a linear decoding model was used (Fig. 2) (Georgopoulos et al.. 2005): N L xitl — x(t - 11=E t bffir calt — lc] (2) n.1 k =0 Yltl - y[t —11 = E E bniy.S„Ir — kj n-1 t=0 (3) where x(rj and All are the horizontal and vertical position of the pen at time sample r respectively. N is the number of MEG sensors. L is the number of time lags, S„lt — kl is the magnetic field strength measured at MEG sensor n at time lag k and the b variables are coefficients obtained through multiple regression. By varying the number of lags and sensors independently in a step-wise fashion, the optimal number of lags (L= 20. corresponding to 200 ms) and the best sensors (N=62; from central and posterior scalp regions) were determined experimentally. The data for each phase of the task were decoded separately. EFTA_R 1_02036241 EFTA02693162 Tr lbodbeiry er al ' Net:mintage. 47 (2009) lit -1700 1693 A B C • Target Cursor . Home Pre-Exposure Early-Exposure Late-Exposure Post-Exposure 60 40 - a 20 a) V UL,P 0 -20- -40- -60- -80 Late Post Pre Early Phase of Task Fig I.Center-out drawing experimental setupand kinerrutin. :A; in the lust and wwnd p.Invb.asubject is shown lying with his head inside the MEG recording dewar and drawing with an optic pen on a sheet of glass.A black curtain used to ocdude vision of the upper limbs is additionally shown in the second panel. The third panel illuurates the subject's view or the computer screen where visual feedback of the pen position (cursor), center location (home). and peripheral targets was displayed. (B) The superimposed pen (black) and cursor :gray) paths for one representative subject confirmed the occurrence of adaptation. Dissociation between the pen (hand) and cursor (eye) movements due to hand-cursor rotation was evident. (C) The mean SD of the IDE calculated across subjects for each phase of the task further confirmed adaptation. Assessment of decoding accuracy M-fold cross-validation was used to assess the decoding accuracy. In this procedure, the data were divided into m parts (each with approximately 12 s of continuous data, or four trials). m — 1 parts were used for training, and the remaining part was used for testing. The procedure was considered complete when each of them combinations of training and testing data were exhausted. and the mean CC between EFTA_R 1_02036242 EFTA02693163 1694 T.J. Bra*my er Nearofmne 47 (2009) 1691-1700 MEG Sensor I MEG Sensor 62 Time Sensor Weights for X Velocity Sensor Weights for Y Velocity E E — Decoded — Measured 4 Nrr.r. ,r r Velocey Reconstrudion from MEG Data at 4100 ms (snow above) Y velocity Reconstruceon from MEG Oats at 1.100 rre (Maim above) x Velocity Recomuructon from MEG Data from mi b 6200 ms V velocity Recorstrucson from MEG Data from tlf) b 1-200 rra Fig. 2. Didactic model of the linear decoding method. The top raster plot contains time series of 62 MEG sensors extracted 100 ms prior to the current velocity sample of interest. Through multiple linear regression. sensor weights were computed separately for x and y velocity that transformed 11w top raster plot to the lower left and right raster plots. The transformed time series of the sensors were then summed to produce the reconSnucted velocity profiles (red) that overlay the measured velocity profiles (black). The upper velocity profiles are associated with the MEG data shown In the example (100 ms prior to the current velocity sample of interest) and the lower ones with MEG data from0 to 200 ms prior to the anent velocity sample of interest. measured and decoded hand velocity was computed across folds. Prior to computing the CC, the kinematic signals were smoothed with a fourth- order, low-pass Butterworth filter with a cutoff frequency of 0.6 HL Cross- validation was executed with m= 9 for all phases of the task except for post-exposure where m = 5. For Fig. 3B, standardized velocity profiles were computed with I% ' with s, replaced by a velocity profile. Sensor sensitivity curves A curve depicting the relationship between decoding accuracy and the number of sensors was computed for the x and y dimensions of hand velocity for each subject for each phase of the task. A similar method to examine this relationship has been used to analyze neuronal recordings (tiancliti ci al., 20114). First, for each subject and each phase of the task, each sensor was assigned a rank according to I q. '4': `Al R" Ma + 1 /bm^kw - + bmitLY for all n from I to N 4) where R„ is the rank of sensor n and M is the number of folds of the cross-validation procedure. Second, the decoding model was iteratively executed with only the highest-ranked sensor, the four highest-ranked sensors, the seven highest-ranked sensors. etc. until all sensors were used. For each phase of the task the mean SD of the CCs computed across subjects was plotted against the number of sensors. Finally, each plot was fitted to a double-exponential curve. and the coefficient of determination. le. was calculated as a measure of the goodness of fit. Scalp snaps of sensor contributions To graphically assess the relative contributions of scalp regions to the reconstruction of hand velocity, the across-subject means of the b (from Eqs. X and ; .l .) vector magnitude were projected onto a time series ( — 200 to 0 ms in increments of 10 ms) of scalp maps for each phase of the task. These spatial renderings of sensor contributions were produced by the topoplot function of EEG AB version 6.01b, an open-source MATLAB toolbox for electrophysiological data proces- sing (Deli-nine and Maketg. 2004: 'thy wen uccd edu eegial) ), that performs biharmonic spline interpolation of the sensor values before plotting them (Sandwell. 198?). To examine which time lags were the most important for decoding. for each scalp map. the percentage of reconstruction contribution for a phase of the task was computed as N 4)b„ + bfly 2 GTI=100%x faa tN for all i from 0 to L (5) E E Veenk. + No 2 k=0 where %I, is the percentage of reconstruction contribution for a scalp map at time lag i. Comparison of scalp maps across adaptation Right-tailed. paired t-tests determined statistically significant (p<0.05) changes in sensor contributions between phases of the task. Three contrasts between the scalp maps were computed for increases from baseline (pre-exposure): early-exposure - pre-expo- sure. late-exposure - pre-exposure. and post-exposure - pre-expo- sure: and three contrasts were computed for decreases from baseline: pre-exposure - early-exposure. pre-exposure - late-exposure, and pre-exposure - post-exposure. The resultant r scores were converted to z scores and then rendered onto scalp maps with the topoplot function of EEGLAB Mdkelg, 21.11.14) with increases and decreases represented with hot and cool colors respectively. EFTA_R1_02036243 EFTA02693164 T.J. Brodbeny cl ot / NeurfOrnoge 47 (2009) 1691-1700 1695 Conical source localization To better estimate the cortical sources of hand velocity encoding in each phase of the task, we used standardized low-resolution brain electromagnetic tomography (sLORETA) software version 20081104 (Pascual-Marqui. 2002; http:kwww.uzh.ch. keyinst/loretatilm). sLORETA computes instantaneous. 3-D linear, distributed and discrete solutions for the MEG/EEG inverse problem, which compare well with respect to linear inverse algorithms like minimum norm solution. weighted minimum norm solution, and weighted resolution optimi- zation (Pascual-Maryut. 2002). These solutions are computed within a three-shell spherical head model that uses a lead field computed with a boundary element method applied to the MNI52 template (Fuchs et al.. 2002). The head model includes scalp. skull, and brain compart- ments. The brain compartment is restricted to the conical matter of a head model co-registered to the Talairach brain atlas (Talairach and Tournoux. 1988). This compartment includes 6239 voxels at 5 mm resolution with each voxel containing a current dipole representing the integrated activity within the corresponding spatial vicinity. The sensor coordinates of the MEG helmet that were entered into sLORETA had been previously measured in the KIT-Maryland MEG laboratory. To identify sources that were sensitive to velocity encoding, we found the sources that best correlated with the most meaningful sensors from the decoding analysis using the following method. Pre- processed MEG signals from all 157 channels for each subject and each phase of the task were fed to sLORETA to estimate current sources. These MEG signals had been pre-processed in the same manner as for decoding: standardized. downsampled. and low-pass filtered. From the scalp map with the highest percentage of reconstruction contribution ( — 100 ms). the fifteen sensor weights possessing the highest values were selected. The CCs were then computed between the squared time series from the fifteen sensors with the 6239 time series from the sLORETA solutions and averaged across subjects. Each CC was multiplied by the magnitude of the regression weight b (from Fos. (2) and (3)) vector of the sensor in the correlation analysis. The reason that fifteen sensors were chosen for the correlation analysis was because of the observation that the sensor sensitivity curves began to plateau around fifteen sensors (Fig. 4). Next the highest 5% of the CCs (weighted by b) were set to the value one, and the rest of the CCs were set to zero. Finally these binary-thresholded CCs were A • 0.8 T.) 0 8 E 0.6 • 0.4 O (7, ▪ 0.2 • 0 X Velocity =1 9 Velocity Post Ea ly Late Phase of Exposure plotted onto an axial slice of the brain (z = 55 mm) from the Colin27 volume (Holmes et al.. 1998). the MRI template that best illustrated our regions of interest. All reported coordinates of regions of interest are in Talairach space. Results Hand kinematics confirmed adaptation During early-exposure to the cursor rotation, we observed curved hand paths due to the subjects' effort to counteract the imposed rotation (Fig. IB). Hand paths became straighter in late-exposure as subjects adapted to the novel environment. In post-exposure, after-effects, which consisted of hand paths curved in the opposite direction from those in early-exposure, indicated that adaptation had occurred. We also confirmed the occurrence of adaptation quantitatively by computing the mean IDE across subjects for each phase of the task and comparing it between phases (Fig. IC). The IDE was not significantly different from zero in pre-exposure (two-tailed [-test: p =0.34). The IDE increased in early-exposure relative to pre-exposure. decreased in late-exposure relative to early-exposure, and increased again in post-exposure relative to pre-exposure (one-tailed, paired t-tests, p<0.001). MEG signals contained decidable hand velocity information We employed a linear decoding model (Fos. (2) and (3)) to reconstruct the horizontal (x) and vertical (y) velocity components of hand movement from the activity of the MEG sensors (Fig. 2). The mean CC of x velocity decreased during each consecutive phase of the adaptation task (Fig. 3A). Interestingly the mean CC of y velocity increased until post-exposure at which point it drastically decreased. In terms of individual subjects. the mean CC ranged from 023 to 0.56 (Table I), and examples of smoothed, reconstructed hand velocity profiles matched the measured velocity profiles well (Fig. 38). Number of sensors and decoding accuracy were exponentially related The linear decoding model produced one weight per sensor per time lag: therefore, the importance of the contribution of a sensor to the decoding process at a particular time lag could be considered the B r8 2 e 0 .2 el 2 P i tn O O 2 X Velocity for Lete•Exposure a%0 19S44 \r hea-- rocrwegased340S g44\es vair— cz 8 -2 r veiVeAerV A t 8 o 4 e 12 2 n ca 2 ▪ 0 -2 X O 0 a ca -2 t"' 2 a 4 Fat 2N 0 8- o Y Velocity for Late-Exposure .trzatt-IV-N 4 B 12 Time (s) Time (s) Fig. 3. Decoding accuracy for hand velocity. (A) The across-subject mean SD of the CCs between measured and decoded hand velocity profiles it plotted separately for x (horizontal. black) and y (vertical. white) velocity breach phase affix task. (B) Examples of smoothed and standardized measured (black) and decoded (gray) hand velocity profiles for late- exposure exhibited high decoding accur.xy.Tbe left and right columns contain x and y velocity profiles respectively. Each row contains data fora single subject, and the CC between the measured and decoded velocity is listed to the left of each plot. E FTA_R 1_02036244 EFTA02693165 1696 74. Brodberry tr of Neurohnege 47 (2009) 1691-1700 08 06 0 — 0.4 X 0.2 O 0 0 CC for Y Velocity •0 2 0.8 0.6 0.4 02 0 -0 2 1 13 25 37 49 61 ..... • ,.•••• 08 06 0.4 02 0 R2 = 0.95 -0.2 Early-Exposure Late-Exposure Post-Exposure 13 25 37 49 61 1 13 25 37 49 61 1 13 25 37 49 61 mean SD 0.8 ............ 0.6 .......... 0.4 , ...... .. R2 = 0.99 13 25 37 49 61 13 25 37 49 61 Number of Sensors Number of Sensors 0.2 0 -0.2 1 13 25 37 49 61 1 13 25 37 49 61 Number of Sensors Number of Sensors Flg.4. Decoding amirary vs. number of sensors. The top and bottom rows comma of mean s() (gray; of the CCs anon tobycts vs. the number of sensor for x and y velocity respectively. Columns organize the plots by phase of the task. R' values between the mean CC curve and a fitted double-exponential curve are displayed at the bottom of each plot. vector magnitude of its regression weights at that time lag. We ranked the sensors and reran the decoding procedure with the most important sensor, the four most important sensors. the seven most important sensors. etc. until all sensors were used. These sensor sensitivity curves of mean CC vs. the number of sensors fit a double- exponential function well (R2 =0.95-1.00) (Fit:. •1). For all phases of the task the curves peaked then plateaued, or nearly plateaued, near 15 sensors. A macroscale sensorimotor network encoded hand velocity To graphically assess the relative contributions of scalp regions to the reconstruction of hand velocity• we projected the across-subject means of the vector magnitudes of the sensor weights onto a time series (-200 to 0 ms in increments of 10 ms) of scalp maps for each phase of the adaptation task The scalp maps for each phase of the task resembled each other. so only those for pre-exposure are shown (Fig. SA). A network of sensors over central and posterior scalp areas contributed to decoding hand velocity with a salient member of the network over the contralateral motor area. Although the scalp maps of the different phases appeared similar upon visual inspection, we investigated the presence of statistically significant increases and decreases in early-, late-. and post-exposure relative to baseline (pre- exposure). We observed notable focal differences between phases of the task in scalp areas over mediolateral premotor and posterior parietal cortices in particular (Fig. 58). To better estimate the cortical Table 1 Mean and SO (in parentheses) of CCs for each subject during each phase of the visuomotor adaptation task. Pre Early Late Post X Vel Y Vel X Vet Y Vel X Vel Y Vet X Vel Y Vel Subject I 064 (0.09) 0.47 (036) 0.44 (QM 0.62 (0.13) 0.53 (0.13) 0.73 (0.12) 0.10 (021) -0.02 (0.13) Subject 2 0.45 (0.16) 029 (0.14) 0.56 (0.10) OAS (021) 0.40 (0.18) 032 (011) Q10 (007) 0.37 (0.13) Subject 3 0.48 (0.14) 023 (021) 046 (0.16) 053(0.18) 0.49 (0121 0.63 (024) 042 (0.16) 026 (0.14) Subject 4 0.60 (0.08) 0.33 (022) 021 (0.20) 023 (0.11) 021 (0.111) 0.44 (015) 035 (0.07) 0.46 (0.13) Subject 5 0.17 (021) 026 (0.30) 026 (0.13) 0.56 (0.14) 024 (015) 0.47 (022) 0.17 (0.32) 0.02 (0.13) Grand mean 0.48 (0.15) 032(008) 0.40 (0.121 0.49 (0.13) 038 (0.12) 036 (OLIO) 028 (0.17) 023 (0.17) EFTA R1_02036245 EFTA02693166 y. Brndbeny el at I Neurohnoge 47 (7009) let -1700 A 8.6% 10.8% ® • 12.4% -100 ms -110 ny, -120 ms -130 r -70 ms -80 ms -90 ms Pre-exposure 12.8% 12.1% Sensor Contribution B Early -exposure — Pre -exposure • • • • 411 • Late -exposure — Pre-exposure • • • • ® • -70 ins -80 ms C Pre Post -exposure — Pre -exposure • -90 ms • -100 ms • Ito m5 -3 -2 -I 0 1 2 3 z score Early Late 10 3' 8.0 • • • • • • -120ms .130 ms Post Fitt Sensonrnocor networks associated with hand velocity during visumnotor adaptation. :A: Int clas.ehmles oldie scii.ui ‘‘c:);it :meat decoding model revealed the importance of neural regions when interpolated and protected onto a time series ( 200 to 0 ins in increments of 10 msl of scalp maps for the pre-exposure phase (other phases were similar). Light and dark colon represent high and low contributors respectively. The highest sensor weighting of the MEC signals led the velocity output by 100 ms, so the display of scalp maps are centered around — 100 ms. The percentage of reconstruction contribution (kr) is displayed above each scalp map. Due to space limitations, only seven of the twenty-one scalp maps arc shown. (BI The rows respectively contain the z scores of differences between early- and pre-exposure. late- and pre- exposure, and post- and pre-exposure Increased ( t ) and derreased ( ) contributions of sensors are napped to hot and cool colon respectively. IC) The estimated conical sources involved in hand velocity encoding during the task were represented on an axial slice from an MRI template (z= 55). The sources and their Talairach coordinates (x. y. z) were the PrC (-41. — 1145), PoC (-45. —17.55).51'3(30, —46.55), PCu (3. —61. 55),IPI. (-41. — 41.55).5MA (5, — 2.55). MEC (19.18.55 and —24.20.55). and SEC (19.12.55). EFTA_R1_02036246 EFTA02693167 1698 El.. Ona:Chewy et at / NeurOMage 47 (2009) 1691-1700 sources that gave rise to the scalp maps at — 100 ms (the highest percentage of reconstruction contribution), we correlated the fifteen best sensors with the sources estimated by sLORETA. After weighting the CCs by the vector magnitudes of the sensor weights, the top 5% were binary-thresholded and plotted on an axial slice (Fig. 5C). In all phases of the task the contralateral precentral gyrus (PIG) and postcentral gyrus ( PoG) and the ipsilateral superior parietal lobule (SPL) and precuneus (Ku) encoded for hand velocity. The contral- ateral inferior parietal lobule (IPL) and ipsilateral medial frontal gyrus. containing the supplementary motor area (SMA). additionally encoded for hand velocity in all phases except pre-exposure. Finally the lateral premotorconex of the bilateral middle frontal gyrus (MFG) and ipsilateral superior frontal gyrus (SFG) ) were involved in hand velocity encoding only in early- and post-exposure. Discussion Our results demonstrate that we can continuously decode information about hand velocity from natural, multi-joint, center- out movements from MEG signals collected during a drawing task that requires visuomotor adaptation to a hand-cursor rotation. With the systematic addition of sensors to the decoding model, the decoding accuracy exponentially increases before reaching a plateau. Addition- ally, a macroscale sensorimotor network composed of central and posterior scalp regions encodes for hand velocity in all phases of adaptation, and the differences in MEG sensor importance between phases capture the evolution of cortical involvement during adapta- tion. Furthermore, localization of cortical sources permits a more detailed investigation into the conical regions that encode for hand velocity in different adaptation phases. Hand velocity information is represented on multiple spatial scales Researchers have firmly established the existence of a population code for hand position and velocity at the microscale level via neuronal recordings (Georgopoulos et at.. 1986: Kettner et al.. 1988: van Hemmen and Schwartz. 2008). Recently, some electrocortico- graphy (ECoG) studies demonstrated that a population code for these kinematic parameters also exists on a mesoscale (Schalk et al. 2007: Pistohl et al. 2008: Sanchez et al. 2008).The most striking result of our study is that a sensorimotor network on a larger spatial scale encodes hand kinematics during natural, multi-joint center-out movements. and. furthermore, does so during adaptation to a screen cursor-hand rotation. In sensor space, this network spans central and posterior sensor areas. Each MEG sensor reflects the contributions of millions of neurons. but yet, we can still decode information about hand velocity. Further regarding spatial scale, we asked whether a denser sampling of the scalp space could improve decoding accuracy. Since the curves of mean CC vs. the number of sensors reveal there to bean optimal, or near optimal, number of sensors less than 62 for all phases of the task (Fig. 4), we conclude that the addition of more sensors would not substantially improve the decoding accuracy. The decreased mean decoding accuracy and increased SD of the CC during post-exposure is likely due to the relatively small amount of data collected and analyzed during this phase of the task The overall increased mean decoding accuracy of y velocity during adaptation was potentially due to the fact that, during exposure, the 60-degree rotation had a greater affect on hand movement in they direction than the x direction, and thus may have recruited more neural resources to handle the y direction (Contreras-Vidal and Kerick, 2004). Several interesting pieces of evidence serve to validate the interpretation of our decoding results. First. the greatest sensor contributions across time lags occur at 100 ms prior to the current kinematic sample under reconstruction for all phases of the task (Fig. SA). Given that prior research has established approximately 100 ms of neural data in the past to be important for planning the current movement (Mehring et al.. 2004: Paninski et al., 2003), this finding is not unexpected. In our previous report leading up to this study (Bradberry et al.. 2008). we discovered that hand velocity was better decoded than position (post-publication analysis: two-tailed, paired t-test; p<0.0001). This is another confirmatory finding, given that the motor cortex represents velocity better than position as has been demonstrated, in particular, by studies aimed at decoding kinematic parameters for neuroprosthetic control (Schwartz et al.. 2001). Furthermore, the salient region of high activation over the left motor area is expected since subjects drew with their right hands. Regional comparison to non-decoding studies of visuomotor adaptation In sensor space, across adaptation we find significant contributions to hand velocity decoding over the mediolateral premotor and posterior parietal scalp areas with respect to pre-exposure (Fig. 58). Previous studies demonstrated that the parietal and premotor cortices are involved in a visuomotor network for reaching (Wise et al.. 1997; Burnod et al.. 1999), and an EEG study of visuomotor adaptation reported fronto-parietal shifts (Contreras-Vidal and Kerick. 2004). To speak more specifically about the conical areas involved with visuomotor adaptation and encoding of hand kinematics, we performed source localization (Fig. 5C). Multiple similarities exist between the conical regions found in our study and those of fMRI and PET studies of visuomotor adaptation. The left PrG. PoG. and IPL have been shown to be involved during visuomotor adaptation to a rotation of visual feedback by a IMRI studies by Graydon et al. (2005) and Seidler et al. (2006;. In KT studies, the right SPL has been observed to increase in activation during visuomotor adaptation tasks by loom. et al. (2000). Ghilardi et al. (20001, and Krakauer et al. (2004). Inoue et al.. Krakauer et al„ and Seidler et al. have also revealed an increase in activation of SMA/preSMA during visuomotor adapta- tion. Finally the MFG and SFG (lateral premotor cortex) have been shown to be active in visuomotor adaptation by Inoue et al. and Seidler et al. Regional comparison to other decoding studies Regarding decoding of hand kinematics. the common involvement across tasks of the PrG. PoG, Sit and PCu implies that these areas form the core for hand velocity encoding in familiar and unfamiliar environments while the SMA, lateral premotor cortex. and IPL encode for hand velocity only during adaptation. Decoding of hand kinematics has been reported for PrG and PoG at a microscale (Georgopoulos et 4..1986; Moran and Schwartz. 1999; Wessberg et al.. 2000; Serruya et al.. 2002; Schwartz et al., 2004), mesoscale (Schalk et al.. 2007; Pistohl et al. 2008: Sanchez et al. 2008). and macroscale (Jerbi et al., 2007). This decoding role has also been ascribed to the SPL at the microscale (Averbeck et al., 2005. 2009: Mulliken et al., 2008) and macroscale (Jerbi et al., 2007). The SMA/preSMA. lateral premotor cortex, and IPL have also been observed to encode movement kinematics (Moran and Schwartz. 1999; Schwartz et al.. 2004; Jerbi et al., 2007: Tankus et al.. 2009). On a slightly different note, a PET study that examined the control of movement velocity, discovered the involvement of left PrG, left PoG. right SPI... and mediolateral premotor cortex (turner et al., 1998). To our knowledge, we are the first to report that the PCu plays a role in the encoding of detailed hand kinematics. Could eye movements haw inadvertently aided hand velocity decoding? Unintended contributions of eye movements to the decoding of hand movement is a potential confound in all MEG. EEG, and ECoG studies, including our study. We did not experimentally control eye movements; however, there is reason to conclude that they do not subvert our interpretations. In an ancillary analysis ( Sul) *mental)? EFTA_R1_02036247 EFTA02693168 T). Bradbury el al 1 Neon:4mage 47 (2009) 1691-1700 1699 Mciliocts), we ran our decoding method with the same central and posterior sensors after removing ocular, muscular, and cardiac artifacts with a method based on independent component analysis (ICA) (Ring and Cunt re p as-Vida I. 2005). Although there was a notable drop in decoding accuracy (or y velocity in pre- and post-exposure. there was no statistically significant difference in the resultant mean CCs of the subjects for any phase of the task (two-tailed, paired t-test: p >0.05) ( I.ihle Si ). Potential application to neuromotor prosthetic control Most studies involving non-invasive BCI systems have focused on 1) the classification of mental tasks to form a low bandwidth communication channel (rfuri‘cheller et al.. :t0061 Mellinger et al_ 2007) or 2) continuous control of a cursor by subjects who, through relatively lengthy biofeedback training, learn to modulate the power of one or more frequency bands of neural signals to control one or more dimensions of cursor movement (Wolpaw and Nit loilantl, 2004: Mcfai land er al . 2008). The lack of focus on decoding detailed kinematics of natural hand movements could be partly due to the unfounded presumption that this information cannot be decoded from non-invasive signals recorded from the scalp (I theclev awl Nit ()tells. 2001;). Despite this presumption, there exist several important exceptions to the lack of non-invasive studies aimed at developing decoding methods for controlling neuromotor prostheses. One study has decoded continuous joystick coordinates from MEG signals acquired during continuous pentagon drawing in the absence of visual feedback of movement (cum ponlos et al., 2011')). and another study has decoded information regarding hand tangential velocity from MEG signals acquired during trackball movements in two dimensions (lei In et al.. 2007). Our study primarily differs from the two aforementioned studies in that we decode continuous hand velocity from multi-joint move- ments during a center-out drawing task that requires adaptation to a novel screen-cursor rotation. The center-out nature of our task is meaningful because it allows comparison to invasive decoding studies for neuromotor prostheses and emphasizes a desired function of the first generation of these devices. In terms of the visuomotor adaptation component. further investigation may pro- vide insight into how the brain adapts to a tool such as a neuromotor prosthesis (I atedev el al.. :'005), and, hence, poten- tially advance the understanding of how to achieve efficient co- adaptation of the brain and decoding model. On a final comparative note, we ran each iteration of our decoding model with a relatively small set of training data composed of 16 (post-exposure) to 32 (pre-. early-, and late-exposure) trials. This small amount of training data is meaningful because it may translate to a substantial reduction in the time required for a patient to gain mastery over the control of a neuromotor prosthesis. What remains to be elucidated is whether the decoding method presented in this report will also be applicable to EEG. which is better suited than MEG for an ambulatory prosthetic system. In terms of EEG-based decoding of movement parameters, several recent studies have decoded the direction of hand movement ( I iantinon et al., 2008. Waldo. I et al.. 200S), but, to our knowledge. researchers have yet to report successful decoding of continuous hand position or velocity from EEG (a comprehensive search in peer-reviewed journals did not produce any studies). In the future. we will apply our decoding method to EEG signals to examine the application of this non-invasive modality to continuous, complex control of a neuromotor prosthesis. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version. at dor 10 In I ti 1. nett, onnage.2009.06 02 ). References Averbeck B.B.. Chafer. M.V_ Crowe. DA. Georgopoulos, AP., 2005. Parietal representa- tion of hand velocity in a copy talk. J, Neurophysiol. 93.508-518. Averbeck. BJL Crave. DA. Chake. Georgopoulos, Al'. 2009. Differential contribution of superior parietal and dorsal-lateral prefrontal conker in coning Cortex 45.432-441. Bradbury. T.J.. Contreras-VidaL J.l.. Rong. F.. 2008. Decoding hand and cursor kinematics from magncioencephalographic signals during tool use. Conf. Proc. IEEE Eng. Bled. Biol. Soc. 2008. 5306-5309. Burned. Y.. Baraduc. P.. Banaglia-Mayer. A. Guigon. E.. Koechlin, L Ferraina. 5.. Lacquaniti. F, Caminiti, R. 1999. Parietofrontal coding of reaching: an integrated framework. Exp. Brain Res. 129. 325-346. Contreras-Vidal. J.L Kerick. S.E.. 2004. Independent component analysis of dynamic brain responses during visuornotor adaptation. Neuroimage 21.936-945. Delorme. A.. Matteis, 5.. 2004. EEGLAB: an open source toolbox for analysis of single. trial EEC dynamics including independent component analysis. J. Neurosci. Methods 134.9-21. Fuchs. M.. Kastner. J. Wagner. M. Hawn. S.. Ebersole, JS. 2002. A standardized boundary element method volume conductor model. Clin. Neurophysiol. 113.702-712. Georgopoulos. AP, Schwan:. A.B.. Kennet. R.E. 1986. Neuronal population coding of movement direction. Science 233.1416-1419. Georgopoulos. AP.. Langheim. FJ.. Leuthold. AC Mende. AN. 2005. Magnetoence- phalographk signals predict movement trajectory in space. Exp. Bram Res. 167. 132-135. Ghilardi. M.. Chez. C.. Dhawan. V.. Moeller. J. Menus. M_ Nakamura. T. Antonini. A.. Eidelberg. 0, 2000. Patterns of regional brain activation associated with different forms of motor laming. Brain Res.871.127-145. Guaydon. FX. ninon. KJ.. Thomas. CA.. Brooks. V.B. Meson. RS.. 2005. Learning- related (MRI activation associated with a rotational visuo-motor transformation. Brain Res. Cugn. Brain Res. 22. 373-383. Hammon. PS.. Makmg. S.. Poimer.n.. Todorov. E. de Sa, Vt. 2008. Predicting reaching targets from human EEG. IEEE Signal. Proc. Mag. 25.69-77. Hochberg Lt. Serruya. M.D.. Friehs.GAL Mukand. JA. Saleh. fit, Caplan. AIL Branner. A.. Chen. D.. Penn. to, Donoghue. J.P., 2006. Neuronal ensemble control of prosthetic devices by a human with tetrapkgia. Nature 442.164-171. Holmes, C.J., Ibge, R.. Collins, L. Woods. It, Toga, A.W., Evans, AC, 1998. Enhancement of MR images using registration for signal averaging. J. Comps. Assist. Tomogr. 22. 324-333. Inoue. K., Kawashlma.R. Satoh. K., Kinomum. 5, Sugiura. M.. C040, R. Ito, M. Fukuda.II, 2000. A PET study of visuomotor learning under optical rotation. Neuroimage It. 505-516. K. Lichaux.J.P.NOuye. P.11114ZiS, D. Leahy, RM.. Camera. L. Bailin. S. 2007. Coherent neural representation of hand speed in humans revealed by MEG imaging. Proc. Nail Acad. Sri. USA 104.7676-7681. Kelso. JA. Fuchs. A. Lancaster. IL Holroyd. Cheyne. D.. Weinberg. H..1998. Dynamic Cortical activity in the human brain reveals motor equivalence. Nature 392. 814-818. Kenner, R.E. Schwan:. A.B.. Georgopoulos. AP.. 1988. Primate motor cortex and free arm movements to visual targets in three-dimensional space. III. Positional gradients and population coding of movement direction from various movement origins. J. Neurosci. 8.2938-2947. Krakauer, J.W. Chilardi, M.F., Monis. M.. Barnes, A. linesman. K. Eidelberg, D. Ghee. C.. 2004. Differential cortical and subcottical activations in learning rotations and gains for reaching: a PET study. J. Neurophysiol. 2.924-933. lebedev. M.A., Nicolehs, MA.2006. Bram-machine interfaces: past, present and future. Trends Neurosci. 29.536-546. tebedev. MA. Camiena. J.M.. °Doherty. J.E. Zacksenhouse. M. Henriques. CS.. Principe. J.C.. Nicolelts. M.A.. 2005. Cortical ensemble adaptation to represent velocity of an anificial actuator controlled by a brain-machine interface. J. Neurosci. 25. 4681-4693. Leuthardt. E.C.. Schalk. G.. Wolpaw. J.R_ Ojemann. J.G.. Moran. U.W. 2004. A brain- computer interface using electroconicographic signals in humans. J. Neural. Eng. I. 63-71. McFarland. DJ.. Knisienski, D.J.. Samacki. WA. Wolpaw. J.K. 2008. Emulation of computer mouse control with a noninvasive brain-computer interface. J. Neural. Eng 5.101-110. Mehring, C.. Rickert. Vaadia. E.. Cardosa de Oliveira S.. Aensen. A.. Boiler. S.. 2003. Inference of hand movements from local field potentials in monkey motor cortex. Nat. Neurosci. 6. 1253-1254. Metering C. Nawrot. M.P.. Cardosa de Oliveira. 5.. Vaadia. E. Schulte.Bonhage. A.. nensen.A.. Ball, T., 2004.comparing information about arm movement direction in single channels of local and epiconical field potentials from monkey and human motor toner.. I. Physiol Paris 98.498-506. Mellinger, J.. Schalk. C.. Braun. C.. Mersa It, Rosenstiel, Birbaumer. N.. KObkr. A.. 2007. An MEG-based brain-computer interface (80). Neuroimage 36. 581-593. Moran. D.W. Schwartz. A.B. 1999. Motor conical activity during drawing move- ments: population representation during spiral tracing. J. NeurophysioL 82. 2693-2704. Mulliken. GIL htusallam.i. Andersen. RA, 2008. Decoding tratenohes from posterior parietal cortex ensembles. J. Neurosci. 28,12913-12926. O'Suilkabhain. P.E. Lagetlund. TD.. Matsumoto. J.Y.. 1999. Conical potentials at the frequency of absolute wrist velocity become phase-locked during slow sinusoidal tracking movements. Exp. Brain Res. 126.529-535. Paninsb. L. Fellows, MR, Hatsopoulos, N.G.. Donoghue, J.P., 2003. Spate/temporal tuning of mowconical neurons for hand position and velocity. J. Neurophysiol. 91. 515-532. EFTA_R1_02036248 EFTA02693169 1700 TJ. Bradbury er at / Newolmage 47 (2009) 169/-1700 Pascual-Margot RD.. 2002. Standardized low resolution brain electromagnetic tomo- graphy (sLORETA): technical details. Methods Ind. Exp.Clin Pturmacol. 24 (Suppi. D1.5-12. Pfunscheller. G.. Brunner. C.. Schltml. A.. Lopes da Alva, F.H.. 2006. Mu rhythm (del synchronization and EEG single-trial classilkation of different motor imagery tasks. Neuroimage 31.153-159. Pistohl. T. Ball. T.. Schulze-Bonhage. A.. Aensen.A. Mehsing, C., 2008. Prediction of arm movement trajectories from ECoG-recordings in humans. J. Neuroses. Methods 167. 105-114. Ricked. J- Cardosa de Oliveira. S. Vaadia. Aertsen, A. Rater. S., Mehring. C., 2005. Encoding of movement direction in different frequency ranges of motor conical local field potentials J. Neurosci. 25. 8815-8824. Ron& F.. Contreras.Vidat J.1_, 2006. Magnetoencephalographic artifact identification and automatic removal based on independent component analysis and categoriza- tion approaches. J. Wormer. Methods 157. 337-354. Sanchez. J.C.. [amen& J.M.. Lebedev. Nicolelis. MA. Harris. J.G., Principe. Jr.. 2004. Ascertaining the importance of neurons to develop better brain-machine interfaces. IEEE Trans. Bloated. Eng. 51.943-953. Sanchez, J.C.. Condor. A. Carney. PA Principe. J.C. 2008. Extraction and localization of mesoscopic motor control signals for human ECoG neuroprosthetics. J. Neurosci. Methods 167. 63-81. Sandwell. DX_ 1987. Biliarrrionic spline interpolation of GEOS-3 and SEASAT altimeter data Geophys. Res. lett. 2.139-142. Santhartort G, Ryu, S.L., Yu. B.M. Afshar. A. Shenoy K.V. 2006. A high-performance brain-computer interface. Nature 442.195-198. Schalk. C, Kubanek. J. Miller. KJ.. Anderson. N.R. Leuthardt. EC.. Ojemann. J.G. Limbrick. IX. Moran. D.. Gerhardt. U. Wolpaw. J.R. 2007. Decoding two- dimensional movement trajectories using ekrtrocorticographic signals in humans J. Neural. Eng. 4. 264-275. Scherberger. H.. Jarvis. M.R. Andersen. RA. 2005. Conical local field potential encodes movement intentions in the posterior parietal cortex. Neuron 46.347-354. Schwartz. Alt.. Taylor. D.M., Helms Illlery. Si., 2001. Extraction algorithms for cortical control of arm prosthetics. Curt. Opin. Neurobiol. 11. 701-707. Schwartz. A.B.. Moran. Reina. GA. 2004. Differential representation of perception and action in the frontal cortex. Science 303.380-383. Scott SM.. 2008. Inconvenient truths about neural processing in primary motor cortex. J. Physic4. 5.1217-1224. Seidler. RD_ Noll. U.C.. Chintalapati. P. 2006. Bilateral basal ganglia activation associated with sensorinxitor adaptation. Exp. Brain Res 175. 544-555. Sentry& M.D., Ilatsopoulos. N.G., Paninski. t. FCROWN, MIL Donoghue. J.P., 2002. Instant neural control of a movement signal. Nature 416.141-142. Talairach. J.. Toumoux. P 1988. Coplanar Stereotaxic Atlas of the Human Brain: 1. Dimensional Proportional System — An Approach to Cerebral Imaging. fluent Medical Publishers, New York. Tankus A. Yeshurun, V.. Flash. T.. Fried. L. 2009. Encoding of speed and direction of movement in the human supplementary motor area. Neurosurg. 110.1304-1316. Taylor. D.M.. Helms Tillery. St Schwartz. AIL 2002. Direct cortical control of 3D neuroprosthetic devices. Science 296.1829-1832. Truccolo. W. Erwhs, G.ht Donoghue. J.P.. Hochberg. tit. 2008. Primary motor cortex tuning to intended movement kinematics in humans with tetraplegia. J. Neurosci. 28,1163-1178. Turner. RS.. Grafton. 5J.. Votaw, J.R. Delong. AIR_ Hoffman. J.M., 1998. Motor subcircuits mediating the control of movement velocity: a PET study. J. Neurophysiol. 80 2162-2176. van Hemmer,. J.L. Schwartz. At. 2008. Population vector code: a geometric universal as actuator Biol. Cybern. 98.509-518. Velliste. M. Pereki Spalding. MC_ Whitford. AS. Schwartz. A.B.2008. Cortical control of a prosthetic arm for self-feeding. Nature 453.1098-1101. Waldeit S. Preisst. It, Derma& E., Braun, C. Birbaunwr. N.. batten, A. Mehring C.. 2008. Hand movement direction decoded from MEG and EEG. J. Neurosci. 28. 1000-1008. Wessberg, J.. Stambaugh. C.R.. Ktalik, J.D.. Beck PD. 'attract). M.. Chapin. J.K., Kim J.. Biggs. Srinivasan. MA. Nicolelis. M.A.. 2000. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408. 361-365. Wise. Si'.. Boossaoud. D.. Johnson. R..1997. Premotor and parietal cortex: conicocortical connectivity and combinatorial computations. Annu. Rev. Neurosci. 20 25-42. Wolpaw. J.R. McFarland. D.J. 2001. Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Roc. Natl. Acad. Set U S. A 101. 17849-17854. EFTA_R1_02036249 EFTA02693170