REPORTS

REPORTS abwdance (3). We normalized the AD values to 'fre- quency' values by referring the AD values on each chip to a calibration curve constructed from the AD values for the 11 control transaipts with known abundances that were spiked into each hybridization (9). This 'fre- quency normalization" allowed comparison of tran- script measurements across multiple way experiments. Frequency values for each gene %we expressed in num- ber concentrations (transcripts per million or ppei) under the assumptions described (9). 13. P. Tamayo et at, Proc. Nett Aced. Sc). U.SA. 96, 2907 (1999). 14. Supplemental information is available at www sciencernagorg/feature/data/1053496.shl 15, L L Johnstone. J. D. Barry, EM8O). 15. 3633 (1996). 16. M. C Costanzo et at. Nucleic Acids Res. 28, 73 (2000 The Proteome WormPD database is available at 17. C. E. Rocheleau et at, Cell 90, 707 (1997). 18. P. W. Carter*, J. M. Roos, K. J. Kemphues, Mot. Gen. Genet. 221, 72 (1990). 19. Additional details are available as supplemental in- formation (II). 20. J. H. Xin, 8. P. Brandhont, R. J. Britten, E. H. Davidson, Dev. Slat. 89. 527 (1982). 21. S. A. Cherritz et at, Science 282, 2022 (1998). 22. G. M. Rubin et at, Science 287, 2204 (2003). 23. To classify ?KOMI genes, we used the sequence com- parison results of Rubin et at (22). In the period between the date of our chip design (early 1999) and the date of this more recent sequence comparison (early 2000). some of the worm ORFs that had been included in the chip designs had been deleted or altered by worm sequence curators (for details on the °Nation of the Wcempep database, see http://vemy sangersc.uk/Projects/C_elegans/wcempept). To rec- oncile worm ORFs in the sequence comparison with worm ORFs on the arrays, we limited our analysis to worm ORFs on the arrays that were unchanged be- tween the two data sets, using creation lists main- tained at the Sanger Centre (the Wonnpephistory file). Thus, we identified on the arrays 2905 worm genes shared between yeast, worm and fly (core genes); 3150 warn genes shared between worm and fly, but not yeast (animal genes); and 8741 worm genes that were unique to the worm (worm genes). Song Replay During Sleep and Computational Rules for Sensorimotor Vocal Learning Amish S. Dave and Daniel Margoliash. Songbirds learn a correspondence between vocal-motor output and auditory feedback during development. For neurons in a motor cortex analog of adult zebra finches, we show that the timing and structure of activity elicited by the playback of song during sleep matches activity during daytime singing. The motor activity leads syllables, and the matching sensory response depends on a sequence of typically up to three of the preceding syllables. Thus. sensori- motor correspondence is reflected in temporally precise activity patterns of single neurons that use long sensory memories to predict syllable sequences. Additionally, "spontaneous" activity of these neurons during sleep matches their sensorimotor activity, a form of song "replay." These data suggest a model whereby sensorimotor correspondences are stored during singing but do not modify behavior, and off-line comparison (e.g., during sleep) of rehearsed motor output and predicted sensory feedback is used to adaptively shape motor output. In reinforcement learning, systems learn through interaction with the environment by trying to optimize some measure of perfor- mance. Biological systems may experience a substantial delay between prentotor activity and assessment of performance through sen- sory feedback (I). This delay poses the prob- lem of how to reward or punish a premotor circuit when that circuit is participating in a different task by the time the reward or pun- ishment is computed. Reinforcement learning is further complicated in systems such as vocal learning, where the mapping of sensory feedback (fundamentally represented as fre- quency versus time) onto motor output (mus- Department of Organismal Biology and Anatomy, University of Chicago, 1027 East 57 Street, Chicago, IL 60637, USA. • To whom correspondence should be addressed. E- mail: dantebigbird.udskago.edu cle dynamics) is of high dimensionality (a many-to-many dynamic mapping). Methods developed in the field of machine learning solve the problem of reinforcement learning with delayed reward (2), and a variety of biological solutions have been proposed to the problem of learning sequences of actions (3). Here, we report on neuronal data that represent a solution to the problem of senso- rimotor mapping in the bird vocal-motor ("song") system. The physiological proper- ties observed during sleep also suggest an algorithmic implementation for reinforce- ment learning of song. Zebra finch songs are organized hierarchi- cally, with one or more notes composing a syllable, and sequences of syllables forming a motif, which are repeated to form song. We investigated neurons in the forebrain nucleus robustus archistriatalis (RA), whose descend- ing projections represent the output of the Shared worm genes were defined as those that had significant homology to one or more genes in yeast or fly (BLASTP expectation values E < 1 x 10 bk, as described by Rubin et at) Discrepancies between these numbers of ORFs and temperable counts in Rubin et at are accounted for by the facts (I) that 2% of worm ORFs were not monitored directly by the arrays and (ii) that to be conservative, we excluded from consideration ORFs that had either been delet- ed or altered in the Worrnpep database in the interval between the chip design and the sequence compari- son of Rubin et at. 24. We thank M. Whitley fee advice on array experi- ments; K. Griffiths fee array bioinformatics support E. Wilson A. Velasco, and H. Horton for technical as- sistance; J. Freeman for bioinformatics support during the chip design protest G. Sherlock for sharing yeast and worm sequence comparison dattc 11. Yandelt for sharing yeast, fly, and worm sequence comparison data; and D. Moots for providing purified oocytes. 26 June 2003; accepted 29 September 2000 forebrain song system. During singing, RA neurons exhibit short bursts of activity, whose identity varies with the note that im- mediately follows the burst (4). In awake birds, outside the context of vocalizations, RA neurons are regularly firing. RA neurons also prominently burst "spontaneously" and respond to sounds, but only during sleep (5). With the goal of comparing motor, auditory, and ongoing bursting activity, we recorded single neurons in the RA of singing male zebra finches, permitted the animals to fall asleep by turning off the lights, and then tested the sante neurons' sensory and ongoing discharge properties (6. 7). The spiking patterns of RA neurons in singing birds consisted of phasic patterns of premotor excitation superimposed over a background of profound inhibition (4) (Fig. I, B and C). This premotor activity was virtually invariant for multiple occurrences of the same sound. After the lights were turned off, RA auditory responses were initially weak but gained strength with time, reflect- ing the gradual transition into sleep (5). Re- sponses to playback of the bird's own song (BOS) also consisted of phasic patterns of excitation separated by inhibition that were similar for multiple occurrences of the same sound, differing mainly in the strength of response rather than pattern (8). The timing of auditory responses to the BOS was very well aligned to the timing of premotor activity (Fig. IF). The only excep- tions were instances of silence following the end of a motif or the end of song, where the auditory response could include an additional burst that corresponded with the syllable that would have followed if the song had continued without pause. To compare motor and auditory activity, we analyzed the singing-related activ- ity surrounding each syllable of sang (4, 9). The spike patterns from the response to the BOS playback were then compared with the spike patterns from premotor activity derived from 812 27 OCTOBER 2000 VOL 290 SCIENCE www.sciencemagorg EFTA01075547 REPORTS Fig. 1. (A) Schematic of the song system. Audi- A tory and premotor activity converge onto the HVc The HVc projects directly to the RA, which projects to brainstem motor centers. The HVc also projects to area X, which projects to the DIM. The DIM projects to the IMAN, which projects to the RA. Feedback loops arise from the RA and IMAN. (B and C) Activity of RA single neurons during singing is premotor (I.e., neuronal activity leads syllables). Spectrographs of the sound that the bird produced are shown in a color scale (frequency, 0 to 10 kHz, is on the ordinate; time is on the abscissa). Corresponding raw traces of the neural activity (amplitude ver- sus time) are shown below the spectrographs. Data from one neuron for two similar examples of singing are shown in (B). The neuron's activity patterns are the same for the two examples of singing, except where the vocalizations differ, and the difference in neuronal discharge (at arrow) precedes the difference in the vocaliza- tions. Both sequences of vocalizations occurred frequently; the neuronal pattern associated with each sequence was stereotyped. In (C) [different neuron, same bird as in (8)1, the bird produced syllable "C (marked by arrows) twice (each syllable is identified by a letter), with the song ending prematurely after the second occurrence. The neuronal discharge following the second C was affected. Activity during calling (not shown) also clearly demonstrates that RA activity leads vocalizations [see also (4)). (D) Cross-correlation of auditory and motor activity for the first neu- ron shown in (E) (positive time shifts imply that auditory response lags premotor activity). (E) Examples of the match between auditory and motor activity from one neuron in each of two birds. The spectrographs show the BOS used as stimuli during playback experiments, and each syllable is identified by a letter. The rasters marked "Aud." represent the neuron's auditory response during playback while the birds were asleep. The rasters marked "Mot." were con- structed from neuronal activity of the same neurons during singing (4). The correspondence between the two patterns of activity is visually striking. In contrast to singing, however, during song playback, neurons exhibited ongoing discharge, not inhibition, for some syllables. (F) Example of raw traces showing the match between activity during playback and singing [same neuron as in (B)). C IIII I IIIIIII • E I A B C DE FGH A B C 11411r A I C 000ms F 500ms IA B COE 1IK B 200ms D 0.6 41 111. 1 I I ii 0.5 4 OA 0.3 01 I I / 1 4 t ill 0.2 -60 0 50 100 -100 Auditory Lag (ms) 200ms AN 8 I A B C DE FGH h I...1 32.s La -A 6 At I s all I -C o a . s . A a s s r a i r s a l l a e l a . . . . . - a . s a t u a l I. 1 -E la ` 1 200 ms Target Syllable 1 1 5 1 6 6 2 7 6 6 7 8 9 1 1 3 2 4 Fig. 2. Deletion experiments. (A) The bird was 7 presented songs during sleep. Below the spec- trograph of the last motif of the bird's song are nine histograms of the response of one neuron, D 7 representing 30 repetitions each of nine differ- ent stimuli. The BOS histogram is the response E 8 to the unaltered motif. For each of the eight other stimuli, one of the syllables from A to H was replaced with background noise. For syllable 9 F, for example, the neuron responded with two bursts, with both bursts occurring during syllable F. The first burst (but not the second) is statis- tically significantly reduced (8) by the elimina- tion of syllables C, D. E. or F. The second burst is affected by the elimination of syllable F. The burst at syllable H is affected by eliminating syllables F, G, and H. (B) in one bird with the most complex song, 10 neurons were tested with deletion stimuli. Each cell of the matrix gives the number of neurons in which the deletion of a syllable (specified by the column) significantly altered the response during the target syllable (specified by the row). The last syllable of the song, H, was excluded because appropriate control data were unavailable (an earlier syllable had always been deleted). The matrix diagonal represents the effect of deleting a syllable on the neuronal response during that same syllable. The numbers to the right of the diagonal are the number of neurons for which there were a statistically significant response during the target syllable. It can be seen that the deletion of a syllable commonly affected the neuronal response several syllables later. For example, of eight neurons responding to syllable E, the response was suppressed for one, six. seven, and six neurons when syllables B, C, D, and E, respectively, were deleted. F G • 8 A B C D E F G Deleted Sy table wwwsciencemag.org SCIENCE VOL 290 27 OCTOBER 2000 813 EFTA01075548 REPORTS the corresponding syllables, showing that the timing of excitation and inhibition during audi- tory stimulation was well aligned to such tinting during singing (Fig. I, D and E). A cross- correlation procedure revealed a strong, signif- icant (P < 0.02) correlation (10) between pre- motor and sensory spike patterns in all 17 neu- rons (from three birds) (mean normalized peak correlation = 0.49 -1- 0.13 SD). Thus, sensori- motor transformations in the song system result in a correspondence between temporally pre- cise sensory and motor activity observed at the level of individual cells. A B The auditory activity was only slightly delayed in relation to motor activity (by 8 -± 2 ms; range, 4 to 13 nu). Because premotor activity in RA can lead the onset of syllables by up to —40 ms (4), this was surprising and suggested that the sensory patterns represent- ing subsequent syllables were generated by responses to previous syllables. To character- ize the extent of temporal integration in the auditory responses, we presented stimuli in which a syllable chosen at random front the final motif of the BOS was substituted by a background of equal duration, and we as- Set Sate tAllk C D E F G H l ilt C D E F h - -Rd 4 t NV Ili l 1 100ms AS OW ire, ims Ow- A B C A B I ill-IrkRh ,4-11i1\ \w l l d -1-41I ,4111 11 Ilil Mil 100m8 Fig. 3. Neuronal replay during undisturbed sleep. (A) Raw traces of neuronal activity (900 ms) during sleep ("Spoil in two different neurons for one bird. For each sample, a representative corresponding sample of premotor activity ("Mot.") and a color spectrograph of the song that the bird sang are shown. (B) Raw traces (1400 ms) of simultaneous recordings from two neurons (-400 µm apart) in another bird. (The second neuron's activity is visible in the background of the first neuron's signal, an artifact of the pairing of signals used to achieve differential recordings resistant to movement-induced artifacts.) Both neurons simultaneously burst during sleep, with complex burst structures that match premotor activity. Apparent temporal expansion (first motif: A, B, and C) and compression (second motif: A, B, and C) is highlighted by the blue lines. This phenomenon has also been reported in population activity of hippocampal neurons (23). sensed the effect on the neuronal activity during the same or subsequent syllables (11). The deletion of a syllable substantially re- duced the neuronal activity occurring one to three syllables later (Fig. 2A), up to —250 ins (8). This property was ubiquitous for all RA neurons that were auditory (14 neurons from three birds) (12) (Fig. 2B). These response properties are suggestive of temporal combi- nation sensitivity, in which a sensory neu- ron's response is nonlinearly dependent on the temporal sequence of preceding syllables. Such responses have also been described for neurons in the nucleus HVc, which projects to the RA (13). Thus, in the RA as well as in the liVc (4, 13), the integration time of indi- vidual neurons appears to be considerably greater when in the sensory (auditory) state than during singing. Given the alignment of auditory and motor activity in the RA, one way of interpreting these results is that audi- tory responses to song syllables represent a prediction of subsequent premotor activity. We also searched for similarities between ongoing bursting activity during sleep (5) and the sensorimotor patterns of RA neurons. For each cell, a visual inspection of samples of activity from long stretches (15 to 60 min) of undisturbed sleep identified repeated exam- ples of one or more complex burst patterns, suggestive of the patterns that we had ob- served in the cell's pre: motor activity. To quantify this match, we developed a proce- dure to automate burst detection (14), con- sidering only bursts of eight spikes or more to ease the computational burden and to allow for statistical analysis. By this procedure, 7.1 ± 5.3% of all spikes (14 neurons from three birds) occurred in bursts, an average of 175.4 ± 144.6 (range, 38 to 581) bursts per cell. For each cell, a measure of similarity between each burst and the single longest bout of the cell's premotor activity (4 to 8 s, consisting of several motifs or songs) was computed and tested for significance (15, 16). The results showed that 15.3 ± 6.5% of bursts (range, 2.6 to 26.8%) significantly matched prentotor activity. Only the cell with 2.6% matching bursts failed to exceed the 5% level expected by chance (16). Examples of matches between longer sequences of com- plex ongoing bursts and premotor activity were particularly compelling (Fig. 3A). In an exceptional case when two RA neurons were recorded simultaneously from different elec- trodes during sleep, both neurons commonly exhibited simultaneous bursting, with the dif- ferent burst patterns for each neuron corre- sponding to the same sequences of syllables (Fig. 3B). This suggests that populations of RA neurons burst in a coordinated fashion during sleep. Bursts (and matching bursts) preferentially occurred during periods when the rate of ongoing discharge was lower and more variable (Fig. 4). Such modulation may 814 27 OCTOBER 2000 VOL 290 SCIENCE www.sciencemag.org EFTA01075549 REPORTS correspond to specific phases of the sleep cycle. In the sensorimotor phase of vocal learn- ing, the mapping between auditory feedback and vocal output is the fundamental compu- tational problem to be solved (17). A solution to this problem is reflected in the sensorimo- tor activity patterns of RA neurons. Precision of spike timing has been observed in a num- ber of systems and provides evidence for temporally based neural codes in sensory pro- cessing, although only in a few cases has the behavioral relevance been directly demon- strated (18). The observed correspondence between auditory activity and vocal output demonstrates that, in the RA, sensorimotor mapping is based on a temporal code. This correspondence is likely to arise from audi- tory input recruiting similar components of the RA pattern-generating circuits as those recruited during singing. In the hierarchical organization of the song system (4, 19), the sensorimotor correspondence may first emerge at the single-cell level within the RA. The data suggest that, during vocal develop- ment, the song system learns to generate pre- motor commands by association with a pre- diction of future commands based on the tinting of auditory feedback from preceding syllables. This can be interpreted as learning the match between the auditory response to a sequence of syllables with the premotor pat- tern for a subsequent syllable or as learning the match between the prediction of a sensory representation of a syllable with the premotor representation of the same syllable. In the birdsong system, RA receives input from the IIVc and from an anterior forebrain pathway (AFP) (Fig. IA). Sensorimotor song learning could result in part from "online" Time (min) Fig. 4. (Top) The firing rate during recordings of RA ongoing activity over almost 1 hour of sleep, estimated from a 100-point moving average of the interspike intervals (there was a gap in the data collection of —3.25 min). (Bottom) A histo- gram (30-s bins) of the number of bursts identi- fied by a burst-finding procedure. The number of bursts that significantly matched the premotor activity is shown in blue. mechanises, whereby during singing, IIVc ac- tivity in response to auditory feedback from sequences of syllables is delayed through the AFP to produce a prediction of activity in the RA of a subsequent syllable. The data collected during sleep, however, also suggest "off-line" models for learning that address the problems of feedback delay and sequence generation. Such models share some similarities with tem- poral-difference models of reinforcement learn- ing and sequence generation that are prominent in mammalian work on basal ganglia and the cerebellum, in that they reward or modify the system on the basis of its overall performance, not on the basis of the performance of individ- ual components or movements (3). The AFP has been likened to a mammalian cortienlywkl ganglia-thalarnocortical loop (20, 21). In the vocal learning model motivated by the present data, signals that arise in the RA during singing train the AFP to gener- ate a prediction of auditory feedback; dur- ing sleep rehearsal, the AFP's predicted feedback provides reinforcement to RA neurons. During singing, sensorimotor ef- ference copy signals (premotor output and expected auditory feedback) traverse the AFP, and via the lateral subdivision of the magnocellular nucleus of the anterior neo- striatum (IMAN) projection onto area X, are compared with real auditory feedback arriving in area X from the HVc (Fig. IA). Efference copy is brought into temporal register with auditory feedback using the long (-50 ms) synaptic delays observed in the medial subdivision of the dorsolateral nucleus of the thalamus (DLM) (21). This stimulates area X neurons that are sensitive to temporally coincident input. The output of the IMAN onto the RA has a reduced effect because the IMAN is not in temporal register with driving input from the HVc. During sleep, replay of song premotor pat- terns via ongoing bursting generates coher- ent activity throughout the song system that is similar to singing in the absence of actual sound production and perception. The out- put of the IMAN represents a prediction of the real auditory feedback that would have resulted from the burst-generated motor command, is in near coincidence with IlVc bursts driving the RA, and is used to mod- ify RA neurons that are sensitive to tempo- rally coincident input. The proposed algorithm for birdsong learning depends on circadian modulation of neuronal activity patterns (22). Our observa- tion of neuronal replay of sensorimotor pat- terns during sleep is consistent with data front hippocampal studies suggesting that sleep is important for the consolidation of neuronal temporal codes for spatial memory (23, 24). The fundamental prediction of our model is that birdsong learning depends on sleep or other off-line computations. References and Notes 1. In songbirds, there is a minimum delay of —70 ms or longer between the production of a burst of neuronal activity contributing to song generation and the reception of processed auditory input from the resulting vocalization. In zebra finches, sylla- bles are typically SO to 200 ms in duration and may comprise one or several notes. The minimal delay is the sum of premotor lead (-50 ms) and sensory lag. Sensory lag is the sum of the minimal latency for response (-20 ms) and the sensory integration time, which, for song system neurons, can be tens to hundreds of milliseconds. 2. R. S. Sutton, A. G. Berta, Reinforcement Learning: An Introduction (MIT Press, London, 1998). 3. R. E. Sur!, W. Schultz. Exp. Brain Res. 121, 350 (1998); D. G. Beiser. J. C. Houk, f. NewrophysieL 79, 3168 (1998); G. S. Berns, T. J. Sejnowski, f. foga. Nearest). 10, 108 (1998); J. Brown, D. Bullock, S. Grossberg f. Neumset 19, 10502 (1999). 4. A C. Yu, D. Margoliash, Science 273, 1871 (1996). 5. A S. Dave, A C Yu. D. Margoliash, Science 282, 2250 (1998) 6. All neuronal data reported in this report are from single units. Eighteen neurons (10. 5, and 3) from three birds contributed to the data reported herein. Two additional neurons from a fourth bird exhibited a match between auditory responses and ongoing bursts. These data were the genesis for this study but did not include premotor data. 7. The chronic recording procedures and conduct of the experiments are described in detail on Science Online (24). 8, Additional procedures and data regarding the audi- tory responses of the netecos are described on Sci- ence Online (24). 9. The pattern of RA neuronal activity depends on notes. which are constituents of syllables. By defini- tion, the same pattern of notes is repeated for a given syllable type. We analyzed RA activity at the syllable level to ease the analysis and to minimize the number of deletion stimuli required in subsequent experi- ments. FolloiMng established procedures, we identi- fkd each syllable that the bird sang, and the times of asset and offset were manually determined from spectrographs and oscillographs of the acoustic sig- nal. The spectrographs of all exemplars for each syllable type were cross-correlated to establish opti- mal time shifts; these were then applied to the corresponding spike bursts. The aligned time-shifted spikes were the basis for further analysis. 10. Peak cross-correlation values were tested for signifi- cance by using a bootstrap procedure, described in detail on Science Online (24). 11. Multiple stimuli were derived from the 8O5, one for each syllable deleted from the last complete motif of the SOS; these were presented in one block during the night while the bird was asleep. Deleted syllables were replaced with samples from silent intervals between motifs or syllables. Spike rates were com- puted over the duration of each syllable. For each syllable from the second to the penultimate, we compared the spike rates over the interval of the target syllable when a previous syllable or the target syllable itself was deleted (experimental data) against the spike rates for the target syllable when only a subsequent syllable was deleted (control data), using a Mann-Whitney U test at the 95% confidence level The last syllable in the song was excluded because, by definition, control data were not Orval- able. (The response to the SOS. presented at the beginning of the night's recordings as the bird tran- sitioned into sleep, showed fluctuations in the strength of response but not in the timing of spikes. Thus, it was appropriate for correlation testing of sensorimotor comparisons but not for a comparison with spike counts of deleted syllable stimuli.) 12. The three birds presented with deletion stimuli had songs with eight, five, and three syllables per motif. All 10 neurons tested in the eight-syllable bird (Fig. 28) and all 3 neurons tested in the three-syllable bird showed a loss of excitatory response when the target or a previous syllable was deleted. One of two neurons tested in the five-syllable bird showed no response to any syllable of the 8O5; the other wwwsciencemag.org SCIENCE VOL 290 27 OCTOBER 2000 815 EFTA01075550 REPORTS neuron responded at the second syllable, and the response was suppressed by deletion of the first or second syllable. 11 D. Margolies/O. Necutod. 3. 1039 (1983); D. Mar- goliash, E. S. Fortune, J. Neustod. 12, 4309 (1992). 14. We gathered sufficient data from 14 neurons (from three birds) to permit quantitative comparisons be- tween ongoing discharge and singing. Interspike in- terval histograms of the ongoing discharge of RA neurons during sleep are binadal the longer interval peak is related to nonbursting activity. Thus, bursts were defined as continuous sequences of interspike intervals falling outside of the normal (nonbursting) distribution of intervals. 15. Two tests of significance were devised. For each neuron, a distance was computed between each ongoing burst and the exemplar stretch of activity during singing. Each best match (lowest distance) was tested for significance by using a bootstrap procedure that assessed the probability of occur- rence of the exact sequence of intervals observed in the burst. In the second test, for the one bird with the greatest number of recordings, we also compared each neuron's ongoing discharges during sleep with the premotor data from other neurons. For seven out of eight neurons, there were more matches with the neuron's own cremator data than with the premotor data from the other neu- rons. The same neuron failed to achieve signifi- cance in both tests. 16. The procedures for identifying bursts in ongoing ac- tivity, matching bursts to premotor activity, and as- sessing the significance of the matches are described in detail on Science Online (24). 17. M. Konishi. Z. Thecpsychol. 22, 770 (1965). 18. C. E. Carr, W. Heiligenberg. G. J. Rose, J. Neurosd. 6, 107 (1986); J. Neunuct 6, 1372 (1986); A. Molten, M. Konishi, J. Neumsci. 1, 40 (1981); C E. Carr, M. Konishi, J. Netwoul. 10, 3227 (1990); G. Laurent, M. Welts, H. Davidowitz, J. NeuroseJ. 16, 3837 (1996); R. C. deCharms, M. M. Merzenich, Nature 381, 610 (1996); A. K. Engel P. R. Roelfsema P. Fries, M. Brecht, W. Singer, fere& Cortex 7. 571 (1997); Y. Prut et at., J. Neon:physic,. 79, 2857 (1998). Structure of Murine CTLA-4 and Its Role in Modulating T Cell Responsiveness David A. Ostrov," Wuxian Shi,2 Jean-Claude D. Schwartz,1 Steven C. Almo,Z* Stanley G. Nathensonl as The effective regulation of T cell responses is dependent on opposing signals transmitted through two related cell-surface receptors, CD28 and cytotoxic T lymphocyte—associated antigen 4 (CTLA-4). Dimerization of CTLA-4 is required for the formation of high-avidity complexes with 87 ligands and for transmis- sion of signals that attenuate T cell activation. We determined the crystal structure of the extracellular portion of CTLA-4 to 2.0 angstrom resolution. CTLA-4 belongs to the immunoglobulin superfamily and displays a strand topology similar to Va domains, with an unusual mode of dimerization that places the B7 binding sites distal to the dimerization interface. This organization allows each CTLA-4 dimer to bind two bivalent B7 molecules and suggests that a periodic arrangement of these components within the immunological synapse may contribute to the regulation of T cell responsiveness. T cell—dependent immune processes require cell-surface interactions that mediate the ini- tiation, modulation, and the ultimate course of the response. The specificity of T cell recognition is determined by the engagement of the T cell receptor (TCR) on T cells with cognate peptide-major histocompatibility complex (MHC) complexes presented by an- tigen-presenting cells (APCs) (1, 2). Addi- tional signals are required to sustain and en- hance T cell activity, the most important of which is provided by the engagement of CD28 on T cells with its ligands B7-I (CD80) and B7-2 (CD86) on APCs (3, 4). In contrast, the interaction of B7 isoforms with 'Department of Microbiology and Immunology, Wre- partment of Biochemistry, 'Department of Cell Biol- ogy. Albert Einstein College of Medicine. Bronx, NY 10461. USA. 'To whom correspondence should be addressed. E- mail almofeaecorn.yu.edu or nathenscSaecom.yu. edu CTLA-4, a CD28 homolog (31% identity), provides inhibitory signals required for down-regulation of the response (5). Unlike CD28, which is expressed on rest- ing T cells, CTLA-4 is not detected on the cell surface until 24 hours after activation, peaking at 36 to 48 hours after activation (6). In addition, CTLA-4 exhibits an affinity for the B7 isoforms that is 10 to 100 times that for CD28 (7). On the basis of these differ- ences in expression patterns and affinities, it is likely that CTLA-4 directly competes with CD28 for binding B7 and also directs the assembly of inhibitory signaling complexes that lead to quiescence or anergy (8). Consis- tent with the inhibitory role of CTLA-4, mice deficient in CTLA-4 die as a consequence of unchecked polyclonal T cell expansion, which results in fatal lymphoproliferative dis- orders (9). Thus, the balance between the opposing signals elicited by CD28 and CTLA-4 is central to the regulation of T cell responsiveness and homeostasis (10). 19. E. T. vu, M. E. Mazur& Kuo, J. Neurosct 14, 6924 (1994). 2a S. W. Bottler, F. Johnson, J. Neurobiot 33, 602 (1997). 21. M. Luo, D. J. Perkel, J. Neurosc). 19. 6700 (1999). 2Z G. E. Hinton, P. Days; 8. J. Frey, R. M. Neal. Science 268. 1158 (1995). 21 G. Bundki, Neunucience 31. 551 (1989); M. A. Wil- son, B. L McNaughton Science 265, 676 (1994); W. E. Skaggs, B. L McNaughton Science 271, 1870 (1996); L NiSdasdy, H. Hirase, A. Curka, J. CSitSvarl, G. Buzsdki,./. /Yet/fwd. 19. 9497 (1999). 24. Supplemental data are available at www.sclencemag. orefeaturefdata/10511399shl 25. We thank. T. Q. Gerstner, ).-M. Ramirez, P. S. Ulinski, and especialty two anonymous reviewers for valuable comments on the manuscript. This work was sup- ported by grants from the NIH (MH59831 and MH60276) to and (MH1161S) to =. 6 April 2000; accepted 8 September 2000 Because of its dominant role in modulating T cell activity, CTLA-4 has received consider- able attention as a therapeutic agent (M. The soluble CTLA-4 —inununoglobulin (CTLA-4 — Ig) fusion protein acts as an inhibitor of CD28- B7 costimulation and has specific inhibitory effects in animal models of autoinununity, transplant rejection, asthma, and allergy (3, 12). The efficacy of CTLA-4-Ig treatment of hu- man disease has been demonstrated in clinical trials on patients with psoriasis vulgaris (13). This approach may well extend to a variety of T cell-mediated diseases including autoimmune diabetes, rheumatoid arthritis, systemic lupus erythematosus, and graft-versus-host disease (13). In contrast to strategies that interfere with the CD28-B7 association, reagents that inter- fere with the CTLA-4 -B7 interaction intensify specific T cell responses. For example, block- ing antibodies directed against CTLA-4 en- hance rejection of preestablished tumors and protect against secondary challenge in animal models of prostate cancer and colon carcinoma (14). The structure of the soluble extracellular domain of murine CTLA-4 (15) revealed two independent copies of the CTLA-4 dimer in the asymmetric unit of the crystal (Fig. I, A and B, and Table I) (16). The CTLA-4 monomer is a two-layer I3-sandwich that ex- hibits the chain topology found in the immu- noglobulin variable domains (Fig. IA) (17). The front and back sheets, composed of strands A'GFCC1 and ABEDC', respectively, are connected by two intersheet disulfide bonds. The disulfide bond between the B and F strands is a signature for the immunoglob- ulin fold; the disulfide bond joining strands C and D is unique to the CD28/CTLA-4 family (Fig. 1A) (17, 18). The nuclear magnetic resonance (NMR) structure of a monomeric form of human CTLA-4 (18) shows the same overall topol- ogy as the murine homolog, with a root mean square (mu) deviation of 2.4 A between equivalent Ca atoms. The most significant 816 27 OCTOBER 2000 VOL 290 SCIENCE www.sciencemag.org EFTA01075551 Supplementary Material All birds were sexually mature males >120 days of age, originally obtained from a commercial breeder. Before experiments, birds were outfitted with a chronic recording apparatus [A. Dave, A. C. Yu, J. J. Gilpin, D. Margoliash, in Methods for Simultaneous Neuronal Ensemble Recordings, M. Nicolelis, Ed. (CRC Press, Boca Raton, FL, 1999), pp. 101- 120]. Subsequently, birds were maintained on a 16.18-hour light/dark schedule, and experiments began near the end of the day to facilitate the bird's falling asleep. Electrodes were advanced or retracted by manually turning a screw while the bird was restrained until RA single units were isolated. The bird was then released and induced to sing by presenting a female in an adjoining half-cage or by broadcasting female calls. The lights were then turned off, and the bird fell asleep. Infrared video monitoring of sleeping birds facilitated assessment of the bird's sleeping state (including lack of movements, closed eyes, and low respiratory rate). BOS playback commenced soon after the lights were turned off, as this seemed to decrease the movements of the birds in the initial minutes of darkness, reducing the risk of losing unit isolation. After a variable number of presentations of the BOS (30 to 220 repetitions, 12 s per repetition), syllable-deleted versions of the BOS were presented in randomized order. After auditory stimulation, up to 60 min of ongoing activity was recorded in silence and darkness. When possible, thereafter, either additional repetitions of auditory stimuli were presented or the bird was awakened and more vocal motor activity was obtained until the unit signal isolation was lost. Auditory responses of RA neurons were analyzed on the basis of the following procedure. Interspike interval histograms of ongoing discharge of RA neurons during sleep are bimodal; the longer interval peak is related to nonbursting activity [A. S. Dave, A. C. Yu, D. Margoliash, Science 282 , 2250 (1998)]. We fit a Gaussian to the longer interval peak using a nonlinear least squares curve-fitting procedure (Matlab, The Math Works). During playback of the BOS, a unit was considered responsive to a syllable when the distribution of interspike intervals occurring between the start of that syllable and the start of the next syllable was different from the Gaussian interval distribution of the nonbursting ongoing activity (Mann-Whitney U test, P<0.05). In 86% (93 out of 108) of syllables, units were responsive to two or more occurrences in the song of the same syllable. In 46% (43 out of 93) of these cases, the strength of response (number of spikes) differed significantly (paired r test, P< 0.05) across occurrences of the same syllable in song (average difference = 21% ± 14%). Each of 18 units analyzed showed this effect. We compared premotor and auditory response data for each of 17 units (from three birds) by constructing vectors (histograms binned at 1 ms) from auditory responses to song playback and from premotor activity. Cross-correlations of the two vectors were calculated with lags ranging over ±100 ms. The cross- correlation was normalized by the square root of the product of the zero-lag autocorrelation values of the two vectors. Peak values ranged from 0.23 to 0.66. The peak of the cross- correlogram was tested for significance by randomly shuffling the values of bins in the auditory response vector and then repeating the cross-correlation (200 times). The resultant distribution of peak correlations defined a normal distribution, which was used to derive the probability of obtaining the actual peak value. For one site, deletion stimuli (see report) but not the intact BOS were presented because cell isolation began unexpectedly during stimuli presentation with a simultaneous recording site on another electrode. Thus, instead of the BOS, the response to an auditory stimulus with the last syllable of the BOS replaced with silence was compared with the premotor activity of that unit We also compared premotor and ongoing discharges of the single units. Bursts occurring in ongoing activity during sleep were defined as continuous sequences of intervals that fell outside of the 99% confidence interval of the distribution of the longer interspike intervals. To match ongoing bursts against syllable-level activity recorded during singing, we removed long intervals from the beginnings and ends of bursts. We also determined the longest duration between syllable onsets within a motif and split any bursts at intervals greater than this duration (185 to 220 ms). One site was excluded because it did not burst sufficiently (rate of ongoing discharge of only 2.46 spikes/s); another site required lowering the confidence threshold for finding burst from 99 to 95% to obtain any bursts. Sufficient data (at least 15 min of sleep, some bursts, and sufficient premotor activity) were therefore available for 14 units (from three birds). For each unit, we computed distances (0.5-ms resolution) between each burst from ongoing activity and the unit's premotor data. The distance D between the two spike trains at a given shift was defined as the average interval from each spike in one train to the nearest spike in the other train I Min (5.0, Min7,,In —0+ iMin(5.0, hi) D— t'1 Supplemental Figure 1. where / and j index the two spike trains and n, and ri) are the number of bursts in each train. A ceiling of 5 ms was set to increase the robustness of the algorithm to outlier spikes. That is, the duration between any spike and the nearest spike in the other train, at any given shift, was allowed to range from 0 to EFTA01075552 5 ms, with any values greater than 5 ms being treated as exactly 5 ms. Two measures of significance were developed to determine the significance of the best match (lowest distance). Using a bootstrap procedure, we shuffled the intervals within an ongoing burst, and the resulting burst was similarly matched against the premotor activity. The distribution of distances from 1000 shuffled bursts was approximately normal. Across all neurons and all bursts, skewness values ranged from •1.5 to 0.5, with a mean of •0.4 ± 0.3. Similarly, kurtosis values ranged from -1.6 to 7.4, with a mean of 3.1 ± 0.6. Bursts were considered to have a significant match to the premotor activity if there was less than a 5% probability of obtaining as strong a correlation by chance. This procedure relied on precise matches between the ongoing and the premotor activity to find matches and could not detect ongoing activity that matched premotor activity only with higher order statistics. The data also show evidence of compression/expansion of the time axis of ongoing discharge during sleep as compared to premotor patterns (see text and Fig. 3B of report). Time "warping" is a nonlinear phenomenon that our matching procedure was not designed to detect; hence, our estimate of matching between bursts and premotor activity is a lower bound estimate. For the bird with the largest number of recordings, we also evaluated the significance of matches by matching a unit's bursts from ongoing discharge to the premotor data belonging to different units recorded from the same bird. Data from eight units provided seven of these pairings for each unit. For seven units, the greatest percentage of bursts matched the unit's own motor activity, with the difference between this percentage and the mean of the percentages for the heteronymous mappings being 8.2 ± 4.9%. For the other site, the unit's own premotor data provided the worst percentage of matching bursts (average difference of 11.7%). The premotor activity of this unit was not apparently unusual; however, the ongoing discharge of this unit had the fewest bursts (in only 1.7% of time in undisturbed sleep did this unit exhibit bursts), the highest firing rate, and was the most regular (lowest standard deviation of the interspike interval distribution). EFTA01075553