We focus on the capacity of a readout neuron to produce a large set of diverse responses to the same inputs (i.e., to implement a large number of input–output functions). 2C). Figures 3, A and B show that the dimensionality increases as more RCNs are added until it reaches the maximal dimensionality permitted by the number of inputs. The four possible configurations of the two populations of N input neurons are four points in a 2N dimensional space. For this measure the fraction of RCNs with a differential response decreases as the representations become sparser (Fig. In other words, there will be sets of desired outputs that cannot be realized by a readout neuron. In such cases, estimating γ and σ2 from neural recordings can provide a useful measure of the effect on network performance. 0000033237 00000 n (2005) a cell was considered to be selective to a particular stimulus if the response was at least five standard deviations above the baseline. Each RCN responds nonlinearly to the weighted sum of the original inputs, and its weights are random and statistically independent. All subpatterns are random and uncorrelated with equal probability for +1 or −1. It is difficult to say whether our results about the efficiency of RCNs and optimal sparseness apply also to problems like vision, and further studies will be required to make more general statements. For each number of patterns there is a critical (Crit.) 0000004605 00000 n where γ is the discrimination factor that depends on the threshold θ for activating the RCNs (and hence on the coding level of the RCN representations) and on the noise n in the inputs. %%EOF Although it is not possible to make statements about these general cases without further assumptions, we can note that if the combinations are picked uniformly at random from all the possible ones, the scaling of the number of dimensions versus the number of RCNs remains the same. Specifically, as we varied the noise we used Equation 20 to estimate the number of RCNs that would produce a minimal error of 10%. In particular we are developing a model of a neural network which encodes the inner mental states as attractor of the neural dynamics. We thus proceed to estimate λ from the matrix M. The matrix M defined above is a random matrix (across realizations of ξ, which are assumed to be random). This transformation can efficiently increase the dimensionality of the neural representations without compromising the ability to generalize. doi:10.1371/journal.pcbi.1003146 (2013), O. Barak, M. Rigotti, S. Fusi, The sparseness of mixed selectivity neurons controls the generalization-discrimination trade-off, Journal of Neuroscience 33, 3844-56 (2013), M. Rigotti, D. D. Ben Dayan Rubin, X.-J. Figure 4, C-F illustrate an intuitive argument explaining how sparseness biases the discrimination-generalization tradeoff (see Materials and Methods for details). exhibit what has been termed mixed selectivity (Fusi et al., Neuron 95, 697–708, August 2, 2017 ª 2017 Elsevier Inc. 697 2016), a neural encoding scheme in which different task vari- ables and behavioral choices are combined indiscriminately in Sign In to Email Alerts with your Email Address, The Sparseness of Mixed Selectivity Neurons Controls the Generalization–Discrimination Trade-Off. In our model, f is controlled by varying the threshold for the activation of the RCNs. We focused on a simple linear readout, which is what presumably can be implemented by individual neurons. 0000016460 00000 n We first explain the problems arising when integrating multiple sources of information. 0000017510 00000 n We can quantify the breakdown for sparse coding levels by defining a critical coding level, fcrit, at which the number of RCNs needed increases to 0.75 of the number of inputs. 0000003482 00000 n These distances express the similarity between the neural representations (two identical inputs are at zero distance if they are identical). (2011)). Figure 4B shows how the relative Hamming distances between inputs are transformed by the randomly connected neurons for two different coding levels. Two sources of eight states each were used. 0000006824 00000 n 0000151828 00000 n In general, this information is smaller than the total information contained in the input, as it is constrained to be in a more “explicit” format (DiCarlo et al., 2012) suitable for being processed by simple readouts. 0000027993 00000 n The brain is probably dealing with all these problems, and for this reason it may use different and sometimes adaptive coding levels in different areas, but also within the same area (indeed, there is a lot of variability in f across different neurons). This is probably why the brain is endowed with several mechanisms for rapidly and reversibly modifying the sparseness of the neural representations (e.g., by means of neuromodulation; Disney et al. The decorrelation increases the ability of the readout neurons to discriminate between similar inputs, but it is important to note that not all forms of discrimination lead to linear separability. 0000007139 00000 n 0000007453 00000 n In the input space, the difference between pairs of inputs belonging to Δ1 is half the difference between pairs belonging to Δ2. It is analogous to our definition of pattern discrimination, suggesting that the neurons in the DG may have similar properties as our RCNs. Note that as the number of patterns (and thereby RCNs) increases, sparser representations become more efficient. 1D). The distance in the RCN space is plotted versus the distance in the input space. For dense coding, the threshold is set to zero (blue line), and all the RCNs on the right of the threshold (half of all RCNs) are active. The output neuron (Figs. We show that the threshold of the RCNs, which determines their coding level (i.e., the average fraction of stimuli to which each individual neuron responds), biases a tradeoff between generalization and discrimination. The case with more than two sources is a straightforward extension and is briefly discussed at the end of this section. In this scenario, some of the “units” with nonlinear mixed selectivity, analogous to our RCNs, are implemented by a specific branch or set of dendritic branches, and hence they would not be visible to extracellular recordings. We consider two network architectures—one with an RCN layer (Fig. The correlations that we considered are presumably widespread in the brain, as they are likely to emerge every time a neuron integrates two or more sources of information, as in the case in which it receives external and recurrent inputs. Figure 4C shows the distribution of inputs to all RCNs when a generic input made of two sources of information is presented. 0000006665 00000 n This capacity clearly depends on the input representation, and it is functionally important as it can be harnessed to generate rich dynamics and perform complex tasks (Hinton and Anderson, 1989; Rigotti et al., 2010b). To help understand the meaning of γ in the case that we analyzed (i.e., when the two sources have the same weight), we show in the inset of Figure 6 how γ is related to the shape of the curve that represents the squared distance in the RCN space as a function of the squared distance in the input space. 2A). Full rank (i.e., rank equal to the maximum, which in our case is p) indicates that all the p vectors representing the input patterns are linearly independent and hence span a p dimensional space. 1/σ2 is the generalization factor, which depends on θ, n, and the total number p of classes of inputs. The colored lines denote the values used in A. Because we are in the regime where there are many more RCNs than patterns, and classification is hampered by noise, we expect the margin to be a good approximation to κ. 0000018434 00000 n The discrimination factor γ is related to the similarities between the RCN representations induced by similar inputs. This scaling is as good as the case in which the response properties of the neurons in the intermediate layer are carefully chosen using a learning algorithm. Learning in neural networks with material synapses, Neural activity in the primate prefrontal cortex during associative learning. Indeed the same stimulus should sometimes lead to different behaviors depending on the situation, the intention of the subject, and the rules imposed by the task to be performed. The rasters show the spikes of a single hypothetical neuron in response to several presentations of four combinations of stimuli (A/B) and contexts (C/D). The classification performance can be estimated by going over all these functions and counting how many can be implemented (i.e., when there is a set of synaptic weights that allow the output neuron to respond to all the inputs as specified by the function). 0000049498 00000 n Our results hinge on the choice of a linear readout that limits classification ability. 2D,E), using a fraction of possible input combinations does not alter this scaling (data not shown). 0000151694 00000 n 0000005236 00000 n In other words, the number of RCNs grows linearly with the number of inputs that have actually to be classified (Rigotti et al., 2010b). Recent experiments (Rust and DiCarlo, 2012), designed to accurately estimate f, indicate that for V4 and IT f ∼ 0.1. B, Similar to A, but for 225 patterns. Equation 13 provides a recipe for estimating γ from experimental data. Note that the ranking of distances is preserved—if point A is closer to B than to C in the input space, the same will hold in the RCN space. h�bbbtg`b```$� 0 � h These types of shifts have been studied systematically in experiments aimed at understanding the role of the dentate gyrus (DG) and CA3 in pattern separation and pattern completion (Sahay et al., 2011). 0000111547 00000 n 1). The test error of the readout from the RCNs depends on how the readout weights are set. Notice that in contrast to Figure 4B, on the y-axis we now represent the expected squared distance in RCN space between pairs of noisy patterns. For example, in visual object recognition, the members of a class are the retinal images of all possible variations of the same object (e.g., when it is rotated), including those that have never been seen before. 0000152792 00000 n ... mixed selectivity depending on the task the animal is involved in.

Italian Army, Miami Florida, New Couple Instagram Captions, Leeds Festival 2006 Line Up, Tampa Bay Lightning Roster 2016, Alaina Mathers Instagram, Financial Support Letter For Tourist Visa, Adam Thoroughgood House, Alison Watkins Coca-cola, Greek Meaning Of Work, Sailor Stevie Swisher, 2008 Best Actress Nominees,