The entropic brain

  • cc icon

    Uncertainty of incoming information increases the amount of mental effort. Scientists might have been interested in quantifying the amount of this mental effort, as evidenced by the great amount of research devoted to computing the energy used to deal with the uncertainty of an event. One of these measures is Shannon’s entropy, which was initially used as a measure of the level of uncertainty with respect to the outcome of an event (Shannon, 1948). As defined by Shannon, entropy is expected to increase as the level of disorder (uncertainty or task complexity) increases. Therefore, to evaluate how the brain deals with entropy, one could either locate the focal area showing activity that is positively correlated with the degree of uncertainty or consider information transfer. We speculated that a functionally effective brain would modulate its network to fit to the varying entropy, since computation in the brain is probabilistic. Therefore, we put forward two hypotheses: 1) the functional brain architecture will reflect the varying entropy, and 2) the entropy will be modality-specific. In order to test these hypotheses, we estimated several network properties of the magnetoencephalography time-series signals obtained from healthy monolingual listeners. We particularly focused on whether entropy reflects the modality-specific processing load, especially when the degrees of the processing load of the two events were similar in terms of the rate of accuracy and the length of response time between tasks. In order to manipulate both entropy and modality, we varied the processing complexity (easy vs. difficult) in either a linguistic or non-linguistic (pitch change detection vs. word intelligibility test) task. Using graph-theoretical measures, the global organisation measures of the network, such as its small-worldness, correlation-coefficient, global efficiency, and characteristic path-lengths, were compared within the network extracted as a set of 78 brain atlas nodes. The results showed a significant main effect of task complexity on the brain network properties, demonstrating that task load is indeed ubiquitous regardless of task modality. Equally importantly, we also found a pronounced task-specific difference in the network properties between linguistic and non-linguistic modalities. Regardless of modality, in the effortful tasks, the characteristic path-length and the correlation-coefficient were significantly larger, whereas the linguistic tasks resulted in significantly higher small-worldness, with the hubs located at the usual language nodes. Our findings collectively suggest that task load is ubiquitous but is also modality-specific in the brain network properties, as evidenced by the specific network graph measures.


    Entropy , brain network , task complexity , language

  • 1. Introduction

    Computation in the brain is complex but nonetheless effective in terms of the size and range of interregional interactions used in order to achieve goals. For example, Karl Friston (2002), along with many other neuroscientists, has emphasised the role of the brain’s distributed circuitry in effectively accomplishing an intended goal. Efficiency is determined from testing how well a system can grasp the rule underlying a task while minimising prediction errors. To a business investor, efficiency is equivalent to the best long-term returns from the least immediate investment (Montague 2007). However, in reality, there is little certainty of the occurrence of upcoming events. Under such a scenario of uncertainty, mental effort must increase in order to reach the best decision. Shannon’s entropy, described by Shannon (1947), is a measure proposed to quantify this mental effort, based on probabilistic computation under the assumption that the larger the entropy, the higher the uncertainty. Therefore, entropy is expected to increase when a task is difficult or unfamiliar. This concept has been explored in various fields of study. Frank (2013) quantified information entropy using a sentence-reading task. The link between probability and phonological form processing has long been confirmed (Lizier et al. 2011, Shannon 1997), and is clearly stated in the principle of least effort proposed by George Kingsly Zipf (1949). As such, human behaviour and the brain are assumed to operate to best cope with uncertain or unknown events; i.e., those that make decisions more difficult. The concept of entropy was also used to explain the state of human brain functionality, including nonordinary states such as REM (Rapid Eye Movement) sleeping (Carhart Harris et al. 2014) or discourses in schizophrenia (Cabana et al. 2011). As neuroscientists, we are generally interested in how the processing load is reflected in the brain. However, investigating the processing-load effect in the brain is far from novel, and has been studied using various methods to date. One method is to locate the brain regions that are most highly involved in the process. For example, Zou et al. (2011) evaluated processing loaddependent brain activity by using pseudo-continuous arterial spin labelling perfusion imaging, while overcoming the shortcomings of ordinary functional magnetic resonance imaging, positron emission tomography, or others. Using a classical N-back task while varying the processing load, they showed that a more difficult task recruits a wider range of brain regions covering the frontal, temporal, and parietal as well as the midline areas. The cerebral blood flow level was positively correlated with task difficulty in the regions of the key default-mode network, the posterior cingulate cortex and the medial prefrontal cortex, which are in line with the results from other imaging systems (Karbasi et al. 2014, Rolls & Webb 2014). Although these localisations can help to explore where a certain behaviour is processed in the brain, the global brain architecture has also been explored using various other measures. Most recently, topological network properties of the brain have been investigated using graph-theoretical approaches, both in resting state (Bullmore & Sporns 2012, He & Evans 2010) and in working state (Ruhnau et al. 2014, Yu et al. 2014). Certain network properties have already been reported as functional markers in diseases such as Alzheimer’s disease (Seo et al. 2013) and Parkinson’s disease (Baggio et al. 2014).

    Brain network graph measures are often computed based on the Granger causality (Rubinov & Sporns 2010): a network graph is a set of nodes (or brain regions) and edges that connect pairs of nodes (or connections). Therefore, the question ‘which area in the brain is primarily involved when performing a given task?’ can be asked from a graph-theory perspective as ‘how crucial is a given area to the brain network?’ In describing a graph, several measures are helpful. The small-worldness index indicates the balance between functional integration and segregation, which is an optimal status associated with efficiency in information transfer. The clustering coefficient measures the immediate connectedness of a node, and thus estimates the contribution of a node to an efficient network. The characteristic path length is an indicator of a network’s global integration. Therefore, a randomly organised network is characterised as having a low clustering coefficient and a shorter path length (Wu et al. 2012). The centrality measure, which is commonly obtained by the node degree, enables a straightforward neurobiological interpretation: a node with a high degree is one that interacts with many other nodes in the network (Rubinov & Sporns 2010). When a node (or edge) has the largest normalized betweenness value, it is considered to be pivotal and is called a hub (or bridge), as defined in Gong et al. (2009).

    In the present study, we aimed to quantify the neural entropy dynamics using the graph-theoretical network properties mentioned above in tasks with varying degrees of difficulty, either in speech or in general (non-speech) conditions. Given that the clearer a sound the lower the processing load, we manipulated the amount of the signal by reducing the spectral details of an ordinary speech sound. Evans et al. (2014) have shown that spectrally degraded speech leads to poorer performance compared to clear, ordinary speech. The general condition involved detection of a pitch change. Task complexity was manipulated by presenting subjects with a two-tone beep, either with a dramatic or subtle tonal change between the two tones. Kyong et al. (2014) reported that tonal variation affected performance when speech was presented either with or without tonal variation. In the present study, we focused specifically on the overall network organisation with respect to several graph measures of the network, including the clustering coefficient, characteristic path length, small-worldness index, and centrality.

    2. Methods and Materials

       2.1 Participants

    Seventeen healthy listeners (8 female, 19~31 age range, of longer than 13 years of official education) participated in the study series. They were all right-handed and monolingual, in the sense that they had no exposure to any language other than Korean before the age of 8. All subjects confirmed no medical history of neurological, neuropsychological, hearing, and speech conditions, and provided written consent. All subjects received 30,000 Korean Won as travel expenses. All the procedures were carried out in accordance with the guidelines of the Institutional Research Board of Seoul National University Hospital (IRB No. H-1109-084-378).

       2.2 Stimuli and Tasks

    Stimuli were 80 two-syllable words that were chosen to range from an easy to intermediate level of a word pool (Lee et al. 2010), and were recorded by a female speaker. Participants listened to a sound and were asked to judge whether or not the sound heard was a word. The word sound presented was either clear or spectrally degraded. As a non-speech counterpart, we presented two-tone beeps, whose pitch change was either easily or hardly detectable. Each stimulus was 500-ms long. To familiarise listeners with the various stimulus types and tasks, they were given a 5-minute practice session with feedback, followed by a 12-minute test. Accuracy and response onset time were recorded at each condition from the test session.

       2.3 Magnetoencephalography (MEG) Recording

    Brain signals were recorded in supine position using 306-channel whole- brain magnetoencephalography (MEG, VectorViewTM , Elekta Neuromag; Oy, Helsinki, Finland) while participants were listening to the sound of a word or a two-tone beep. The participants were encouraged to blink after each trial before a sound was heard. In the testing session, no feedback was given and no stimulus was heard twice. Signals were recorded at a sampling rate of 600.165 Hz and digitised at a range of 0.01–200 Hz. An epoch was defined as an event of −100 to 1000 ms. Artefact-rejected and baselinecorrected (−100 to 0 ms) averaged event-related fields per condition were fed into the amplitude and network analyses.

       2.4 Data Analyses

    2.4.1. Amplitude Analysis

    Amplitude values from 78 brain regions were collected from the 4 conditions (2 × 2), and the grand mean was calculated for each region and condition. Regions were grouped into the frontal, temporal, parietal, and occipital lobes for ease of interpretation. A 2 × 2-factor analysis was performed and the main effect was tested among the lobes.

    2.4.2. Network Character Analysis

    Functional connectivity matrices were determined by computing the correlation between all pair-wise combinations of cortical regions (Zhou et al. 2011). In our study, the nodes were represented by regions of interest defined using an automated anatomical labelling atlas template (TzourioMazoyer et al. 2002). The cerebral cortex was parcelled into 78 regions (39 per hemisphere), and then a 78 × 78-inter-regional correlation matrix was constructed by correlation analysis, controlling for the effects of age and gender. The correlation index ranges from 0 to 1, and further analyses were based on an undirected and unweighted binary matrix transformed using a fixed network density threshold. Binary links (edges) indicate the presence or absence of connections, and the network density denotes the total wiring cost of a network (Rubinov & Sporns 2010). Seventeen 78 × 78 connectivity metrics containing the baseline- and artefact-corrected average coherence values in the beta- (14–25 Hz) and low gamma (30–70 Hz)-frequency bands were generated for the 2 × 2 conditions. The undirected binary matrices were constructed for 2 × 2 conditions within a range of the network density threshold D. Since there is no gold standard for a single threshold, a wide network density range was applied (4% ≤ D ≤ 60%, with increments of 1). Calculation of the graph measures was conducted as described in Seo et al. (2013) and Bolanos et al. (2011). In order to identify hubs in the network, we followed the definition used in Gong et al. (2009), by thresholding the betweenness values to at least 1 standard deviation (SD) greater than the average betweenness of the whole network. The subsequent analyses of network connectivity were carried out using the Brain Connectivity Toolbox (Rubinov & Sporns 2010).

    Scores acquired from the behavioural task were fed into a statistical model using a 2 × 2-factor analysis implemented in SPSS13 software (SPSS Inc.; Chicago, IL, USA). Graph measures between conditions were tested using a non-parametric permutation test (1000 permutations, p < 0.001) against the random networks generated, since statistical analyses for normality might not be useful given the sample size.

    3. Results

      >  Behavioural data

    Accuracy was obtained after rejecting the trials with no response or with commission errors as well as trials with only partial correct answers (i.e., when only one of the two syllables was correctly answered). Accuracy was significantly higher in the easy tasks regardless of condition (95 ± 1 in the speech task and 97 ± 0.5 in the non-speech task, p = 0.05). Response onset time was longest in the difficult-speech condition (920 ± 7.5 ms). Complexity was a significant main factor in both tasks (p = 0.05). In the speech-intelligibility task, subjects responded more easily and quickly to clear speech than to degraded speech when compared by a response onset time and correct rate. The same pattern was observed in the pitch change detection task, although with less of a difference in the response onset time, presumably due to the relatively early resolution of a second ‘tone’ compared to a second ‘syllable.’ A significant modality effect was found both in the response onset time and the accuracy (p = 0.05), and a significant effect of task load was found specifically for accuracy (p = 0.05).

      >  Amplitude data

    The main effect of the lobe was significant, with the frontal lobe having the greatest mean amplitude in all conditions. We compared the amplitude in the frontal lobe among the 4 conditions. The main effect of the task complexity factor was prominent (p = 0.001), whereas the effect of modality was only marginally significant (p = 0.052); hard tasks elicited greater frontal amplitude than the easy tasks while speech task elicited greater frontal amplitude than the non-speech one.

      >  Connectivity data

    All of the networks were found to be fully connected at a network density of 18%. The network of the easy-speech condition was fully connected at a network density of 8%, and that of the difficult-non-speech condition was connected at a density of 11%. Therefore, the lowest density in which the largest component size was 78 (i.e., all 78 nodes are connected) was 18% in the current study. Since the data is not normally distributed, we carried out a non-parametric testing with 1000 surrogate datasets randomly generated. We chose to report the resulting data when the significance (p) was smaller than 0.01. The repeated Wilcoxon’s signed rank tests (p = 0.001) showed that the most significant difference in the clustering coefficient was detected at D = 23% and D = 24% between the easy-speech and difficultspeech conditions. Difficult conditions commonly had higher clustering coefficients, regardless of the task type (18% < D < 32%). The characteristic path length was significantly different at D = 6–12% and D = 16–23% between the easy-speech and difficult-speech and between the easy-non- speech and difficult-non-speech conditions, respectively.

    The constructed cortical network had a small-world topological organisation in all of the tested conditions (σ > 1). Specifically, in the easy- speech condition, the small-world topology was prominent at densities of 6–22% (p = 0.001). There was no difference of tasks in σ over the rest of the density range. However, to identify hubs in the network, we used a threshold of 18% when estimating the small-worldness of a network, making sure that all of the networks have the same number of edges (He et al. 2008). Common hub nodes in the difficult condition were the inferior frontal regions (inferior frontal gyrus, IFG), superior temporal lobes (anterior, posterior, and middle temporal gyri STG), and Heschl’s gyrus, whereas the common hubs in the easy condition were the superior temporal lobes (posterior and middle) and Heschl’s gyrus. A further betweencondition comparison showed that the betweenness index of the IFG was significantly higher in the difficult-speech than the easy-speech condition (p < 0.001), and was also higher in the difficult-non-speech than the easy-nonspeech condition (p < 0.05). The comparison between beta- and gammafrequency bands revealed that the node degree of the speech tasks was significantly higher in the beta band than that of the non-speech tasks in the frontal and temporal regions, regardless of task load (p < 0.001). The comparisons in other conditions did not reach the significant level (p < 0.053), presumably due to the sample size.

    4. Discussion

    The main question of this study was whether the brain network could reflect the degree of mental effort or entropy, and if so, whether languagespecific entropy would replicate the known brain network in the literature. Our results first demonstrated that the brain network is functionally organised to deal with the task load depending on the task modality, as represented by the graph measures. The constructed cortical network had a small-world topological organisation in all of the tested conditions. Specifically, in the easy-speech condition, the small-world topology was most prominent. In the present study, we only evaluated the global clustering coefficient, which is determined by the number of closed triples in the network and indicates the possibility of clustering. Our results showed that the clustering coefficient was higher in the difficult conditions, regardless of task modality. The characteristic path length was also longer in the difficult conditions, although the density range differed depending on the condition. Furthermore, hub regions, computed based on the node betweenness index, revealed that the speech-related tasks were associated with the usual language areas such as the superior temporal, parietal, and inferior frontal regions, unlike the non-speech condition (simple pitch change detection). Apart from the network properties, the frontal amplitude was compared using a 2 by 2 factor (task difficulty x modality) analysis and both the factors were found to contribute the difference. We may expect that the difficult speech condition recruits the most resource from the inferior frontal area.

    Taken together, our study revealed that varying entropy by manipulating task complexity was reflected in the topological organisation of the graph-theoretic network, and that the degree of entropy depended on the task modality (speech vs. non-speech). It would further be interesting to determine how information transfer within the hubs depends on task complexity and task type. Therefore, we might next focus on unravelling how the brain optimises its resource allocation depending on the degree of entropy and task modality.

  • 1. Baggio HC, Sala-Llonch R, Segura B, Marti MJ, Valldeoriola F 2014 Functional brain networks and cognitive deficits in Parkinson's disease. [Human brain mapping] Vol.35 P.4620-34 google doi
  • 2. Bolanos ME, Bernat EM, Aviyente S 2011 Multivariate synchrony modules identified through multiple subject community detection in functional brain networks. Conference proceedings, Annual International Conference of the IEEE Engineering in Medicine and Biology Society. [IEEE Engineering in Medicine and Biology Society Annual Conference 2011] P.2534-7 google
  • 3. Bullmore E, Sporns O 2012 The economy of brain network organization. Nature reviews. [Neuroscience] Vol.13 P.336-49 google
  • 4. Cabana A, Valle-Lisboa JC, Elvevag B, Mizraji E 2011 Detecting order-disorder transitions in discourse, implications for schizophrenia. [Schizophrenia research] Vol.131 P.157-64 google doi
  • 5. Carhart-Harris RL, Leech R, Hellyer PJ, Shanahan M, Feilding A 2014 The entropic brain, a theory of conscious states informed by neuroimaging research with psychedelic drugs. [Frontiers in human neuroscience] Vol.8 P.20 google doi
  • 6. Evans S, Kyong JS, Rosen S, Golestani N, Warren JE 2014 The pathways for intelligible speech, multivariate and univariate perspectives. [Cerebral cortex] Vol.24 P.2350-61 google doi
  • 7. Frank SL 2013 Uncertainty reduction as a measure of cognitive load in sentence comprehension. [Topics in cognitive science] Vol.5 P.475-94 google doi
  • 8. Friston K 2002 Beyond phrenology, what can neuroimaging tell us about distributed circuitry? [Annual review of neuroscience] Vol.25 P.221-50 google doi
  • 9. Gong G, He Y, Concha L, Lebel C, Gross DW 2009 Mapping anatomical connectivity patterns of human cerebral cortex using in vivo diffusion tensor imaging tractography. [Cerebral cortex] Vol.19 P.524-36 google doi
  • 10. He Y, Chen Z, Evans A 2008 Structural insights into aberrant topological patterns of large-scale cortical networks in Alzheimer's disease. [The Journal of neuroscience , the official journal of the Society for Neuroscience] Vol.28 P.4756-66 google doi
  • 11. He Y, Evans A 2010 Graph theoretical modeling of brain connectivity. [Current opinion in neurology] Vol.23 P.341-50 google
  • 12. Karbasi A, Salavati AH, Shokrollahi A, Varshney LR 2014 Noise facilitation in associative memories of exponential capacity. [Neural computation] Vol.26 P.2493-526 google doi
  • 13. Kyong JS, Scott SK, Rosen S, Howe TB, Agnew ZK, McGettigan C 2014 Exploring the roles of spectral detail and intonation contour in speech intelligibility, an FMRI study. [Journal of cognitive neuroscience] Vol.26 P.1748-63 google doi
  • 14. Lee SY, Kim JS, Chung CK, Lee SK, Kim WS 2010 Assessment of language dominance by event-related oscillatory changes in an auditory language task, magnetoencephalography study. [Journal of clinical neurophysiology] Vol.27 P.263-9 google doi
  • 15. Lizier JT, Heinzle J, Horstmann A, Haynes JD, Prokopenko M 2011 Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity. [Journal of computational neuroscience] Vol.30 P.85-107 google doi
  • 16. Rolls ET, Webb TJ 2014 Finding and recognizing objects in natural scenes, complementary computations in the dorsal and ventral visual systems. [Frontiers in computational neuroscience] Vol.8 P.85 google doi
  • 17. Rubinov M, Sporns O 2010 Complex network measures of brain connectivity, uses and interpretations. [NeuroImage] Vol.52 P.1059-69 google doi
  • 18. Ruhnau P, Hauswald A, Weisz N 2014 Investigating ongoing brain oscillations and their influence on conscious perception - network states and the window to consciousness. [Frontiers in psychology] Vol.5 P.1230 google doi
  • 19. Seo EH, Lee DY, Lee JM, Park JS, Sohn BK 2013 Whole-brain functional networks in cognitively normal, mild cognitive impairment, and Alzheimer's disease. [PloS one] Vol.8 P.e53922 google doi
  • 20. Shannon CE 1997 The mathematical theory of communication. 1963. M.D. [computing , computers in medical practice] Vol.14 P.306-17 google
  • 21. Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O 2002 Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. [NeuroImage] Vol.15 P.273-89 google doi
  • 22. Wu J, Zhang J, Liu C, Liu D, Ding X, Zhou C 2012 Graph theoretical analysis of EEG functional connectivity during music perception. [Brain research] Vol.1483 P.71-81 google doi
  • 23. Yu S, Taya F, Yu C, Delgado Martinez I, Thakor N, Bezerianos A 2014 Topological changes of the effective connectivity during the working memory training. Conference proceedings, Annual International Conference of the IEEE Engineering in Medicine and Biology Society. [IEEE Engineering in Medicine and Biology Society Annual Conference 2014] P.6242-5 google
  • 24. Zhou Z, Wang X, Klahr NJ, Liu W, Arias D 2011 A conditional Granger causality model approach for group analysis in functional magnetic resonance imaging. [Magnetic resonance imaging] Vol.29 P.418-33 google doi
  • 25. Zipf GK 1949 Human behavior and the principle of least ef fort; an introduction to human ecology. P.573 google