Facial expressions alter the fundamental sound properties of speech

Authors

Lucas Tessaro Laurentian University
Cynthia Whissell Laurentian University, Department of Pscyhology

DOI:

https://doi.org/10.34050/elsjish.v4i1.12786

Keywords:

PRAAT, phonology, phoneme, gender differences, pitch

Abstract

Literature from across academic disciplines has demonstrated significant links between emotional valence and language. For example, Whissell’s Dictionary of Affect in Language defines three dimensions upon which the emotionality of words is describable, and Ekman’s Theories of Emotion include the perception and internalization of facial expressions. The present study seeks to expand upon these works by exploring whether holding facial expressions alters the fundamental speech properties of spoken language. Nineteen (19) participants were seated in a soundproof chamber and were asked to speak a series of pseudowords containing target phonemes. The participants spoke the pseudowords either holding no facial expression, smiling, or frowning, and the utterances recorded using a high-definition microphone and phonologically analysed using PRAAT analysis software. Analyses revealed a pervasive gender differences in frequency variables, where males showed lower fundamental but higher formant frequencies compared to females. Significant main effects were found within the fundamental and formant frequencies, but no effects were discerned for the intensity variable. While intricate, these results are indicative of an interaction between the activity of facial musculature when reflecting emotional valence and the sound properties of speech uttered simultaneously.

References

Ackrill, J. L. (1975). Categories and De Interpretatione: Clarendon Press.

Allen, E. J., Burton, P. C., Olman, C. A., & Oxenham, A. J. (2016). Representations of Pitch and Timbre Variation in Human Auditory Cortex. Journal of Neuroscience, 2336-2316.

Aristotle. (2009). The Basic Works of Aristotle (R. McKeon, Trans.): Modern Library.

Bennett, S. (1981). Vowel formant frequency characteristics of preadolescent males and females. The Journal of the Acoustical Society of America, 69(1), 231-238.

Blasi, D. E., Moran, S., Moisik, S. R., Widmer, P., Dediu, D., & Bickel, B. (2019). Human sound systems are shaped by post-Neolithic changes in bite configuration. Science, 363(6432), eaav3218.

Blasi, D. E., Wichmann, S., Hammarström, H., Stadler, P. F., & Christiansen, M. H. (2016). Sound–meaning association biases evidenced across thousands of languages. Proceedings of the National Academy of Sciences, 201605782.

Busby, P. A., & Plant, G. L. (1995). Formant frequency values of vowels produced by preadolescent boys and girls. The Journal of the Acoustical Society of America, 97(4), 2603-2606.

Cahn, J. E. (1990). The generation of a ect in synthesized speech. Journal of the American Voice I/O Society, 8, 1-19.

Chevalier-Skolnikoff, S. (1973). Facial expression of emotion in nonhuman primates. Darwin and facial expression: A century of research in review, 11-89.

Childers, D., Wu, K., Bae, K., & Hicks, D. (1988). Automatic recognition of gender by voice. Paper presented at the Acoustics, Speech, and Signal Processing, 1988. ICASSP-88., 1988 International Conference on.

Chomsky, N. (1959). A review of BF Skinner's Verbal Behavior. Language, 35(1), 26-58.

Chomsky, N., & Lightfoot, D. W. (2009). Syntactic Structures: De Gruyter.

De Saussure, F. (2011). Course in general linguistics: Columbia University Press.

Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of personality and social psychology, 17(2), 124.

Ekman, P., Friesen, W. V., & Scherer, K. R. (1976). Body movement and voice pitch in deceptive interaction. Semiotica, 16(1), 23-28.

Ekman, P., & Oster, H. (1979). Facial expressions of emotion. Annual review of psychology, 30(1), 527-554.

Feld, S. L. (1981). The focused organization of social ties. American journal of sociology, 86(5), 1015-1035.

Fernald, A. (1989). Intonation and communicative intent in mothers' speech to infants: Is the melody the message? Child development, 1497-1510.

Fine, C. (2008). The Britannica Guide to the Brain: A Guided Tour of the Brain - Mind, Memory, and Intelligence: Robinson.

Frege, G. (1948). Sense and reference. The philosophical review, 57(3), 209-230.

Frith, C. D., & Frith, U. (2006). The neural basis of mentalizing. Neuron, 50(4), 531-534.

Grisoni, L., Miller, T. M., & Pulvermüller, F. (2017). Neural correlates of semantic prediction and resolution in sentence processing. Journal of Neuroscience, 37(18), 4848-4858.

Hirano, M., Vennard, W., & Ohala, J. (1970). Regulation of register, pitch and intensity of voice. Folia Phoniatrica et Logopaedica, 22(1), 1-20.

Hodgson, D. (2019). The origin, significance, and development of the earliest geometric patterns in the archaeological record. Journal of Archaeological Science: Reports, 24, 588-592.

Huber, J. E., Stathopoulos, E. T., Curione, G. M., Ash, T. A., & Johnson, K. (1999). Formants of children, women, and men: The effects of vocal intensity variation. The Journal of the Acoustical Society of America, 106(3), 1532-1542.

Huth, A. G., de Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016a). Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600), 453-458.

Huth, A. G., De Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016b). Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600), 453.

Huth, A. G., Nishimoto, S., Vu, A. T., & Gallant, J. L. (2012). A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron, 76(6), 1210-1224.

Jaynes, J. (2000). The Origin of Consciousness in the Breakdown of the Bicameral Mind: HMH Books.

Kocagoncu, E., Clarke, A., Devereux, B., & Tyler, L. K. (2016). Decoding the cortical dynamics of sound-meaning mapping. Journal of Neuroscience, 2858-2816.

Komiyama, S., Watanabe, H., & Ryu, S. (1984). Phonographic relationship between pitch and intensity of the human voice. Folia Phoniatrica et Logopaedica, 36(1), 1-7.

Kotti, M., & Kotropoulos, C. (2008). Gender classification in two emotional speech databases. Paper presented at the Pattern Recognition, 2008. ICPR 2008. 19th International Conference on.

Leslie, A. M. (1987). Pretense and representation: The origins of" theory of mind.". Psychological review, 94(4), 412.

Maddieson, I., & Coupé, C. (2015). Human language diversity and the acoustic adaptation hypthesis. Paper presented at the Proceedings of Meetings on Acoustics 170ASA.

Mitchell, J. P., Macrae, C. N., & Banaji, M. R. (2006). Dissociable medial prefrontal contributions to judgments of similar and dissimilar others. Neuron, 50(4), 655-663.

Modrak, D. K. W. (2001). Aristotle's Theory of Language and Meaning: Cambridge University Press.

Murray, I. R., & Arnott, J. L. (1993). Toward the simulation of emotion in synthetic speech: A review of the literature on human vocal emotion. The Journal of the Acoustical Society of America, 93(2), 1097-1108.

Neel, A. T. (2008). Vowel space characteristics and vowel identification accuracy. Journal of Speech, Language, and Hearing Research, 51(3), 574-585.

Pestilli, F. (2018). Human white matter and knowledge representation. PLoS biology, 16(4), e2005758.

Plato. (2015). Cratylus (T. Taylor, Trans.): CreateSpace Independent Publishing Platform.

Puts, D. A., Gaulin, S. J., & Verdolini, K. (2006). Dominance and the evolution of sexual dimorphism in human voice pitch. Evolution and Human Behavior, 27(4), 283-296.

Ramiro, C., Srinivasan, M., Malt, B. C., & Xu, Y. (2018). Algorithms in the historical emergence of word senses. Proceedings of the National Academy of Sciences, 115(10), 2323-2328.

Regier, T., Carstensen, A., & Kemp, C. (2016). Languages support efficient communication about the environment: words for snow revisited. PloS one, 11(4), e0151138.

Roy, D., & Pentland, A. (1996). Automatic spoken affect classification and analysis. Paper presented at the Automatic Face and Gesture Recognition, 1996., Proceedings of the Second International Conference on.

Sauter, D. A., Eisner, F., Ekman, P., & Scott, S. K. (2010). Cross-cultural recognition of basic emotions through nonverbal emotional vocalizations. Proceedings of the National Academy of Sciences, 107(6), 2408-2412.

Scherer, K. R. (1985). Vocal affect signaling: A comparative approach. Advances in the study of behavior, 15, 189-244.

Scherer, K. R., Koivumaki, J., & Rosenthal, R. (1972). Minimal cues in the vocal communication of affect: Judging emotions from content-masked speech. Journal of Psycholinguistic Research, 1(3), 269-285.

Sedaaghi, M. (2009). A comparative study of gender and age classification in speech signals. Iranian Journal of Electrical and Electronic Engineering, 5(1), 1-12.

Simonyan, K., Ackermann, H., Chang, E. F., & Greenlee, J. D. (2016). New Developments in Understanding the Complexity of Human Speech Production. Journal of Neuroscience, 36(45), 11440-11448.

Skinner, B. F. (2014). Verbal behavior: BF Skinner Foundation.

Sperber, D., Wilson, D., 何自然, & 冉永平. (1986). Relevance: Communication and cognition (Vol. 142): Citeseer.

Taylor, J., Davis, M. H., & Rastle, K. (2019). Mapping visual symbols onto spoken language along the ventral visual stream. Proceedings of the National Academy of Sciences, 201818575.

Ververidis, D., & Kotropoulos, C. (2004). Automatic speech classification to five emotional states based on gender information. Paper presented at the Signal Processing Conference, 2004 12th European.

Wells, R. S. (1945). The pitch phonemes of English. Language, 27-39.

Whissell, C. (1989). The dictionary of affect in language. emotion: theory, research and experience 4. The Measurement of Emotions. In: Academic Press, New York.

Whissell, C. (2000). Phonoemotional profiling: a description of the emotional flavour of English texts on the basis of the phonemes employed in them. Perceptual and Motor Skills, 91(2), 617-648.

Whissell, C. (2001). Sound and emotion in given names. Names, 49(2), 97-120.

Whissell, C. (2006). The flow of emotion through Beowulf. Psychological reports, 99(3), 835-850.

Whissell, C. (2009). Using the revised dictionary of affect in language to quantify the emotional undertones of samples of natural language. Psychological reports, 105(2), 509-521.

Wittgenstein, L. (2010). Philosophical investigations: John Wiley & Sons.