A Social-Semiotic Approach to Color Analysis in Digital Media: Theory and Practice / Uma abordagem sociossemiótica para análise de cores em mídias digitais: teoria e prática

André de Oliveira Matumoto; Paulo Roberto Gonçalves-Segundo

doi:10.17851/2237-2083.31.4.2009-2065

A Social-Semiotic Approach to Color Analysis in Digital Media: Theory and Practice / Uma abordagem sociossemiótica para análise de cores em mídias digitais: teoria e prática

André de Oliveira Matumoto, Paulo Roberto Gonçalves-Segundo

Abstract

Abstract: This article aims to discuss the study of color from a social-semiotic perspective. The article begins with an overview of Gunther Kress and Theo van Leeuwen’s distinctive feature approach to color in their Grammar of Visual Design (2021) and concludes that the authors base their claims primarily on paintings and print media, which may skew the categories toward analog media. Drawing on color theory, mainly the contributions of Rhyne (2017), this article argues that social-semiotic analyzes can benefit from systematized and quantitative categories for color analysis, especially for digital media corpora. The article introduces several analytic categories, such as color harmony and the RGB color space, that are relevant to understanding how color works in digital spaces. The paper proposes the use of the free and open source software GIMP and ImageMagick for a qualitative and quantitative approach to the distinctive features of color and for a better understanding of the complexity of colors and their meaning potential. Based on these methodological procedures, the distinctive feature approach is revised with Rhyne’s contribution. Finally, the categories are applied to two stock images as a case study.

Resumo: Este artigo visa discutir o estudo da cor a partir da abordagem sociossemiótica. Começando com uma visão geral da abordagem da cor por Gunther Kress e Theo van Leeuwen, na sua Gramática de design visual (2021), conclui-se que os autores baseiam as suas constatações principalmente em pinturas e mídias impressas, o que pode enviesar as categorias em direção aos meios analógicos. Baseado na teoria das cores, principalmente nas contribuições de Rhyne (2017), o artigo argumenta que as análises que partem da semiótica social podem se beneficiar de categorias especializadas para a análise da cor, principalmente para corpora de meios digitais. O artigo introduz uma série de categorias analíticas, tais como a harmonia de cores e o espaço de cor RGB, que são relevantes para compreender como funciona a cor nos espaços digitais. Para demonstrar as categorias aqui apresentadas, são utilizados os softwares gratuitos e de código aberto GIMP e ImageMagick. Eles permitem uma abordagem qualitativa e quantitativa das características distintivas da cor e permitem uma melhor compreensão da complexidade das cores e do seu potencial de significado. Com base nesses procedimentos metodológicos, a abordagem de traços distintivos é revista através das contribuições de Rhyne. Finalmente, como um estudo de caso, as categorias são aplicadas em duas stock images.

Keywords

social semiotics; color theory; grammar of visual design; semiótica social; teoria das cores; gramática do design visual.

Full Text:

PDF

References

BATEMAN, J.; WILDFEUER, J.; HIIPPALA, T. Multimodality: Foundations, Research and Analysis – A Problem-Oriented Introduction. Berlin: De Gruyter Mouton, 2017.

COLOUR vision deficiency (colour blindness). NHS, England, UK, 01 Apr. 2019. Available at: https://www.nhs.uk/conditions/colour-vision-deficiency/#:~:text=It’s%20a%20common%20problem%20that,to%20someone%20with%20normal%20vision. Visited on: 21 Jan. 2023.

GIMP DOCUMENTATION TEAM. GIMP Documentation. GIMP. Available at: https://www.gimp.org/docs/. Visited on: 21 Jan. 2023.

GIMP main page. GIMP, 2023. Available at: https://www.gimp.org/. Visited on: 21 Jan. 2023.

HALLIDAY, M.A.K. Language as Social Semiotic. London: Arnold, 1978.

HERON, P.; CRAMERI, F.; SHEPHARD, G. How rainbow colour maps can distort data and be misleading. The conversation, 06 Oct. 2021. Available at: https://theconversation.com/how-rainbow-colour-maps-can-distort-data-and-be-misleading-167159. Visited on: 21 Jan. 2023.

HODGE, R.; KRESS, G. Social Semiotics. Cambridge: Polity Press, 1988.

IMAGEMAGICK main page. ImageMagick Studio LLC, 2023. Available at: https://imagemagick.org/index.php. Visited on: 21 Jan. 2023.

INKSCAPE main page. Inkscape, 2023. Available at: https://inkscape.org/. Visited on: 21 Jan. 2023.

JAKOBSON, R.; HALLE, M. Fundamentals of Language. Mouton: The Hague, 1956.

JEWITT, C.; BEZEMER, J.; O’HALLORAN, K. Introducing Multimodality. New York: Routledge, 2016.

JOHANNESSEN, C. M.; TVEDE, M. L.; BOESEN, K. C.; HIIPPALA, T. A Corpus-Based Approach to Color, Shape, and Typography in Logos. In: PFLAEGING, J.; WILDFEUER, J.; BATEMAN, J. A. (eds.). Empirical Multimodality Research: Methods, Evaluations, Implications. Berlin/ Boston: De Gruyter, 2021. p. 159–185.

JOHNSON, J.; KARPATHY, A.; FEI-FEI, L. Densecap: Fully convolutional localization networks for dense captioning. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, Las Vegas, US. Proceedings […]. Las Vegas, USA: IEEE, p. 4565-4574, DOI: https://doi.org/10.1109/CVPR.2016.494.

JUNCTIONS (username). Optical grey squares orange brown. Wikimedia Commons, 10 Apr. 2007. Available at: https://commons.wikimedia.org/wiki/File:Optical_grey_squares_orange_brown.svg. Visited on: 21 Jan. 2023.

KANDINSKY, W. Concerning the Spiritual in Art. New York: Dover Publications, 1977.

KARPATHY, A.; FEI-FEI, L. Deep visual-semantic alignments for generating image descriptions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, Boston, USA. Proceedings […]. Boston, US: IEEE, p. 3128-3137, DOI: 10.1109/CVPR.2015.7298932.

KINGDON Jonathan. The Kingdon Field Guide to African Mammals. Academic Press: San Diego, California, 1997.

KRESS, G.; VAN LEEUWEN, T. Colour as a semiotic mode: notes for a grammar of colour. Visual communication, [S.l], v. 1, n. 3, p. 343-368, 2002. DOI: https://doi.org/10.1177/147035720200100306. Available at: https://www.researchgate.net/publication/237623450_Colour_as_a_Semiotic_Mode_Notes_for_a_Grammar_of_Colour. Visited on: 21 Jan. 2023.

KRESS, G.; VAN LEEUWEN, T. Reading Images. Geelong, Australia: Deakin University Press, 1990.

KRESS, G.; VAN LEEUWEN, T. Reading Images: The Grammar of visual design. 1 ed. London/ New York: Routledge, 1996.

KRESS, G.; VAN LEEUWEN, T. Reading Images: The Grammar of Visual Design. 2 ed. London/ New York: Routledge, 2006.

KRESS, G.; VAN LEEUWEN, T. Reading Images: The Grammar of Visual Design. 3 ed. London/ New York: Routledge, 2021.

MATUMOTO, A. de O. A construção brasilidade em Street Fighter II: uma perspectiva sociossemiótica. In: GONÇALVES-SEGUNDO, P. R.; ARAES, C. R.; CASTANHEIRA, C.; ISOLA-LANZONI, G.; SILVA, L. P. da; MITSUNARI, N. A. S.; DOMINGUES, T. (eds.). Estudos do texto e do discurso: perspectivas contemporâneas. São Paulo, SP: FFLCH/USP, 2022a. p. 109–133. Available at: https://www.livrosabertos.sibi.usp.br/portaldelivrosUSP/catalog/book/935.

MATUMOTO, A. de O. Sobrevoando a Amazônia: a construção verbo-imagética do Brasil em três jogos shmup. Entrepalavras, Fortaleza, v. 12, n. 3, p. 92–122, 2022b. DOI: https://doi.org/10.22168/2237-6321-32512.

MATUMOTO, A. de O.; GONÇALVES-SEGUNDO, P. R. Towards a social-semiotic approach to visual analysis of two-dimensional games: a toolkit. Texto Livre, Belo Horizonte, vol. 15, no. 1, p. e39398, 2022a. DOI: https://doi.org/10.35699/1983-3652.2022.39398.

MATUMOTO, A. de O.; GONÇALVES-SEGUNDO, P. R. Uma Proposta Sociossemiótica para a Análise Visual de Jogos Bidimensionais: a construção de significado em Ducktales. Entreletras, [S.l.], v. 13, n. 1, p. 344–369, 2022b. DOI: https://doi.org/10.20873/uft2179-3948.2022v13n1p344-369.

PAVLIDIS, G. A Brief History of Colour Theory: Foundations of Colour Science. Cham, CH: Springer International Publishing, 2021.

POSSEBON, E. L. A teoria das cores de Goethe hoje. 2009. 168 f. Doutorado em Design e Arquitetura – Universidade de São Paulo, São Paulo, 2009. DOI 10.11606/T.16.2009.tde-10052010-144639. Available at: http://www.teses.usp.br/teses/disponiveis/16/16134/tde-10052010-144639/. Accessed on: 13 Jul. 2023.

PSIKYO. Strikers 1945 II. Kyoto, Japan: Psikyo, 1997.

RHYNE, T.-M. Applying color theory to digital media and visualization. Boca Raton: CRC Press, Taylor &Francis Group, 2017.

SARAVANAN, G.; YAMUNA, G.; NANDHINI, S. Real time implementation of RGB to HSV/HSI/HSL and its reverse color space models. In: International Conference on Communication and Signal Processing (ICCSP), 2016, Melmaruvathur, India. Proceedings […]: IEEE, p. 0462-0466, DOI: 10.1109/ICCSP.2016.7754179.

VAN LEEUWEN, T. Introducing Social Semiotics. London: Routledge, 2005.

VAN LEEUWEN, T. The Language of Colour: An Introduction. London: Routledge, 2011.

Video System. Aero Fighters 2 (Sonic Wings 2). Kyoto, Japan: SNK, 1994.

Video System. Sonic Wings Special. Kyoto, Japan: Media Quest/Phoenix, 1996.

VON GOETHE, J. W. Zur Farbenlehre. Tübingen, DE: Cotta, 1810. Available at: https://archive.org/details/gri_c00233125012873531. Accessed on: 13 Jul. 2023.

VON GOETHE, J. W. Goethe’s theory of colours. trans. by Charles Lock Eastlake. London, UK: John Murray, 1840. Available at: https://archive.org/details/goethestheoryco01goetgoog. Accessed on: 13 Jul. 2023.

DOI: http://dx.doi.org/10.17851/2237-2083.31.4.2009-2065