A study of the semiotic role of sound in interactive media
Sound: an underestimated medium in interactive design
Jan Meinem
1.3. Sound as a language
1.3.1. Semiotics
To look at sound as a language we can make some links with semiotics.
Semiotics
1
is the study of how signs communicate meanings with linguistic signs
(words) being one a part of this. Ferdinand de Saussure, a Swiss academic, in his
work Course in General Linguistics
2
describes a study of linguistic signs. Saussure
claimed that this study could be applied to other types of signs that communicate
meanings, like images, and study them in similar ways.
Signs, according to Saussure, consist of two parts, firstly the signal or signifier
which could be the sound patterns of a word and secondly the signification or
signified, which is the actual concept of a word, it's meaning. He also states that
the link between these, signal and signification, is arbitrary in the sense that the
word could have been any other word (e.g. table could have been the word to
describe the concept of a door). In other words the sound pattern door has no
direct connection to the concept of a piece of wood used to close a walkthrough
passage in a wall. Saussure stated:
"The psychological nature of our sound patterns becomes clear when we
consider our own linguistic activity. Without moving either lips or tongue,
we can talk to ourselves or recite silently a piece of verse. We grasp the10
words of language as sound patterns." (Saussure, 1915, p 11)
This applies to sound as well. We can silently 'hum' a melody from a well-known
song or 'imagine' what broken glass would sound like.
Another important point Saussure made was that culture and society agree on
what a certain sign means Although individuals might have slightly varying
understandings of signs, there seems to be a commonly agreed meaning attached
to them.
1.3.2. Music as a language
If we would consider a piece of music to be a signal, used to communicate a
certain message, than the concept or signification of that musical signal is not as
easy to describe. The interpretation will probably vary a lot from individual to
individual, much more so than with linguistic signs within a certain language
3
For music we need to learn a new language. Music communicates in a different
way. It's content is much less specific than linguistic language and yet it is a very
powerful language.
Roland Barthes, a French critic describes in his book Mythologies that signs do11
often not just denote a certain concept, like the door being a piece of wood used as
a separation between rooms, but have other connotative meanings included with
them. For example in a television advert; a shot of a woman driving a small car in
Paris does not denote just that, but might also connote things like independence,
cheekiness and confidence. These are all associative ideas formed by society and
culture.
Music especially relies on these connotative meanings. The associative links within
each individual, evoking emotional reactions linked to an individuals past
experiences with hearing a type of music in a certain context. Film music and
television have both probably become a huge provider of these connotative
meanings. So very much like with linguistics, a collectively agreed language that
seems to be much less specific in its description than words are. Nicholas Cook
describes this when he refers to music;
"Its ‘powers’ of transference are limited – it can convey sadness but cannot
distinguish sadness from depression, despair or gloom." (Cook, 1998, p )
However music (or perhaps sound in general) has less 'language' barriers and is a
more international language than linguistic language. But perhaps the best
examples when sound is used to communicate meanings, becomes apparent
when it is combined with other mediums like visuals or text.
When sound is combined with visuals the audience is practically not aware of the
presence of the music (provided that it is stopped and started at appropriate
moments). They simply forget that it is there and get immersed in the total
experience. Sound and visuals have also the ability to transfer their specific
'qualities' onto one another.
"By working with the image the sound seeks to explain the events,
emotions and meaning, to transfer its clarity of meaning to the other. This
meaning is not contained ‘within’ the music, in the same way as sad14
music does not ‘contain’ sadness within it. It transfers qualities of sadness
into an image and onto the spectator." - Cook
Music is the unseen character, in any media form it does become
something that the listener may not really be thinking about, but yet affects
their perception of what is actually happening on screen, whether that is a
computer screen, television screen or a film screen.
To improve sound design in media, one needs to be aware of what
sound can do and start viewing it as a medium that has huge potential to
communicate messages to the end user. It participates, in a 'hidden' way, in
constructing meanings in interactive works through; juxtaposing sound images
against other media, the use of non-diegetic sound to comment on visual events
and the powerful immersing qualities of environmental sounds. These are tools that
are rarely used to their maximum potential.
A more comprehensive evaluation will be completed in due course as i develop the concept further..
No comments:
Post a Comment