Modality semiotics: Encyclopedia II - Modality semiotics - Discussion of sign-type

Modality semiotics - Discussion of sign-type

The psychology of perception seems to suggest the existence of a common cognitive system which treats all or most sensorily conveyed meanings in the same way. If all signs must also be objects of perception, there is every reason to believe that their modality will determine at least part of their nature. Thus, the sensory modalities will be visual, auditory, tactile, olfactory, kinetic, etc. A list of sign types would include: text, symbol, index, image, map, graph, diagram, etc. Some combinations of signs can be multi-modal, i.e. different types of signs grouped together for effect. But the distinction between a medium and a modality should be clarified:

• text is a medium for presenting the modality of natural language;
• image is both a medium and a modality;
• music is a modality for the auditory media.

So, the modality refers to a certain type of information and/or the representation format in which information is stored. The medium is the means whereby this information is delivered to the senses of the interpreter. Natural language is the primary modality, having many invariant properties across the auditory media as spoken language, the visual media as written language, the tactile media as Braille, and kinetic media as sign language. When meaning is conveyed by spoken language, it is converted into sound waves broadcast by the speaker and received by another's ears. Yet this stimulus cannot be divorced from the visual evidence of the speaker's manner and gestures, and the general awareness of the physical location and its possible connotative significance. Similarly, meaning that is contained in a visual form cannot be divorced from the iconicity and implications of the form. If handwritten, is the writing neat or does it evidence emotion in its style. What type of paper is used, what colour ink, what kind of writing instrument: all such questions are relevant to an interpretation of the significance of what is represented. But images are distinguishable from natural language. For Roland Barthes (1915-80), language functions with relatively determinate meanings whereas images "say" nothing. Nevertheless, there is a rhetoric for arranging the parts which are to signify, and an emerging, if not yet generally accepted, syntax that articulates their parts and binds them into an effective whole.

Adapted from the Wikipedia article "Discussion of sign-type", under the G.N U Free Docmentation License. Please also see http://en.wikipedia.org/wiki