A Semiotic Definition of Multimedia Communication by Helen C. Purchase

The aim in this article is to provide a sound definition of multimedia that is based on an extension to a common classification of semiotic representational systems, in both the visual and aural modalities. In addition, the differing nature of communication devices is considered, providing a theoretical framework and terminology for consistent and appropriate discussion about multimedia texts and devices that are used to transmit them.

By separating the classification of the nature of the message to be communicated from the nature of the device, the intension is to focus on the importance of appropriate matching of text with technology. In this article, multimedia is defined independently of communication devices.

Ultimately, the message will be appropriately matched with the technology. This matching needs to be done with respect to the semiotic nature of both the message and the intented device.

Jerome Bruner classify semiotics system in three types:

Type of Semiotic System Explanation Example
Enactive based on physical movement and the learning of the responses the actions required for riding a bike
Iconic depends on imagery and perception pictures and photos
Symbolic uses symbols which do not have a perceptual relationship with concepts they signify word, traffic lights

Syntactic Categories in the visual modality

Syntactic Categories Explanation Concrete-iconic Abstract-iconic Symbolic
Individual There is only a single object to communicate a single concept, and only a single moment in time is required in order to receive the message photograph of an author on the book road sign of falling rocks EXIT notice on the door
Temporal Time is important, the message cannot be interpreted if only a snapshot is taken. Like the individual, only a single concept is communicated by a single object. continuous film of a waterfall changing the backdrop when a cartoon character is falling, to indicate the distance rotating cursor to indicate that the computer is busy processing
Linear Object is in a purely sequential manner and interpretation of the message depends on the objects being considered in this linear arrangement over time. Unlike temporal, more than one concept may be communicated movies cartoons book
Schematic Use spatial indicators to show the structure of the information. They represent relationship btw the concepts associated with the individual objects in a two- or three-dimensional manner according to a conventional code. diagram representing the hierarchy of the organization. bar chart illustrating population growth a desktop interface displays icons representing directories
Network not describe the syntax for individual objects, but used for the arrangements of minitexts within a composite texts.

Unlike others, does not restrict the order in which the user receives the minitexts.

 video-based interactive fiction story animated version video-based interactive fiction story an online thesaurus with hypertexts links btw related entries

Syntactic Categories in the aural modality

Syntactic Categories Explanation Concrete-iconic Abstract-iconic Symbolic
Individual There is only a single object to communicate a single concept, and only a single moment in time is required in order to receive the message a recording of a car ignition which may be used in radio drama synthesized machine "wrr" sound from computer which may indicate that an inference engine is working on a problem beep sound from a computer that occurs when new mail arrives in an electronic mailbox
Temporal Time is important, the message cannot be interpreted if only a snapshot is taken. Like the individual, only a single concept is communicated by a single object. recording of laughter accompanying a TV commedy synthesized sound of a gunfire in a violent arcade game fire alarm or doorbell
Linear Object is in a purely sequential manner and interpretation of the message depends on the objects being considered in this linear arrangement over time. Unlike temporal, more than one concept may be communicated a recording of a storm, including the build up, height, damage that it causes, and its conclusions synthesized sounds representing a train's approach, passing, and departure continual computer hmmm which changes the pitch depending on the number and nature of the people using the computer network
Schematic Use spatial indicators to show the structure of the information. They represent relationship btw the concepts associated with the individual objects in a two- or three-dimensional manner according to a conventional code. recording of sounds in a forest associate the individual sounds with locations relative to the receiver synthesized sound of footsteps associate the individual sounds with locations relative to the receiver different locations of warning sounds
Network not describe the syntax for individual objects, but used for the arrangements of minitexts within a composite texts.

Unlike others, does not restrict the order in which the user receives the minitexts.

 audio cassette comprising different bird songs demonstration of audio cassette comprising examples of synthesized sounds for possible use in an animated film audio cassette used in language labs and touch tome telephone menus of recorded spoken info.

 

 

Definition of multimedia communication can be defined as the production, transmission, and interpretation of a composite texts, when at least two of the minitexts use different representational systems in either modality.

Examples:

a wall poster with has photos, written texts, and maps
guided tour of a art gallery
a picture of a waterfall with its sound

Communication devices

Classification of communication devices

Classification Non Interactive Interactive
  inflexible, does not allow reader to interact with it.

the reader has no choice but receive the message as it comes, and no input facilities are provided.

 have the flexibility to allow the text to be controlled by the receiver.

this interaction enables the receiver to determine the parts of the message are to be communicated at any time.
Types Static devices do not enable a non-interactive text to be transmitted over time Dynamic devices are capable of transmitting a non-interactive text over time Interaction may take place in two ways
Enactive: the action receiver makes is natural and related to the desired action Symbolic: the action that the receiver makes has no relation to the desired action
Example Paper, poster, road signs Morse code transmitters, musical instruments, and TV. closing a book and opening a new page mouse-based action of browsing in a web-browser, pressing the fast forward button in a VCR.
      books, video machines, newspapers, computers