Chapter 1. Introduction to the Digital Media Libraries
Prev |  | Next

## Chapter 1. Introduction to the Digital Media Libraries

The digital media development environment (DMdev) is a family of libraries that provides application program interfaces (APIs) for digital media I/O, file operations, playback, and conversions.

Figure 1-1 shows how the libraries in the digital media development environment are related to each other and to other development libraries. Lines in Figure 1-1 indicate where some libraries make internal calls to other libraries.

Figure 1-1. Silicon Graphics Digital Media Programming Environment

Together, the family of Digital Media Libraries encompass facilities for data format description, live audio and video I/O with built-in conversion capabilities, file operations such as reading, writing, and editing multimedia files, data conversion and compression, and playback. The following sections describe these facilities.

## Digital Media Data Specification Facilities

What distinguishes the Digital Media Libraries from other multimedia developer software is the ability to accept any data regardless of format. You aren't limited to working with a particular format or the constraints it imposes. One of the things that makes this possible are the extensive data specification facilities offered by the DMdev.

The DM Library, `libdmedia.so`, provides global data type and parameter definitions for specifying digital media data attributes. You can use DM parameters to describe data held in memory, passed among the Digital Media Libraries, and imported and exported externally. File formats for on-disk data are described in “Digital Media File Operation and Conversion Facilities.”

The DM Library features

- type definitions for digital media

- routines for creating and configuring digital media parameters

- routines for creating and configuring digital media buffers

- a debugging version of the library that lets you check for proper usage

See Chapter 3, “Digital Media Parameters,” for a complete explanation of using the DM data types and parameters.

## Digital Media I/O Facilities

The Audio Library (AL), the Video Library (VL), and the MIDI Library enable real-time I/O by providing the interface between your application program, the workstation CPU, and external devices. An overview of the digital media I/O capabilities follows, and Chapter 4, “Digital Media I/O,” provides a complete explanation of using the digital media I/O APIs.

### Audio I/O

The AL provides a device-independent C language API for programming audio I/O on all Silicon Graphics workstations. It provides routines for configuring the audio hardware, managing audio I/O between the application program and the audio hardware, specifying attributes of digital audio data, and facilitating real-time programming.

Use the AL for

- capturing audio from your workstation's microphone, line-level inputs, or a digital audio input source

- playing audio to your workstation's internal speaker, line-level outputs, headphones, or a digital output

- managing audio I/O between multiple audio devices

- adding audio to any application program

### Video I/O

The VL provides an API for transporting live video on Silicon Graphics workstations equipped with on-board video and video options. The VL enables live video flow into a program.

Use the VL for

- displaying live video input in an onscreen window

- capturing video from your workstation's camera input, S-video input, composite analog inputs, or a digital input port into program memory

- playing video from program memory to your workstation's analog or digital video output

- combining video with computer graphics

| Note: The range of video and VL capabilities you can use depends on the capabilities of your workstation and the video options installed in it.

### MIDI I/O

The MIDI Library provides an API for sending, receiving, processing, and synthesizing musical instrument digital interface (MIDI) messages through the serial interface of Silicon Graphics workstations.

The MIDI Library enables

- content creation through an API to sound synthesizer and sequencer tools

- live interaction through a virtual 3D keyboard

## Digital Media Live Data Transport Facilities

The digital media buffers live data transport system provides data types and operations for sharing and exchanging time-sensitive visual data in real time between video I/O devices, compression devices, graphics rendering and texturing operations, and the host processor(s). It includes

- Digital media buffers (DMbuffers) for carrying digital representations of images

- Digital media buffer pools (DMbufferpools) for reserving, apportioning, and allocating dedicated system and physical memory under direct application control for live visual data processing/transport devices

See Chapter 4, “Digital Media I/O,” for a complete explanation of using DMbuffers.

## Digital Media File Operation and Conversion Facilities

Both the Movie Library and the Audio File Library provide file operations (identifying, opening, reading, writing, and editing files). These routines are capable of supporting a variety of file formats. Note the distinction between a file or container format, which is associated with media stored on disk (or tape), and the data format, which is a collection of attributes that describes the data. File format information is typically contained in a header that immediately precedes the data.

### Audio Files

The Audio File Library, `libaudiofile`, provides a uniform C language API for indentifying, opening, reading, writing, and converting digital audio files of a variety of storage formats.

Use the Audio File Library for

- identifying audio files (multimedia file recognition)

- opening and creating audio files

- seeking, reading, and writing audio files

- setting and retrieving information in audio file headers

- setting and retrieving characteristics of the audio file or the data it contains

- converting audio file formats

### Movie Files

The Movie File Library, `libmoviefile`, provides a file format-independent C language API for reading, writing, editing, and playing movies on Silicon Graphics workstations.

Use the Movie Library for:

- reading, writing, and editing movie files

- converting movie files from one container format to another

- compressing and decompressing movie files

- supporting movies embedded in applications programs

The Movie File Library provides a uniform interface to movies of various formats and lets you convert movies from one format to another. Currently, the Movie Library supports the following file formats:

- Apple® Computer QuickTime™ movie format, including uncompressed data, and JPEG, Indeo®, Cinepak®, Apple Animation, and Apple Video, compression

- Microsoft® audio-video interleaved (AVI) format, including uncompressed data, and JPEG, Indeo, and Cinepak compression

- MPEG-1 systems and video bitstreams

- Silicon Graphics movie format

| Note: The Digital Media Libraries do not provide a QuickTime API, rather, they provide a file format-independent API that supports QuickTime and several other formats.

| Note: Some QuickTime track types are not supported by the Movie File Library.

## Digital Media Data Conversion Facilities

The Digital Media Library, `libdmedia`, provides data conversion support for real-time I/O and file operations. Many of the file operations and real-time I/O routines perform automatic data format conversions, by calling these lower level converter APIs, which are also accessible directly from your application:

- dmAC, an audio conversion API that performs audio compression and decompression and audio sample rate conversion

- dmIC, an image conversion API that performs image compression and decompression and color space conversion.

## Digital Media Playback Facilities

The Digital Media Libraries provide a playback API, capable of playing audio, video, movies, and MIDI.

### Movies

The Movie Playback Library, `libmovieplay`, is implemented using calls from the OpenGL and Digital Media Libraries. It provides a scheduler and modules to communicate with output devices. You can take advantage of its built-in playback support in your application.

The movie library playback engine provides:

- asynchronous playback support

- flexible playback control (start, stop, speed, looping)

- the ability to properly combine and blend multiple image and audio tracks

- software scaling of audio

The main advantage of the built-in playback support is that it performs audio and video synchronization for you. Otherwise you would have to calculate the rate for each track and determine the proper display timing. You can still take advantage of this synchronization capability even if you want to use your own display method, by turning off the movie library display and using your own event loop to respond to events. You may also choose to create your own playback using routines from the other libraries.

Prev | Table of Contents | Next
About This Guide |  | Chapter 2. Digital Media Essentials