Information Technology — MPEG-21 Multimedia Framework

MPEG-21 (g)BSD-based multimedia publishing

Part 7 of MPEG-21 Digital entitled Digital Item Adaptation (DIA) specifies normative description schemes (also known as tools) enabling device and coding-format independent multimedia resource adaptation.

Device independence is achieved by means of the Usage Environment Description (UED) tools. As such, UED allows to describe the User’s characteristics (User info, usage preferences, usage history, audio/display/graphics presentation preferences, color preference, stereoscopic video conversion, conversion preferences, presentation priority preferences, focus of attention, auditory/visual impairment, color vision deficiency, mobility characteristics, and destination), the terminal’s capabilities (codec/display/audio output capabilities, user interaction input, device class, power characteristics, storage, data I/O, benchmarks, and IPMP tools), dynamic (available bandwidth, delay, and error) and static (maximum capacity, minimum guaranteed, in sequence delivery, error delivery, and error correction) network characteristics and the natural environment (location, time, audio environment, and illumination characteristics.) where Digital Items may be consumed.

Coding-format independence is achieved by means of describing the high-level structure of the bitstream’s syntax using XML. As such, scalable formats like MPEG-4 and JPEG2000 as well as the emerging MPEG-21 Scalable Video Coding (SVC) can be described, i.e., how they are organized in frames, layers, or packets. The resulting XML document is called a (generic) Bitstream Syntax Description (for the difference between gBSD and BSD see Figure 1). The adaptation is performed by editing style operations (e.g., discarding gBSD portions corresponding to certain frame types or updating specific layer information) on the gBSD with the subsequent actual bitstream adaptation process by means of the generic-Bitstream-Syntax-Description-to-Binary (gBSDtoBin) processor. The actual adaptation parameters for the gBSD transformation are provided by the Adaptation Quality of Service (AQoS) description which describes the relationship between, for example, device constraints, feasible adaptation operations satisfying these constraints, and associated utilities (qualities) of the multimedia resource. As such it takes the UED into account which may be further constraint by Universal Constraints Descriptions (UCD). For example, the UED might describe a 1,400 x 1,050-pixel resolution, and the UCD constrains this further, e.g., only 70% of this is available. Note that UCD can be also used to further constrain the AQoS, i.e., the usage of the resource with respect to adaptation possibilities.

Figure 1 — Differences between gBSD and BSD.
 (*** click on figure to enlarge ***)

Components

• generic Bitstream Syntax Description (gBSD). Coding-format independent description of the bitstream’s syntax, i.e., how the bitstream is organized in frames, layers, or packets.
• generic-Bitstream-Syntax-Description-to-Binary (gBSDtoBin) processor. Adaptation processor which behaviour is normatively specified in MPEG-21 DIA, i.e., it produces an possibly adapted bitstream by means of parsing and interpreting the information contained in the generic Bitstream Syntax Description.
• Adaptation Quality of Service (AQoS). Describes the relationship between constraints, feasible adaptation operations satisfying these constraints, and possibly associated utilities (qualities).
• Universal Constraints Description (UCD). Further constraining the ‘use of Digital Items’ as well as the ‘usage environment’ where the Digital Item is consumed.
• Extensible Stylesheet Language Transformations (XSLT) and Streaming Transformations for XML (STX). Standardized and open source XML transformation techniques.
• Usage Environment Description (UED). Device independent description of user characteristics, terminal capabilities, network characteristics, and natural environment characteristics.

Multimedia publishing

The high-level architecture for the gBSD-based multimedia publishing is depicted in Figure 2.

Figure 2 — High-level architecture for gBSD-based multimedia publishing.
 (*** click on figure to enlarge ***)

A User requests a Digital Item including her/his/its context information in form of an Usage Environment Description (UED). The UED together with the AdaptationQoS (AQoS) description is used to perform the adaptation decision-taking process, i.e., to determine the actual adaptation parameters. These parameters are used as input for the XML transformation style sheet, e.g., XSLT or STX, responsible for transforming the gBSD which describes the actual multimedia resource. The resulting transformed gBSD is used by the gBSDtoBin processor to generate the adapted bitstream which is forwarded to the user where it is consumed according to the usage environment.

Example scenario

Assume someone requests a Digital Item with JPEG2000 images using different devices, e.g., notebook or mobile devices. According to architecture from above following steps are performed:

• Step 1: BSDLink contains or refers to all required information assets (cf. Figure 3).

  

  Figure 3 — BSDLink description as a wrapper for all required information assets.
   (*** click on figure to enlarge ***)

• Step 2: Adaptation decision-taking according to the usage environment description (cf. Figure 4).

  

  Figure 4 — Adaptation decision-taking according to the display’s resolution.
   (*** click on figure to enlarge ***)

• Step 3: gBSD transformation using XSLT or STX (cf. Figure 5).

  

  Figure 5 — gBSD transformation using XSLT or STX.
   (*** click on figure to enlarge ***)

• Step 4: gBSDtoBin generates the adapted image (cf. Figure 6).

  

  Figure 6 — gBSDtoBin generates the adaptated image.
   (*** click on figure to enlarge ***)

Acknowledgements

• Jörg Heuer, Gabriel, Panis, and Andreas Hutter from Siemens AG
• Eric Delfosse from IMEC
• Debargha Mukherjee from HP Labs
• Sylvain Devillers from France Telecom R&D
• Harald Kosch from Klagenfurt University

References

• ISO/IEC 21000-7:2004, Information technology — Multimedia framework (MPEG-21) — Part 7: Digital Item Adaptation

© 2004-2005 Department of Information Technology (ITEC), Klagenfurt University, Austria
last updated: 2005/01/03 08:45am CET by