December 08, 2014 by Romain Bouqueau

Apple HLS: technical in-depth

Content

• Master, Variants and Renditions
• Version 1
• Version 2
• Version 3
• Version 4
• Version 5
• Version 6
• Versions 7, 8, 9 (functionaly equivalent)
• Version 10

An introduction to Apple HLS is available as: Apple HLS: introduction.

A summary table of the versions and revisions is available as: Apple HLS: comparing versions.

Master, Variants and Renditions

A complex presentation can be described by a Master Playlist.

The Master Playlist provides a set of Variant Streams, each of which describes a different version of the same content.

A Variant Stream can also specify a set of Renditions. Renditions are alternate versions of the content, such as audio produced in different languages or video recorded from different camera angles.

Version 1

The initial version defines nine new tags:

EXTM3U

An Extended M3U file is distinguished from a basic M3U file by its first line which MUST be #EXTM3U.

EXTINF

An Extended Info Marker describing the media file that follows it:

#EXTINF:<duration>,<title>

EXT-X-TARGETDURATION

Approximate duration of the next media file (the real duration must be less or equal). The EXTINF duration segment file must be less than or equal than EXT-X-TARGETDURATION:

#EXT-X-TARGETDURATION:<seconds>

EXT-X-MEDIA-SEQUENCE

Sequence number:

#EXT-X-MEDIA-SEQUENCE:<number>

Starting at version 5, a client shall not assume that segments with the same media sequence number in different masters, variants or renditions contain matching content.

EXT-X-KEY

Optional tag. The encryption algorithm is AES-128 CBC with PKCS7 padding:

#EXT-X-KEY:METHOD=<method>[,URI="<URI>"]

EXT-X-PROGRAM-DATE-TIME

Associates the beginning of the next media file with an absolute time:

#EXT-X-PROGRAM-DATE-TIME:<YYYY-MM-DDThh:mm:ssZ>

Starting at revision 20 (version 7) it should provides milliseconds:

#EXT-X-PROGRAM-DATE-TIME:<YYYY-MM-DDThh:mm:ss.SSSZ>

EXT-X-ALLOW-CACHE

Specifies whether the client may cache downloaded media files for later replay:

#EXT-X-ALLOW-CACHE:<YES|NO>

EXT-X-STREAM-INF

The next URI in the Playlist is another Playlist file:

#EXT-X-STREAM-INF:[BANDWIDTH=<n>],[PROGRAM-ID=<i>],[CODECS="[format][,format]*"] <URI>

EXT-X-ENDLIST

We reached the end of the playlist:

#EXT-X-ENDLIST

EXT-X-DISCONTINUITY

Added in revision 2.

Indicates that the next media file has different characteristics than the previous one:

#EXT-X-DISCONTINUITY

Version 2

EXT-X-STREAM-INF (additions)

The RESOLUTION=<N>x<M> attribute is added.

EXT-X-VERSION

Indicates the compatibility version of the Playlist file:

#EXT-X-VERSION:<n>

EXT-X-KEY (additions)

Added the optional IV attribute.

Version 3

EXTINF (additions)

The duration can be expressed with a floating point argument.

EXT-X-PLAYLIST-TYPE

Optional:

#EXT-X-PLAYLIST-TYPE:<EVENT|VOD>

If the tag is present and has a value VOD, the playlist shall not change. If the tag is present and has a value of EVENT, the server may only append lines to the playlist.

Version 4

Rendition groups

A set of EXT-X-MEDIA tags with the same GROUP-ID value forms a group of renditions.

EXT-X-STREAM-INF (additions)

New AUDIO and VIDEO attributes.

AUDIO (resp. VIDEO)
The value is a quoted-string. It MUST match the value of the
 GROUP-ID attribute of an EXT-X-MEDIA tag elsewhere in the Playlist
 whose TYPE attribute is AUDIO (resp. VIDEO). It indicates the set of audio
 renditions that MAY be used when playing the presentation.

EXT-X-BYTERANGE

Indicates that a media file is a sub-range of the resource identified by its media URI:

#EXT-X-BYTERANGE:<n>[@o]

With:

• n: length of the sub-range
• o (optional): offset, may be inferred from the previous media segment length and computed offset.

EXT-X-MEDIA

Indicates that playlists contain alternate renditions of the same content. For example two audio languages, or two video camera angles:

#EXT-X-MEDIA:[TYPE={AUDIO,VIDEO}],[URI],[GROUP-ID],[LANGUAGE],[NAME],[DEFAULT={YES,NO}],[AUTOSELECT={YES,NO}]

EXT-X-I-FRAME-STREAM-INF

Identifies a playlist containing the I-frames of a multimedia presentation. It stands alone, in that it does not apply to a particular URI in the playlist:

#EXT-X-I-FRAME-STREAM-INF:<attribute-list>

Same attributes as EXT-X-STREAM-INF, minus the AUDIO attributes, plus a URI attribute to identify the I-frame playlist file.

EXT-X-I-FRAMES-ONLY

Indicates that each media segment in the Playlist describes a single I-frame:

#EXT-X-I-FRAMES-ONLY

Version 5

Subtitle segments

They must use WebVTT.

Each WebVTT segment MUST have an X-TIMESTAMP-MAP metadata header.

EXT-X-KEY (additions)

New encryption method SAMPLE-AES. SAMPLE-AES only encrypts the audio and video payloads. This is how DVB encryption works also.

The legacy encryption in HLS was called AES-128. AES-128 encrypts the full TS packets including headers.

The possible attributes are URI, IV, KEYFORMAT, KEYFORMATVERSIONS.

EXT-X-MEDIA (additions)

The TYPE attribute can have the value SUBTITLES.

New attributes [FORCED={YES,NO}], [CHARACTERISTICS=UTI].

EXT-X-MAP

The EXT-X-MAP tag specifies how to obtain the Transport Stream PAT/

PMT for the applicable media segment.

It applies until the next EXT-X-DISCONTINUITY tag

#EXT-X-MAP:<attribute-list>

With the attributes URI and BYTERANGE.

Version 6

EXT-X-MEDIA (additions)

Added the TYPE attribute value CLOSED-CAPTIONS. The media segments for the video renditions can include closed captions.

The attributes URI, ASSOC-LANGUAGE, CHARACTERISTICS and INSTREAM-ID={CC1,CC2,CC3,CC4} are added.

EXT-X-STREAM-INF (additions and removals)

The CLOSED-CAPTIONS attribute is added.

The PROGRAM-ID attribute is removed.

EXT-X-I-FRAME-STREAM-INF (removals)

The PROGRAM-ID attribute is removed.

EXT-X-DISCONTINUITY-SEQUENCE

Allows synchronization between different renditions of the entire playlist, same variant stream or different variant streams that have EXT-X-DISCONTINUITY tags in their playlists:

#EXT-X-DISCONTINUITY-SEQUENCE:<number>

For revision 13 specifically, a playlist that contains an EXT-X-PLAYLIST-TYPE tag with a value of EVENT or VOD must not contain an EXT-X-DISCONTINUITY-SEQUENCE tag.

EXT-X-START

Indicates a preferred point at which to start playing a playlist:

#EXT-X-START:[TIMEOFFSET],[PRECISE={YES,NO}]

Media Segments (audio only)

Each Elementary Audio Stream segment MUST signal the timestamp of its
 first sample with an ID3 PRIV tag [ID3] at the beginning of the
 segment. The ID3 PRIV owner identifier MUST be
 "com.apple.streaming.transportStreamTimestamp". The ID3 payload MUST
 be a 33-bit MPEG-2 Program Elementary Stream timestamp expressed as a
 big-endian eight-octet number, with the upper 31 bits set to zero.

EXT-X-INDEPENDENT-SEGMENTS

Added at revision 13.

Indicates that all media samples in a segment can be decoded without information from other segments:

#EXT-X-INDEPENDENT-SEGMENTS

Versions 7, 8 and 9

Note: version 8 introduces no new tag/attribute and version 9 does not exist.

The specification was re-written in a more readable way.

Alternative Renditions

EXT-X-STREAM-INF tag containing an AUDIO, VIDEO, SUBTITLES, or CLOSED-CAPTIONS attribute indicates that alternative Renditions are available for playback of that Variant Stream.

EXT-X-SESSION-DATA

Allows arbitrary session data to be carried in a Master Playlist:

#EXT-X-SESSION-DATA:<attribute list>

The attributes DATA-ID, VALUE, LANGUAGE and URI are defined.

EXT-X-STREAM-INF (additions)

The AVERAGE-BANDWIDTH attribute was added.

The FRAME-RATE attribute was added in revision 15.

The HDCP-LEVEL attribute was added in revision 20.

The VIDEO-RANGE attribute was added in revision 00 of rfc8216bis.

The ALLOWED-CPC attribute was added in revision 06 of rfc8216bis.

EXT-X-ALLOW-CACHE (removal)

The tag was removed.

EXT-X-MEDIA (additions)

INSTREAM-ID value SERVICE indicated a CEA-708 Digital Television Closed Captioning.

(Revision 20): the attribute CHANNELS is added.

EXT-X-DATERANGE (revision 19)

associates a time/date range (oddly named "Date Range") with an attribute/value pair:

#EXT-X-DATERANGE:<attribute list>

The attributes ID, CLASS, START-DATE, END-DATE, DURATION, PLANNED-DURATION, X-<client-attribute> (client defined), SCTE35-CMD, SCTE35-OUT, SCTE35-IN, END-ON-NEXT, and CUE (since revision 12 of rfc8216bis) are defined.

NB: as of revision 06 of rfc8216-bis, this can be removed from playlists.

EXT-X-KEY

Starting at revision 20 (version 7) it should provides milliseconds:

#EXT-X-PROGRAM-DATE-TIME:<YYYY-MM-DDThh:mm:ss.SSSZ>

EXT-X-MAP

Made mandatory for MPEG-4 Fragmented (starting at revision 20).

Version 10

The format of this update is shorter. That's because the HLS specification itself is better structured. This document is only intended to help people navigate in HLS history.

Master Playlist is now called "Multivariant Playlist".

The #EXT-X-BITRATE, #EXT-X-GAP, #EXT-X-MEDIA STABLE-RENDITION-ID, #EXT-X-SERVER-CONTROL, #EXT-X-RENDITION-REPORT, #EXT-X-SESSION-KEY and #EXT-X-DEFINE tag were added. That's a lot.

#EXT-X-STREAM-INF: additions of new SCORE (estimated QoE) and SUPPLEMENTAL-CODECS (codecs containing a base and enhancements layers e.g. Dolby Vision) attributes.

Low latency:

• The EXT-X-SKIP, EXT-X-PART-INF, EXT-X-PRELOAD-HINT, EXT-X-PART tags were added.
• Playlist Delta Update: CAN-SKIP-UNTIL, CAN-SKIP-DATERANGES, and CAN-BLOCK-RELOAD=YES attribute to the EXT-X-SERVER-CONTROL tag.

Content steering: the #EXT-X-CONTENT-STEERING tag was added with the PATHWAY-ID and SERVER-URI attributes.