Getting Insights into the Delivery Path of Media with Breadcrumbs
Media streaming typically involves three main components: a player, the origin, and a CDN. Typically, the content provider owns the player and the origin stack and hence is able to diagnose and troubleshoot issues, but the CDN for the most part is a black box.
The player interacts with the edge network while the origin is only aware of a limited number of machines of the CDN that interact with it to fetch content. Everything else in the middle owned by the CDN is completely opaque to the player and the origin side.
This black box nature of the CDN hides the delivery path taken to serve individual players. Consequently, for any performance or availability issues, the content provider is unaware of the geo, ASN, or specific caching layer of the CDN that might be the source of these issues, resulting in manual escalations and mitigation steps.
Breadcrumbs is a new feature of Akamai’s media delivery products and provides customers with visibility into how each player request was handled and served from within the CDN.
For every request made by the player, the response will additionally contain a header named “Akamai-Request-BC.” This header provides details about the different IPs involved in handling the request to eventually serve the requested object back to the player.
Improving Visibility into Delivery Performance
With Breadcrumbs, a player has visibility into the tier that finally served the data for the response. This can be paired with existing client-side metrics such as TTFB and throughput metrics to get insights, as follows:
Figure 1 — TTFB and throughput to player bucketed into tier serving data
The player-side data can be aggregated to create a holistic view of overall delivery performance, as shown below:
Figure 2 — Traffic Split View showing percent of requests served from each tier
The Breadcrumbs Format
The format of a single Breadcrumb is a comma-delimited list of strings or fields, where each field represents some aspect of the IP that was involved in handling the current player request. The complete string for this Breadcrumb is wrapped in a pair of square brackets.
Example Syntax of a Single Breadcrumb:[<field-a1>, <field-a2>, <field-a3>]
where each field is a key/value pair of the form key=value
Given that multiple IPs within a CDN can be involved in handling a request, the Breadcrumb of each IP is concatenated together to create the final Breadcrumbs (plural), making it a comma-delimited list of square bracket encapsulated strings.
Example Syntax of Breadcrumbs:[<field-a1>, <field-a2>, <field-a3>],[<field-b1>, <field-b2>, <field-b3>],[<field-c1>, <field-c2>, <field-c3>]
Definition of Breadcrumb Fields
The Breadcrumbs response header that’s sent to a player consists of the following fields
A unique identifier assigned to the request. (This helps Akamai support teams quickly pull the relevant log line associated with the request.)
The IP address of the node/host currently processing the request.
If the node or host is neither an edge server nor an origin, then the IP is excluded.
This is a single-letter identifier for the Akamai network component that was involved during that phase of the request. (Your Akamai representative can use this value to determine where in the request phase the Breadcrumb was gathered.)
Geographic details associated with the node/host currently processing the request.
The autonomous system number associated with the current node/host.
Following are some example values of the Breadcrumbs data returned in the response header:
Example 1 — when served from within Akamai
In this example, the request flow is from Edge → Peer so the Breadcrumbs format will be:[<component-ip>,<request-id>,<component-letter>,<component-geo>,<component-asn>],[<component-letter>,<component-geo>,<component-asn>]
Example Breadcrumbs Response to Player:[a=188.8.131.52,b=198488236,c=g,n=US_CA_SANJOSE,o=20940],[c=p,n=US_CA_SANJOSE,o=20940]
Example 2 — when served from origin
In this example, the request flow is from Edge → Cache-H → Origin so the Breadcrumbs format will be:[<component-ip>,<request-id>,<component-letter>,<component-geo>,<component-asn>],[<component-letter>,<component-geo>,<component-asn>],[<component-ip>,<component-letter>]
Example Breadcrumbs Response to Player:[a=184.108.40.206,b=36411806,c=g,n=US_CA_SANJOSE,o=20940],[c=c,n=US_CA_SANJOSE,o=20940],[a=220.127.116.11,c=o]
Enabling Breadcrumbs in Property Manager
Breadcrumbs is a self-service feature that can be enabled for a particular property using a new Property Manager behavior, “Breadcrumbs.”
By enabling Breadcrumbs, responses to a player will receive an additional response header with the Breadcrumbs data, as described in the above sections.
Here’s an example of a request/response showing the Breadcrumbs response header:
$ curl -i http://amd-pa-live-1.akamaized.net/hls/live/518979/pankaj/muxed/master.m3u8
HTTP/1.1 200 OK
Last-Modified: Thu, 13 Feb 2020 17:38:17 GMT
Cache-Control: max-age=0, no-cache, no-store
Date: Thu, 27 Feb 2020 15:39:48 GMT
... truncated for brevity ...
... truncated for brevity ...
Breadcrumbs Opt Mode — Controlling Delivery of Breadcrumbs Response Header
After enabling Breadcrumbs via Property Manager, the default configuration will send the Breadcrumbs response header to all player requests.
For a player to opt-out from receiving the Breadcrumbs response header, a player can send the query param ak-bc=off.
If the default behavior of sending Breadcrumbs response header is not desirable for some reason, this can be changed by toggling the “Opt Mode” UI button to the value “Opt-In.”
In the opt-in mode, a player has to explicitly opt into receiving the Breadcrumbs response header by passing the query param ak-bc=on.
Find out more about how you can use Breadcrumbs in the Adaptive Media Delivery (AMD), Download Delivery (DD), and Object Delivery (OD) media projects by following this link: https://learn.akamai.com/en-us/webhelp/adaptive-media-delivery/adaptive-media-delivery-implementation-guide/GUID-3940CDAF-07B0-4855-9D40-74F379D6A1CB.html
*** This is a Security Bloggers Network syndicated blog from The Akamai Blog authored by Pankaj Chaudhari. Read the original post at: http://feedproxy.google.com/~r/TheAkamaiBlog/~3/8XeO1fK8-5U/getting-insights-into-the-delivery-path-of-media-with-breadcrumbs.html