Internet-Draft Koen Holtman/TUE Expires: 13 December 1996 13 June 1996 Transparent Content Negotiation draft-holtman-http-negotiation-01.txt STATUS OF THIS MEMO This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress". To learn the current status of any Internet-Draft, please check the "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). Distribution of this document is unlimited. Please send comments to the HTTP working group at . Discussions of the working group are archived at . General discussions about HTTP and the applications which use HTTP should take place on the mailing list. ABSTRACT HTTP/1.1 will allow one to put multiple versions of the same information under a single URL. Transparent content negotiation is a mechanism, layered on top of HTTP/1.1, for selecting the best version when a retrieval request is made. This document outlines the planned transparent negotiation mechanism, and identifies some issues and tradeoffs that have yet to be addressed. [Page 1] Internet-Draft Content Negotiation 13 June 1996 1 Introduction Transparent content negotiation is a mechanism layered on top of the HTTP/1.1 protocol [1]. HTTP/1.1 will allow one to put multiple versions of the same information under a single URL. Each of these versions is called a `variant'. Transparent content negotiation is a mechanism for selecting the best variant when a retrieval request is made. This document outlines the planned transparent content negotiation mechanism, and identifies some issues and tradeoffs that have yet to be addressed. 2 Overview of HTTP transparent content negotiation This section contains an extended version of the HTTP content negotiation presentation given by the author at the Fifth International WWW Conference in Paris. This section tries to avoid the use of terminology local to the IETF http-wg. For example, the term `URL' is used instead of `URI'. 2.1 Content negotiation HTTP/1.1 will allow one to put multiple versions of the same information under a single URL. Each of these versions is called a `variant'. For example, a URL http://x.org/paper could bind to three different variants of a paper: 1. HTML, English 2. HTML, French 3. Postscript, English Content negotiation is the process by which the best variant is selected if the URL is accessed. It has always been possible under HTTP to have multiple representations available for one URL, and to return the most appropriate representation for each subsequent request. However, HTTP/1.1 is the first version of HTTP which has provisions for doing this in a cache-friendly way. These provisions include the Vary response header, entity tags, and the If-None-Match request header. In content negotiation, the selection of the best variant for a particular request is based on a matching of the properties of the available variants to the capabilities of the user agent and the preferences of the user. 2.2 HTTP/1.0 variant selection scheme Under the HTTP/1.0 scheme, the variant selection happens as follows: [Page 2] Internet-Draft Content Negotiation 13 June 1996 Server _____ proxy _____ proxy _____ user x.org cache cache agent < ---------------------------------- | GET http://x.org/paper | Accept headers choose | ---------------------------------- > Best variant When the URL is accessed, the user agent sends, along with its request, various Accept headers which express the user agent capabilities and the user preferences. Then, the origin server uses these Accept headers to choose the best variant, which is returned in the response. The biggest problem with this scheme is that it does not scale well. For all but the most minimal user agents, Accept headers expressing all capabilities and preferences would be very large, and sending them in every request would be hugely inefficient, in particular because only a small fraction of the URLs on the web will have multiple variants. In current practice, user agents send short Accept headers which inadequately describe capabilities and preferences, except maybe for inline image requests. Servers providing multiple representations of the same information usually do so under different URLs, and allow users to manually select a representation by clicking a particular link. Also in current practice, servers often use the contents of the User-Agent request header for selection purposes. These contents sometimes allow the server to infer information about capabilities which cannot be expressed with existing Accept headers. 2.3 Transparent content negotiation scheme The transparent negotiation scheme eliminates the need to send huge Accept headers, and nevertheless allows for a selection process that always yields either the best variant, or an error message indicating that user agent is not capable of displaying any of the available variants. Under the transparent content negotiation scheme, the server sends a list with the available variants and their properties to the user agent. An example of a list with three variants is {"paper.html.en" 0.9 {type text/html} {language en}}, {"paper.html.fr" 0.7 {type text/html} {language fr}}, {"paper.ps.en" 1.0 {type application/postscript} {language en}} The syntax and semantics of the variant descriptions in this list are covered in depth in Section 3. When the list is received, the user [Page 3] Internet-Draft Content Negotiation 13 June 1996 agent can choose the best variant and retrieve it. Graphically, the communication can be represented as follows: Server _____ proxy _____ proxy _____ user x.org cache cache agent < ---------------------------------- | GET http://x.org/paper | ----------------------------------- > return of list | choose | < ---------------------------------- | GET http://x.org/paper.html.en | ---------------------------------- > return of html.en With this scheme, the information about capabilities and preferences is only used by the user agent itself. Therefore, the sending of such information in large Accept headers is unnecessary. Accept headers do have a limited use in transparent content negotiation however: the sending of small Accept headers can often speed up the negotiation process, this is covered in Section 2.4. According to current plans, the response that returns the list in the above picture will look as follows, with the list of variants in the Alternates header: HTTP/1.1 300 Multiple Choices Date: Tue, 11 Jun 1996 19:39:48 GMT Alternates: {"paper.html.en" 0.9 {type text/html} {language en}}, {"paper.html.fr" 0.7 {type text/html} {language fr}}, {"paper.ps.en" 1.0 {type application/postscript} {language en}} Vary: accept, accept-language Content-Type: text/html Content-Length: 227

Multiple Choices:

The HTML page included in the response is intended for compatibility with (existing) user agents which do not support transparent content negotiation. Such user agents will ignore the computer-readable list in the Alternates header and just display the HTML page, allowing the user to select the best variant by hand. [##Issue to be resolved: Some existing HTTP/1.0 user agents crash when getting a 300 response without a Location header. This problem [Page 4] Internet-Draft Content Negotiation 13 June 1996 may be big enough to warrant use of a response code outside of the 3xx class, at least for responses to HTTP/1.0 clients.##] 2.4 Cutting corners The basic transparent negotiation scheme involves two HTTP transactions: one to retrieve the list, and a second one to retrieve the chosen variant. There are however several ways to `cut corners' in the data flow path of the basic scheme. First, caching proxies can cache both lists and responses from variant URLs. Such caching can reduce the communication overhead, as shown in the following example: Server _____ proxy _____ proxy __________ user x.org cache cache agent < -------------- | GET ../paper | has the list in cache | ------------- > list | | choose | < -------------------------- | GET ../paper.html.en | has the variant in cache | -------------------------- > return of html.en Second, the user agent can send small Accept headers which could allow a server to determine the choice the user agent would make on receiving the list. If the Accept headers contain enough information, the origin server can send back the variant directly: Server _____ proxy _____ proxy _____ user x.org cache cache agent < ---------------------------------- | GET http://x.org/paper | small Accept headers | able to choose on behalf of user agent | ---------------------------------- > return of html.en and list [Page 5] Internet-Draft Content Negotiation 13 June 1996 According to current plans, the response that returns the html.en variant and the list in the above picture will look as follows: HTTP/1.1 200 OK Date: Tue, 11 Jun 1996 19:51:02 GMT Alternates: {"paper.html.en" 0.9 {type text/html} {language en}}, {"paper.html.fr" 0.7 {type text/html} {language fr}}, {"paper.ps.en" 1.0 {type application/postscript} {language en}} Vary: accept, accept-language Content-Location: paper.html.en Content-Type: text/html Last-Modified: Mon, 10 Jun 1996 10:01:14 GMT Etag: "gonkyyyy" Content-Length: ... ... Finally, the above two kinds of optimization can be combined: a caching proxy which has the list will sometimes be able to choose on behalf of the user agent. This could lead to the following communication pattern: Server _____ proxy _____ proxy __________ user x.org cache cache agent < --------------- | GET ../paper | small Accept | able to choose on behalf | < ---------- | GET ../paper.html.en | ----------- html.en | ---------------- > html.en and list 2.5 The network negotiation algorithm To allow for the second optimization above, choosing on behalf of the user agent, transparent content negotiation will standardize the so-called network negotiation algorithm. The algorithm will take as input - the list of variants of a transparently negotiable URL - the (small) Accept headers of an incoming request on that URL [Page 6] Internet-Draft Content Negotiation 13 June 1996 and will yield as output the appropriate action to be performed by the proxy cache or origin server. Proxy caches are required to either perform this appropriate action or to perform any other action allowed by the HTTP/1.1 standard. Origin servers are not required to follow the network negotiation algorithm. If the request was made by a user agent capable of transparent negotiation, the algorithm can yield two possible actions: A) Choice on behalf of the user agent: A particular variant X may be retrieved and returned on behalf of the user agent. B) No choice possible, return the list: The Accept headers do not contain sufficient information to make a choice on behalf of the user agent possible: the list of variants should be returned so that the user agent can make the choice itself. If the request was made by an (old) user agent not capable of transparent negotiation, the algorithm can yield three possible actions: C) Choice on behalf of the origin server: A particular variant X may be retrieved and returned on behalf of the origin server D) No choice possible, return a cached 300 response: The Accept headers do not contain sufficient information to make a choice on behalf of the origin server possible, and a cached or freshly retrieved 300 response should be returned, allowing the user to manually select the best variant E) No choice possible, forward request towards the origin server: The Accept headers do not contain sufficient information to make a choice on behalf of the origin server possible, and the request should be forwarded towards the origin server so that it can choose the best variant with a custom negotiation algorithm. [##Issue to be resolved: though several definitions of subsets of the network negotiation algorithm exist, the algorithm has not been fully defined yet. A mechanism for deciding between D) and E) has not yet been discussed.##] As the network negotiation algorithm is only an optimization device, complete or partial implementation of the algorithm by proxies is optional. To support transparent content negotiation, a HTTP/1.1 proxy need only follow the caching rules defined in the HTTP/1.1 specification. The core computations performed by the network negotiation algorithm are based on quality factors, which are short real numbers in the [Page 7] Internet-Draft Content Negotiation 13 June 1996 range 0.000 -- 1.000. These factors appear in variant lists and in Accept headers. 2.5.1 Network negotiation algorithm example 1 As an example, if a variant list contains the variant description {"paper.html.en" 0.7 {type text/html} {language fr}} and if the request contains the following Accept headers: Accept: text/html:q=1.0, */*:q=0.8, transparent Accept-Language: en;q=1.0, fr;q=0.5 the network negotiation algorithm will compute an `overall quality' for the variant by multiplying quality factors as follows: {"paper.html.en" 0.7 {type text/html} {language fr}} | | | | | | V V V 0.7 * 1.0 * 0.5 = 0.35 The 1.0 above is the quality factor assigned to "text/html" by the Accept header, the 0.5 is the quality factor assigned to "fr" by the Accept-Language header. The overall quality of this variant is 0.35. With the above Accept headers, the complete variant list {"paper.html.en" 0.9 {type text/html} {language en}}, {"paper.html.fr" 0.7 {type text/html} {language fr}}, {"paper.ps.en" 1.0 {type application/postscript} {language en}} would yield the following computations: paper.html.en: 0.9 * 1.0 * 1.0 = 0.9 paper.html.fr: 0.7 * 1.0 * 0.5 = 0.35 paper.ps.en: 1.0 * 0.8 * 1.0 = 0.8 The variant with the highest overall quality, "paper.html.en" in this case, is the variant that seems most acceptable to the user agent. [##Aside for mathematicians: note that this calculation scheme has similarities with fuzzy set theory. As far as I know, fuzzy set theory was not involved when the scheme was thought up.##] The inclusion of the word `transparent' in the received Accept header Accept: text/html:q=1.0, */*:q=0.8, transparent indicates that the user agent is capable of transparent negotiation. The second stage of the network negotiation algorithm will therefore have to choose between two possible end results: [Page 8] Internet-Draft Content Negotiation 13 June 1996 A) Choice on behalf of the user agent: The variant paper.html.en may be retrieved and returned on behalf of the user agent. B) No choice possible, return the list: The Accept headers do not contain sufficient information to make a choice on behalf of the user agent possible. The choice is made by examining the Accept header components used to derive the best overall quality value of 0.9 for paper.html.en: text/html:q=1.0 en;q=1.0 If none of these components contain a wildcard character *, as is the case here, the appropriate action is A). If a wildcard character is present, the appropriate action is B). 2.5.2 Network negotiation algorithm example 2 If the same variant list is matched to the following shorter Accept headers: Accept: */*:q=1.0, transparent Accept-Language: en;q=1.0, fr;q=0.5 the calculations would as follows: paper.html.en: 0.9 * 1.0 * 1.0 = 0.9 paper.html.fr: 0.7 * 1.0 * 0.5 = 0.35 paper.ps.en: 1.0 * 1.0 * 1.0 = 1.0 Now, paper.ps.en is the best candidate. However, if the Accept header components used to derive the best overall quality value of 1.0 are examined: */*:q=1.0 en;q=1.0 it is found that a wildcard character * is present. Therefore, the algorithm yields the result B) No choice possible, return the list: The Accept headers do not contain sufficient information to make a choice on behalf of the user agent possible. Note that the interpretation of */*:q=1.0, if sent by a user agent supporting transparent negotiation, is therefore not `I accept all media types with a quality of 1.0' but [Page 9] Internet-Draft Content Negotiation 13 June 1996 `I accept several media types, and assign quality factors from 0.0 up to 1.0 to them. If this information is insufficient to make a choice on my behalf, do not make a choice but send the list of variants'. This means that a user agent can always send the short Accept: */*, transparent by default, without this affecting the quality of the negotiation process. If it is found that requests to the server often return 300 responses, the user agent can try to dynamically extend the Accept header sent in requests to that server, for example to Accept: text/html, application/postscript:q=0.8, */*, transparent to increase the chance that the network negotiation algorithm will be able to optimize the retrieval process. A good strategy for dynamic extension would be to extend the Accept header with those media types mentioned in the variant lists of past (300) responses from the server. 2.6 Retrieving a variant by hand If a transparently negotiated resource is accessed, the user agent will always at some point receive the list of available variants. The user agent can use this list to make available a menu (in HTML or not) which lists all variants and their characteristics to the user. Such a menu will allow the user to randomly browse other variants, and will also make it possible to manually correct any sub-optimal choice made by the automatic negotiation process. 2.7 Dimensions of negotiation Transparent content negotiation defines four dimensions of negotiation: 1. Media type (MIME type) 2. Charset 3. Language 4. Features The first three dimensions have traditionally been present in HTTP, the fourth dimension is added by the transparent content negotiation specification. Additional dimensions, beyond the four mentioned above, could be added by future specifications. Negotiation on the content encoding of a response (gzipped, compressed, etc.) is left outside of the realm of transparent negotiation. Transparent negotiation does not prohibit proxies to [Page 10] Internet-Draft Content Negotiation 13 June 1996 encode or decode a relayed or cached response on the fly: the response still contains the same variant as far as transparent negotiation is concerned. 2.8 Feature negotiation Feature negotiation intends to provide for all areas of negotiation not covered by the first three dimensions. Examples are negotiation on * HTML extensions * Extensions of non-HTML media types * Color capabilities of the user agent * Output medium (screen, paper, ...) * Preference for speed vs. preference for graphical detail The feature negotiation framework is the main means by which transparent negotiation offers extensibility: a new dimension of negotiation (really a sub-dimension of the feature dimension) can be added without the need for a new standards effort, by the simple registration of a `feature tag'. 2.8.1 Feature tags A feature tag identifies a capability of a user agent or a preference of a user. A feature is said to be `present' in a user agent if the corresponding capability is implemented, or if the user has expressed corresponding preference. Examples of feature tags are ns_tables, fonts, monochrome, textonly Note that it sometimes depends on context whether a feature tag expresses a capability or a preference. For a text-only browser, the `textonly' tag is naturally present, but the user of a graphical browser could set the tag to be present if text-only content is preferred to graphical content. The `feature set' of a user agent contains all tags of the features present in the user agent for the current request. In general, the part of the feature set which reflects capabilities will be hard-coded in the user agent binary or in a user agent library file, and the part which reflects preferences will have been configured by the user. As feature registration will be an ongoing process, it is generally not possible for a user agent to know the meaning of all feature tags it can possibly encounter in a variant description. A user agent should assume that all feature tags unknown to it are not in its feature set. [Page 11] Internet-Draft Content Negotiation 13 June 1996 2.8.2 Feature tag registration According to current plans, everybody will be allowed to register feature tags: registration only requires that the tag follows the syntax rules, and that a definition of the meaning of the tag is supplied. In particular, registration will not require actual implementation of a feature, and there will be no test on whether the feature definition overlaps with another feature definition. The feature negotiation mechanism is designed not to break if 1000+ features, partially overlapping, are registered. It is expected that after the introduction of feature negotiation, an explosion in feature tag registration will occur, and that only some of these tags will end up being actively used to label variants. Web indexing robots could, while traversing the web, gather statistics about actual use of feature tags. These statistics could be used by individuals to compile lists, intended for content authors, of useful feature tags for particular areas of negotiation. Note that this indexing activity is orthogonal to the feature registration process. It is expected that, once an area of negotiation is well-understood, this process will converge on a commonly-used and commonly-recognized set of feature tags for that area. [##Issue to be resolved: it is unclear yet whether it is desirable for feature tags to have some hierarchical structure, which would make it easier, for example, to trace who registered the feature. All that is known now is that it is desirable to have short feature tags. A naming scheme like that for SGML-DTDs or PEP protocol names would probably yield feature tags which are too long. The main advantage of having a registration process at all is that it allows for short tags without running the risk of `tag collision'.##] 2.8.3 Use of feature tags Feature tags can be used in variant lists to express the quality degradation associated with the presence or absence of certain features. This is covered in depth in Section 4.4. One example is {"index.html.plain" 0.7 {type text/html} }, {"index.html" 1.0 {type text/html} {features ns_tables ns_frames} } Here, the "{features ns_tables ns_frames}" part expresses that index.html uses the features tagged as ns_tables and ns_frames. If these features are absent, the overall quality of index.html degrades to 0. Another example is {"home.graphics" 1.0 {type text/html} {features !textonly} }, {"home.textonly" 0.7 {type text/html} } where the "{features !textonly}" part expresses that home.graphics requires the absence of the textonly feature. If the feature is present, the overall quality of home.graphics degrades to 0. [Page 12] Internet-Draft Content Negotiation 13 June 1996 The absence of a feature need not always degrade the overall quality to 0. In the example {"x.html.1" 1.0 {type text/html} {features fonts/0.7}} the absence of the fonts feature degrades the quality with a factor of 0.7. "fonts/0.7" can be pronounced as "fonts, or a degradation of 0.7". In the example {"x.html.2" 0.5 {type text/html} {features fonts:1.5}} the presence of the fonts feature improves the overall quality with a factor of 1.5. If the fonts feature is absent, the overall quality is not affected. "fonts:1.5" can be pronounced as "fonts improves with 1.5". Finally, in the example {"y.html" 1.0 {type text/html} {features [blebber wolx] } } The "[blebber wolx]" expresses that y.html.1 requires the presence of the blebber feature or the wolx feature. This construct can be used in a number of cases: 1. blebber and wolx actually tag the same feature, but they were registered by different people, and some browsers say they support blebber while others say they support wolx. 2. blebber and wolx are HTML tags of different vendors which implement the same functionality, and which were used together in y.html without interference. 3. blebber and wolx are HTML tags of different vendors which implement the same functionality, and y.html uses conditional HTML to provide versions using both tags [##Note: conditional HTML does not yet exist, but it is something people are thinking about.##] 4. blebber is a complicated HTML tag with only a sketchy definition, implemented by one browser vendor, and wolx indicates implementation of a well-defined subset of the blebber tag by some other vendor(s). y.html uses only this well-defined subset. 2.9 Current status of transparent content negotiation There is no complete specification of transparent content negotiation yet. Documents completely specifying some subset of transparent content negotiation have been written, and have had some review by the http-wg, but none of these define feature negotiation, and none of them take the specific caching mechanisms defined by the latest version of the HTTP/1.1 specification [1] into account. The author of this document plans to finish a complete specification of transparent content negotiation, including feature negotiation, within the time frame of a few months. Many aspects of the design of [Page 13] Internet-Draft Content Negotiation 13 June 1996 feature negotiation are still in flux. Some tradeoffs between complexity and expressive power have yet to be discussed. No serious efforts have yet been made to invent and define feature tags for particular complicated areas of negotiation. The final shape of the feature negotiation mechanism will in part depend on the result of such efforts. It is expected that feature negotiation will solve many of the current negotiation problems. However, it is also expected that there will remain negotiation problems not solved by feature negotiation. 2.9.1 Feature negotiation in numeric areas Feature tags as covered in Section 2.8 have a boolean nature, because they serve as indicators of the simple presence or absence of some feature. Many areas of negotiation however, like negotiation on screen size, color depth, or bandwidth, have a numeric nature. A numeric area like negotiation on color depth can however be trivially mapped onto the feature negotiation framework. One can define a collection of four feature tags colordepth_1 colordepth_4 colordepth_8 colordepth_24 to span the numeric range of color depth negotiation. Each of these tags would indicate the capability to render pictures up to the given color depth. Under this scheme, a user agent rendering on a screen with 8 bit color depth would have all three features colordepth_1, colordepth_4, colordepth_8 present in its feature set, so that a variant using 10 distinct colors can be safely described as {"button.gif" 0.5 {type image/gif} {features colordepth_4}} While this technique seems adequate for to many numeric areas of negotiation, it is unknown at this point whether it provides sufficient expressive power to cover all areas feature negotiation intends to cover. Section 4 will introduce additional feature negotiation protocol elements which provide more direct support for negotiation on numbers. 2.9.2 Bandwidth negotiation With a bandwidth negotiation mechanism, the time needed to retrieve a particular variant over the network can be taken into account during the negotiation process. Work on mechanisms for bandwidth negotiation has been done since at least 1993, but this work has not yet yielded a successful standard mechanism for bandwidth negotiation. Cascaded proxy caches introduce additional complexity in this area. [Page 14] Internet-Draft Content Negotiation 13 June 1996 The planned transparent content negotiation specification will not attempt to solve the problem of bandwidth negotiation. There is some hope that the feature negotiation framework will allow the introduction of an adequate solution for bandwidth negotiation. 2.9.3 Content transformation by proxies Recently, there has been some attention to content transformation by proxies. An example is the transformation of an image/gif response to a shorter image/jpeg response, with potential degraded quality, to save bandwidth. Another example is the transformation of an application/postscript response to a text/html response, with definite degraded quality, to allow viewing by a user agent which cannot handle postscript. The impact of such transformations on the quality of content on the web, and the impact on the current model for the allocation of trust to various parties in a HTTP transaction, has not yet been fully explored. Content transformation could be a useful optimization for transparent content negotiation. A response with a variant list {"map.gif" 1.0 {type image/gif} {features !monochrome}}, {"map.mono.gif" 0.8 {type image/gif} {features monochrome}} could include an additional response header Allow-Transform: "map.gif" -> "map.mono.gif" with "dither-2" to allow a proxy which has cached "map.gif" to create "map.mono.gif" on demand, by applying a particular standardized transformation to the "map.gif" data. Another interesting possibility would be Allow-Transform: "map.gif" -> "map.mono.gif" with "http://x.org/java/filters/my_dither" 3 Variant descriptions Variant descriptions have been informally introduced in Section 2.3. This section defines variant descriptions more formally. The version of BNF used in the rest of this document is taken from [1], and many of the nonterminals used here are also defined in [1]. One new notation is added: 1%rule stands for one or more instances of "rule", separated by whitespace: 1%rule = rule *( 1*LWS rule ) . [Page 15] Internet-Draft Content Negotiation 13 June 1996 3.1 Syntax A variant can be described in a machine-readable way with a variant description. variant-descr = "{" <"> URI <"> source-quality [ "{" "type" media-type "}" ] [ "{" "language" 1#language-tag "}" ] [ "{" "length" 1*DIGIT "}" ] [ "{" "features" feature-list "}" ] [ "{" "description" quoted-string "}" ] [ extension-attribute ] "}" source-quality = qvalue extension-attribute = "{" extension-name extension-value "}" extension-name = token extension-value = *( token | quoted-string | LWS | ) Examples are {"paper.html.fr" 0.7 {type text/html} {language fr}} {"paper.html.tables" 0.9 {type text/html} {features tables}} The various attributes which can be present in a variant description are covered in the subsections below. [##Issue to be resolved: earlier versions had quotes (<">) around the media-type and 1#language-tag. Are these necessary for some reason unknown to me?##] 3.2 URI The URI attribute gives the URI of the un-negotiated resource from which the variant can be retrieved with a GET request. It can be absolute or relative to the Request-URI. 3.3 Source-quality The source-quality attribute gives the quality of the variant as a representation of the content of the negotiable URI when the variant is rendered with a perfect rendering engine on the best possible output medium. If it is less than 1, this attribute often expresses degradation caused by lossy conversion to a particular data format. For example, a picture originally in JPEG form would have a lower source quality [Page 16] Internet-Draft Content Negotiation 13 June 1996 when translated to the XBM format, and a much lower source quality when translated to an ASCII-art variant. Note, however, that degradation is a function of the source -- an original piece of ASCII-art may degrade in quality if it is captured in JPEG form. Content providers should use the following table as a guide when assigning source quality values: 1.000 no degradation 0.999-0.900 no noticeable degradation 0.899-0.700 noticeable, but acceptable degradation 0.699-0.500 barely acceptable degradation 0.499-0.000 unacceptable degradation [##Issue to be resolved: can we come up with a word other than `degradation' that covers better the case of variants not converted from one source?##] It is important that content providers do not assign very low source quality values without good reason, as this would limit the ability of users to influence the negotiation process with their own preference settings. When assigning source-quality values, content providers must not account for the size of the variant and its impact on transmission and rendering delays. Source-quality values are assigned assuming instantaneous transmission and rendering. This rule ensures that a future mechanism for bandwidth negotiation (Section 2.9.2) can be cleanly added. 3.4 Type and charset, language, length The type, language, and length attributes of a variant description refer to their Content-* header counterparts as defined in [1]. Note that the type attribute also holds the charset of the variant, if appropriate. An example is {type text/html;charset=ISO-8859-4} Though all of these attributes are optional, it is often desirable to include as many attributes as possible as this will increase the quality of the negotiation process. The length attribute, if present, must reflect the length of the variant only, not the length of the variant plus the length of any objects inlined or included by the variant. 3.5 Features The features attribute specifies how the presence or absence of particular features in the user agent affects the overall quality of the variant. This attribute is covered in Section 4.4. [Page 17] Internet-Draft Content Negotiation 13 June 1996 3.6 Description The description attribute is meant to provide a textual description of some properties of the variant, to be displayed by a user agent when showing a menu of available variants (Section 2.6). This attribute can be included if the URI and normal attributes of a variant are considered too opaque to allow interpretation by the user. 3.7 Extension-attribute The extension-attribute allows future specifications to incrementally define new dimensions of negotiation (Section 2.7), and eases content negotiation experiments under HTTP/1.1. In experimental situations, servers must only generate extension-attributes whose names start with "x-". User agents should ignore all extension attributes they do not recognize. Proxies must not run the network negotiation algorithm if an unknown attribute is present in the variant list. 4 Feature negotiation Feature negotiation has been informally introduced in Section 2.8. This section defines feature negotiation more formally. Note that much of the material in this section is still in flux. Some tradeoffs between complexity and expressive power have yet to be discussed. 4.1 Feature tags A feature tag (ftag) identifies a capability of a user agent or a preference of a user. ftag = 1* tspecials = "(" | ")" | "<" | ">" | "@" | "," | ";" | ":" | "\" | <"> | "/" | "[" | "]" | "?" | "=" | "{" | "}" | SP | HT Examples are ns_tables, fonts, blebber, wolx, screenwidth, colordepth Feature-tags are case-insensitive. (Case-sensitivity would add a huge factor of potential confusion to a process already designed to be chaotic). The definition of a feature tag may state that a feature tag, if present, must be have associated with it a numeric value which reflects a particular capability or preference. For example, a feature tag `screenwidth' could be present with a value of 640. [Page 18] Internet-Draft Content Negotiation 13 June 1996 4.2 Accept-Features header The Accept-Features request header can be used by a client to give information about the presence or absence of certain features. Accept-Features = "Accept-Features" : 1#( ftag | "!" ftag | ftag "=" number | "*" ) number = 1*DIGIT An example is: Accept-Features: blex, !blebber, colordepth=5, * A feature tag must never occur twice in an Accept-Features header. In the Accept-Features header, an ftag without a "!" indicates that the corresponding feature is present, but that is has no numeric value associated with it. An ftag with a "!" indicates that the corresponding feature is absent. An ftag with a number indicates that the corresponding feature is present with the given numeric value. The special construct "*" is best interpreted in terms of its effect on the network negotiation algorithm calculations. As an approximation, the "*" indicates that all features not named explicitly in the Accept-Features header are present, present with any numeric value, and absent, all at the same time. Absence of the Accept-Features header in a request is equivalent to the inclusion of Accept-Features: * 4.3 Feature predicates Feature predicates are used in the features attribute of a variant description. fpred = [ "!" ] ftag [ "=" number ] Examples feature predicates are blebber, !blebber, colordepth=5, !blex=54 The network negotiation algorithm can compute the truth value of a feature predicate by using the contents of the Accept-Features header of the current request. [Page 19] Internet-Draft Content Negotiation 13 June 1996 If the ftag in a feature predicate appears in the Accept-Features header, the truth value of the predicate is defined as follows, depending on its form: ftag true if the feature is present according to the Accept-Features header, false otherwise. !ftag true if the feature is absent according to the Accept-Features header, false otherwise. ftag=N true if the feature is present with a value greater than or equal to N according to the Accept-Features header, false otherwise. !ftag=N true if the feature is present with a value less than N according to the Accept-Features header, false otherwise. If the ftag in a feature predicate does not appear in the Accept-Features header, the value of the predicate is true if there is a "*" in the Accept-Features header, false otherwise. As an example, the header Accept-Features: blex, !blebber, colordepth=5, !screenwidth, * makes the following predicates true: blex, colordepth=4, !colordepth=6, colordepth, !screenwidth, frtnbf, !frtnbf, frtnbf=4, !frtnbf=4 and makes the following predicates false: !blex, blex=0, blebber, colordepth=6, !colordepth, screenwidth, screenwidth=640, !screenwidth=640 4.4 Features attribute The features attribute "{" "features" feature-list "}" is used in a variant description to specify how the presence or absence of particular features in the user agent affects the overall quality of the variant. [Page 20] Internet-Draft Content Negotiation 13 June 1996 feature-list = 1%feature-list-element feature-list-element = ( fpred | fpred-bag ) [ ":" present-improvement ] [ "/" absent-degradation ] fpred-bag = "[" 1%fpred "]" absent-degradation = 1*3DIGIT [ "." 0*3DIGIT ] present-improvement = 1*3DIGIT [ "." 0*3DIGIT ] Examples are: {features !textonly [blebber !wolx] colordepth=3:0.7 } {features !blink/0.5 background:1.5 [blebber !wolx]:1.4/0.8 } The default value for the present-improvement is 1. The default value for the absent-degradation is 0, or 1 if a present-improvement value is given. The network negotiation algorithm can compute the quality degradation factor associated with the features attribute by multiplying all quality degradation factors of the elements of the feature-list. Note that the result can be a factor greater than 1. A feature list element yields its present-improvement value if the corresponding feature predicate is true, or if at least one element of the corresponding fpred-bag is true. The element yields its absent-degradation value otherwise. [##Issue to be resolved: It is unknown yet whether this features attribute definition makes the right tradeoff between complexity and (ease of) expressive power. The attribute grammar above is designed to be parsable with simple non-recursive parsers. The present-improvement construct does not add expressive power in a theoretical sense, but does make the (automatic) construction of variant lists more straightforward in many cases.##] [##Aside for mathematicians: note the similarity between the feature-list [f1 !f2] [!f1 f3] and the conjunctive normal form of a boolean expression. This similarity implies good things about the expressive power of the features attribute. I have not yet explored how far this power extends into the non-boolean domain.##] 4.5 Example of negotiation in a numeric area As an example of negotiation in a numeric area, the following variant list describes four variants with title graphics designed for increasing screen widths: [Page 21] Internet-Draft Content Negotiation 13 June 1996 {"home.pda" 1.0 {type text/html} {features !screenwidth=200} }, {"home.narrow" 1.0 {type text/html} {features screenwidth=200 !screenwidth=600} }, {"home.normal" 1.0 {type text/html} {features screenwidth=600 !screenwidth=1000} }, {"home.wide" 1.0 {type text/html} {features screenwidth=1000} } The above variant list has the disadvantage that a user agent not supporting screenwidth negotiation is given no guidance on which variant to select: all would have an overall quality value of 0 if the screenwidth feature is not present. A feature list which solves this problem is {"home.pda" 1.0 {type text/html} {features !screenwidth=200} }, {"home.narrow" 1.0 {type text/html} {features screenwidth=200 !screenwidth=600} }, {"home.normal" 0.95 {type text/html} }, {"home.wide" 0.1 {type text/html} {features screenwidth=1000} } with this list, a user agent which does not support the screenwidth feature will always select the third variant. A user agent which does support the screenwidth feature will only select the third variant if the overall qualities of all other variants are 0. 4.6 Content-Features header The Content-Features response header can be used by a server to indicate how the presence or absence of particular features in the user agent affects the overall quality of the response. Content-Features = "Content-Features" : feature-list 5 Security and privacy considerations 5.1 Accept headers revealing information of a private nature Accept headers, in particular Accept-Language headers, may reveal information that the user rather keeps private unless it will directly improve the quality of service. For example, a user may not want to send language preferences to sites which do not offer multilingual content. The transparent content negotiation mechanism allows user agents to omit sending of the Accept-Language header by default, without this affecting the outcome of the negotiation process if transparently negotiated multilingual content is accessed. However, even if the Accept headers never send any information other than that the user agent is capable of transparent negotiation, automatic selection and retrieval of a variant by a user agent will reveal a preference for this variant to the server. A malicious service author could provide a page with `fake' negotiability on (ethnicity-correlated) languages, with all variants actually being the same English document, as a means of extracting privacy-sensitive [Page 22] Internet-Draft Content Negotiation 13 June 1996 information. Such a plot would however be visible to an alert victim if the menu of available variants and their properties made available by the user agent is reviewed. Some additional privacy considerations connected to Accept headers are discussed in [1]. 5.2 Spoofing of responses from variant URLs In the second optimization of the transparent negotiation process outlined in Section 2.4, a request on a transparently negotiable URL http://x.org/paper can lead to a response which must be interpreted as belonging to a variant URL http://x.org/paper.html.en. According to current plans, a proxy cache will be allowed to return this response, after removal of the list of variants and the Content-Location header, if a direct request on the variant URL http://x.org/paper.html.en is received. This caching shortcut could, for some negotiable resources, save up to a factor 2 in bandwidth usage for requests made by the cache. However, this shortcut also allows the author of the URL http://x.org/paper to control the data cached for the URL http://x.org/paper.html.en. If the variant URL belongs to another author, this is a security problem. Therefore, it is planned to restrict the use of the second optimization by origin servers to cases where the negotiable URL and the variant URL are equal up to the last slash. If an origin server generates a response which does not satisfy this restriction on use, the recipient must reject it and signal a server error. 6 Acknowledgments This document builds on an earlier specification of content negotiation as recorded in [2], and directly incorporates text from [2] in some places. Many members of the HTTP working group have contributed to the work reflected in this document. The author wishes to thank the individuals who have commented on earlier versions of the transparent content negotiation specification, including Brian Behlendorf, Daniel DuBois, Larry Masinter, and Roy T. Fielding. 7 References [1] Fielding et al, Hypertext Transfer Protocol -- HTTP/1.1. Internet-Draft draft-ietf-http-v11-spec-05.txt, HTTP Working Group, June, 1996. [2] Roy T. Fielding, Henrik Frystyk Nielsen, and Tim Berners-Lee. Hypertext Transfer Protocol -- HTTP/1.1. Internet-Draft draft-ietf-http-v11-spec-01.txt, HTTP Working Group, January, 1996. [Page 23] Internet-Draft Content Negotiation 13 June 1996 8 Author's address Koen Holtman Technische Universiteit Eindhoven Postbus 513 Kamer HG 6.57 5600 MB Eindhoven (Holland) e-mail: koen@win.tue.nl Expires: 13 December 1996 [Page 24]