2004/id/draft-ietf-http-negotiation-00.txt


HTTP Working Group                                     Koen Holtman, TUE
Internet-Draft                              Andrew Mutz, Hewlett-Packard
Expires: August 5, 1997                                 February 5, 1997


                 Transparent Content Negotiation in HTTP

                   draft-ietf-http-negotiation-00.txt


STATUS OF THIS MEMO

        This document is an Internet-Draft. Internet-Drafts are
        working documents of the Internet Engineering Task Force
        (IETF), its areas, and its working groups. Note that other
        groups may also distribute working documents as
        Internet-Drafts.

        Internet-Drafts are draft documents valid for a maximum of
        six months and may be updated, replaced, or obsoleted by
        other documents at any time. It is inappropriate to use
        Internet-Drafts as reference material or to cite them other
        than as "work in progress".

        To learn the current status of any Internet-Draft, please
        check the "1id-abstracts.txt" listing contained in the
        Internet-Drafts Shadow Directories on ftp.is.co.za
        (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific
        Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US
        West Coast).

        Distribution of this document is unlimited.  Please send
        comments to the HTTP working group at
        <http-wg@cuckoo.hpl.hp.com>.  Discussions of the working
        group are archived at
        <URL:http://www.ics.uci.edu/pub/ietf/http/>.  General
        discussions about HTTP and the applications which use HTTP
        should take place on the <www-talk@w3.org> mailing list.

        HTML and change bar versions of this document, are available
        at <URL:http://gewis.win.tue.nl/~koen/conneg/>.


ABSTRACT

        HTTP allows web site authors to put multiple versions of the
        same information under a single URL.  Transparent content
        negotiation is a mechanism, layered on top of HTTP, for
        automatically selecting the best version when the URL is
        accessed.  This enables the smooth deployment of new web data
        formats and markup tags.


OVERVIEW OF THE TRANSPARENT CONTENT NEGOTIATION DOCUMENT SET

   An up-to-date overview of documents related to transparent content
   negotiation is maintained on the web page
   <URL:http://gewis.win.tue.nl/~koen/conneg/>.

   The transparent content negotiation document set currently consists
   of three series of internet drafts.

     1. draft-ietf-http-negotiation-XX.txt (this document)

        `Transparent Content Negotiation in HTTP'

        Defines the core mechanism. Standards track.

     2. draft-ietf-http-rvsa-v10-XX.txt

        `HTTP Remote Variant Selection Algorithm -- RVSA/1.0'

        Defines the remote variant selection algorithm version 1.0.
        Standards track.

     3. draft-ietf-http-feature-reg-XX.txt

        `Feature Tag Registration Procedures'

        Defines feature tag registration.  Best Current Practice track.

   An additional document about `the core feature set', which may
   later become an informational RFC, may also appear.  Currently,
   there are two internet drafts which discuss parts of what could be
   a core feature set: draft-mutz-http-attributes-XX.txt and
   draft-goland-http-headers-XX.txt

   Older versions of the text in documents 1 and 2 may be found in the
   draft-holtman-http-negotiation-XX.txt series of internet drafts.


TABLE OF CONTENTS

    1  Introduction
     1.1 Background
     1.2 Revision history

    2  Terminology
     2.1 Terms from HTTP/1.1
     2.2 New terms

    3  Notation

    4  Overview
     4.1 Content negotiation
     4.2 HTTP/1.0 style negotiation scheme
     4.3 Transparent content negotiation scheme
     4.4 Optimizing the negotiation process
     4.5 Downwards compatibility with non-negotiating user agents
     4.6 Retrieving a variant by hand
     4.7 Dimensions of negotiation
     4.8 Feature negotiation

    5  Variant descriptions
     5.1 Syntax
     5.2 URI
     5.3 Source-quality
     5.4 Type, charset, language, and length
     5.5 Features
     5.6 Description
     5.7 Extension-attribute

    6  Feature negotiation
     6.1 Feature tags
     6.2 Accept-Features header
     6.3 Feature predicates
     6.4 Features attribute

    7  Remote variant selection algorithms
     7.1 Version numbers

    8  Content negotiation response codes and headers
     8.1 506 Variant Also Negotiates
     8.2 Accept-Charset
     8.3 Accept-Features
     8.4 Alternates
     8.5 Content-Features
     8.6 Negotiate
     8.7 Variant-Vary

    9 Cache validators
     9.1 Variant list validators
     9.2 Structured entity tags
     9.3 Assigning entity tags to variants

    10 Content negotiation responses
     10.1 List response
     10.2 Choice response
     10.3 Ad hoc response
     10.4 Reusing the Alternates header
     10.5 Extracting a normal response from a choice response
     10.6 Elaborate Vary headers
     10.6.1 Construction of an elaborate Vary header
     10.6.2 Caching of an elaborate Vary header
     10.7 Adding an Expires header to ensure HTTP/1.0 compatibility
     10.8 Negotiation on content encoding

    11 User agent support for transparent negotiation
     11.1 Handling of responses
     11.2 Presentation of a transparently negotiated resource

    12 Origin server support for transparent negotiation
     12.1 Requirements
     12.2 Negotiation on transactions other than GET and HEAD

    13 Proxy support for transparent negotiation

    14 Security and privacy considerations
     14.1 Accept headers revealing information of a private nature
     14.2 Spoofing of responses from variant resources

    15 Acknowledgments

    16 References

    17 Authors' addresses

    18 Appendix: feature negotiation examples
     18.1 Use of feature tags
     18.2 Use of numeric feature tags
     18.3 Feature tag design

    19 Appendix: origin server implementation considerations
     19.1 Implementation with a CGI script
     19.2 Direct support by HTTP servers
     19.3 Web publishing tools


1  Introduction

   HTTP allows web site authors to put multiple versions of the same
   information under a single URI.  Each of these versions is called a
   `variant'.  Transparent content negotiation is a mechanism for
   automatically and efficiently retrieving the best variant when a
   GET or HEAD request is made.  This enables the smooth deployment of
   new web data formats and markup tags.

   This specification defines transparent content negotiation as an
   extension on top of the HTTP/1.1 protocol [1].  However, use of
   this extension does not require use of HTTP/1.1: transparent
   content negotiation can also be done if some or all of the parties
   are HTTP/1.0 [3] systems.

   Transparent content negotiation is called `transparent' because it
   makes all variants which exist inside the origin server visible to
   outside parties.

      Note: Though this specification is limited to negotiation on
      HTTP transactions, elements of this specification could also be
      used in other contexts.  For example, feature predicates could
      be used in conditional HTML, and variant descriptions could be
      used in multipart mail messages.  Such use in other contexts is
      encouraged.


1.1 Background

   The addition of content negotiation to the web infrastructure has
   been considered important since the early days of the web.  Among
   the expected benefits of a sufficiently powerful system for content
   negotiation are

     * smooth deployment of new data formats and markup tags will
       allow graceful evolution of the web

     * eliminating the need to choose between a `state of the art
       multimedia homepage' and one which can be viewed by all web
       users

     * enabling good service to a wider range of browsing
       platforms (from low-end PDA's to high-end VR setups)

     * eliminating error-prone and cache-unfriendly
       User-Agent based negotiation

     * enabling construction of sites without `click here for the X
       version' links

     * internationalization, and the ability to offer multi-lingual
       content without a bias towards one language.


1.2 Revision history

   Most text in this draft was taken from the
   draft-holtman-http-negotiation-04.txt internet draft.  Major
   changes are:

     - The scope of the `network negotiation algorithm' has been
       limited to variant selection by servers on behalf of the user
       agent only.  The algorithm has been renamed to `remote variant
       selection algorithm', and has been moved to a separate draft.

     - A mechanism to negotiate on the use of remote variant selection
       algorithms was added.

     - The two cases `request from negotiating user agent' and
       `request from non-negotiating user agent' have been decoupled
       completely.

   Some appendices were cut, and some existing text has been improved.

   In the text version of this document with change bars, all changes
   with respect to the corresponding sections in
   draft-holtman-http-negotiation-04.txt are marked, except changes in
   formatting and punctuation.

   In the HTML version of this document with changes marked, all
   changed words and symbols are typeset in bold text.  Deletions and
   changes in punctuation are not marked in the HTML version.


2  Terminology

2.1 Terms from HTTP/1.1

   This specification mostly uses the terminology of the HTTP/1.1
   specification [1].  The definitions below were reproduced from [1].

   request
     An HTTP request message.

   response
     An HTTP response message.

   resource
     A network data object or service that can be identified by a URI.
     Resources may be available in multiple representations
     (e.g. multiple languages, data formats, size, resolutions) or
     vary in other ways.

   content negotiation
     The mechanism for selecting the appropriate representation when
     servicing a request.

   variant
     A resource may have one, or more than one, representation(s)
     associated with it at any given instant.  Each of these
     representations is termed a `variant.'  Use of the term `variant'
     does not necessarily imply that the resource is subject to
     content negotiation.

   client
     A program that establishes connections for the purpose of sending
     requests.

   user agent
     The client which initiates a request.  These are often browsers,
     editors, spiders (web-traversing robots), or other end user
     tools.

   server
     An application program that accepts connections in order to
     service requests by sending back responses.  Any given program may
     be capable of being both a client and a server; our use of these
     terms refers only to the role being performed by the program for
     a particular connection, rather than to the program's
     capabilities in general.  Likewise, any server may act as an
     origin server, proxy, gateway, or tunnel, switching behavior
     based on the nature of each request.

   origin server
     The server on which a given resource resides or is to be created.

   proxy
     An intermediary program which acts as both a server and a client
     for the purpose of making requests on behalf of other
     clients.  Requests are serviced internally or by passing them on,
     with possible translation, to other servers.  A proxy must
     implement both the client and server requirements of this
     specification.

   age
     The age of a response is the time since it was sent by, or
     successfully validated with, the origin server.

   fresh
     A response is fresh if its age has not yet exceeded its freshness
     lifetime.


2.2 New terms

   transparently negotiable resource
     A resource, identified by a single URI, which has multiple
     representations (variants) associated with it.  When servicing a
     request on its URI, it allows selection of the best
     representation using the transparent content negotiation
     mechanism.  A transparently negotiable resource always has a
     variant list bound to it, which can be represented as an
     Alternates header.

   variant list
     A list containing variant descriptions, which can be bound to a
     transparently negotiable resource.

   variant description
     A machine-readable description of a variant resource, usually
     found in a variant list.  A variant description contains the
     variant resource URI and various attributes which describe
     properties of the variant.  Variant descriptions are defined in
     section 5.

   variant resource
     A resource from which a variant of a negotiable resource can be
     retrieved with a simple GET request.

   list response
     A list response contains the variant list of the negotiable
     resource, but no variant data.  It is generated when the server
     does not (perhaps cannot) choose a particular best variant for the
     request.  List responses are defined in section 10.1.

   choice response
     A choice response contains both the variant list of the
     negotiable resource and a representation of the best variant for
     the request.  Choice responses are defined in section 10.2.

   ad hoc response
     An ad hoc response contains the variant list of the negotiable
     resource, and any other data the origin server wants to send.  It
     can be generated as a response to a non-negotiating user agent if
     the server does not (perhaps cannot) choose any particular
     variant.  Ad hoc responses are defined in section 10.3.

   Accept headers
     The request headers: Accept, Accept-Charset, Accept-Language, and
     Accept-Features.

   remote variant selection algorithm
     A standardized algorithm by which a server can sometimes choose a
     best variant on behalf of a negotiating user agent.  The
     algorithm typically computes whether the Accept headers in the
     request contain sufficient information to allow a choice, and if
     so, which variant must be chosen.  The use of a remote algorithm
     can speed up the negotiation process.

   neighbor
     Two resources are called neighbors if the absolute URI of the
     first resource up to its last slash equals the absolute URI of
     the second resource up to its last slash.  The neighboring
     relation is important because of security considerations; see
     section 14.2.


3  Notation

   The version of BNF used in this document is taken from [1], and
   many of the nonterminals used are defined in [1].

   One new BNF construct is added:

      1%rule

   stands for one or more instances of "rule", separated by
   whitespace:

      1%rule =  rule *( 1*LWS rule )

   This specification also introduces

     number = 1*DIGIT

     short-float = 1*3DIGIT [ "." 0*3DIGIT ]


4  Overview

   This section gives an overview of transparent content negotiation.
   It starts with a more general discussion of negotiation as provided
   by HTTP.


4.1 Content negotiation

   HTTP/1.1 allows web site authors to put multiple versions of the
   same information under a single resource URI.  Each of these
   versions is called a `variant'. For example, a resource
   http://x.org/paper could bind to three different variants of a
   paper:

         1. HTML, English
         2. HTML, French
         3. Postscript, English

   Content negotiation is the process by which the best variant is
   selected if the resource is accessed.  The selection is done by
   matching the properties of the available variants to the
   capabilities of the user agent and the preferences of the user.

   It has always been possible under HTTP to have multiple
   representations available for one resource, and to return the most
   appropriate representation for each subsequent request.  However,
   HTTP/1.1 is the first version of HTTP which has provisions for
   doing this in a cache-friendly way.  These provisions include the
   Vary response header, entity tags, and the If-None-Match request
   header.


4.2 HTTP/1.0 style negotiation scheme

   The HTTP/1.0 protocol elements allow for a negotiation scheme as
   follows:

      Server _____ proxy _____ proxy _____ user
      x.org        cache       cache       agent

        < ----------------------------------
        |      GET http://x.org/paper
        |          Accept headers
      choose
        |
         ---------------------------------- >
                    Best variant

   When the resource is accessed, the user agent sends (along with its
   request) various Accept headers which express the user agent
   capabilities and the user preferences.  Then the origin server uses
   these Accept headers to choose the best variant, which is returned
   in the response.

   The biggest problem with this scheme is that it does not scale
   well.  For all but the most minimal user agents, Accept headers
   expressing all capabilities and preferences would be very large,
   and sending them in every request would be hugely inefficient, in
   particular because only a small fraction of the resources on the
   web have multiple variants.


4.3 Transparent content negotiation scheme

   The transparent content negotiation scheme eliminates the need to
   send huge Accept headers, and nevertheless allows for a selection
   process that always yields either the best variant, or an error
   message indicating that user agent is not capable of displaying any
   of the available variants.

   Under the transparent content negotiation scheme, the server sends
   a list with the available variants and their properties to the user
   agent.  An example of a list with three variants is

     {"paper.html.en" 0.9 {type text/html} {language en}},
     {"paper.html.fr" 0.7 {type text/html} {language fr}},
     {"paper.ps.en"   1.0 {type application/postscript} {language en}}

   The syntax and semantics of the variant descriptions in this list
   are covered in section 5.  When the list is received, the user
   agent can choose the best variant and retrieve it.  Graphically,
   the communication can be represented as follows:

      Server _____ proxy _____ proxy _____ user
      x.org        cache       cache       agent

        < ----------------------------------
        |      GET http://x.org/paper
        |
        ----------------------------------- >         [list response]
                  return of list            |
                                         choose
                                            |
        < ----------------------------------
        |  GET http://x.org/paper.html.en
        |
         ---------------------------------- >         [normal response]
                return of html.en

   The first response returning the list of variants is called a `list
   response'.  The second response is a normal HTTP response: it does
   not contain special content negotiation related information.  Only
   the user agent needs to know that the second request actually
   retrieves a variant.  For the other parties in the communication,
   the second transaction is indistinguishable from a normal HTTP
   transaction.

   With this scheme, information about capabilities and preferences is
   only used by the user agent itself.  Therefore, sending such
   information in large Accept headers is unnecessary.  Accept headers
   do have a limited use in transparent content negotiation however;
   the sending of small Accept headers can often speed up the
   negotiation process. This is covered in section 4.4.

   List responses are covered in section 10.1.  As an example, the
   list response in the above picture could be:

     HTTP/1.1 300 Multiple Choices
     Date: Tue, 11 Jun 1996 20:02:21 GMT
     Alternates: {"paper.html.en" 0.9 {type text/html} {language en}},
                 {"paper.html.fr" 0.7 {type text/html} {language fr}},
                 {"paper.ps.en"   1.0 {type application/postscript}
                     {language en}}
     Vary: negotiate, accept, accept-language
     ETag: "blah;1234"
     Cache-control: max-age=86400
     Content-Type: text/html
     Content-Length: 227

     <h2>Multiple Choices:</h2>
     <ul>
     <li><a href=paper.html.en>HTML, English version</a>
     <li><a href=paper.html.fr>HTML, French version</a>
     <li><a href=paper.ps.en>Postscript, English version</a>
     </ul>

   The Alternates header in the response contains the variant list.
   The Vary header is included to ensure correct caching by HTTP/1.1
   caches not capable of transparent content negotiation (see section
   10.6).  The ETag header allows the response to be revalidated by
   caches, the Cache-Control header controls this revalidation.  The
   HTML entity included in the response allows the user to select the
   best variant by hand if desired.


4.4 Optimizing the negotiation process

   The basic transparent negotiation scheme involves two HTTP
   transactions: one to retrieve the list, and a second one to retrieve
   the chosen variant.  There are however several ways to `cut corners'
   in the data flow path of the basic scheme.

   First, caching proxies can cache both variant lists and variants.
   Such caching can reduce the communication overhead, as shown in the
   following example:

      Server _____ proxy _____ proxy __________ user
      x.org        cache       cache            agent

                                 < --------------
                                 |  GET ../paper
                                 |
                               has the list
                               in cache
                                 |
                                  -------------  >  [list response]
                                           list  |
                                                 |
                                              choose
                                                 |
                     < --------------------------
                     |   GET ../paper.html.en
                     |
                  has the variant
                  in cache
                     |
                      -------------------------- >  [normal response]
                         return of html.en

   Second, the user agent can send small Accept headers, which may
   contain enough information to allow the server to choose the best
   variant and return it directly.

      Server _____ proxy _____ proxy _____ user
      x.org        cache       cache       agent

        < ----------------------------------
        |      GET http://x.org/paper
        |       small Accept headers
        |
      able to choose on
      behalf of user agent
        |
         ---------------------------------- >    [choice response]
              return of html.en and list

   This choosing based on small accept headers is done with a `remote
   variant selection algorithm'.  Such an algorithm takes the variant
   list and the Accept headers as input.  It then computes whether the
   Accept headers contain sufficient information to choose on behalf
   of the user agent, and if so, which variant must be chosen.

   A server may only choose on behalf of the user agent if the user
   agent explicitly allows the use of a particular remote variant
   selection algorithm in the Negotiate request header.  User agents
   with sophisticated internal variant selection algorithms may want
   to disallow a remote choice, or may want to allow it only when
   retrieving inline images.  If the local algorithm of the user agent
   is superior in only some difficult areas of negotiation, it is
   possible to enable the remote algorithm for the easy areas only.
   More information about the use of a remote variant selection
   algorithm can be found in [5].

   The response in the above diagram is called a choice response. It
   transmits both the chosen variant and the list of all variants
   bound to the negotiable resource.  Choice responses are covered in
   section 10.2.  For example, the choice response in the above
   picture could be:

     HTTP/1.1 200 OK
     Date: Tue, 11 Jun 1996 20:05:31 GMT
     Content-Type: text/html
     Last-Modified: Mon, 10 Jun 1996 10:01:14 GMT
     Content-Length: 5327
     Cache-control: max-age=604800
     Content-Location: paper.html.en
     Alternates: {"paper.html.en" 0.9 {type text/html} {language en}},
                 {"paper.html.fr" 0.7 {type text/html} {language fr}},
                 {"paper.ps.en"   1.0 {type application/postscript}
                     {language en}}
     Etag: "gonkyyyy;1234"
     Vary: negotiate, accept, accept-language
     Expires: Thu, 01 Jan 1980 00:00:00 GMT

     <title>A paper about ....

   Finally, the above two kinds of optimization can be combined; a
   caching proxy which has the list will sometimes be able to choose on
   behalf of the user agent.  This could lead to the following
   communication pattern:

      Server _____ proxy _____ proxy __________ user
      x.org        cache       cache            agent

                                 < ---------------
                                 |  GET ../paper
                                 |  small Accept
                                 |
                              able to choose
                                on behalf
                                 |
                     < ----------
                     |  GET ../paper.html.en
                     |
                      ---------- >   [normal response]
                        html.en  |
                                  ---------------- >  [choice response]
                                   html.en and list

   Note that this cutting of corners not only saves bandwidth, it also
   eliminates delays due to packet round trip times, and reduces the
   load on the origin server.


4.5 Downwards compatibility with non-negotiating user agents

   To handle requests from user agents not capable of transparent
   content negotiation, transparent content negotiation allows the
   origin server to revert to a HTTP/1.0 style negotiation scheme.
   The specification of heuristics for such schemes is beyond the
   scope of this document.


4.6 Retrieving a variant by hand

   If a transparently negotiated resource is accessed, the user agent
   will always at some point receive the list of available variants.
   The user agent can use this list to make available a menu of all
   variants and their characteristics to the user.  Such a menu allows
   the user to randomly browse other variants, and makes it possible
   to manually correct any sub-optimal choice made by the automatic
   negotiation process.


4.7 Dimensions of negotiation

   Transparent content negotiation defines four dimensions of
   negotiation:

     1. Media type (MIME type)
     2. Charset
     3. Language
     4. Features

   The first three dimensions have traditionally been present in HTTP.
   The fourth dimension is added by this specification.  Additional
   dimensions, beyond the four mentioned above, could be added by
   future specifications.

   Negotiation on the content encoding of a response (gzipped,
   compressed, etc.) is left outside of the realm of transparent
   negotiation.   See section 10.8 for more information.


4.8 Feature negotiation

   Feature negotiation intends to provide for all areas of negotiation
   not covered by the type, charset, and language dimensions.
   Examples are negotiation on

       * HTML extensions
       * Extensions of other media types
       * Color capabilities of the user agent
       * Screen size
       * Output medium (screen, paper, ...)
       * Preference for speed vs. preference for graphical detail

   The feature negotiation framework (section 6) is the principal
   means by which transparent negotiation offers extensibility; a new
   dimension of negotiation (really a sub-dimension of the feature
   dimension) can be added without the need for a new standards effort
   by the simple registration of a `feature tag'.  Feature tag
   registration is discussed in [4].


5  Variant descriptions

5.1 Syntax

   A variant can be described in a machine-readable way with a variant
   description.

       variant-description =
                  "{" <"> URI <"> source-quality *variant-attribute"}"

       source-quality = qvalue

       variant-attribute = "{" "type" media-type "}"
                         | "{" "charset" charset "}"
                         | "{" "language"  1#language-tag "}"
                         | "{" "length" 1*DIGIT "}"
                         | "{" "features" feature-list "}"
                         | "{" "description" quoted-string "}"
                         | extension-attribute

       extension-attribute = "{" extension-name extension-value "}"
       extension-name      = token
       extension-value     = *( token | quoted-string | LWS
                              | extension-specials )

       extension-specials  =
                          <any element of tspecials except <"> and "}">

   Examples are

      {"paper.html.fr" 0.7 {type text/html} {language fr}}

      {"paper.html.tables"  0.9 {type text/html} {features tables}}

      {"paper.html.en"}

   The various attributes which can be present in a variant
   description are covered in the subsections below.  Each attribute
   may appear only once in a variant description.


5.2 URI

   The URI attribute gives the URI of the resource from which the
   variant can be retrieved with a GET request.  It can be absolute or
   relative to the Request-URI.  The variant resource may vary (on the
   Cookie request header, for example), but must not engage in
   transparent content negotiation itself.


5.3 Source-quality

   The source-quality attribute gives the quality of the variant, as a
   representation of the negotiable resource, when this variant is
   rendered with a perfect rendering engine on the best possible
   output medium.

   If the source-quality is less than 1, it often expresses a quality
   degradation caused by a lossy conversion to a particular data
   format.  For example, a picture originally in JPEG form would have
   a lower source quality when translated to the XBM format, and a
   much lower source quality when translated to an ASCII-art variant.
   Note however, that degradation is a function of the source; an
   original piece of ASCII-art may degrade in quality if it is
   captured in JPEG form.

   The source-quality could also represent a level of quality caused
   by skill of language translation, or ability of the used media type
   to capture the intended artistic expression.

   It is important that content providers do not assign very low
   source quality values without good reason, as this would limit the
   ability of users to influence the negotiation process with their
   own preference settings.  The following table should be used as a
   guide when assigning source quality values:

       1.000  perfect representation
       0.900  threshold of noticeable loss of quality
       0.800  noticeable, but acceptable quality reduction
       0.500  barely acceptable quality
       0.300  severely degraded quality
       0.000  completely degraded quality

   Note that most meaningful values in this table are close to 1.
   This is due to the fact that quality factors are generally combined
   by multiplying them, not by adding them.

   When assigning source-quality values, content providers must not
   account for the size of the variant and its impact on transmission
   and rendering delays.  Any constant rendering delay for a
   particular media type (for example due to the startup time of a
   helper application) should be accounted for by the user agent, when
   assigning a quality factor to that media type.


5.4 Type, charset, language, and length

   The type attribute of a variant description carries the same
   information as its Content-Type response header counterpart defined
   in [1], except for any charset information, which must be carried
   in the charset attribute.  For, example, the header

      Content-Type: text/html; charset=ISO-8859-4

   has the counterpart attributes

     {type text/html} {charset ISO-8859-4}

   The language and length attributes carry the same information as
   their Content-* response header counterparts in [1].  The length
   attribute, if present, must thus reflect the length of the variant
   alone, and not the total size of the variant and any objects
   inlined or embedded by the variant.

   Though all of these attributes are optional, it is often desirable
   to include as many attributes as possible, as this will increase
   the quality of the negotiation process.

      Note: A server is not required to maintain a one-to-one
      correspondence between the attributes in the variant description
      and the Content-* headers in the variant response.  For example,
      if the variant description contains a language attribute, the
      response does not necessarily have to contain a Content-Language
      header. If a Content-Language header is present, it does not
      have to contain an exact copy of the information in the language
      attribute.


5.5 Features

   The features attribute specifies how the presence or absence of
   particular feature tags in the user agent affects the overall
   quality of the variant.  This attribute is covered in section 6.4.


5.6 Description

   The description attribute gives a textual description of the
   variant.  It can be included if the URI and normal attributes of a
   variant are considered too opaque to allow interpretation by the
   user.  If a user agent is showing a menu of available variants
   compiled from a variant list, and if a variant has a description
   attribute, the user agent should show the description attribute of
   the variant instead of showing the normal attributes of the
   variant.


5.7 Extension-attribute

   The extension-attribute allows future specifications to
   incrementally define new dimensions of negotiation, and eases
   content negotiation experiments.  In experimental situations,
   servers must only generate extension-attributes whose names start
   with "x-".  User agents should ignore all extension attributes they
   do not recognize.  Proxies must not run a remote variant selection
   algorithm if an unknown extension attribute is present in the
   variant list.


6  Feature negotiation

   This section defines the feature negotiation mechanism.  Feature
   negotiation has been introduced in section 4.8.  Appendix 18
   contains examples of feature negotiation.


6.1 Feature tags

   A feature tag (ftag) identifies a capability of a user agent or a
   preference of a user.  A feature is said to be `present' in a user
   agent if the corresponding capability is implemented, or if the
   user has expressed corresponding preference.

       ftag = 1*<any CHAR except CTLs or tspecials or "!">

       tspecials      = "(" | ")" | "<" | ">" | "@"
                      | "," | ";" | ":" | "\" | <">
                      | "/" | "[" | "]" | "?" | "="
                      | "{" | "}" | SP | HT

        (tspecials definition reproduced from [1])

   Examples are

        tables, fonts, blebber, wolx, screenwidth, colordepth

   An example of the use of feature tags in a variant description is:

     {"index.html" 1.0 {type text/html} {features tables frames}}

   Feature tags are case-insensitive.  The definition of a feature tag
   may state that a feature tag, if present, can have associated with
   it one or more values which reflect a particular capability or
   preference.  For example, a feature tag `paper' could be present
   with the values `A4' and `A5'.

   Note that context may determine whether a feature tag expresses a
   capability or a preference.  The `textonly' tag is naturally
   present for a text-only user agent, but the user of a graphical
   user agent could set the tag to be present if text-only content is
   preferred to graphical content.

   As feature registration [4] will be an ongoing process, it is
   generally not possible for a user agent to know the meaning of all
   feature tags it can possibly encounter in a variant description.  A
   user agent should treat all features with tags unknown to it as
   absent.


6.2 Accept-Features header

   The Accept-Features request header can be used by a client to give
   information about the presence or absence of certain features.

       Accept-Features = "Accept-Features" ":"
                   #( feature-expr *( ";" feature-extension ) )

       feature-expr = [ "!" ] ftag
                    | ftag [ "!" ] "=" tag-value
                    | ftag "=" "{" tag-value "}"
                    | ftag "<=" number
                    | ftag "=" "<" numeric-range ">"
                    | "*"

       tag-value  = token | quoted-string

       numeric-range = [ number ] "-" [ number ]

       feature-extension = token [ "=" ( token | quoted-string ) ]

   Tag values must be compared case-insensitively, and a token value
   XYZ is equal to a quoted-string value "XYZ".  No feature extensions
   are defined in this specification.  An example is:

       Accept-Features: blex, !blebber, colordepth<=5, !screenwidth,
                  UA-media={stationary}, paper=a4, paper!="a0",
                  x_version=<100-205>, *

   The different feature expressions have the following meaning:

      ftag       ftag is present

      !ftag      ftag is absent

      ftag=V     ftag is present with the value V (it may also be
                 present with other values)

      ftag!=V    ftag is present, but not with the value V

      ftag={V}   ftag is present with the value V, and not with any
                 other values

      ftag<=N    ftag is present with the numeric values from 0 up to
                 and including N, and not with any other values

      ftag=<N-M> ftag is present with the numeric values from N up to
                 and including M, and not with any other values.  If N
                 is missing, the lower bound is 0.  If M is missing,
                 the upper bound is infinity.

      *          makes true all feature predicates (section 6.3) which
                 were not assigned truth values by other elements of
                 the header

   Absence of the Accept-Features header in a request is equivalent to
   the inclusion of

       Accept-Features: *


6.3 Feature predicates

   Feature predicates are used in the features attribute of a variant
   description.

      fpred = [ "!" ] ftag
            | ftag [ "!" ] "=" tag-value
            | ftag "=" "<" numeric-range ">"

   Examples of feature predicates are

      blebber, !blebber, paper=a4, colordepth=5, blex!=54,
      dpi=<300-599>, colordepth=<24->

   A server can compute the truth value of a feature predicate by
   using the knowledge gained from the Accept-Features header in the
   current request.  The truth value must be assigned as follows,
   depending on the form of the predicate:

      ftag       true if the feature is known to be present

                 false if the feature is known to be absent

      !ftag      true if the feature is known to be absent

                 false if the feature is known to be present

      ftag=V     true if the feature is known to be present with
                 the value V,

                 false if the feature is known not to be present with
                 the value V

      ftag!=V    true if the feature is known to be present, but known
                 not to be present with the value V,

                 false if the feature is known to be absent or present
                 with the value V

      ftag=<N-M> true if the feature is known to be present with some
                 numeric values, while the highest value with which it
                 is present is known and in the range N-M,

                 false if the feature is known to be absent, or if it
                 is known to be present with some numeric values,
                 while the highest value with which it is present is
                 known and not in the range N-M.

                 If N is missing, the lower bound is 0.  If M is
                 missing, the upper bound is infinity.

   If the information in the Accept-Features header does not provide
   sufficient knowledge to assign a value to a predicate using the
   above rules, then the value is true if there is a "*" in the
   Accept-Features header, false otherwise.

   As an example, the header

       Accept-Features: blex, !blebber, colordepth<=5, !screenwidth,
                  UA-media={stationary}, paper=a4, paper!="a0",
                  x_version=<100-205>, *

   makes the following predicates true:

       blex, colordepth=4, colordepth!=6, colordepth, !screenwidth,
       UA-media=stationary, !UA-media=screen, paper=a4, paper=!a0,
       colordepth=<4-6>, x_version="101"

   The * in the header makes all of the following predicates true:

       blex=wox, blex!=wox,  paper=a5,
       frtnbf, !frtnbf, frtnbf=4, frtnbf!=4, frtnbf=<1-42>

   The header makes the following predicates false:

       !blex, blebber, colordepth=6, colordepth=foo, !colordepth,
       screenwidth, screenwidth=640, screenwidth!=640, x_version=99,
       UA-media=screen, paper=a0


6.4 Features attribute

   The features attribute

            "{" "features" feature-list "}"

   is used in a variant description to specify how the presence or
   absence of particular feature tags in the user agent affects the
   overall quality of the variant.

       feature-list = 1%feature-list-element

       feature-list-element = ( fpred | fpred-bag )
                              [ ":" true-improvement ]
                              [ "/" false-degradation ]

       fpred-bag = "[" 1%fpred "]"

       true-improvement   =  short-float
       false-degradation  =  short-float

   Examples are:

       {features !textonly [blebber !wolx] colordepth=3:0.7}

       {features !blink/0.5 background:1.5  [blebber !wolx]:1.4/0.8}

   The default value for the true-improvement is 1.  The default value
   for the false-degradation is 0, or 1 if a true-improvement value is
   given.

   A remote variant selection algorithm must compute the quality
   degradation factor associated with the features attribute by
   multiplying all quality degradation factors of the elements of the
   feature-list.  Note that the result can be a factor greater than 1.

   A feature list element yields its true-improvement factor if the
   corresponding feature predicate is true, or if at least one element
   of the corresponding fpred-bag is true. The element yields its
   false-degradation factor otherwise.


7  Remote variant selection algorithms

   A remote variant selection algorithms is a standardized algorithm
   by which a server can choose a best variant on behalf of a
   negotiating user agent.  The use of a remote algorithm can speed up
   the negotiation process by eliminating a request-response round
   trip.

   A remote algorithm typically computes whether the Accept headers in
   the request contain sufficient information to allow a choice, and
   if so, which variant must be chosen.  This specification does not
   define any remote algorithms, but does define a mechanism to
   negotiate on the use of such algorithms.


7.1 Version numbers

   A version numbering scheme is used to distinguish between different
   remote variant selection algorithms.

      rvsa-version = major "." minor

      major = number
      minor = number

   An algorithm with the version number X.Y, with Y>0, must be
   downwards compatible with all algorithms from X.0 up to X.Y.
   Downwards compatibility means that, if supplied with the same
   information, the newer algorithm must make the same choice, or a
   better choice, as the old algorithm.  There are no compatibility
   requirements between algorithms with different major version
   numbers.


8  Content negotiation response codes and headers

   This specification adds one new HTTP response code, and introduces
   five new HTTP headers.  It also extends the semantics of an
   existing HTTP/1.1 header.


8.1 506 Variant Also Negotiates

   The 506 response code indicates that the server has an internal
   configuration error: the chosen variant resource is configured to
   engage in transparent content negotiation itself, and is therefore
   not a proper end point in the negotiation process.

8.2 Accept-Charset

   The Accept-Charset header is defined in the HTTP/1.1 specification
   [1].  HTTP/1.1 allows the following Accept-Charset header to be
   sent:

      Accept-Charset: iso-8859-5;q=0.8, *;q=0.9

   but HTTP/1.1 does not assign any special meaning to the charset
   "*".

   This specification does assign a special meaning: servers and
   clients capable of transparent content negotiation must take "*" as
   a wildcard matching every character set not explicitly mentioned
   elsewhere in the Accept-Charset header.  As an example, the above
   header assigns a quality value of 0.9 to the iso-8859-2 charset.

   If no "*" is present in an Accept-Charset header, then all
   character sets not explicitly mentioned get a quality factor of 0,
   except for ISO-8859-1, which gets a quality factor of 1 if not
   explicitly mentioned.

       Note: The omission of a wildcard from the Accept-Language
       header in [1] is believed to be due to an oversight during the
       design of HTTP/1.1.  A future revision of [1] may correct this
       oversight, and make this section redundant.


8.3 Accept-Features

  This request header was defined in section 6.2.


8.4 Alternates

   The Alternates response header is used to convey the list of
   variants bound to a negotiable resource.  It can also contain other
   directives for the content negotiation process.

       Alternates = "Alternates" ":" 1#( variant-description
                                         fallback-variant
                                       | alt-directive )

       fallback-variant = "{" <"> URI <"> "}"

       alt-directive = ( "proxy-rvsa" "=" <"> 0#rvsa-version <"> )
                       | extension-alt-directive

       extension-alt-directive = token [ "=" ( token | quoted-string ) ]

   An example is

     Alternates: {"paper.html.en" 0.9 {type text/html} {language en}},
                 {"paper.html.fr" 0.7 {type text/html} {language fr}},
                 {"paper.ps.en"   1.0 {type application/postscript}
                     {language en}},
                 proxy-rvsa="1.0, 2.5"

   Any relative URI specified in a variant-description or
   fallback-variant field is relative to the request-URI.  Only one
   fallback-variant field may be present.  If the variant selection
   algorithm of the user agent finds that all described variants are
   unacceptable, then it should choose the fallback variant, if
   present, as the best variant.  If the user agent computes the
   overall quality values of the described variants, and finds that
   several variants share the highest value, then the first variant
   with this value in the list should be chosen as the best variant.

   The proxy-rvsa directive restricts the use of remote variant
   selection algorithms by proxies. If present, a proxy may only use
   the algorithms which have one of the version numbers listed, or
   have the same major version number and a higher minor version
   number as one of the versions listed.  Any restrictions set by
   proxy-rvsa come on top of the restrictions set by the user agent in
   the Negotiate request header.  The directive proxy-rvsa="" will
   disable variant selection by proxies entirely.  Clients should
   ignore all extension-alternates-directives they do not understand.

   A variant list may contain multiple differing descriptions of the
   same variant.  This can be convenient if the variant uses
   conditional rendering constructs, or if the variant resource
   returns multiple representations using a multipart media type.


8.5 Content-Features

   The Content-Features response header can be used by a server to
   indicate how the presence or absence of particular feature tags in
   the user agent affects the overall quality of the response.

       Content-Features = "Content-Features" ":" feature-list

     Note: This header mainly exists because of symmetry
     considerations.  It is the counterpart of the features attribute
     which can be present in variant descriptions.  If present in a
     response, the header will therefore not in general specify all
     user agent capabilities used by the response.


8.6 Negotiate

   The Negotiate request header can contain directives for any content
   negotiation process initiated by the request.

      Negotiate = "Negotiate" ":" 1#negotiate-directive

      negotiate-directive = "trans" | rvsa-version | "*"
                          | negotiate-extension

      negotiate-extension = token [ "=" token ]

   Examples are

      Negotiate: 1.0, 2.5
      Negotiate: *

   The negotiate directives have the following meaning

      "trans"
        The user agent supports transparent content negotiation for
        the current request.

      rvsa-version
        The user agent allows origin servers and proxies to run the
        remote variant selection algorithm with the indicated version
        number, or with the same major version number and a higher
        minor version number.  If the algorithm has sufficient
        information to choose a best variant, the origin server or
        proxy may return a choice response with this variant.  Implies
        "trans".

      "*"
        The user agent allows origin servers and proxies to run any
        remote variant selection algorithm.  The origin server may
        even run algorithms which have not been standardized.  If the
        algorithm has sufficient information to choose a best variant,
        the origin server or proxy may return a choice response with
        this variant.  Implies "trans".

   Servers should ignore all negotiate-directives they do not
   understand.


8.7 Variant-Vary

   The Variant-Vary response header can be used in a list response to
   record any vary information which applies to the variant data
   contained in the response, rather than to the response as a whole.

      Variant-Vary  = "Variant-Vary" ":" ( "*" | 1#field-name )

   Use of the Variant-Vary header is discussed in section 10.2.


9 Cache validators

   To allow for correct and efficient caching and revalidation of
   negotiated responses, this specification extends the caching model
   of HTTP/1.1 [1] in various ways.

   Under the rules in this specification, the maximum age (time since
   last revalidation) of a variant list bound to a negotiable
   resource, as received from a cache in an Alternates header, is the
   maximum of

      1. the freshness lifetimes (max-age values) in the responses
         from the negotiable resource itself, and

      2. the freshness lifetimes (max-age values) of the variant
         resources of the negotiable resource which are also neighbors
         of the negotiable resource.

   If no freshness lifetimes are assigned by the origin server, the
   maximum age of a variant list is the maximum of the freshness
   lifetime values which were heuristically assigned by the cache.


9.1 Variant list validators

   A variant list validator is an opaque value which acts as the cache
   validator of a variant list bound to a negotiable resource.

      variant-list-validator = <quoted-string not containing any ";">

   If two responses contain the same variant list validator, a cache
   can treat the Alternates headers in these responses as equivalent
   (though the headers themselves need not be identical).


9.2 Structured entity tags

   A structured entity tag consists of a normal entity tag of which
   the opaque string is extended with a semicolon followed by a
   variant list validator:

        normal      |  variant list  |   structured
        entity tag  |  validator     |   entity tag
       -------------+----------------+-----------------
         "etag"     |     "vlv"      |   "etag;vlv"
        W/"etag"    |     "vlv"      |  W/"etag;vlv"

   Note that a structured entity tag is itself also an entity tag.
   The structured nature of the tag allows caching proxies capable of
   transparent content negotiation to perform certain optimizations.
   Examples of structured entity tags are:

      "xyzzy;1234"  W/"xyzzy;1234"  "gonkxxxx;1234"  "a;b;c;;1234"

   In the last example, the normal entity tag is "a;b;c;" and the
   variant list validator is "1234".

   If a transparently negotiated response includes an entity tag, it
   must be a structured entity tag.  The variant list validator in the
   structured tag must act as a validator for the variant list
   contained in the Alternates header.  The normal entity tag in the
   structured tag must act as a validator of the entity body in the
   response and of all entity headers except Alternates.


9.3 Assigning entity tags to variants

   To allow for correct revalidation of transparently negotiated
   responses by clients, origin servers must generate all normal
   entity tags for the variant resources which are neighbors of the
   negotiable resource in such a way that

     1. the same tag is never used by two different variants,
        unless this tag labels exactly the same entity on all
        occasions,

     2. if one normal tag "X" is a prefix of another normal tag "XY",
        then "Y" must never be a semicolon followed by a variant list
        validator.


10 Content negotiation responses

   If a request on a transparently negotiated resource yields a
   response with a 2xx status code or any 3xx status code except 304,
   this response must always be either a list response, a choice
   response, or an ad hoc response.  These responses always include
   the Alternates header bound to the negotiable resource.

   Transparently negotiated responses with other status codes may also
   include an Alternates header, if this is allowed by the HTTP/1.1
   specification [1].  Note that HTTP/1.1 does not allow an Alternates
   header in a 304 (Not Modified) response.  When generating a
   response from a resource which does not support transparent content
   negotiation, a server must never include an Alternates header.

   A list response always has the 300 (Multiple Choices) response
   code.  A choice response never has the 300 code, and always has a
   Content-Location header.  An ad hoc response never has the 300
   code, and never has a Content-Location header.

   After having constructed a list, choice, or ad hoc response, a
   server may process any If-No-Match or If-Range headers in the
   request message and shorten the response to a 304 (Not Modified) or
   206 (Partial Content) response, following the rules in the HTTP/1.1
   specification [1].  In this case, the entity-ID of the shortened
   response will identify it as belonging to a list, choice, or ad-hoc
   response.


10.1 List response

   A list response has the 300 response status code.  It must contain
   (besides the normal headers required by HTTP) the Alternates header
   bound to the negotiable resource, a Vary header and (unless it was
   a HEAD request) an entity body which allows the user to manually
   select the best variant.  It is generated as a response to a user
   agent capable of transparent content negotiation if the server does
   not (perhaps cannot) choose a particular best variant for the
   request.

   An example of a list response is

     HTTP/1.1 300 Multiple Choices
     Date: Tue, 11 Jun 1996 20:02:21 GMT
     Alternates: {"paper.html.en" 0.9 {type text/html} {language en}},
                 {"paper.html.fr" 0.7 {type text/html} {language fr}},
                 {"paper.ps.en"   1.0 {type application/postscript}
                     {language en}}
     Vary: negotiate, accept, accept-language
     ETag: "blah;1234"
     Cache-control: max-age=86400
     Content-Type: text/html
     Content-Length: 227

     <h2>Multiple Choices:</h2>
     <ul>
     <li><a href=paper.html.en>HTML, English version</a>
     <li><a href=paper.html.fr>HTML, French version</a>
     <li><a href=paper.ps.en>Postscript, English version</a>
     </ul>

   The Vary header in the response should ensure correct handling by
   HTTP/1.1 caching proxies not capable of transparent content
   negotiation.  This header can either be

        Vary: *

   or a more elaborate header; see section 10.6.1.

   Only the origin server may construct list responses.  List
   responses are cacheable unless indicated otherwise.

   According to the HTTP/1.1 specification [1], a user agent not
   capable of transparent content negotiation will, when receiving a
   list response, display the entity body included in the response.
   If the response contains a Location header, however, the user agent
   may automatically redirect to this location.

   The handling of list responses by clients supporting transparent
   content negotiation is described in sections 11.1 and 13.

      Note: Some existing versions of HTTP/1.0 clients are known to
      silently ignore list responses, instead of handling them
      according to the HTTP/1.0 specification [3].  Servers should
      therefore be careful in sending list responses to
      non-negotiating HTTP/1.0 user agents, and in making these
      responses cacheable.


10.2 Choice response

   A choice response merges a normal HTTP response from the chosen
   variant, a Content-Location header giving the location of the
   variant, and the Alternates headers bound to the negotiable
   resource.  Depending on the response code, a choice response is
   cacheable unless indicated otherwise.

   Origin servers and proxy caches must construct choice responses
   with the following algorithm (or any other algorithm which gives
   equal end results for the client).

   In this algorithm, `the current Alternates header' refers to the
   Alternates header containing the variant list which was used to
   choose the best variant, and `the current variant list validator'
   refers to the validator of this list.  Section 10.3 specifies how
   these two items can be obtained by a proxy cache.

   The algorithm consists of four steps.

     1. Construct a HTTP request message on the best variant resource
        by rewriting the request-URI and Host header (if appropriate)
        of the received request message on the negotiable resource.

     2. Generate a valid HTTP response message, but not one with the
        304 (Not Modified) code, for the request message constructed
        in step 1.

        In a proxy cache, the response can be obtained from cache
        memory, or by passing the constructed HTTP request towards the
        origin server.  If the request is passed on, the proxy may
        add, modify, or delete If-None-Match and If-Range headers to
        optimize the transaction with the upstream server.

           Note: the proxy must be careful not to add entity tags of
           non-neighboring variants to the request, as there are no
           global uniqueness requirements for these tags.

     3. Check for an origin server configuration error. If the HTTP
        response message generated in step 2 contains an Alternates
        header, a Content-Location header, or has the 300 status code,
        then the best variant resource is not a proper end point in
        the negotiation process, and a 506 (Variant Also Negotiates)
        error response message should be generated instead of going to
        step 4.

     4. Add a number of headers to the HTTP response message generated
        in step 2.

        a. Add a Content-Location header giving the location of the
           chosen variant.

               Note: According to the HTTP/1.1 specification [1], if
               the Content-Location header contains a relative URI,
               this URI is relative to the URI in the Content-Base
               header, if present.

        b. If any Vary headers are present in the response message
           from step 2, add, for every Vary header, a Variant-Vary
           header with a copy of the contents of this Vary header.

        c. Add the current Alternates header.

        d. Add a Vary header to ensure correct handling by HTTP/1.1
           caching proxies not capable of transparent content
           negotiation.  This header can either be

               Vary: *

           or a more elaborate header, see section 10.6.

        e. To ensure compatibility with HTTP/1.0 caching proxies which
           do not recognize the Vary header, an Expires header with a
           date in the past may be added. See section 10.7 for more
           information.

        f. If an ETag header is present in the response message from
           step 2, then extend the entity tag in that header with the
           current variant list validator, as specified in section
           9.2.

        g. Only in proxy caches: set the Age header of the response to

              max( variant_age , alternates_age )

           where variant_age is the age of the variant response
           obtained in step 2, calculated according to the rules in
           the HTTP/1.1 specification [1], and alternates_age is the
           age of the Alternates header added in step c, calculated
           according to the rules in section 10.4.

   Note that a server can shorten the response produced by the above
   algorithm to a 304 (Not Modified) response if an If-None-Match
   header in the original request allows it.  If this is the case, an
   implementation of the above algorithm can avoid the unnecessary
   internal construction of full response message in step 2, it need
   only construct the parts which end up in the final 304 response.  A
   proxy cache which implements this optimization can sometimes
   generate a legal 304 response even if it has not cached the variant
   data itself.

   An example of a choice response is:

     HTTP/1.1 200 OK
     Date: Tue, 11 Jun 1996 20:05:31 GMT
     Content-Type: text/html
     Last-Modified: Mon, 10 Jun 1996 10:01:14 GMT
     Content-Length: 5327
     Cache-control: max-age=604800
     Content-Location: paper.html.en
     Alternates: {"paper.html.en" 0.9 {type text/html} {language en}},
                 {"paper.html.fr" 0.7 {type text/html} {language fr}},
                 {"paper.ps.en"   1.0 {type application/postscript}
                     {language en}}
     Etag: "gonkyyyy;1234"
     Vary: negotiate, accept, accept-language
     Expires: Thu, 01 Jan 1980 00:00:00 GMT

     <title>A paper about ....

   An example of forwarding by a proxy cache: if a proxy receives the
   request

     GET /paper HTTP/1.1
     Host: x.org
     User-Agent: WuxtaWeb/2.4
     Negotiate: 1.0
     Accept: text/html, *
     Accept-Language: en
     If-None-Match: "gonkyyyy;1234", W/"a;b;1234"

   and if it can reuse a cached variant list with the validator
   "1234", taken from a cached response with an age of 8000 seconds,
   to choose paper.html.en as the best variant, then the proxy can
   pass on the request

     GET /paper.html.en HTTP/1.1
     Host: x.org
     User-Agent: WuxtaWeb/2.4
     Negotiate: 1.0
     Accept: text/html, *
     Accept-Language: en
     If-None-Match: "gonkyyyy", W/"a;b"
     Via: 1.1 fred

   to an upstream server.  On receipt of the response

     HTTP/1.1 304 Not Modified
     Date: Tue, 11 Jun 1996 20:05:31 GMT
     Etag: "gonkyyyy"

   from the upstream server, it can return

     HTTP/1.1 304 Not Modified
     Date: Tue, 11 Jun 1996 20:05:31 GMT
     Etag: "gonkyyyy;1234"
     Content-Location: paper.html.en
     Vary: negotiate, accept, accept-language
     Expires: Thu, 01 Jan 1980 00:00:00 GMT
     Via: 1.1 fred
     Age: 8000

   to its own client.


10.3 Ad hoc response

   An ad hoc response never has the 300 response status code and never
   has a Content-Location header.  It must contain the Alternates
   header bound to the negotiable resource, and a Vary header if the
   response is cacheable.  It may be generated by an origin server as
   a response to a non-negotiating user agent, if the server cannot or
   does not want to send a list or choice response.

   The Vary header in the response should ensure correct handling by
   HTTP/1.1 caching proxies not capable of transparent content
   negotiation.  This header can either be

        Vary: *

   or a more elaborate header, see section 10.6.1.
   Depending on the response code, a choice response is
   cacheable unless indicated otherwise.

   An example of an ad hoc response is:

     HTTP/1.1 200 OK
     Date: Tue, 11 Jun 1996 20:02:26 GMT
     Alternates: {"paper.html.en" 0.9 {type text/html} {language en}},
                 {"paper.html.fr" 0.7 {type text/html} {language fr}},
                 {"paper.ps.en"   1.0 {type application/postscript}
                     {language en}}
     Vary: negotiate, accept, accept-language
     Etag: "gonkzzzz;1234"
     Cache-control: max-age=86400
     Content-Type: text/html
     Content-Length: 227

     <h2>Multiple Choices:</h2>
     <ul>
     <li><a href=paper.html.en>HTML, English version</a>
     <li><a href=paper.html.fr>HTML, French version</a>
     <li><a href=paper.ps.en>Postscript, English version</a>
     </ul>

   Another example is

     HTTP/1.1 302 Moved Temporarily
     Date: Tue, 11 Jun 1996 20:02:28 GMT
     Alternates: {"paper.html.en" 0.9 {type text/html} {language en}},
                 {"paper.html.fr" 0.7 {type text/html} {language fr}},
                 {"paper.ps.en"   1.0 {type application/postscript}
                     {language en}}
     Location: paper.html.en
     Content-Type: text/html
     Content-Length: 59

     This document is available <a href=paper.html.en>here</a>.


10.4 Reusing the Alternates header

   If a proxy cache has available a negotiated response which is
   cacheable, fresh, and has an ETag header, then it may extract the
   Alternates header and associated variant list validator from the
   response, and reuse them (without unnecessary delay) to negotiate
   on behalf of the user agent (section 13) or to construct a choice
   response (section 10.2).  The age of the extracted Alternates
   header is the age of the response from which it is extracted,
   calculated according to the rules in the HTTP/1.1 specification
   [1].


10.5 Extracting a normal response from a choice response

   If a proxy receives a choice response, it may extract and cache the
   normal HTTP response contained therein.  The normal response can be
   extracted by taking a copy of the choice response and then deleting
   the Content-Location, Alternates, and Vary headers, renaming any
   Variant-Vary headers to Vary headers, and shortening the structured
   entity tag in any ETag header to a normal entity tag.

   This normal response may be cached (as a HTTP response to the
   variant request as constructed in step 1. of section 10.2) and
   reused to answer future direct requests on the variant resource,
   according to the rules in the HTTP/1.1 specification [1].  This
   caching of extracted responses can increase overall efficiency with
   up to a factor 2.

   For security reasons (see section 14.2), an extracted normal
   response may only be cached if the negotiable resource and the
   variant resource are neighbors.  If they are not neighbors, the
   proxy should reject the choice response as a probable spoofing
   attempt and pass on a 502 (bad gateway) error response instead.


10.6 Elaborate Vary headers

   If a HTTP/1.1 [1] server can generate varying responses for a
   request on some resource, then the server must include a Vary
   header in these responses if they are cacheable.  This Vary header
   is a signal to HTTP/1.1 caches that something special is going on.
   It prevents the caches from returning the currently chosen response
   for every future request on the resource.

   Servers engaging in transparent content negotiation will generate
   varying responses.  Therefore, cacheable list, choice, and ad hoc
   responses must always include a Vary header.

   The most simple Vary header which can be included is

       Vary: *

   This header leaves the way in which the response is selected by the
   server completely unspecified.

   A more elaborate Vary header can be used to allow for certain
   optimizations in HTTP/1.1 caches which are not capable of
   transparent content negotiation, but which do cache multiple
   variant responses for one resource.  Such a more elaborate Vary
   header lists all request headers which can be used by the server
   when selecting a response for a request on the resource.

10.6.1 Construction of an elaborate Vary header

   Origin servers can construct a more elaborate Vary header in the
   following way.  First, start with the header

       Vary: negotiate

   `negotiate' is always included because servers use the information
   in the Negotiate header when choosing between a list, choice, or
   ad-hoc response.

   Then, if any of the following attributes is present in any variant
   description in the Alternates header, add the corresponding header
   name to the Vary header

         attribute  |   header name to add
         -----------+---------------------
          type      |   accept
          charset   |   accept-charset
          language  |   accept-language
          features  |   accept-features

   The Vary header constructed in this way specifies the response
   variation which can be caused by the use of a variant selection
   algorithm in proxies.  If the origin server will in some cases, for
   example if contacted by a non-negotiating user agent, use a custom
   negotiation algorithm which takes additional headers into account,
   these names of these headers should also be added to the Vary
   header.

10.6.2 Caching of an elaborate Vary header

   A proxy cache cannot construct an elaborate vary header using the
   method above, because this method requires exact knowledge of any
   custom algorithms present in the origin server.  However, when
   extracting an Alternates header from a response (section 10.4)
   caches may also extract the Vary header in the response, and reuse
   it along with the Alternates header.  A clean Vary header can
   however only be extracted if the variant does not vary itself,
   i.e. if a Variant-Vary header is absent.


10.7 Adding an Expires header to ensure HTTP/1.0 compatibility

   To ensure compatibility with HTTP/1.0 caching proxies which do not
   recognize the Vary header, an Expires header with a date in the
   past can be added to the response, for example

        Expires: Thu, 01 Jan 1980 00:00:00 GMT

   If this is done by an origin server, the server should usually also
   include a Cache-Control header for the benefit of HTTP/1.1 caches,
   for example

              Cache-Control: max-age=604800

   which overrides the freshness lifetime of zero seconds specified by
   the included Expires header.


10.8 Negotiation on content encoding

   Negotiation on the content encoding of a response is orthogonal to
   transparent content negotiation.  The rules for when a content
   encoding may be applied are the same as in HTTP/1.1: servers may
   content-encode responses that are the result of transparent content
   negotiation whenever an Accept-Encoding header in the request
   allows it.  When negotiating on the content encoding of a cacheable
   response, servers must add the accept-encoding header name to the
   Vary header of the response, or add `Vary: *'.

   Servers should always be able to provide unencoded versions of
   every transparently negotiated response.  This means in particular
   that every variant in the variant list must at least be available
   in an unencoded form.

   Like HTTP/1.1, this specification allows proxies to encode or
   decode relayed or cached responses on the fly: the response still
   contains the same variant as far as transparent content negotiation
   is concerned.  Note that HTTP/1.1 requires proxies to add a Warning
   header if the encoding of a response is changed.


11 User agent support for transparent negotiation

   This section specifies the requirements a user agent must satisfy
   in order to support transparent negotiation.  If the user agent
   contains an internal cache, this cache must satisfy the
   requirements for proxy caches in section 13.


11.1 Handling of responses

   If a list response is received when a resource is accessed, the
   user agent must be able to automatically choose, retrieve, and
   display the best variant, or display an error message if none of
   the variants are acceptable.

   If a choice response is received when a resource is accessed, the
   usual action is to automatically display the enclosed entity.
   However, if a remote variant selection algorithm which was enabled
   could have made a choice different from the choice the local
   algorithm would make, the user agent may apply its local algorithm
   to the variant list in the response, and automatically retrieve and
   display another variant if the local algorithm makes an other
   choice.

   When receiving a choice response, a user agent should check if the
   negotiable resource and the chosen variant resource are neighbors.
   If this is not the case, the user agent should reject the choice
   response as a probable spoofing attempt and display an error
   message, for example by internally replacing the choice response
   with a 502 (bad gateway) response.


11.2 Presentation of a transparently negotiated resource

   If the user agent is displaying a variant which is not an embedded
   or inlined object and which is the result of transparent
   negotiation, the following requirements must be met.

    1. The user agent should allow the user to review a list of all
       variants bound to the negotiable resource, and to manually
       retrieve another variant if desired.  There are two general
       ways of providing such a list.  First, the information in the
       Alternates header of the negotiable resource could be used to
       make an annotated menu of variants.  Second, the entity
       included in a list response of the negotiable resource could be
       displayed.  Note that a list response can be obtained by doing
       a GET request which only has the "trans" directive in the
       Negotiate header.

    2. The user agent should make available though its user interface
       some indication that the resource being displayed is a
       negotiated resource instead of a plain resource.  It should
       also allow the user to examine the variant list included in the
       Alternates header.  Such a notification and review mechanism is
       needed because of privacy considerations, see section 14.1.

    3. If the user agent shows the URI of the displayed information to
       the user, it should be the negotiable resource URI, not the
       variant URI that is shown.  This encourages third parties, who
       want to refer to the displayed information in their own
       documents, to make a hyperlink to the negotiable resource as a
       whole, rather than to the variant resource which happens to be
       shown.  Such correct linking is vital for the interoperability
       of content across sites.

    4. Similarly, if the user agent stores a reference to the
       displayed information for future use, for example in a hotlist,
       it should store the negotiable resource URI, not the
       variant URI.

   It is encouraged, but not required, that some of the above
   functionality is also made available for inlined or embedded
   objects, and when a variant which was selected manually is being
   displayed.


12 Origin server support for transparent negotiation

12.1 Requirements

   To implement transparent negotiation on a resource, the origin
   server must be able to send a list response when getting a GET
   request on the resource.  It should also be able to send
   appropriate list responses for HEAD requests.  A list response must
   always be sent if the request includes a Negotiate header with only
   a "trans" directive.  If the Negotiate header allows it, the origin
   server may run a remote variant selection algorithm, and if the
   algorithm has sufficient information to choose a best variant, the
   origin server may return a choice response with this variant.

   When getting a request without a Negotiate header indicating
   support for transparent content negotiation, the origin server may
   use a custom algorithm to select between sending a list, choice, or
   ad hoc response.  The origin server must never return a response
   with a 2xx status code or any 3xx status code, except 304, which is
   not a list, choice, or ad hoc response.

   Negotiability is a binary property: a resource is either
   transparently negotiated, or it is not.  Origin servers should not
   vary the negotiability of a resource, or the variant list bound to
   that resource, based on the request headers which are received.
   The variant list and the property of being negotiated may however
   change through time.  The Cache-Control header can be used to
   control the propagation of such time-dependent changes through
   caches.

   It is the responsibility of the author of the negotiable resource
   to ensure that all resources in the variant list serve the intended
   content, and that the variant resources do not engage in
   transparent content negotiation themselves.


12.2 Negotiation on transactions other than GET and HEAD

   If a resource is transparently negotiable, this only has an impact
   on the GET and HEAD transactions on the resource.  It is not
   possible (under this specification) to do transparent content
   negotiation on the direct result of a POST request.

   However, a POST request can return an unnegotiated 303 (See Other)
   response which causes the user agent to do a GET request on a
   second resource.  This second resource could then use transparent
   content negotiation to return an appropriate final response.  The
   figure below illustrates this.

      Server ______ proxy ______ proxy ______ user
      x.org         cache        cache        agent

        < -------------------------------------
        |     POST http://x.org/cgi/submit
        |     <form contents in request body>
        |
        -------------------------------------- >
              303 See Other                    |
              Location: http://x.org/result/OK |
                                               |
        < -------------------------------------
        |     GET http://x.org/result/OK
        |      small Accept headers
        |
      able to choose on
      behalf of user agent
        |
         ------------------------------------- >
              choice response with             |
              ..result/OK.nl variant           |
                                           displays OK.nl

   See the HTTP/1.1 specification [1] for details on the 303 (See
   Other) response code.  Note that this response code is not
   understood by most HTTP/1.0 clients.


13 Proxy support for transparent negotiation

   Transparent content negotiation is designed to work through any
   proxy which only implements the HTTP/1.1 specification [1].  If
   Expires headers are added as discussed in section 10.7, negotiation
   will also work though HTTP/1.0 proxies.  Thus, in a sense, every
   HTTP proxy supports transparent content negotiation.

   Plain HTTP/1.1 allows proxies to cache list, choice, and ad hoc
   responses, and to efficiently revalidate them by using the
   If-None-Match header.  This specification defines additional
   optimization mechanisms.

   First, when getting a request on a transparently negotiable
   resource from a user agent which is capable of transparent content
   negotiation (from a user agent which sends a Negotiate header), the
   proxy may return a cached, fresh list response from that resource.

   Second, when allowed by the user agent and origin server, a proxy
   may reuse an Alternates header taken from a previous response
   (section 10.4) to run a remote variant selection algorithm.  If the
   algorithm has sufficient information to choose a best variant, the
   origin server may return a choice response with this variant.

   Third, if a proxy receives a choice response, it may extract and
   cache the normal response embedded therein, as described in section
   10.5.


14 Security and privacy considerations

14.1 Accept headers revealing information of a private nature

   Accept headers, in particular Accept-Language headers, may reveal
   information which the user would rather keep private unless it will
   directly improve the quality of service.  For example, a user may
   not want to send language preferences to sites which do not offer
   multi-lingual content.  The transparent content negotiation
   mechanism allows user agents to omit sending of the Accept-Language
   header by default, without adversely affecting the outcome of the
   negotiation process if transparently negotiated multi-lingual
   content is accessed.

   However, even if Accept headers are never sent, the automatic
   selection and retrieval of a variant by a user agent will reveal a
   preference for this variant to the server.  A malicious service
   author could provide a page with `fake' negotiability on
   (ethnicity-correlated) languages, with all variants actually being
   the same English document, as a means of obtaining
   privacy-sensitive information.  Such a plot would however be
   visible to an alert victim if the list of available variants and
   their properties is reviewed.

   Some additional privacy considerations connected to Accept headers
   are discussed in [1].


14.2 Spoofing of responses from variant resources

   The caching optimization in section 10.5 gives the implementer of a
   negotiable resource control over the responses cached for all of
   its variant resources which are neighbors.  This is a security
   problem if a neighboring variant resource belongs to another
   author.  To provide security in this case, the HTTP server will
   have to filter the Content-Location headers in the choice responses
   generated by the negotiable resource implementation.


15 Acknowledgments

   Work on HTTP content negotiation has been done since at least 1993.
   The authors are unable to trace the origin of many of the ideas
   incorporated in this document.  This specification builds on an
   earlier incomplete specification of content negotiation recorded in
   [2].  Many members of the HTTP working group have contributed to
   the negotiation model in this specification.  The authors wish to
   thank the individuals who have commented on earlier versions of
   this document, including Brian Behlendorf, Daniel DuBois, Ted
   Hardie, Larry Masinter, and Roy T. Fielding.


16 References

   [1] R. Fielding, J. Gettys, J. C. Mogul, H. Frystyk, and
       T. Berners-Lee.  Hypertext Transfer Protocol -- HTTP/1.1.  RFC
       2068, HTTP Working Group, January, 1997.

   [2] Roy T. Fielding, Henrik Frystyk Nielsen, and Tim Berners-Lee.
       Hypertext Transfer Protocol -- HTTP/1.1.  Internet-Draft
       draft-ietf-http-v11-spec-01.txt, HTTP Working Group, January,
       1996.

   [3] T. Berners-Lee, R. Fielding, and H. Frystyk.  Hypertext
       Transfer Protocol -- HTTP/1.0.  RFC 1945.  MIT/LCS, UC Irvine,
       May 1996.

   [4] K. Holtman, A. Mutz.  Feature Tag Registration Procedures.
       Internet-Draft draft-ietf-http-feature-reg-00.txt, HTTP Working
       Group, October 30, 1996.

   [5] K. Holtman, A. Mutz.  HTTP Remote Variant Selection Algorithm
       -- RVSA/1.0.  Internet-Draft draft-ietf-http-rvsa-v10-00.txt,
       HTTP Working Group.


17 Authors' addresses

   Koen Holtman
   Technische Universiteit Eindhoven
   Postbus 513
   Kamer HG 6.57
   5600 MB Eindhoven (The Netherlands)
   Email: koen@win.tue.nl

   Andrew H. Mutz
   Hewlett-Packard Company
   1501 Page Mill Road 3U-3
   Palo Alto CA 94304, USA
   Fax +1 415 857 4691
   Email: mutz@hpl.hp.com


18 Appendix: feature negotiation examples

   This appendix contains examples of the use of feature tags in
   variant descriptions.  The tag names used here are examples only,
   they do not in general reflect the tag naming scheme proposed in
   [4].

18.1 Use of feature tags

   Feature tags can be used in variant lists to express the quality
   degradation associated with the presence or absence of certain
   features.  One example is

     {"index.html.plain" 0.7 },
     {"index.html"       1.0 {features tables frames}}

   Here, the "{features tables frames}" part expresses that index.html
   uses the features tagged as tables and frames.  If these features
   are absent, the overall quality of index.html degrades to 0.
   Another example is

     {"home.graphics" 1.0 {features !textonly}},
     {"home.textonly" 0.7 }

   where the "{features !textonly}" part expresses that home.graphics
   requires the absence of the textonly feature.  If the feature is
   present, the overall quality of home.graphics degrades to 0.

   The absence of a feature need not always degrade the overall quality
   to 0.  In the example

     {"x.html.1" 1.0 {features fonts/0.7}}

   the absence of the fonts feature degrades the quality with a factor
   of 0.7.  "fonts/0.7" can be pronounced as "fonts, or a degradation
   of 0.7".  Finally, in the example

      {"y.html" 1.0 {features [blebber wolx] }}

   The "[blebber wolx]" expresses that y.html requires the presence of
   the blebber feature or the wolx feature.  This construct can be
   used in a number of cases:

     1. blebber and wolx actually tag the same feature, but they were
        registered by different people, and some user agents say they
        support blebber while others say they support wolx.

     2. blebber and wolx are HTML tags of different vendors which
        implement the same functionality, and which are used
        together in y.html without interference.

     3. blebber and wolx are HTML tags of different vendors which
        implement the same functionality, and y.html uses the tags in
        a conditional HTML construct.

     4. blebber is a complicated HTML tag with only a sketchy
        definition, implemented by one user agent vendor, and wolx
        indicates implementation of a well-defined subset of the
        blebber tag by some other vendor(s).  y.html uses only this
        well-defined subset.


18.2 Use of numeric feature tags

   As an example of negotiation in a numeric area, the following
   variant list describes four variants with title graphics designed
   for increasing screen widths:

     {"home.pda"    1.0 {features screenwidth=<-199> }},
     {"home.narrow" 1.0 {features screenwidth=<200-599> }},
     {"home.normal" 1.0 {features screenwidth=<600-999> }},
     {"home.wide"   1.0 {features screenwidth=<1000-> }},
     {"home.normal"}

   The last element of the list specifies a safe default for user
   agents which do not implement screen width negotiation.  Such user
   agents will reject the first four variants as unusable, as they
   seem to rely on a feature which they do not understand.


18.3 Feature tag design

   When designing a new feature tag, it is important to take into
   account that existing user agents, which do not recognize the new
   tag will treat the feature as absent.  In general, a new feature
   tag needs to be designed in such a way that absence of the tag is
   the default case which reflects current practice.  If this design
   principle is ignored, the resulting feature tag will generally be
   unusable.

   As an example, one could try to support negotiation between
   monochrome and color content by introducing a `color' feature tag,
   the presence of which would indicate the capability to display
   color graphics.  However, if this new tag is used in a variant
   list, for example

      {"rainbow.gif"      1.0 {features color} }
      {"rainbow.mono.gif" 0.6 {features !color}}

   then existing user agents, which would not recognize the color tag,
   would all display the monochrome rainbow.  The color tag is
   therefore unusable in situations where optimal results for existing
   user agents are desired.  To provide for negotiation in this area,
   one must introduce a `monochrome' feature tag; its presence
   indicates that the user agent can only render (or the user prefers
   to view) monochrome graphics.


19 Appendix: origin server implementation considerations

19.1 Implementation with a CGI script

   Transparent content negotiation has been designed to allow a broad
   range of implementation options at the origin server side.  A very
   minimal implementation can be done using the CGI interface.  The
   CGI script below is an example.

      #!/bin/sh

      echo "$HTTP_NEGOTIATE" | awk '$0~ \
      "^(|.*,)[\t ]*(trans|\*|[0-9]+.[0-9]+)[\t ]*(|,.*)$" \
      { print "Status: 300 Multiple Choices" }' -

      cat - <<'blex'
      Alternates: {"stats.tables.html" 1.0 {type text/html} {features
      tables}}, {"stats.html" 0.8 {type text/html}}, {"stats.ps" 0.95
      {type application/postscript}}
      Vary: *
      Content-Type: text/html

      <title>Multiple Choices for Web Statistics</title>
      <h2>Multiple Choices for Web Statistics:</h2>
      <ul>
      <li><a href=stats.tables.html>Version with HTML tables</a>
      <p>
      <li><a href=stats.html>Version without HTML tables</a>
      <p>
      <li><a href=stats.ps>Postscript version</a>
      </ul>
      blex

   The Alternates header in the above script must be read as a single
   line.  The script generates a list response for user agents capable
   of transparent content negotiation, and an ad hoc 200 (OK) response
   for all non-negotiating agents.


19.2 Direct support by HTTP servers

   Sophisticated HTTP servers could make a transparent negotiation
   module available to content authors.  Such a module could
   incorporate a remote variant selection algorithm and an
   implementation of the algorithm for generating choice responses
   (section 10.2).  The definition of interfaces to such modules is
   beyond the scope of this specification.


19.3 Web publishing tools

   Web publishing tools could automatically generate several variants
   of a document (for example the original TeX version, a HTML version
   with tables, a HTML version without tables, and a Postscript
   version), together with an appropriate variant list in the
   interface format of a HTTP server transparent negotiation module.
   This would allow documents to be published as transparently
   negotiable resources.


Expires: August 5, 1997
