2004/id/draft-holtman-http-negotiation-01.txt

Internet-Draft                                          Koen Holtman/TUE
Expires: 13 December 1996                                   13 June 1996


                     Transparent Content Negotiation

                  draft-holtman-http-negotiation-01.txt


STATUS OF THIS MEMO

        This document is an Internet-Draft. Internet-Drafts are
        working documents of the Internet Engineering Task Force
        (IETF), its areas, and its working groups. Note that other
        groups may also distribute working documents as
        Internet-Drafts.

        Internet-Drafts are draft documents valid for a maximum of
        six months and may be updated, replaced, or obsoleted by
        other documents at any time. It is inappropriate to use
        Internet-Drafts as reference material or to cite them other
        than as "work in progress".

        To learn the current status of any Internet-Draft, please
        check the "1id-abstracts.txt" listing contained in the
        Internet-Drafts Shadow Directories on ftp.is.co.za
        (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific
        Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US
        West Coast).

        Distribution of this document is unlimited.  Please send
        comments to the HTTP working group at
        <http-wg@cuckoo.hpl.hp.com>.  Discussions of the working
        group are archived at
        <URL:http://www.ics.uci.edu/pub/ietf/http/>.  General
        discussions about HTTP and the applications which use HTTP
        should take place on the <www-talk@w3.org> mailing list.


ABSTRACT

        HTTP/1.1 will allow one to put multiple versions of the same
        information under a single URL.  Transparent content negotiation
        is a mechanism, layered on top of HTTP/1.1, for selecting the
        best version when a retrieval request is made.  This document
        outlines the planned transparent negotiation mechanism, and
        identifies some issues and tradeoffs that have yet to be
        addressed.


                                                              [Page 1]

Internet-Draft          Content Negotiation                 13 June 1996

1  Introduction

   Transparent content negotiation is a mechanism layered on top of the
   HTTP/1.1 protocol [1].  HTTP/1.1 will allow one to put multiple
   versions of the same information under a single URL.  Each of these
   versions is called a `variant'.  Transparent content negotiation is a
   mechanism for selecting the best variant when a retrieval request is
   made.  This document outlines the planned transparent content
   negotiation mechanism, and identifies some issues and tradeoffs that
   have yet to be addressed.


2  Overview of HTTP transparent content negotiation

   This section contains an extended version of the HTTP content
   negotiation presentation given by the author at the Fifth
   International WWW Conference in Paris.  This section tries to avoid
   the use of terminology local to the IETF http-wg.  For example, the
   term `URL' is used instead of `URI'.


2.1 Content negotiation

   HTTP/1.1 will allow one to put multiple versions of the same
   information under a single URL.  Each of these versions is called a
   `variant'. For example, a URL http://x.org/paper could bind to three
   different variants of a paper:

         1. HTML, English
         2. HTML, French
         3. Postscript, English

   Content negotiation is the process by which the best variant is
   selected if the URL is accessed.

   It has always been possible under HTTP to have multiple
   representations available for one URL, and to return the most
   appropriate representation for each subsequent request.  However,
   HTTP/1.1 is the first version of HTTP which has provisions for doing
   this in a cache-friendly way.  These provisions include the Vary
   response header, entity tags, and the If-None-Match request header.

   In content negotiation, the selection of the best variant for a
   particular request is based on a matching of the properties of the
   available variants to the capabilities of the user agent and the
   preferences of the user.


2.2 HTTP/1.0 variant selection scheme

   Under the HTTP/1.0 scheme, the variant selection happens as follows:


                                                              [Page 2]

Internet-Draft          Content Negotiation                 13 June 1996

      Server _____ proxy _____ proxy _____ user
      x.org        cache       cache       agent

        < ----------------------------------
        |      GET http://x.org/paper
        |          Accept headers
      choose
        |
         ---------------------------------- >
                    Best variant

   When the URL is accessed, the user agent sends, along with its
   request, various Accept headers which express the user agent
   capabilities and the user preferences.  Then, the origin server uses
   these Accept headers to choose the best variant, which is returned
   in the response.

   The biggest problem with this scheme is that it does not scale well.
   For all but the most minimal user agents, Accept headers expressing
   all capabilities and preferences would be very large, and sending
   them in every request would be hugely inefficient, in particular
   because only a small fraction of the URLs on the web will have
   multiple variants.

   In current practice, user agents send short Accept headers which
   inadequately describe capabilities and preferences, except maybe for
   inline image requests.  Servers providing multiple representations of
   the same information usually do so under different URLs, and allow
   users to manually select a representation by clicking a particular
   link.

   Also in current practice, servers often use the contents of the
   User-Agent request header for selection purposes. These contents
   sometimes allow the server to infer information about capabilities
   which cannot be expressed with existing Accept headers.


2.3 Transparent content negotiation scheme

   The transparent negotiation scheme eliminates the need to send huge
   Accept headers, and nevertheless allows for a selection process that
   always yields either the best variant, or an error message indicating
   that user agent is not capable of displaying any of the available
   variants.

   Under the transparent content negotiation scheme, the server sends a
   list with the available variants and their properties to the user
   agent.  An example of a list with three variants is

     {"paper.html.en" 0.9 {type text/html} {language en}},
     {"paper.html.fr" 0.7 {type text/html} {language fr}},
     {"paper.ps.en"   1.0 {type application/postscript} {language en}}

   The syntax and semantics of the variant descriptions in this list are
   covered in depth in Section 3.  When the list is received, the user

                                                              [Page 3]

Internet-Draft          Content Negotiation                 13 June 1996

   agent can choose the best variant and retrieve it.  Graphically, the
   communication can be represented as follows:

      Server _____ proxy _____ proxy _____ user
      x.org        cache       cache       agent

        < ----------------------------------
        |      GET http://x.org/paper
        |
        ----------------------------------- >
                  return of list            |
                                         choose
                                            |
        < ----------------------------------
        |  GET http://x.org/paper.html.en
        |
         ---------------------------------- >
                return of html.en

   With this scheme, the information about capabilities and preferences
   is only used by the user agent itself.  Therefore, the sending of
   such information in large Accept headers is unnecessary.  Accept
   headers do have a limited use in transparent content negotiation
   however: the sending of small Accept headers can often speed up the
   negotiation process, this is covered in Section 2.4.

   According to current plans, the response that returns the list in the
   above picture will look as follows, with the list of variants in the
   Alternates header:

     HTTP/1.1 300 Multiple Choices
     Date: Tue, 11 Jun 1996 19:39:48 GMT
     Alternates: {"paper.html.en" 0.9 {type text/html} {language en}},
                 {"paper.html.fr" 0.7 {type text/html} {language fr}},
                 {"paper.ps.en"   1.0 {type application/postscript}
                     {language en}}
     Vary: accept, accept-language
     Content-Type: text/html
     Content-Length: 227

     <h2>Multiple Choices:</h2>
     <ul>
     <li><a href=paper.html.en>HTML, English version</a>
     <li><a href=paper.html.fr>HTML, French version</a>
     <li><a href=paper.ps.en>Postscript, English version</a>
     </ul>

   The HTML page included in the response is intended for compatibility
   with (existing) user agents which do not support transparent content
   negotiation.  Such user agents will ignore the computer-readable list
   in the Alternates header and just display the HTML page, allowing the
   user to select the best variant by hand.

   [##Issue to be resolved: Some existing HTTP/1.0 user agents crash
   when getting a 300 response without a Location header.  This problem

                                                              [Page 4]

Internet-Draft          Content Negotiation                 13 June 1996

   may be big enough to warrant use of a response code outside of the
   3xx class, at least for responses to HTTP/1.0 clients.##]


2.4 Cutting corners

   The basic transparent negotiation scheme involves two HTTP
   transactions: one to retrieve the list, and a second one to retrieve
   the chosen variant.  There are however several ways to `cut corners'
   in the data flow path of the basic scheme.

   First, caching proxies can cache both lists and responses from
   variant URLs.  Such caching can reduce the communication overhead, as
   shown in the following example:

      Server _____ proxy _____ proxy __________ user
      x.org        cache       cache            agent

                                 < --------------
                                 |  GET ../paper
                                 |
                               has the list
                               in cache
                                 |
                                  -------------  >
                                           list  |
                                                 |
                                              choose
                                                 |
                     < --------------------------
                     |   GET ../paper.html.en
                     |
                  has the variant
                  in cache
                     |
                      -------------------------- >
                         return of html.en

   Second, the user agent can send small Accept headers which could
   allow a server to determine the choice the user agent would make on
   receiving the list.  If the Accept headers contain enough
   information, the origin server can send back the variant directly:

      Server _____ proxy _____ proxy _____ user
      x.org        cache       cache       agent

        < ----------------------------------
        |      GET http://x.org/paper
        |       small Accept headers
        |
      able to choose on
      behalf of user agent
        |
         ---------------------------------- >
              return of html.en and list

                                                              [Page 5]

Internet-Draft          Content Negotiation                 13 June 1996


   According to current plans, the response that returns the html.en
   variant and the list in the above picture will look as follows:

     HTTP/1.1 200 OK
     Date: Tue, 11 Jun 1996 19:51:02 GMT
     Alternates: {"paper.html.en" 0.9 {type text/html} {language en}},
                 {"paper.html.fr" 0.7 {type text/html} {language fr}},
                 {"paper.ps.en"   1.0 {type application/postscript}
                     {language en}}
     Vary: accept, accept-language
     Content-Location: paper.html.en
     Content-Type: text/html
     Last-Modified: Mon, 10 Jun 1996 10:01:14 GMT
     Etag: "gonkyyyy"
     Content-Length: ...

     ...
 
   Finally, the above two kinds of optimization can be combined: a
   caching proxy which has the list will sometimes be able to choose on
   behalf of the user agent.  This could lead to the following
   communication pattern:

      Server _____ proxy _____ proxy __________ user
      x.org        cache       cache            agent

                                 < ---------------
                                 |  GET ../paper
                                 |  small Accept
                                 |
                              able to choose
                               on behalf
                                 |
                     < ----------
                     |  GET ../paper.html.en
                     |
                      -----------
                        html.en  |
                                  ---------------- >
                                   html.en and list


2.5 The network negotiation algorithm

   To allow for the second optimization above, choosing on behalf of the
   user agent, transparent content negotiation will standardize the
   so-called network negotiation algorithm.

   The algorithm will take as input

        - the list of variants of a transparently negotiable URL

        - the (small) Accept headers of an incoming request on that URL

                                                              [Page 6]

Internet-Draft          Content Negotiation                 13 June 1996


   and will yield as output the appropriate action to be performed by
   the proxy cache or origin server.  Proxy caches are required to
   either perform this appropriate action or to perform any other action
   allowed by the HTTP/1.1 standard.  Origin servers are not required to
   follow the network negotiation algorithm.

   If the request was made by a user agent capable of transparent
   negotiation, the algorithm can yield two possible actions:

      A) Choice on behalf of the user agent:
         A particular variant X may be retrieved and returned on behalf
         of the user agent.

      B) No choice possible, return the list:
         The Accept headers do not contain sufficient information to
         make a choice on behalf of the user agent possible: the list of
         variants should be returned so that the user agent can make the
         choice itself.

   If the request was made by an (old) user agent not capable of
   transparent negotiation, the algorithm can yield three possible
   actions:

      C) Choice on behalf of the origin server:
         A particular variant X may be retrieved and returned on behalf
         of the origin server

      D) No choice possible, return a cached 300 response:
         The Accept headers do not contain sufficient information to
         make a choice on behalf of the origin server possible, and a
         cached or freshly retrieved 300 response should be returned,
         allowing the user to manually select the best variant

      E) No choice possible, forward request towards the origin server:
         The Accept headers do not contain sufficient information to
         make a choice on behalf of the origin server possible, and the
         request should be forwarded towards the origin server so that
         it can choose the best variant with a custom negotiation
         algorithm.

   [##Issue to be resolved: though several definitions of subsets of the
   network negotiation algorithm exist, the algorithm has not been fully
   defined yet.  A mechanism for deciding between D) and E) has not yet
   been discussed.##]

   As the network negotiation algorithm is only an optimization device,
   complete or partial implementation of the algorithm by proxies is
   optional.  To support transparent content negotiation, a HTTP/1.1
   proxy need only follow the caching rules defined in the HTTP/1.1
   specification.

   The core computations performed by the network negotiation algorithm
   are based on quality factors, which are short real numbers in the


                                                              [Page 7]

Internet-Draft          Content Negotiation                 13 June 1996

   range 0.000 -- 1.000.  These factors appear in variant lists and in
   Accept headers.

2.5.1 Network negotiation algorithm example 1

   As an example, if a variant list contains the variant description

     {"paper.html.en" 0.7 {type text/html} {language fr}}

   and if the request contains the following Accept headers:

     Accept: text/html:q=1.0, */*:q=0.8, transparent
     Accept-Language: en;q=1.0, fr;q=0.5

   the network negotiation algorithm will compute an `overall quality'
   for the variant by multiplying quality factors as follows:

     {"paper.html.en" 0.7 {type text/html} {language fr}}
                       |           |                 |
                       |           |                 |
                       V           V                 V
                      0.7   *     1.0        *      0.5  = 0.35

   The 1.0 above is the quality factor assigned to "text/html" by the
   Accept header, the 0.5 is the quality factor assigned to "fr" by the
   Accept-Language header.  The overall quality of this variant is 0.35.

   With the above Accept headers, the complete variant list

     {"paper.html.en" 0.9 {type text/html} {language en}},
     {"paper.html.fr" 0.7 {type text/html} {language fr}},
     {"paper.ps.en"   1.0 {type application/postscript}
                             {language en}}

   would yield the following computations:

      paper.html.en: 0.9 * 1.0 * 1.0 = 0.9
      paper.html.fr: 0.7 * 1.0 * 0.5 = 0.35
      paper.ps.en:   1.0 * 0.8 * 1.0 = 0.8

   The variant with the highest overall quality, "paper.html.en" in this
   case, is the variant that seems most acceptable to the user agent.

   [##Aside for mathematicians: note that this calculation scheme has
   similarities with fuzzy set theory.  As far as I know, fuzzy set
   theory was not involved when the scheme was thought up.##]

   The inclusion of the word `transparent' in the received Accept header

     Accept: text/html:q=1.0, */*:q=0.8, transparent

   indicates that the user agent is capable of transparent negotiation.
   The second stage of the network negotiation algorithm will therefore
   have to choose between two possible end results:


                                                              [Page 8]

Internet-Draft          Content Negotiation                 13 June 1996

      A) Choice on behalf of the user agent:
         The variant paper.html.en may be retrieved and returned on
         behalf of the user agent.

      B) No choice possible, return the list:
         The Accept headers do not contain sufficient information to
         make a choice on behalf of the user agent possible.

   The choice is made by examining the Accept header components used to
   derive the best overall quality value of 0.9 for paper.html.en:

          text/html:q=1.0
          en;q=1.0

   If none of these components contain a wildcard character *, as is the
   case here, the appropriate action is A).  If a wildcard character is
   present, the appropriate action is B).

2.5.2 Network negotiation algorithm example 2

   If the same variant list is matched to the following shorter Accept
   headers:

      Accept: */*:q=1.0, transparent
      Accept-Language: en;q=1.0, fr;q=0.5

   the calculations would as follows:

      paper.html.en: 0.9 * 1.0 * 1.0 = 0.9
      paper.html.fr: 0.7 * 1.0 * 0.5 = 0.35
      paper.ps.en:   1.0 * 1.0 * 1.0 = 1.0

   Now, paper.ps.en is the best candidate.  However, if the Accept
   header components used to derive the best overall quality value of
   1.0 are examined:

      */*:q=1.0
      en;q=1.0

   it is found that a wildcard character * is present.  Therefore, the
   algorithm yields the result

      B) No choice possible, return the list:
         The Accept headers do not contain sufficient information to
         make a choice on behalf of the user agent possible.

   Note that the interpretation of */*:q=1.0, if sent by a user agent
   supporting transparent negotiation, is therefore not

     `I accept all media types with a quality of 1.0'

   but


                                                              [Page 9]

Internet-Draft          Content Negotiation                 13 June 1996

     `I accept several media types, and assign quality factors from 0.0
     up to 1.0 to them.  If this information is insufficient to make a
     choice on my behalf, do not make a choice but send the list of
     variants'.

   This means that a user agent can always send the short

      Accept: */*, transparent

   by default, without this affecting the quality of the negotiation
   process.  If it is found that requests to the server often return 300
   responses, the user agent can try to dynamically extend the Accept
   header sent in requests to that server, for example to

      Accept: text/html, application/postscript:q=0.8,
              */*, transparent

   to increase the chance that the network negotiation algorithm will be
   able to optimize the retrieval process.  A good strategy for dynamic
   extension would be to extend the Accept header with those media types
   mentioned in the variant lists of past (300) responses from the
   server.


2.6 Retrieving a variant by hand

   If a transparently negotiated resource is accessed, the user agent
   will always at some point receive the list of available variants.
   The user agent can use this list to make available a menu (in HTML or
   not) which lists all variants and their characteristics to the user.
   Such a menu will allow the user to randomly browse other variants,
   and will also make it possible to manually correct any sub-optimal
   choice made by the automatic negotiation process.


2.7 Dimensions of negotiation

   Transparent content negotiation defines four dimensions of
   negotiation:

     1. Media type (MIME type)
     2. Charset
     3. Language
     4. Features

   The first three dimensions have traditionally been present in HTTP,
   the fourth dimension is added by the transparent content negotiation
   specification.

   Additional dimensions, beyond the four mentioned above, could be
   added by future specifications.

   Negotiation on the content encoding of a response (gzipped,
   compressed, etc.) is left outside of the realm of transparent
   negotiation.  Transparent negotiation does not prohibit proxies to

                                                              [Page 10]

Internet-Draft          Content Negotiation                 13 June 1996

   encode or decode a relayed or cached response on the fly: the
   response still contains the same variant as far as transparent
   negotiation is concerned.


2.8 Feature negotiation

   Feature negotiation intends to provide for all areas of negotiation
   not covered by the first three dimensions.  Examples are negotiation
   on

       * HTML extensions
       * Extensions of non-HTML media types
       * Color capabilities of the user agent
       * Output medium (screen, paper, ...)
       * Preference for speed vs. preference for graphical detail

   The feature negotiation framework is the main means by which
   transparent negotiation offers extensibility: a new dimension of
   negotiation (really a sub-dimension of the feature dimension) can be
   added without the need for a new standards effort, by the simple
   registration of a `feature tag'.

2.8.1 Feature tags

   A feature tag identifies a capability of a user agent or a preference
   of a user.  A feature is said to be `present' in a user agent if the
   corresponding capability is implemented, or if the user has expressed
   corresponding preference.

   Examples of feature tags are

           ns_tables, fonts, monochrome, textonly

   Note that it sometimes depends on context whether a feature tag
   expresses a capability or a preference.  For a text-only browser, the
   `textonly' tag is naturally present, but the user of a graphical
   browser could set the tag to be present if text-only content is
   preferred to graphical content.

   The `feature set' of a user agent contains all tags of the features
   present in the user agent for the current request.  In general, the
   part of the feature set which reflects capabilities will be
   hard-coded in the user agent binary or in a user agent library file,
   and the part which reflects preferences will have been configured by
   the user.

   As feature registration will be an ongoing process, it is generally
   not possible for a user agent to know the meaning of all feature tags
   it can possibly encounter in a variant description.  A user agent
   should assume that all feature tags unknown to it are not in its
   feature set.


                                                              [Page 11]

Internet-Draft          Content Negotiation                 13 June 1996

2.8.2 Feature tag registration

   According to current plans, everybody will be allowed to register
   feature tags: registration only requires that the tag follows the
   syntax rules, and that a definition of the meaning of the tag is
   supplied.  In particular, registration will not require actual
   implementation of a feature, and there will be no test on whether the
   feature definition overlaps with another feature definition.

   The feature negotiation mechanism is designed not to break if 1000+
   features, partially overlapping, are registered.  It is expected that
   after the introduction of feature negotiation, an explosion in
   feature tag registration will occur, and that only some of these tags
   will end up being actively used to label variants.  Web indexing
   robots could, while traversing the web, gather statistics about
   actual use of feature tags.  These statistics could be used by
   individuals to compile lists, intended for content authors, of useful
   feature tags for particular areas of negotiation.  Note that this
   indexing activity is orthogonal to the feature registration process.
   It is expected that, once an area of negotiation is well-understood,
   this process will converge on a commonly-used and commonly-recognized
   set of feature tags for that area.

   [##Issue to be resolved: it is unclear yet whether it is desirable
   for feature tags to have some hierarchical structure, which would
   make it easier, for example, to trace who registered the feature.
   All that is known now is that it is desirable to have short feature
   tags.  A naming scheme like that for SGML-DTDs or PEP protocol names
   would probably yield feature tags which are too long.  The main
   advantage of having a registration process at all is that it allows
   for short tags without running the risk of `tag collision'.##]

2.8.3 Use of feature tags

   Feature tags can be used in variant lists to express the quality
   degradation associated with the presence or absence of certain
   features.  This is covered in depth in Section 4.4.  One example is

     {"index.html.plain" 0.7 {type text/html} },
     {"index.html"       1.0 {type text/html}
                               {features ns_tables ns_frames} }

   Here, the "{features ns_tables ns_frames}" part expresses that
   index.html uses the features tagged as ns_tables and ns_frames.  If
   these features are absent, the overall quality of index.html degrades
   to 0.  Another example is

     {"home.graphics" 1.0 {type text/html} {features !textonly} },
     {"home.textonly" 0.7 {type text/html} }

   where the "{features !textonly}" part expresses that home.graphics
   requires the absence of the textonly feature.  If the feature is
   present, the overall quality of home.graphics degrades to 0.


                                                              [Page 12]

Internet-Draft          Content Negotiation                 13 June 1996

   The absence of a feature need not always degrade the overall quality
   to 0.  In the example

     {"x.html.1" 1.0 {type text/html} {features fonts/0.7}}

   the absence of the fonts feature degrades the quality with a factor
   of 0.7.  "fonts/0.7" can be pronounced as "fonts, or a degradation of
   0.7".  In the example

     {"x.html.2" 0.5 {type text/html} {features fonts:1.5}}

   the presence of the fonts feature improves the overall quality with a
   factor of 1.5. If the fonts feature is absent, the overall quality is
   not affected.  "fonts:1.5" can be pronounced as "fonts improves with
   1.5".  Finally, in the example

      {"y.html" 1.0 {type text/html} {features [blebber wolx] } }

   The "[blebber wolx]" expresses that y.html.1 requires the presence of
   the blebber feature or the wolx feature.  This construct can be used
   in a number of cases:

     1. blebber and wolx actually tag the same feature, but they
        were registered by different people, and some browsers say
        they support blebber while others say they support wolx.

     2. blebber and wolx are HTML tags of different vendors which
        implement the same functionality, and which were used
        together in y.html without interference.

     3. blebber and wolx are HTML tags of different vendors which
        implement the same functionality, and y.html uses conditional
        HTML to provide versions using both tags [##Note: conditional
        HTML does not yet exist, but it is something people are thinking
        about.##]

     4. blebber is a complicated HTML tag with only a sketchy
        definition, implemented by one browser vendor, and wolx
        indicates implementation of a well-defined subset of the blebber
        tag by some other vendor(s).  y.html uses only this well-defined
        subset.


2.9 Current status of transparent content negotiation

   There is no complete specification of transparent content negotiation
   yet.  Documents completely specifying some subset of transparent
   content negotiation have been written, and have had some review by
   the http-wg, but none of these define feature negotiation, and none
   of them take the specific caching mechanisms defined by the latest
   version of the HTTP/1.1 specification [1] into account.

   The author of this document plans to finish a complete specification
   of transparent content negotiation, including feature negotiation,
   within the time frame of a few months.  Many aspects of the design of

                                                              [Page 13]

Internet-Draft          Content Negotiation                 13 June 1996

   feature negotiation are still in flux.  Some tradeoffs between
   complexity and expressive power have yet to be discussed.

   No serious efforts have yet been made to invent and define feature
   tags for particular complicated areas of negotiation.  The final
   shape of the feature negotiation mechanism will in part depend on the
   result of such efforts.  It is expected that feature negotiation will
   solve many of the current negotiation problems.  However, it is also
   expected that there will remain negotiation problems not solved by
   feature negotiation.

2.9.1 Feature negotiation in numeric areas

   Feature tags as covered in Section 2.8 have a boolean nature, because
   they serve as indicators of the simple presence or absence of some
   feature.  Many areas of negotiation however, like negotiation on
   screen size, color depth, or bandwidth, have a numeric nature.  A
   numeric area like negotiation on color depth can however be trivially
   mapped onto the feature negotiation framework.  One can define a
   collection of four feature tags

       colordepth_1
       colordepth_4
       colordepth_8
       colordepth_24

   to span the numeric range of color depth negotiation.  Each of these
   tags would indicate the capability to render pictures up to the given
   color depth.  Under this scheme, a user agent rendering on a screen
   with 8 bit color depth would have all three features

       colordepth_1, colordepth_4, colordepth_8

   present in its feature set, so that a variant using 10 distinct
   colors can be safely described as

     {"button.gif" 0.5 {type image/gif} {features colordepth_4}}

   While this technique seems adequate for to many numeric areas of
   negotiation, it is unknown at this point whether it provides
   sufficient expressive power to cover all areas feature negotiation
   intends to cover.  Section 4 will introduce additional feature
   negotiation protocol elements which provide more direct support for
   negotiation on numbers.


2.9.2 Bandwidth negotiation

   With a bandwidth negotiation mechanism, the time needed to retrieve
   a particular variant over the network can be taken into account
   during the negotiation process.  Work on mechanisms for bandwidth
   negotiation has been done since at least 1993, but this work has
   not yet yielded a successful standard mechanism for bandwidth
   negotiation.  Cascaded proxy caches introduce additional complexity
   in this area.

                                                              [Page 14]

Internet-Draft          Content Negotiation                 13 June 1996


   The planned transparent content negotiation specification will not
   attempt to solve the problem of bandwidth negotiation.  There is some
   hope that the feature negotiation framework will allow the
   introduction of an adequate solution for bandwidth negotiation.

2.9.3 Content transformation by proxies

   Recently, there has been some attention to content transformation by
   proxies.  An example is the transformation of an image/gif response
   to a shorter image/jpeg response, with potential degraded quality, to
   save bandwidth.  Another example is the transformation of an
   application/postscript response to a text/html response, with
   definite degraded quality, to allow viewing by a user agent which
   cannot handle postscript.

   The impact of such transformations on the quality of content on the
   web, and the impact on the current model for the allocation of trust
   to various parties in a HTTP transaction, has not yet been fully
   explored.

   Content transformation could be a useful optimization for transparent
   content negotiation.  A response with a variant list

     {"map.gif" 1.0 {type image/gif} {features !monochrome}},
     {"map.mono.gif" 0.8 {type image/gif} {features monochrome}}

   could include an additional response header

     Allow-Transform: "map.gif" -> "map.mono.gif" with "dither-2"

   to allow a proxy which has cached "map.gif" to create "map.mono.gif"
   on demand, by applying a particular standardized transformation to
   the "map.gif" data.  Another interesting possibility would be

     Allow-Transform: "map.gif" -> "map.mono.gif"
                    with "http://x.org/java/filters/my_dither"


3  Variant descriptions

   Variant descriptions have been informally introduced in Section 2.3.
   This section defines variant descriptions more formally.

   The version of BNF used in the rest of this document is taken from
   [1], and many of the nonterminals used here are also defined in [1].
   One new notation is added:

      1%rule

   stands for one or more instances of "rule", separated by
   whitespace:

      1%rule =  rule *( 1*LWS rule ) .


                                                              [Page 15]

Internet-Draft          Content Negotiation                 13 June 1996


3.1 Syntax

   A variant can be described in a machine-readable way with a variant
   description.

       variant-descr =
               "{" <"> URI <">
                   source-quality
                   [ "{" "type"  media-type "}" ]
                   [ "{" "language"  1#language-tag "}" ]
                   [ "{" "length" 1*DIGIT "}" ]
                   [ "{" "features" feature-list "}" ]
                   [ "{" "description" quoted-string "}" ]
                   [ extension-attribute ]
               "}"

       source-quality = qvalue

       extension-attribute = "{" extension-name extension-value "}"
       extension-name      = token
       extension-value     = *( token | quoted-string | LWS
                              | <any element of tspecials except "}"> )

   Examples are

      {"paper.html.fr" 0.7 {type text/html} {language fr}}

      {"paper.html.tables"  0.9 {type text/html} {features tables}}

   The various attributes which can be present in a variant
   description are covered in the subsections below.

   [##Issue to be resolved: earlier versions had quotes (<">) around
   the media-type and 1#language-tag.  Are these necessary for some
   reason unknown to me?##]


3.2 URI

   The URI attribute gives the URI of the un-negotiated resource from
   which the variant can be retrieved with a GET request. It can be
   absolute or relative to the Request-URI.


3.3 Source-quality

   The source-quality attribute gives the quality of the variant as a
   representation of the content of the negotiable URI when the
   variant is rendered with a perfect rendering engine on the best
   possible output medium.

   If it is less than 1, this attribute often expresses degradation
   caused by lossy conversion to a particular data format.  For example,
   a picture originally in JPEG form would have a lower source quality

                                                              [Page 16]

Internet-Draft          Content Negotiation                 13 June 1996

   when translated to the XBM format, and a much lower source quality
   when translated to an ASCII-art variant.  Note, however, that
   degradation is a function of the source -- an original piece of
   ASCII-art may degrade in quality if it is captured in JPEG form.

   Content providers should use the following table as a guide when
   assigning source quality values:

       1.000       no degradation
       0.999-0.900 no noticeable degradation
       0.899-0.700 noticeable, but acceptable degradation
       0.699-0.500 barely acceptable degradation
       0.499-0.000 unacceptable degradation

   [##Issue to be resolved: can we come up with a word other than
   `degradation' that covers better the case of variants not converted
   from one source?##]

   It is important that content providers do not assign very low source
   quality values without good reason, as this would limit the ability
   of users to influence the negotiation process with their own
   preference settings.

   When assigning source-quality values, content providers must not
   account for the size of the variant and its impact on transmission
   and rendering delays.  Source-quality values are assigned assuming
   instantaneous transmission and rendering.  This rule ensures that a
   future mechanism for bandwidth negotiation (Section 2.9.2) can be
   cleanly added.


3.4 Type and charset, language, length

   The type, language, and length attributes of a variant description
   refer to their Content-* header counterparts as defined in [1].  Note
   that the type attribute also holds the charset of the variant, if
   appropriate.  An example is

      {type text/html;charset=ISO-8859-4}

   Though all of these attributes are optional, it is often desirable to
   include as many attributes as possible as this will increase the
   quality of the negotiation process.

   The length attribute, if present, must reflect the length of the
   variant only, not the length of the variant plus the length of any
   objects inlined or included by the variant.


3.5 Features

   The features attribute specifies how the presence or absence of
   particular features in the user agent affects the overall quality of
   the variant.  This attribute is covered in Section 4.4.


                                                              [Page 17]

Internet-Draft          Content Negotiation                 13 June 1996


3.6 Description

   The description attribute is meant to provide a textual description
   of some properties of the variant, to be displayed by a user agent
   when showing a menu of available variants (Section 2.6).  This
   attribute can be included if the URI and normal attributes of a
   variant are considered too opaque to allow interpretation by the
   user.


3.7 Extension-attribute

   The extension-attribute allows future specifications to incrementally
   define new dimensions of negotiation (Section 2.7), and eases content
   negotiation experiments under HTTP/1.1.  In experimental situations,
   servers must only generate extension-attributes whose names start
   with "x-".  User agents should ignore all extension attributes they
   do not recognize.  Proxies must not run the network negotiation
   algorithm if an unknown attribute is present in the variant list.


4  Feature negotiation

   Feature negotiation has been informally introduced in Section 2.8.
   This section defines feature negotiation more formally.  Note that
   much of the material in this section is still in flux.  Some
   tradeoffs between complexity and expressive power have yet to be
   discussed.


4.1 Feature tags

   A feature tag (ftag) identifies a capability of a user agent or a
   preference of a user.

       ftag = 1*<any CHAR except CTLs or tspecials or "!">

       tspecials      = "(" | ")" | "<" | ">" | "@"
                      | "," | ";" | ":" | "\" | <">
                      | "/" | "[" | "]" | "?" | "="
                      | "{" | "}" | SP | HT

   Examples are

        ns_tables, fonts, blebber, wolx, screenwidth, colordepth

   Feature-tags are case-insensitive.  (Case-sensitivity would add a
   huge factor of potential confusion to a process already designed to
   be chaotic).

   The definition of a feature tag may state that a feature tag, if
   present, must be have associated with it a numeric value which
   reflects a particular capability or preference.  For example, a
   feature tag `screenwidth' could be present with a value of 640.

                                                              [Page 18]

Internet-Draft          Content Negotiation                 13 June 1996


4.2 Accept-Features header

   The Accept-Features request header can be used by a client to give
   information about the presence or absence of certain features.

       Accept-Features = "Accept-Features" :
                            1#( ftag
                              | "!" ftag
                              | ftag "=" number
                              | "*"
                              )

       number = 1*DIGIT

   An example is:

       Accept-Features: blex, !blebber, colordepth=5, *

   A feature tag must never occur twice in an Accept-Features header.
   In the Accept-Features header, an ftag without a "!"  indicates that
   the corresponding feature is present, but that is has no numeric
   value associated with it.  An ftag with a "!"  indicates that the
   corresponding feature is absent.  An ftag with a number indicates
   that the corresponding feature is present with the given numeric
   value.

   The special construct "*" is best interpreted in terms of its effect
   on the network negotiation algorithm calculations.  As an
   approximation, the "*" indicates that all features not named
   explicitly in the Accept-Features header are present, present with
   any numeric value, and absent, all at the same time.

   Absence of the Accept-Features header in a request is equivalent to
   the inclusion of

       Accept-Features: *


4.3 Feature predicates

   Feature predicates are used in the features attribute of a variant
   description.

      fpred = [ "!" ] ftag [ "=" number ]

   Examples feature predicates are

      blebber, !blebber, colordepth=5, !blex=54

   The network negotiation algorithm can compute the truth value of a
   feature predicate by using the contents of the Accept-Features
   header of the current request.


                                                              [Page 19]

Internet-Draft          Content Negotiation                 13 June 1996

   If the ftag in a feature predicate appears in the Accept-Features
   header, the truth value of the predicate is defined as follows,
   depending on its form:

       ftag    true if the feature is present according to the
               Accept-Features header, false otherwise.

      !ftag    true if the feature is absent according to the
               Accept-Features header, false otherwise.

       ftag=N  true if the feature is present with a value greater than
               or equal to N according to the Accept-Features header,
               false otherwise.

      !ftag=N  true if the feature is present with a value less than N
               according to the Accept-Features header, false otherwise.

   If the ftag in a feature predicate does not appear in the
   Accept-Features header, the value of the predicate is true if there
   is a "*" in the Accept-Features header, false otherwise.

   As an example, the header

       Accept-Features: blex, !blebber, colordepth=5, !screenwidth, *

   makes the following predicates true:

       blex, colordepth=4, !colordepth=6, colordepth, !screenwidth,
       frtnbf, !frtnbf, frtnbf=4, !frtnbf=4

   and makes the following predicates false:

       !blex, blex=0, blebber, colordepth=6, !colordepth,
       screenwidth, screenwidth=640, !screenwidth=640


4.4 Features attribute

   The features attribute

            "{" "features" feature-list "}"

   is used in a variant description to specify how the presence or
   absence of particular features in the user agent affects the overall
   quality of the variant.


                                                              [Page 20]

Internet-Draft          Content Negotiation                 13 June 1996

       feature-list = 1%feature-list-element

       feature-list-element = ( fpred | fpred-bag )
                              [ ":" present-improvement ]
                              [ "/" absent-degradation  ]

       fpred-bag = "[" 1%fpred "]"

       absent-degradation  =  1*3DIGIT [ "." 0*3DIGIT ]
       present-improvement =  1*3DIGIT [ "." 0*3DIGIT ]

   Examples are:

       {features !textonly [blebber !wolx] colordepth=3:0.7 }

       {features !blink/0.5 background:1.5  [blebber !wolx]:1.4/0.8 }

   The default value for the present-improvement is 1.  The default
   value for the absent-degradation is 0, or 1 if a present-improvement
   value is given.

   The network negotiation algorithm can compute the quality degradation
   factor associated with the features attribute by multiplying all
   quality degradation factors of the elements of the feature-list.
   Note that the result can be a factor greater than 1.

   A feature list element yields its present-improvement value if the
   corresponding feature predicate is true, or if at least one element
   of the corresponding fpred-bag is true. The element yields its
   absent-degradation value otherwise.

   [##Issue to be resolved: It is unknown yet whether this features
   attribute definition makes the right tradeoff between complexity and
   (ease of) expressive power.  The attribute grammar above is designed
   to be parsable with simple non-recursive parsers.  The
   present-improvement construct does not add expressive power in a
   theoretical sense, but does make the (automatic) construction of
   variant lists more straightforward in many cases.##]

   [##Aside for mathematicians: note the similarity between the
   feature-list

            [f1 !f2] [!f1 f3]

   and the conjunctive normal form of a boolean expression.  This
   similarity implies good things about the expressive power of the
   features attribute.  I have not yet explored how far this power
   extends into the non-boolean domain.##]


4.5 Example of negotiation in a numeric area

   As an example of negotiation in a numeric area, the following variant
   list describes four variants with title graphics designed for
   increasing screen widths:

                                                              [Page 21]

Internet-Draft          Content Negotiation                 13 June 1996


     {"home.pda" 1.0 {type text/html} {features !screenwidth=200} },
     {"home.narrow" 1.0 {type text/html}
                  {features screenwidth=200 !screenwidth=600} },
     {"home.normal" 1.0 {type text/html}
                  {features screenwidth=600 !screenwidth=1000} },
     {"home.wide"   1.0 {type text/html} {features screenwidth=1000} }

   The above variant list has the disadvantage that a user agent not
   supporting screenwidth negotiation is given no guidance on which
   variant to select: all would have an overall quality value of 0 if
   the screenwidth feature is not present.  A feature list which solves
   this problem is

     {"home.pda"    1.0 {type text/html} {features !screenwidth=200} },
     {"home.narrow" 1.0 {type text/html}
                  {features screenwidth=200 !screenwidth=600} },
     {"home.normal" 0.95 {type text/html} },
     {"home.wide"   0.1 {type text/html} {features screenwidth=1000} }

   with this list, a user agent which does not support the screenwidth
   feature will always select the third variant.  A user agent which
   does support the screenwidth feature will only select the third
   variant if the overall qualities of all other variants are 0.


4.6 Content-Features header

   The Content-Features response header can be used by a server to
   indicate how the presence or absence of particular features in the
   user agent affects the overall quality of the response.

       Content-Features = "Content-Features" : feature-list


5  Security and privacy considerations

5.1 Accept headers revealing information of a private nature

   Accept headers, in particular Accept-Language headers, may reveal
   information that the user rather keeps private unless it will
   directly improve the quality of service.  For example, a user may not
   want to send language preferences to sites which do not offer
   multilingual content.  The transparent content negotiation mechanism
   allows user agents to omit sending of the Accept-Language header by
   default, without this affecting the outcome of the negotiation
   process if transparently negotiated multilingual content is accessed.

   However, even if the Accept headers never send any information other
   than that the user agent is capable of transparent negotiation,
   automatic selection and retrieval of a variant by a user agent will
   reveal a preference for this variant to the server.  A malicious
   service author could provide a page with `fake' negotiability on
   (ethnicity-correlated) languages, with all variants actually being
   the same English document, as a means of extracting privacy-sensitive

                                                              [Page 22]

Internet-Draft          Content Negotiation                 13 June 1996

   information.  Such a plot would however be visible to an alert victim
   if the menu of available variants and their properties made available
   by the user agent is reviewed.

   Some additional privacy considerations connected to Accept headers
   are discussed in [1].


5.2  Spoofing of responses from variant URLs

   In the second optimization of the transparent negotiation process
   outlined in Section 2.4, a request on a transparently negotiable URL
   http://x.org/paper can lead to a response which must be interpreted
   as belonging to a variant URL http://x.org/paper.html.en.  According
   to current plans, a proxy cache will be allowed to return this
   response, after removal of the list of variants and the
   Content-Location header, if a direct request on the variant URL
   http://x.org/paper.html.en is received.  This caching shortcut could,
   for some negotiable resources, save up to a factor 2 in bandwidth
   usage for requests made by the cache.

   However, this shortcut also allows the author of the URL
   http://x.org/paper to control the data cached for the URL
   http://x.org/paper.html.en.  If the variant URL belongs to another
   author, this is a security problem.  Therefore, it is planned to
   restrict the use of the second optimization by origin servers to
   cases where the negotiable URL and the variant URL are equal up to
   the last slash.  If an origin server generates a response which does
   not satisfy this restriction on use, the recipient must reject it and
   signal a server error.


6  Acknowledgments

   This document builds on an earlier specification of content
   negotiation as recorded in [2], and directly incorporates text from
   [2] in some places.  Many members of the HTTP working group have
   contributed to the work reflected in this document.  The author
   wishes to thank the individuals who have commented on earlier
   versions of the transparent content negotiation specification,
   including Brian Behlendorf, Daniel DuBois, Larry Masinter, and Roy
   T. Fielding.


7  References

   [1]  Fielding et al, Hypertext Transfer Protocol -- HTTP/1.1.
        Internet-Draft draft-ietf-http-v11-spec-05.txt, HTTP Working
        Group, June, 1996.

   [2]  Roy T. Fielding, Henrik Frystyk Nielsen, and Tim Berners-Lee.
        Hypertext Transfer Protocol -- HTTP/1.1.  Internet-Draft
        draft-ietf-http-v11-spec-01.txt, HTTP Working Group, January,
        1996.


                                                              [Page 23]

Internet-Draft          Content Negotiation                 13 June 1996


8  Author's address

   Koen Holtman
   Technische Universiteit Eindhoven
   Postbus 513
   Kamer HG 6.57
   5600 MB Eindhoven (Holland)

   e-mail: koen@win.tue.nl


Expires: 13 December 1996


                                                              [Page 24]

