2004/id/draft-ietf-http-ext-mandatory-00.txt



       INTERNET-DRAFT                Mandatory       H. Frystyk Nielsen, W3C
       draft-ietf-http-ext-mandatory-00                   P. Leach, Microsoft
                                              Scott Lawrence, Agranat Systems
       Expires: August 13, 1998                        Friday, March 13, 1998

                            Mandatory Extensions in HTTP


       Status of this Document

       This document is an Internet-Draft. Internet-Drafts are working
       documents of the Internet Engineering Task Force (IETF), its areas,
       and its working groups. Note that other groups may also distribute
       working documents as Internet-Drafts. Internet-Drafts are draft
       documents valid for a maximum of six months and may be updated,
       replaced, or obsoleted by other documents at any time. It is
       inappropriate to use Internet-Drafts as reference material or to cite
       them other than as "work in progress".

       To learn the current status of any Internet-Draft, please check the
       "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow
       Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe),
       munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or
       ftp.isi.edu (US West Coast).

       Distribution of this document is unlimited. Please send comments to
       the <ietf-http-ext@w3.org> mailing list. This list is archived at
       "http://lists.w3.org/Archives/Public/ietf-http-ext/".

       The contribution of World Wide Web Consortium (W3C) staff is part of
       the W3C HTTP Activity (see "http://www.w3.org/Protocols/Activity").


       Abstract

       HTTP is used increasingly in applications that need more facilities
       than the standard version of the protocol provides, ranging from
       distributed authoring, collaboration, and printing, to various remote
       procedure call mechanisms. This document proposes the use of a
       mandatory extension mechanism designed to address the tension between
       private agreement and public specification and to accommodate
       extension of applications such as HTTP clients, servers, and proxies.
       The proposal associates each extension with a URI[2], and use a few
       new RFC 822[1] style header fields to carry the extension identifier
       and related information between the parties involved in an extended
       transaction.


       Table of Contents

       1. Introduction.....................................................2
       2. Notational Conventions...........................................2
       3. Extension Declarations...........................................3
         3.1 Header Field Prefixes.........................................4

       Frystyk et al                                                 [Page 1]


       INTERNET-DRAFT                Mandatory        Friday, March 13, 1998

       4. Extension Header Fields..........................................4
         4.1 End-to-End Extensions.........................................5
         4.2 Hop-by-Hop Extensions.........................................6
       5. Mandatory HTTP Requests..........................................6
       6. Mandatory HTTP Responses.........................................7
       7. 510 Not Extended.................................................7
       8. Publishing an Extension..........................................8
       9. Security Considerations..........................................8
       10. References......................................................9
       11. Acknowledgements................................................9
       12. Authors Addresses..............................................10
       13. Summary of Protocol Interactions...............................10
       14. Examples.......................................................11
         14.1 Client Queries Server for DAV...............................11
         14.2 Server Uses ZipFlate Compression Extension..................11

       1. Introduction

       The mandatory proposal is designed to accommodate dynamic extension of
       HTTP clients and servers by software components; and to address the
       tension between private agreement and public specification. The kind
       of extensions capable of being introduced range from:

          o extending a single HTTP message;
          o introducing new encodings;
          o initiating HTTP-derived protocols for new applications; to...
          o switching to protocols which, once initiated, run independent of
            the original protocol stack.

       The  proposal is intended to be used as follows:

          o Some party designs and specifies an extension; the party assigns
            the extension an identifier, which is a URI, and makes one or
            more representations of the extension available at that address
            (see section 8).
          o An HTTP client, server, or proxy that implements the Mandatory
            extension mechanism (hereafter called an agent) declares the use
            of the extension by referencing its URI in an extension
            declaration in an HTTP message (see section 3).
          o The ultimate recipient of the extension declaration which can be
            the origin server, the user agent, or any intermediary in the
            request/response chain can based on the extension declaration
            deduce how to properly interpret the extended message.

       The proposal uses features in HTTP/1.1 but is compatible with both
       HTTP/1.0 and HTTP/1.1 applications in such a way that extended
       applications can coexist with existing HTTP applications.


       2. Notational Conventions

       This specification uses the same notational conventions and basic
       parsing constructs as RFC 2068[7]. In particular the BNF constructs

       Frystyk, et al                                                [Page 2]


       INTERNET-DRAFT                Mandatory        Friday, March 13, 1998

       "token", "quoted-string", "field-name", and "URI" in this document are
       to be interpreted as described in RFC 2068[7].

       The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
       "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
       document are to be interpreted as described in RFC 2119[8].

       This proposal does not rely on particular features defined in URLs [3]
       that cannot potentially be expressed using URNs (see section 8).
       Therefore, the more generic term URI[2] is used throughout the
       specification.


       3. Extension Declarations

       An extension declaration can be used to indicate that an extension has
       been applied to a message and possibly to reserve a part of the header
       namespace identified by a header field prefix (see 3.1).

       This specification does not define any ramifications of applying an
       extension to a message nor whether two extensions can or cannot
       logically coexist within the same message. It is strictly a framework
       for describing which extensions have been applied and what the
       ultimate recipient either must or may do in order to properly
       interpret any extension declarations within that message.

       The grammar for an extension declaration is as follows:

           ext-decl        = <"> URI <"> [ ext-params ]
           ext-params      = ";" namespace *( ext-extension )

           namespace       = ";" "ns" "=" prefix
           prefix          = 2*DIGIT "-"
           ext-extension   = ";" token [ "=" ( token | quoted-string ) ]

       An extension is identified by a URI. Extension identifier URIs can be
       either relative or absolute. Relative extension identifiers are
       interpreted relative to the IANA registry (see RFC 1808[4]). Examples
       of extension declarations are

           "rfc2068"; SetCookie2
           "Content-FooBar"
           "New-Registered-Header"
           "http://www.temporary.com/extension"; ns=33-

       An extension declaration can be extended through the use of one or
       more ext-extension parameters. Unrecognized ext-extension parameters
       SHOULD be ignored and MUST NOT be removed by proxies when forwarding
       the extension declaration.


       Frystyk, et al                                                [Page 3]


       INTERNET-DRAFT                Mandatory        Friday, March 13, 1998

       Note: In layered implementations, unknown ext-extension parameters
       should be passed to the upper layers as they may have other mechanisms
       of knowing the semantics of the parameters.


       3.1 Header Field Prefixes

       The header-prefix can be used to indicate that all header fields in
       the message matching the header-prefix value using string prefix-
       matching are introduced by this extension instance. This allows an
       extension instance to dynamically reserve a subspace of the header
       space in a protocol message in order to prevent header field name
       clashes.

       Prefixes are primarily intended to avoid header field name conflicts
       and to allow multiple instances of a single extension using its own
       header fields to be applied to the same message without conflicting
       with each other. Alternatively, the extension declarations can contain
       the header field information as ext-params.

       Agents SHOULD NOT reuse header-prefix values in the same message
       unless explicitly allowed by the extension (see section 4.1 for a
       discussion of the ultimate recipient of an extension declaration).

       Examples of header-prefix values are

           1234-
           546-
           234345653-

       Linear white space (LWS) MUST NOT be used between the digits and the
       "-". The format of the prefix using a combination of digits and the
       dash "-" guarantees that no extension declaration can reserve the
       whole header field name space.

       Old applications may introduce header fields independent of this
       extension mechanism, potentially conflicting with header fields
       introduced by the prefix mechanism. In order to minimize this risk,
       prefixes MUST contain at least 2 digits.


       4. Extension Header Fields

       This proposal introduces two types of extension declaration strength:
       mandatory and optional, and two types of  extension declaration scope:
       hop-by-hop and end-to-end (see section 4.1 and 4.2).

       A mandatory extension declaration indicates that the ultimate
       recipient MUST consult and adhere to the rules given by the extension
       when processing the message or report an error (see section 5 and 7).


       Frystyk, et al                                                [Page 4]


       INTERNET-DRAFT                Mandatory        Friday, March 13, 1998

       An optional extension declaration indicates that the ultimate
       recipient of the extension MAY consult and adhere to the rules given
       by the extension when processing the message, or ignore the extension
       declaration completely. An agent may not be able to distinguish
       whether the ultimate recipient does not understand an extension
       referred to by an optional extension or simply ignores the extension
       declaration.

       The combination of the declaration strength and scope defines a 2x2
       matrix which is distinguished by four new general HTTP header fields:
       Man, Opt, C-Man, and C-Opt (see section 4.1 and 4.2, and appendix 13
       for a table of interactions with origin servers and proxies).

       The header fields are general header fields as they describe which
       extensions actually are applied to an  HTTP message. Optional
       declarations MAY be applied to any HTTP message without any change to
       existing HTTP semantics. Mandatory declarations MUST be applied to a
       request message as described in section 5 and to a response message as
       described in section 6.


       4.1 End-to-End Extensions

       End-to-end declarations MUST be transmitted to the ultimate recipient
       of the declaration. The Man and the Opt general header fields are end-
       to-end header fields and are defined as follows:

           mandatory       = "Man" ":" 1#ext-decl
           optional        = "Opt" ":" 1#ext-decl

       For example

           HTTP/1.1 200 OK
           Content-Length: 421
           Opt: "http://www.digest.org/Digest"; digest="4525dct344v@fsdfsg"
           ...

       The ultimate recipient of an end-to-end extension declaration would
       often be but is not limited to either the origin server or the user
       agent. Proxies MAY act as both the initiator and the ultimate
       recipient of end-to-end extension declarations. It is outside the
       scope of this specification to define how an agreement is reached
       between a party representing the proxy and the party on which behalf
       it can act, but for example, the parties may be within the same trust
       domain.

       If a proxy is the ultimate recipient of a mandatory end-to-end
       extension declaration then it MUST handle that extension declaration
       as described in section 5. The proxy SHOULD remove all parts of the
       extension declaration from the message before forwarding it.


       Frystyk, et al                                                [Page 5]


       INTERNET-DRAFT                Mandatory        Friday, March 13, 1998

       4.2 Hop-by-Hop Extensions

       Hop-by-hop extension declarations are meaningful only for a single
       transport-level connection. The C-Man and the C-Opt general header
       field are hop-by-hop header fields and MUST NOT be communicated by
       proxies over further connections. The two headers have the following
       grammar:

           c-mandatory     = "C-Man" ":" 1#ext-decl
           c-optional      = "C-Opt" ":" 1#ext-decl

       For example

           GET / HTTP/1.1
           Host: some.host
           C-Man: "http://www.digest.org/ProxyAuth";
       Credentials="g5gj262jdw@4df"
           Connection: C-Man

       In HTTP/1.1, the C-Man and the C-Opt header field MUST be protected by
       a Connection header. That is, the header fields are to be included  as
       Connection header directives (see section [7], section 14.10).

       An agent MUST NOT send the C-Man or the C-Opt header field to an
       HTTP/1.0 proxy as it does not obey the HTTP/1.1 rules for parsing the
       Connection header field (see [7]).


       5. Mandatory HTTP Requests

       An HTTP request is called a mandatory request if it includes at least
       one mandatory extension declaration (using the Man or the C-Man header
       fields). The method name of a mandatory request MUST be prefixed by
       "M-". For example, a client might express the binding rights-
       management constraints in an HTTP PUT request as follows:

           M-PUT /a-resource HTTP/1.1
           Man: "http://www.copyright.org/rights-management"; ns=43-
           43-copyright: http://www.copyright.org/COPYRIGHT.html-
           43-contributions: http://www.copyright.org/PATCHES.html
           Host: www.w3.org
           Content-Length: 1203
           Content-Type: text/html

           <!doctype html ...

       An HTTP server MUST NOT return a 2xx status-code without understanding
       and obeying all mandatory extension declaration(s) in a mandatory
       request. A mandatory HTTP request invalidates cached entries as
       described in [7], section 13.10.

       The ultimate recipient of a mandatory HTTP request with the "M-"
       prefix on the method name MUST process the request by performing the
       following actions in the order they occur:

       Frystyk, et al                                                [Page 6]


       INTERNET-DRAFT                Mandatory        Friday, March 13, 1998

          1.  Identify all mandatory extension declarations (both hop-by-hop
              and end-to-end); the server MAY ignore optional declarations
              without affecting the result of the transaction;
          2.  Evaluate and process the extensions identified in 1) or if the
              extension declarations do not match the policy for accessing
              the resource then respond with a 510 (Not Extended) status-code
              (see section 7);
          3.  Strip the "M-" prefix from the method name and process the
              reminder of the request according to the semantics of the
              existing HTTP/1.1 method name as defined in [7].

       An "M-" aware proxy  that does not act as the ultimate recipient of a
       mandatory extension declaration MUST NOT remove the declaration or the
       "M-" method name prefix when forwarding the message.

       An agent receiving an HTTP/1.0 (or lower-version) message that
       includes a Connection header MUST, for each connection-token in this
       field, remove and ignore any header field(s) from the message with the
       same name as the connection-token. Any "M-" method name prefix
       introduced as a result of  discarded hop-by-hop extensions MUST be
       ignored and removed by a proxy when forwarding the message.

       HTTP proxies that do not understand the "M-" method name prefix SHOULD
       return 501 (Not Implemented) or turn themselves into a tunnel ([7]) in
       which case they do not take any part in the communication.

       The "M-" prefix is reserved by this proposal and MUST NOT be used by
       other HTTP extensions.


       6. Mandatory HTTP Responses

       Mandatory extension declarations in HTTP responses are often a result
       of but not limited to mandatory HTTP requests. A server MAY use
       mandatory extension declarations in response to a non-mandatory HTTP
       request. This is primarily intended to facilitate extensions that can
       not be understood without using the extension or are authorized out-
       of-band.

       If a client receives an HTTP response which contains a Mandatory
       extension declaration which it does not understand or does not want to
       use, it SHOULD treat it as if the message was of type
       "application/octet-stream".


       7. 510 Not Extended

       The policy for accessing the resource has not been met in the request.
       The server SHOULD send back all the information necessary for the
       client to issue an extended request. It is outside the scope of this
       specification to specify how the extensions inform the client.


       Frystyk, et al                                                [Page 7]


       INTERNET-DRAFT                Mandatory        Friday, March 13, 1998

       If the initial request already included the extensions requested in
       the 510 response, then the response indicates that access has been
       refused for those extension declarations.

       If the 510 response contains the same set of extension policies as the
       prior response, then the client MAY present any entity included in the
       response to the user, since that entity may include relevant
       diagnostic information.


       8. Publishing an Extension

       While the protocol extension definition should be published at the
       address of the extension identifier, this is not a requirement of this
       specification. The only absolute requirement is that distinct names be
       used for distinct semantics. For example, one way to achieve this is
       to use a mid, cid, or uuid URI. The association between the extension
       identifier and the specification might be made by distributing a
       specification, which references the extension identifier.

       It is strongly recommended that the integrity and persistence of the
       extension identifier is maintained and kept unquestioned throughout
       the lifetime of the extension. Care should be taken not to distribute
       conflicting specifications that reference the same name. Even when a
       URI is used to publish extension specifications, care must be taken
       that the specification made available at that address does not change
       significantly over time. One agent may associate the identifier with
       the old semantics, and another might associate it with the new
       semantics.

       The extension definition may be made available in different
       representations ranging from

          o a human-readable specification defining the extension semantics,
          o downloadable code which implements the semantics defined by the
            extension,
          o a formal interface description provided by the extension, to
          o a machine-readable specification defining the extension
            semantics.

       For example, a software component that implements the specification
       may reside at the same address as a human-readable specification
       (distinguished by content negotiation). The human-readable
       representation serves to document the extension and encourage
       deployment, while the software component allows clients and servers to
       be dynamically extended.


       9. Security Considerations

          o Dynamic installation of extension facilities as described in the
            introduction involves software written by one party (the provider
            of the implementation) to be executed under the authority of

       Frystyk, et al                                                [Page 8]


       INTERNET-DRAFT                Mandatory        Friday, March 13, 1998

            another (the party operating the host software). This opens the
            host party to a variety of "Trojan horse" attacks by the
            provider, or a malicious third party that forges implementations
            under a provider's name. See, for example RFC2046[6], section
            4.5.2 for a discussion of these risks.

       10. References

       [1]  D. H. Crocker. "Standard for the Format of ARPA Internet Text
            Messages", STD 11, RFC 822, UDEL, August 1982
       [2]  T. Berners-Lee, "Universal Resource Identifiers in WWW. A
            Unifying Syntax for the Expression of Names and Addresses of
            Objects on the Network as used in the World-Wide Web", RFC 1630,
            CERN, June 1994.
       [3]  T. Berners-Lee, L. Masinter, M. McCahill. "Uniform Resource
            Locators (URL)" RFC 1738, CERN, Xerox PARC, University of
            Minnesota, December 1994.
       [4]  R. Fielding, "Relative Uniform Resource Locators", RFC 1808, UC
            Irvine, June 1995.
       [5]  T. Berners-Lee, R. Fielding, H. Frystyk, "Hypertext Transfer
            Protocol -- HTTP/1.0", RFC 1945, W3C/MIT, UC Irvine, W3C/MIT, May
            1996.
       [6]  N. Freed, N. Borenstein, "Multipurpose Internet Mail Extensions
            (MIME) Part Two: Media Types", RFC 2046, Innosoft, First Virtual,
            November 1996.
       [7]  R. Fielding, J. Gettys, J. C. Mogul, H. Frystyk, T. Berners-Lee,
            "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2068, U.C. Irvine,
            DEC W3C/MIT, DEC, W3C/MIT, W3C/MIT, January 1997
       [8]  S. Bradner, "Key words for use in RFCs to Indicate Requirement
            Levels", RFC 2119, Harvard University, March 1997
       [9]  Y. Goland et al, "Extensions for Distributed Authoring and
            Versioning", Internet Draft, draft-jensen-webdav-ext-01, 26 March
            1997. This is work in progress.
       [10] H. F. Nielsen, D. Connolly, R. Khare, "PEP - an extension
            mechanism for HTTP", draft-http-pep-05.txt, November 21, 1997

       11. Acknowledgements

       Rohit Khare deserves special recognition for his efforts in commenting
       in the design phase of the protocol. Also thanks to Josh Cohen, Ross
       Patterson, Jim Gettys and all the people who have been involved in
       PEP.


       Frystyk, et al                                                [Page 9]


       INTERNET-DRAFT                Mandatory        Friday, March 13, 1998

       12. Authors Addresses

       Henrik Frystyk Nielsen
       Technical Staff, World Wide Web Consortium
       MIT Laboratory for Computer Science
       545 Technology Square
       Cambridge, MA 02139, USA
       Email: frystyk@w3.org

       Paul J. Leach
       Microsoft Corporation
       1 Microsoft Way
       Redmond, WA 98052, USA
       Email: paulle@microsoft.com

       Scott Lawrence
       Agranat Systems, Inc.
       1345 Main Street
       Waltham, MA 02154, USA
       Email: lawrence@agranat.com


       Appendices


       13. Summary of Protocol Interactions

       The following tables summarize the outcome of strength and scope rules
       of the mandatory proposal of compliant and non-compliant HTTP proxies
       and origin servers. The summary is intended as a guide and index to
       the text, but is necessarily cryptic and incomplete. This summary
       should never be used or referenced separately from the complete
       specification.


                               Table 1: Origin Server

            Scope            Hop-by-hop               End-to-end

           Strength     Optional    Required    Optional    Required
                          (may)       (must)       (may)       (must)

        Mandatory    Standard    501 (Not    Standard    501 (Not
        unsupported  processing  Implemented)processing  Implemented)

        Extension    Standard    510 (Not    Standard    510 (Not
        unsupported  processing  Extended)   processing  Extended)

        Extension    Extended    Extended    Extended    Extended
        supported    processing  processing  processing  processing


       Frystyk, et al                                               [Page 10]


       INTERNET-DRAFT                Mandatory        Friday, March 13, 1998

                                Table 2: Proxy Server

            Scope            Hop-by-hop               End-to-end

           Strength     Optional    Required    Optional    Required
                          (may)       (must)       (may)       (must)

        Mandatory    Strip       501 (Not    Forward     501 (Not
        unsupported  extension   Implemented)extension   Implemented)
                                   or tunnel                or tunnel

        Extension    Strip       510 (Not    Forward     Forward
        unsupported  extension   Extended)   extension   extension

        Extension    Extended    Extended    Extended    Extended
        supported    processing  processing  processing, processing,
                      and strip   and strip   may strip   may strip


       14. Examples

       The following examples show various scenarios using mandatory in
       HTTP/1.1 requests and responses. Information not essential for
       illustrating the examples is left out (referred to as "�")


       14.1 Client Queries Server for DAV

       In this example, the purpose is to determine whether a server
       understands and supports the Distributed Authoring and Versioning
       (DAV) protocol extension [9]. By making the request mandatory (see
       section 5), the client forces the server to process the extension
       declaration and obey the extension or report an error.

           M-GET /some.url HTTP/1.1
           Host: some.host
           Man: "http://www.dav.org"
           ...

           HTTP/1.1 200 OK
           ...

       The response shows that the server does understand. It is not possible
       to distinguish between querying about or using an extension - the
       extension declaration is identical. Whether it in fact is a query may
       depend on the request method name and request modifiers.


       14.2 Server Uses ZipFlate Compression Extension

       This example shows a server using the zipflate compression extension
       in a response:


       Frystyk, et al                                               [Page 11]


       INTERNET-DRAFT                Mandatory        Friday, March 13, 1998

           GET /Index HTTP/1.1
           Host: some.host

           HTTP/1.1 200 OK
           Man: "http://www.encoding.com/zipflate"
           Cache-Control: no-transform
           Vary: *
           ...

       The response shows that the server uses the extension the response.
       The response includes the no-transform cache-control directive in
       order to avoid that proxies add their own content-coding to the
       message and a Vary header field indicating that a cache may not use
       the response to reply to a subsequent request without revalidation.


       Frystyk, et al                                               [Page 12]