HTTP Working Group D. Connolly Internet Draft W3 Consortium March 1997 Expire in six months PEP: an Extension Mechanism for HTTP Status of this Document This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress". To learn the current status of any Internet-Draft, please check the "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). This document is also available as a W3C Working Draft. The most recent release is available at http://www.w3.org/pub/WWW/TR/WD-http- pep. Distribution of this document is unlimited. Please send comments to the HTTP working group at http-wg@cuckoo.hpl.hp.com. Discussions of the working group are archived at http://www.ics.uci.edu/pub/ietf/http/. The editor maintains background information about PEP at http://www.w3.org/pub/WWW/Protocols/PEP/ The contribution of W3 Consortium staff time to the HTTP working group is part of the W3C HTTP Activity. Abstract HTTP is an extensible protocol. PEP is an extension mechanism designed to address the tension between private agreement and public specification and to accommodate extension of HTTP clients and servers by software components. The PEP mechanism is to associate each extension with a URI, and use a few new header fields to carry the extension identifier and related information from HTTP clients, thru proxies and intermediaries, to servers, and back again. Connolly W3C Working Draft [Page 1] WD-pep PEP $Date: 1997/03/21 20:59:19 $ PEP relies on some HTTP 1.1 features, but is intended to be compatible with all versions of HTTP from 1.1 on, and to be compatible with HTTP/1.0 inasmuch as HTTP 1.1 is compatible with HTTP/1.0. The kinds of extension protocol capable of being introduced by PEP are envisioned as ranging from: * a simple qualification of a normal HTTP transaction; * changes to HTTP semantics; * HTTP-like protocols for new applications; * new HTTP methods; * new HTTP syntax, encoding or marshaling; * modification of replacement of lower layers such as TCP; to... * protocols which, once initatiated by a PEP transaction, runcompletely indepdently of HTTP, TCP and IP. Contents Introduction HTTP is a generic request-response protocol, designed to accommodate a variety of applications, from network information retrieval and searching to file transfer and repository access to query and forms processing. HTTP is used increasingly in applications that need more facilities than the standard version of the protocol provides, from distributed authoring, collaboration and printing, to various remote procedure call mechanisms. Many of these applications do not require agreement across the whole Internet about the extended facilities; rather, it suffices: * That conforming HTTP peers supporting a particular protocol extension or feature should be able to employ this in real time with no prior agreement; * That it should be possible for one party having a capability for a new protocol to require that the the other party either understand and abide by the new Connolly W3C Working Draft [Page 2] WD-pep PEP $Date: 1997/03/21 20:59:19 $ protocol or abort the operation; * That the HTTP protocol as extended should still be able to work through proxies - especially caching proxies; * That negotiation of matching capabilities should be possible. This document defines PEP, an extension mechanism for HTTP. The PEP design is the result of analyzing a variety of HTTP extensions and extension mechanisms, and the motivation behind them. PEP relies on some HTTP 1.1 features, but is intended to be compatible with all versions of HTTP from 1.1 on, and to be compatible with HTTP/1.0 inasmuch as HTTP 1.1 is compatible with HTTP/1.0. See section Considerations for Defining Extensions. Operational Overview PEP is intended to be used as follows: * Some party designs and specifies an extension to HTTP; the party assigns the extension an identifier which is a URI, and they make the specification of the protocol available at that address. * Extended clients and servers are implemented per the HTTP specification as extended by the extension specification. * Requests and responses declare the use of the extension by reference to its URI. * If the extension becomes ubiquitous, a new version of the HTTP specification can include the extension, and messages can declare use of the new HTTP version instead of the extension. Editor's note: I have used the term URI throughout this specification, since PEP relies only on the identification of resources, not on location nor resolution. URNs are expected to serve just as well as URLs for use as PEP extension identifiers. Hence, I cite the URL syntax draft in progress. Note that, at the cost of some extra bytes to spell out the URI in Connolly W3C Working Draft [Page 3] WD-pep PEP $Date: 1997/03/21 20:59:19 $ full, the use of a central registry of extension names is avoided. See Considerations for Defining Extensions for more on defining extensions. The PEP mechanism is designed to accommodate extension of clients, servers, and proxies by software components as follows: * Clients and servers are implemented with software component interfaces that allow dynamic installation of extension facilities. * An extension is assigned a URI; in addition to a human- readable specification of an extension, a machine-readable implementation or description of the extension is published at that address. * If a message that refers to an extension is received by a party that has no awareness of the extension, the receiver can dereference the extension's identifier and dynamically load support for the extended facility. Strength, Scope and Semantics of Extended Transactions The agents in an HTTP transaction are a client and a server, which send HTTP messages to each other, with intermediaries between them in some cases. However, semantically, an HTTP transaction is between a client party (for example, the referent of the From: header field) and a the principle responsible for the publication of a given resource. The publishing party is basically the one responsible for the service provided at any particular URI, for example, the mapping between the URI and any representation of the resource to which it refers. Exactly who takes this role is beyond the scope this document, but for example, it may be the writer of a document, the server administrator, the organization running the server, or a combination of these. PEP extensions MAY be used to extend the end-to-end transaction semantics, or, using the Connection header field (see [HTTP] section 14.10 Connection), they MAY be used to extend the hop-by- hop transaction semantics. See The Protocol Header Field and Hop- by-hop Extensions for details. A PEP extension declaration has a strength which MAY be either required or optional. If it is optional, the transaction may succeed even if some of the parties within its scope do not Connolly W3C Working Draft [Page 4] WD-pep PEP $Date: 1997/03/21 20:59:19 $ participate. If it is reuired, it MUST be understood and responded to by all parties within its scope. See The Protocol Header Field and Binding Request for details. The Protocol Header Field The extensions used in a message are declared using the Protocol request/response header field. The syntax is: Protocol = "Protocol" ":" 1#extension-decl | "C-Protocol" ":" 1#extension-decl extension-decl = "{" extension-id 1*LWS *ext-info "}" extension-id = URI ext-info = str | enc | params params = "{" "params" *bagitem "}" str = "{" "str" ("req" | "opt" ) "}" enc = "enc" bag = "{" bagname *bagitem "}" bagname = token ¦ URI bagitem = bag ¦ token ¦ quoted-string URI = 1* Each extension-decl: * MUST contain an extension-id, which is a URI [URLSYN] for an extension specification. * MAY contain a params bag of items whose semantics are specified by the extension specification. * MAY signal contain an enc token, which signals that the extension is an encoding. See Extension Encodings. * MAY specify a strength of optional ({str opt}) or required ({str req}). The strength defaults to optional if it is unspecified. See Strength Note that, since URIs may contain { and } characters, a space is required after the extension-id. For example: Connolly W3C Working Draft [Page 5] WD-pep PEP $Date: 1997/03/21 20:59:19 $ GET /a-document HTTP/1.1 Host: a.host Protocol: {http://some.org/an-extension } HTTP/1.1 200 OK Protocol: {http://some.org/an-extension } Vary: Protocol Content-Type: text/plain Glad you're using an-extension! Note the use of the Vary header to notify proxies that responses to GET /a-document depend on the Protocol header fields used in the request. See [HTTP], section 14.43 Vary. Strength Each extension-decl declares the use of the extension to be one of: Required: {str req} Ignorance of the extension or failure to process the extension declaration by the relevant agent(s) is an error that results in failure of the entire transaction, and SHOULD be reported using the 420 Bad Extension status code. See also: Binding Request. Optional: {str opt} The success of the transaction does not depend on any agents recognizing or processing the extension declaration. This is the default. If any of the extension declarations in a request is required, then the request is a binding request. See: Binding Request. The issue of "relevant agents" is defined in section Hop-by-hop Extensions and summarized in section Summary of Protocol Interactions. Hop-by-hop Extensions Extensions declared with the Protocol header field are end-to-end extensions. Hop-by-hop extensions are declared with the C- Protocol header field, in conjunction with the Connection header ([HTTP], section 13.5.1 and 14.10). The relevant agents in a hop-by-hop extension are the agents at the ends of the connection. The relevant agents in an end-to-end extension are the origin client and server, and depending on the extension in question any intermediaries acting on behalf of the origin client and server in Connolly W3C Working Draft [Page 6] WD-pep PEP $Date: 1997/03/21 20:59:19 $ ccordance with the extension specification and any private agreements. Binding Request A request with {str req} in any of its Protocol header fields is a binding request -- the transaction cannot be succeed without consulting and adhering to the relevant extension specification(s). Because legacy HTTP agents MAY ignore all protocol header fields, the {str req} is not sufficient to evoke the correct behaviour from HTTP agents. The method name of all binding request MUST be prefixed by BINDING-. Legacy HTTP agents (i.e. agents implemented without consulting this specification) SHOULD respond with 501 (Not Implemented) (see [HTTP] section 5.1.1, Method). Other agents MUST process the request resulting from removing the BINDING- from the method name and leaving the rest of the request (request URI, version, header fields, body) as is. NOTE: All method names beginning with BINDING- are reserved for this use. For example, a client might express the binding rights-management constraints on its request as follows: BINDING-PUT /a-resource HTTP/1.2 Protocol: {http://some.org/rights-management {str req} {params {copyright-remains-with-client} {nonexclusive-right-to-redistribute} } Host: some.hose Content-Length: 1203 Content-Type: text/html ... Hop-by-hop Policies The C-Protocol-Info header field provides hop-by-hop policies; that is, it allows a server to express policy(ies) to an agent at the other end of an HTTP connection, rather than to all parties in an HTTP transaction. Other than scope, its semantics are the same as the Protocol-Info header field; the name is distinct so that the Connection header field can distinguish between hop-by-hop and end-to-end protocol information notifications. For example, consider a server whos policy is to access cache usage statistics from clients that connect to it. In response from a client, it might advertise its policy as follows: Connolly W3C Working Draft [Page 11] WD-pep PEP $Date: 1997/03/21 20:59:19 $ HTTP/1.1 200 OK C-Protocol-Info: {http://some.org/provide-stats {for / * }} Connection: C-Protocol-Info Content-Type: text/plain some content The next time that client makes a request to this server, it may provide statistics as follows: GET /some-resource HTTP/1.1 Host: some.org C-Protocol: {http://some.org/provide-stats {params {hits 10}}} Connection: C-Protocol 420: Bad Extensions A server policy MAY require (or refuse) the use of some extensions in some circumstances. If a request fails to fulfill the policy, the server SHOULD respond with a 420 status code (Bad Extensions) and specify the policy using the Protocol-Info header field. Implementors may note the similarity to the way authentication challenges are issued with the 401 (Unauthorized) status code. Considerations for Defining Extensions While the protocol extension definition SHOULD be published at the address of the extension identifier, this is not strictly necessary. The only absolute requirement is that distinct names be used for distinct semantics. For example, one way to achieve this is to use an mid:, cid:, or uuid: URI. The association between the extension identifier and the specification might be made by distributing a specification which references the extension identifier. Care should be taken not to distribute conflicting specifications which reference the same name. Even when the web is used to publish extension specifications, care must be taken that the specification made available at that address does not change significantly over time. One agent may associate the identifier with the old semantics, and another might associate it with the new semantics. Interaction with Caching, Connections, etc. For each aspect of an extension, the interaction with other aspects of HTTP/1.1 SHOULD be fully specified, and the issues of compatibility SHOULD be discussed. For example, any extension headers which are not strictly entity headers require careful consideration. In particular: Connolly W3C Working Draft [Page 12] WD-pep PEP $Date: 1997/03/21 20:59:19 $ Cache-Control, [HTTP] section 14.9 Any interactions with existing and extended cache-control directives MUST be specified [HTTP] 19.7 Compatibility with Previous Versions Interaction with HTTP 1.0 clients and servers SHOULD be clarified. Upgrade (14.41) Extensions which define transitions to new application- layer protocols SHOULD discuss interaction with the Upgrade header, at least to mention that PEP is being used instead of the Upgrade header. Content-Encoding (14.12) Extension encodings SHOULD mention the prohibition against mixing Content-Encoding with extension encodings, among other interactions. Bootstrapping and Dynamic Loading The extension definition MAY be made available in different representations. For example, a software component that implements the specification MAY reside at the same address as a human-readable specification (distinguished by content negotiation). The human-readable representation serves to document the extension and encourage deployment, while the software component to allows clients and servers to be dynamically extended. Security Considerations * The for parameter allows one party to give information about the extensions used by another party's resources. The parties may provide resources on different servers, or at different addresses on the same server. While distinguishing between the parties responsible for different resources at the same server may be infeasible, clients SHOULD ignore information given by one server about another unless they have reason to trust it, or reason to believe that trusting it will have no significant negative consequences. * Connolly W3C Working Draft [Page 13] WD-pep PEP $Date: 1997/03/21 20:59:19 $ Dynamic installation of extension facilities as described in the introduction involves software written by one party (the provider of the implementation) to be executed under the authority of another (the party operating the host software). This opens the host party to a variety of "trojan horse" attacks by the provider, or a malicious third party that forges implementations under a provider's name. See, for example, section 7.4.2 of RFC1521 for a discussion of these risks Future Work The design of some aspects of earlier drafts of this specification are still pending implementation experience. Multi-Transaction Negotiation An earlier draft of PEP included a mechanism for multi-transaction negotiation. Implementation experience showed the need to identify clients across transactions, which the mechanism did not provide. It is possible, within the design specified here, to do multi- transaction negotiation within an extension (for example, by putting information to disambiguate conversation threads in the params). Other possibilities under consideration include the use of state management "cookies" to disambiguate clients, or the use of an analogous PEP-specific mechanism. Appendix: Considerations for the Design of a PEP Software Component Interface This section got blown away in an editing disaster. If requested, the editor will attempt to include it in a future draft. Normative References [URLSYN] T. Berners-Lee, R. Fielding, L. Masinter, Uniform Resource Locators (URL), draft-fielding-url-syntax-03 , MIT/LCS, U.C. Irvine, Xerox Corporation, December 1996, work in progress [HTTP] R. Fielding, J. Gettys, J. C. Mogul, H. Frystyk, T. Berners- Lee, Hypertext Transfer Protocol -- HTTP/1.1. RFC 2068 U.C. Irvine, DEC, DEC, W3C/MIT, W3C/MIT, January 1997 Connolly W3C Working Draft [Page 14] WD-pep PEP $Date: 1997/03/21 20:59:19 $ [BRADNER] @@may/must/should RFC Bibliography: Informative References [CGI] D. Robinson The WWW Common Gateway Interface Version 1.1, work in progress 15 February 1996 [NSAPI] Netscape server API documentation, 1995 [ISAPI] ISAPI documentation, Microsoft Corporation, in ActiveX Alpha SDK, http://www.msn.com/download/sdk/msactivedk.zip, 1996 [Apache] Thau, Robert, Design considerations for the Apache Server API, Fifth International World Wide Web Conference, May 6-10, 1996, Paris, France [OM] OpenMarket server technical overview sometime in 1996. [Spy95] Spyglass Server Application Development Interface Spyglass, Inc. version 1.17 1995/09/11 [MAILCAP] N. Borenstein, RFC 1524: A User Agent Configuration Mechanism For Multimedia Mail Format Information, pp. 12, Sep 1993. [STATE] D. Kristol, L. Montulli, 22 Nov 1996. "HTTP State Management Mechanism", RFC xxxx. Proposed Standard Approved by the IESG, not yet assigned an RFC. [Kristol95] David M. Kristol, A Proposed Extension Mechanism for HTTP, Jan 1995. D. Kristol, A Proposed Extension Mechanism for HTTP, Internet Draft, January 1995 (Work in Progress, Expired). Connolly W3C Working Draft [Page 15] WD-pep PEP $Date: 1997/03/21 20:59:19 $ [RFC822] D. H. Crocker. Standard for the Format of ARPA Internet Text Messages. STD 11, RFC 822, UDEL, August 1982. [UPP] D. Eastlake, "Universal Payment Preamble", Internet Draft CyberCash, March 1996 (Work in Progress). [JEPI] JEPI, "Selecting Payment Mechanisms Over HTTP", Internet Draft, August 1996 (Work in Progress). [Also available as http://www.w3.org/pub/WWW/Payments/JEPI/draft-jepi- uppflow-00.html] [MAILEXT] J. Klensin, N. Freed, M. Rose, E. Stefferud, and D. Crocker. "SMTP Service Extensions." RFC 1869. MCI, Innosoft, Dover Beach Consulting, Network Management Associates, Brandenburg Consulting, November 1995. [PICS] J. Miller. PICS Label Syntax and Communication Protocols (Version 1.1). 31 October 1996 [SpyClient] Spyglass Client Software Development Kit [SpyEcom] Electronic Commerce Standards for the WWW [WN] WN server documentation, 1995 [Spinner] Spinner server technical overview, http://spinner.infovav.se/overview.html, 1995 Connolly W3C Working Draft [Page 16] WD-pep PEP $Date: 1997/03/21 20:59:19 $ Acknowledgements This draft of PEP is the product of a substantial amount of investigation and collaboration. Dave Kristol did some of the first writing on HTTP extension mechanisms. [Kristol95]. Jim Miller and Dave Raggett sketched out an initial design, which Rohit Khare wrote up in a number of drafts. Rohit also coined the term "PEP." This draft is a direct reflection of some implementation work: a client implementation Henrik Frystyk Nielsen et. al. (see the HTPEP module in libwww) and a server implementation by Eui Suk Chung and Anit Chakraborty for the JEPI project. Tim Berners-Lee contributed significantly to the requirements section, and Daniel Dardailler provided extensive review comments. Authors Addresses Dan Connolly Architecture Domain Lead, W3 Consortium MIT Laboratory for Computer Science 545 Technology Square Cambridge, MA 02139, U.S.A. Tel: +1 (512) 310 2971 Email: connolly@w3.org Rohit Khare Technical Staff, W3 Consortium MIT Laboratory for Computer Science 545 Technology Square Cambridge, MA 02139, U.S.A. Tel: +1 (617) 253 5884 Fax: +1 (617) 258 5999 Email: khare@w3.org Henrik Frystyk Nielsen Technical Staff, W3 Consortium MIT Laboratory for Computer Science 545 Technology Square Cambridge, MA 02139, U.S.A. Tel: +1 (617) 253 8143 Fax: +1 (617) 258 5999 Email: frystyk@w3.org Connolly W3C Working Draft [Page 17]