2004/id/draft-ietf-http-negotiate-scenario-01.txt

INTERNET-DRAFT                                              Edward Hardie
Expires: January 18, 1998                                        NASA NIC
<draft-ietf-http-negotiate-scenario-01.txt>                            


     Scenarios for the Delivery of Negotiated Content using HTTP


1.  Status of this Memo

This document is an Internet-Draft.  Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas, and
its working groups.  Note that other groups may also distribute working
documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time.  It is inappropriate to use Internet-Drafts as reference material
or to cite them other than as ``work in progress.''

To learn the current status of any Internet-Draft, please check the
"1id-abstracts.txt" listing contained in the Internet- Drafts Shadow
Directories on ds.internic.net (US East Coast), nic.nordu.net (Europe),
ftp.isi.edu (US West Coast), or munnari.oz.au (Pacific Rim).

Distribution of this memo is unlimited.  Please send comments to the
author.

2.  Abstract

This document describes various problems which have arisen in attempts
to deliver negotiated content using HTTP and examines several
scenarios by which improvements in delivery might be accomplished.

This document does not discuss how HTTP might be used to negotiate the
use of other protocols to deliver content.

3. Problems with content negotiation in HTTP


Content providers on the World Wide Web would like to present their
material in the form most appropriate to each individual user.  In
order to do this effectively, the content provider and content user
must be able negotiate among variants when more than one
representation of a resource is available.  This negotiation entails
the exchange of information between the provider and user about their
respective capabilities and preferences.

The Hypertext Transport Protocol has since version 1.0 [1] used
certain request headers to indicate a list of the responses preferred
by the user agent making the request.  The Accept request header is
used to indicate media types, Accept-Charset a list of preferred
character sets, Accept-Encoding supported content encodings, and
Accept-Language the set of natural languages preferred.  In theory, a
server receiving these headers can use them to determine which of the
available variants of a particular item is most appropriate to the
requesting user agent.

Problems have emerged with this approach.  First, the explosion of
content types in use on the World Wide Web has meant that Accept
headers would have to be extraordinarily long in order to adequately
state the user's Accept preferences.  Increasing the length of the
Accept headers in order to negotiate consumes bandwidth for every
request, whether the resource has variants or not.  Increasing their
complexity in order to account for all of the possible axes of
variation can also consume excessive amounts of server processing
power.  Second, even if using a full list of Accept preferences did
not present resource problems, it could create privacy problems.  A
full list of Accept preferences creates a user profile; in some
situations it could be combined with other information to track a
user's interaction with a server or set of servers even if the user
has taken steps to eliminate that possiblity.

Many content providers have attempted to circumvent the problems
associated with negotiation based on Accept headers.  The most popular
method has been to examine the User-Agent header and to create content
tailored to the typical capabilities of that user agent.  Because many
web user agents are both configurable and extensible, this method has
serious drawbacks.  A user agent capable of using graphics but
configured to display only text may be presented with completely
unusable content.  Conversely, a user agent which has been extended to
handle new media types may never be presented with them.  The
usefulness of this method has been further degraded by some client
authors, who have chosen to use the User-Agent strings associated
with other clients in order to indicate compatibility; commonly, this
has been done even when the two clients are not, in fact, fully
compatible.

Content negotiation based on Accept headers or User-agent strings also
creates difficulties for caching proxies, as they have no way to
determine when the form of content delivered to two different
user-agents would be the same.  The delivery of inappropriate content
can be prevented with the appropriate use of expiration times and
cache control directives, but only at the cost of serious losses in
cache efficiency.


4. Requirements for effective content negotiation

Content negotiation methods should improve the likelihood that the
best available resource variant is served to each user.  If no
available resource variant is appropriate for a particular user, they
should furnish a facility for the content provider to indicate that
lack.

Content negotiation methods should also accomplish their task without
substantially increasing the bandwidth or number of network round
trips required to deliver the negotiated variant over that required to
deliver similar invariant content.  Some increase must be expected,
but the smallest possible increase is preferred, and any overhead
should be limited to those resources for which variants are available.

As a rule of thumb, any method that increases bandwidth requirements
by more than 50% is unacceptable.  For example, a method that
delivered all available variants to a client and expected the client
to discard those that were not used might be very successful in
providing the best available resource and in limiting overhead to
those resources for which variants existed; it would, nonetheless, be
unacceptable for deployment on the global Internet because of the
wastage associated with each delivery.  

Similarly, any method which adds more than two network round trips to
the content delivery is unacceptable.  While some negotiation
exchanges might be appropriately conceived as a series of increasingly
detailed "questions" and "answers" about the respective capabilities
and preferences of the parties involved, the increase in latency such
extended exchanges would entail make them inappropriate for deployment
on the global Internet.

In addition, content negotiation methods should interact well with
caching proxies.  They must prevent the delivery of inappropriate
content to proxy users.  If possible, they should allow caches to
store sufficient information to participate in the negotiation
process.

Lastly, content negotiation methods used in this context will be
preferred if they are extensible to other MIME transport protocols
that have the need to negotiate over the same space.  Not all aspects
of negotiation needed for HTTP fit within the MIME context, however,
and content negotiation should not be limited to just those aspects
which do.


5. Scenarios

5.1 Improvements to the current system

Content negotiation based on Accept headers or User-Agent strings is
undertaken by the server based on information present in all requests
from clients.  The current problems with this method could be
ameliorated in a number of ways.  Registration of feature bundles
could, for example, allow a client to indicate its capabilities
without the increase in packet size that individual enumeration of
capabilities requires.  The use of registered feature bundles does,
however, require a fair amount of maintenance for both server and
clients, especially given their dynamic extendability.

Content providers could also improve the cachability of the variants
served by issuing redirects to unique URLs specific to each variant;
proxy users with equivalent feature bundles or user agents would then
be able to retrieve the cached variant once they received a redirect.
Some content providers already use this method to good effect.

5.2 Client-based negotiation

Improvements in the current method tend, however, to increase demands
on the processing power of the server, which raises serious problems
of scalability.  Shifting all or part of the processing involved in
the negotiation to the client may improve scalability and shifts the
locus of negotiation to the party most aware of both the current and
potential possibilities for presentation to the user.  

Client-based content negotiation can take two basic forms: elective
negotiation and "active content" negotiation.

5.2.1. Elective negotiation

Elective negotiation relies on the idea that the first response to a
request for a resource which has variants should be a statement naming
the choices and/or the axes along which they vary.  The simplest form
of elective negotiation occurs when a content provider creates a
meta-resource with links to several variants and explanations of the
features needed to present each variant.  This basic form presumes a
least common denominator for presentation of the links and an active
user selecting among the presentations.  More complex versions of
elective negotiation might use HTTP protocol elements to present the
axes of variation for a particular resource ("Transparent Content
Negotiation in HTTP" [2] presents one proposal along these lines).

Elective negotiation limits the process of selecting among variants to
the axes along which they vary.  This eliminates the need for clients'
enumerating their capabilities and user preferences.  It also provides
both a bandwidth savings and an increase in privacy. This process may,
however, require a user's intervention to select among the resources
provided, which reduces perceived efficiency and increases perceived
latency.

5.2.2 "Active content" negotiation

"Active content" negotiation takes place when the request for a
resource which has variants returns content for execution in the
client context rather than for presentation to the user.  This active
content determines the capabilities of the user agent and the
preferences of the user and then retrieves the best available variant
or provides an error status if no variants are appropriate.  This
method has several advantages: the negotiation takes place without the
need for user intervention; the processing for selection of variants
is moved completely off the server; and the active content itself
should be cachable.

There are, however, several limitations to active content negotiation.
The most severe problem is that there is not yet a cross-platform
standard for active content.  Without such a standard, servers may
have to engage in Accept header or User-Agent negotiation prior to
using active content in order to identify the correct form of active
content to send in response to a particular request.  Some user agents
will also be unable to use active content, either because they have
limited processing power or because of security considerations (see
Section 7).  For these user agents, a completely different method must
be provided.  As with server-side negotiation, each variant will
require a unique URL to ensure cache correctness.

6.  Summary

Content negotiation using HTTP currently presents problems in
accuracy, scalability, and privacy.  Amelioration of the problems in
the current design is possible, but other content negotiation methods
may provide significant improvements.  The type of content negotiation
most appropriate to a specific context will depend on the type of
variable content being served and the capabilities of the parties to
the negotiation.  Before deployment on the global Internet, however,
any client-based content negotiation method MUST meet the following
tests:

1) It MUST provide at least as good a content negotiation facility as
the current Accept header and User-Agent methods, and it SHOULD
increase the likelihood that the best available variant is served to
each client.

2) It MUST NOT significantly increase the bandwidth required to
deliver the content.

3) It MUST NOT significantly increase the number of network round
trips required to deliver the content.

4) It MUST NOT allow inappropriate content to be delivered to the
users of caching proxies.  It SHOULD reduce overall bandwidth
requirements by providing cachable content and allowing proxies to
participate in the negotiation process.

5) It SHOULD recognize HTTP's relationship to MIME and provide methods
which are extensible to other MIME transports when the space over
which negotiation takes place is the same.

Experimentation with various content negotiation methods may be
necessary to determine their usefulness for different types of
resources and contexts.  Interoperability will be best maintained,
however, if a single method of each type is made standard as soon as
experience permits.


7. Security Considerations

"Active content" negotiation presumes that clients will execute code
provided by servers and that those clients will allow that code to
examine user preferences and user agent capabilities in order to
select among variants.  Any such system would have to be tightly
bounded to prevent the active content from executing arbitrary code on
the client.  The examination of user preferences and user agent
capabilities also presents a possible privacy problem, especially as
the selection of a particular variant provides an opportunity for the
active content to communicate its discoveries.

8. Acknowledgements

Koen Holtman,  Graham Klyne, Scott Lawrence, and Larry Masinter
provided valuable feedback on the initial draft of this document.
Discussions within the HTTP working group and the IETF-FAX mailing
list also provided insights into the scope of negotiation needed in
the HTTP context.  

9. References
 
   [1] Tim Berners-Lee, Roy Fielding, Henrik Frystk.  Hypertext
   Transfer Protocol -- HTTP/1.0.  RFC RFC1945.  May 1996.

   [2] Koen Holtman and Andrew Mutz.  Transport Content Negotiation in
   HTTP.  Internet-Draft draft-ietf-http-negotiation-02.txt, HTTP Working
   Group, May 26, 1997.


9.  Author's Address

   Edward Hardie
   NASA Network Information Center
   MS 204-14
   Moffett Field, CA 94035-1000 USA
   +1 415 604  0134
   hardie@nasa.gov


1	wakaba	1.1	INTERNET-DRAFT Edward Hardie
2			Expires: January 18, 1998 NASA NIC
3			<draft-ietf-http-negotiate-scenario-01.txt>
4
5
6
7
8			Scenarios for the Delivery of Negotiated Content using HTTP
9
10
11
12			1. Status of this Memo
13
14			This document is an Internet-Draft. Internet-Drafts are working
15			documents of the Internet Engineering Task Force (IETF), its areas, and
16			its working groups. Note that other groups may also distribute working
17			documents as Internet-Drafts.
18
19			Internet-Drafts are draft documents valid for a maximum of six months
20			and may be updated, replaced, or obsoleted by other documents at any
21			time. It is inappropriate to use Internet-Drafts as reference material
22			or to cite them other than as ``work in progress.''
23
24			To learn the current status of any Internet-Draft, please check the
25			"1id-abstracts.txt" listing contained in the Internet- Drafts Shadow
26			Directories on ds.internic.net (US East Coast), nic.nordu.net (Europe),
27			ftp.isi.edu (US West Coast), or munnari.oz.au (Pacific Rim).
28
29			Distribution of this memo is unlimited. Please send comments to the
30			author.
31
32			2. Abstract
33
34			This document describes various problems which have arisen in attempts
35			to deliver negotiated content using HTTP and examines several
36			scenarios by which improvements in delivery might be accomplished.
37
38			This document does not discuss how HTTP might be used to negotiate the
39			use of other protocols to deliver content.
40
41			3. Problems with content negotiation in HTTP
42
43
44			Content providers on the World Wide Web would like to present their
45			material in the form most appropriate to each individual user. In
46			order to do this effectively, the content provider and content user
47			must be able negotiate among variants when more than one
48			representation of a resource is available. This negotiation entails
49			the exchange of information between the provider and user about their
50			respective capabilities and preferences.
51
52			The Hypertext Transport Protocol has since version 1.0 [1] used
53			certain request headers to indicate a list of the responses preferred
54			by the user agent making the request. The Accept request header is
55			used to indicate media types, Accept-Charset a list of preferred
56			character sets, Accept-Encoding supported content encodings, and
57			Accept-Language the set of natural languages preferred. In theory, a
58			server receiving these headers can use them to determine which of the
59			available variants of a particular item is most appropriate to the
60			requesting user agent.
61
62			Problems have emerged with this approach. First, the explosion of
63			content types in use on the World Wide Web has meant that Accept
64			headers would have to be extraordinarily long in order to adequately
65			state the user's Accept preferences. Increasing the length of the
66			Accept headers in order to negotiate consumes bandwidth for every
67			request, whether the resource has variants or not. Increasing their
68			complexity in order to account for all of the possible axes of
69			variation can also consume excessive amounts of server processing
70			power. Second, even if using a full list of Accept preferences did
71			not present resource problems, it could create privacy problems. A
72			full list of Accept preferences creates a user profile; in some
73			situations it could be combined with other information to track a
74			user's interaction with a server or set of servers even if the user
75			has taken steps to eliminate that possiblity.
76
77			Many content providers have attempted to circumvent the problems
78			associated with negotiation based on Accept headers. The most popular
79			method has been to examine the User-Agent header and to create content
80			tailored to the typical capabilities of that user agent. Because many
81			web user agents are both configurable and extensible, this method has
82			serious drawbacks. A user agent capable of using graphics but
83			configured to display only text may be presented with completely
84			unusable content. Conversely, a user agent which has been extended to
85			handle new media types may never be presented with them. The
86			usefulness of this method has been further degraded by some client
87			authors, who have chosen to use the User-Agent strings associated
88			with other clients in order to indicate compatibility; commonly, this
89			has been done even when the two clients are not, in fact, fully
90			compatible.
91
92			Content negotiation based on Accept headers or User-agent strings also
93			creates difficulties for caching proxies, as they have no way to
94			determine when the form of content delivered to two different
95			user-agents would be the same. The delivery of inappropriate content
96			can be prevented with the appropriate use of expiration times and
97			cache control directives, but only at the cost of serious losses in
98			cache efficiency.
99
100
101			4. Requirements for effective content negotiation
102
103			Content negotiation methods should improve the likelihood that the
104			best available resource variant is served to each user. If no
105			available resource variant is appropriate for a particular user, they
106			should furnish a facility for the content provider to indicate that
107			lack.
108
109			Content negotiation methods should also accomplish their task without
110			substantially increasing the bandwidth or number of network round
111			trips required to deliver the negotiated variant over that required to
112			deliver similar invariant content. Some increase must be expected,
113			but the smallest possible increase is preferred, and any overhead
114			should be limited to those resources for which variants are available.
115
116			As a rule of thumb, any method that increases bandwidth requirements
117			by more than 50% is unacceptable. For example, a method that
118			delivered all available variants to a client and expected the client
119			to discard those that were not used might be very successful in
120			providing the best available resource and in limiting overhead to
121			those resources for which variants existed; it would, nonetheless, be
122			unacceptable for deployment on the global Internet because of the
123			wastage associated with each delivery.
124
125			Similarly, any method which adds more than two network round trips to
126			the content delivery is unacceptable. While some negotiation
127			exchanges might be appropriately conceived as a series of increasingly
128			detailed "questions" and "answers" about the respective capabilities
129			and preferences of the parties involved, the increase in latency such
130			extended exchanges would entail make them inappropriate for deployment
131			on the global Internet.
132
133			In addition, content negotiation methods should interact well with
134			caching proxies. They must prevent the delivery of inappropriate
135			content to proxy users. If possible, they should allow caches to
136			store sufficient information to participate in the negotiation
137			process.
138
139			Lastly, content negotiation methods used in this context will be
140			preferred if they are extensible to other MIME transport protocols
141			that have the need to negotiate over the same space. Not all aspects
142			of negotiation needed for HTTP fit within the MIME context, however,
143			and content negotiation should not be limited to just those aspects
144			which do.
145
146
147			5. Scenarios
148
149			5.1 Improvements to the current system
150
151			Content negotiation based on Accept headers or User-Agent strings is
152			undertaken by the server based on information present in all requests
153			from clients. The current problems with this method could be
154			ameliorated in a number of ways. Registration of feature bundles
155			could, for example, allow a client to indicate its capabilities
156			without the increase in packet size that individual enumeration of
157			capabilities requires. The use of registered feature bundles does,
158			however, require a fair amount of maintenance for both server and
159			clients, especially given their dynamic extendability.
160
161			Content providers could also improve the cachability of the variants
162			served by issuing redirects to unique URLs specific to each variant;
163			proxy users with equivalent feature bundles or user agents would then
164			be able to retrieve the cached variant once they received a redirect.
165			Some content providers already use this method to good effect.
166
167			5.2 Client-based negotiation
168
169			Improvements in the current method tend, however, to increase demands
170			on the processing power of the server, which raises serious problems
171			of scalability. Shifting all or part of the processing involved in
172			the negotiation to the client may improve scalability and shifts the
173			locus of negotiation to the party most aware of both the current and
174			potential possibilities for presentation to the user.
175
176			Client-based content negotiation can take two basic forms: elective
177			negotiation and "active content" negotiation.
178
179			5.2.1. Elective negotiation
180
181			Elective negotiation relies on the idea that the first response to a
182			request for a resource which has variants should be a statement naming
183			the choices and/or the axes along which they vary. The simplest form
184			of elective negotiation occurs when a content provider creates a
185			meta-resource with links to several variants and explanations of the
186			features needed to present each variant. This basic form presumes a
187			least common denominator for presentation of the links and an active
188			user selecting among the presentations. More complex versions of
189			elective negotiation might use HTTP protocol elements to present the
190			axes of variation for a particular resource ("Transparent Content
191			Negotiation in HTTP" [2] presents one proposal along these lines).
192
193			Elective negotiation limits the process of selecting among variants to
194			the axes along which they vary. This eliminates the need for clients'
195			enumerating their capabilities and user preferences. It also provides
196			both a bandwidth savings and an increase in privacy. This process may,
197			however, require a user's intervention to select among the resources
198			provided, which reduces perceived efficiency and increases perceived
199			latency.
200
201			5.2.2 "Active content" negotiation
202
203			"Active content" negotiation takes place when the request for a
204			resource which has variants returns content for execution in the
205			client context rather than for presentation to the user. This active
206			content determines the capabilities of the user agent and the
207			preferences of the user and then retrieves the best available variant
208			or provides an error status if no variants are appropriate. This
209			method has several advantages: the negotiation takes place without the
210			need for user intervention; the processing for selection of variants
211			is moved completely off the server; and the active content itself
212			should be cachable.
213
214			There are, however, several limitations to active content negotiation.
215			The most severe problem is that there is not yet a cross-platform
216			standard for active content. Without such a standard, servers may
217			have to engage in Accept header or User-Agent negotiation prior to
218			using active content in order to identify the correct form of active
219			content to send in response to a particular request. Some user agents
220			will also be unable to use active content, either because they have
221			limited processing power or because of security considerations (see
222			Section 7). For these user agents, a completely different method must
223			be provided. As with server-side negotiation, each variant will
224			require a unique URL to ensure cache correctness.
225
226			6. Summary
227
228			Content negotiation using HTTP currently presents problems in
229			accuracy, scalability, and privacy. Amelioration of the problems in
230			the current design is possible, but other content negotiation methods
231			may provide significant improvements. The type of content negotiation
232			most appropriate to a specific context will depend on the type of
233			variable content being served and the capabilities of the parties to
234			the negotiation. Before deployment on the global Internet, however,
235			any client-based content negotiation method MUST meet the following
236			tests:
237
238			1) It MUST provide at least as good a content negotiation facility as
239			the current Accept header and User-Agent methods, and it SHOULD
240			increase the likelihood that the best available variant is served to
241			each client.
242
243			2) It MUST NOT significantly increase the bandwidth required to
244			deliver the content.
245
246			3) It MUST NOT significantly increase the number of network round
247			trips required to deliver the content.
248
249			4) It MUST NOT allow inappropriate content to be delivered to the
250			users of caching proxies. It SHOULD reduce overall bandwidth
251			requirements by providing cachable content and allowing proxies to
252			participate in the negotiation process.
253
254			5) It SHOULD recognize HTTP's relationship to MIME and provide methods
255			which are extensible to other MIME transports when the space over
256			which negotiation takes place is the same.
257
258			Experimentation with various content negotiation methods may be
259			necessary to determine their usefulness for different types of
260			resources and contexts. Interoperability will be best maintained,
261			however, if a single method of each type is made standard as soon as
262			experience permits.
263
264
265			7. Security Considerations
266
267			"Active content" negotiation presumes that clients will execute code
268			provided by servers and that those clients will allow that code to
269			examine user preferences and user agent capabilities in order to
270			select among variants. Any such system would have to be tightly
271			bounded to prevent the active content from executing arbitrary code on
272			the client. The examination of user preferences and user agent
273			capabilities also presents a possible privacy problem, especially as
274			the selection of a particular variant provides an opportunity for the
275			active content to communicate its discoveries.
276
277			8. Acknowledgements
278
279			Koen Holtman, Graham Klyne, Scott Lawrence, and Larry Masinter
280			provided valuable feedback on the initial draft of this document.
281			Discussions within the HTTP working group and the IETF-FAX mailing
282			list also provided insights into the scope of negotiation needed in
283			the HTTP context.
284
285			9. References
286
287			[1] Tim Berners-Lee, Roy Fielding, Henrik Frystk. Hypertext
288			Transfer Protocol -- HTTP/1.0. RFC RFC1945. May 1996.
289
290			[2] Koen Holtman and Andrew Mutz. Transport Content Negotiation in
291			HTTP. Internet-Draft draft-ietf-http-negotiation-02.txt, HTTP Working
292			Group, May 26, 1997.
293
294
295			9. Author's Address
296
297			Edward Hardie
298			NASA Network Information Center
299			MS 204-14
300			Moffett Field, CA 94035-1000 USA
301			+1 415 604 0134
302			hardie@nasa.gov
303
304
305