2004/id/draft-ietf-http-negotiate-scenario-00.txt

INTERNET-DRAFT                                              Edward Hardie
Expires: December, 1997                                     NASA NIC
<draft-ietf-http-negotiate-scenario-00.txt>                            


     Scenarios for the Delivery of Negotiated Content using HTTP


1.  Status of this Memo

This document is an Internet-Draft.  Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas, and
its working groups.  Note that other groups may also distribute working
documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time.  It is inappropriate to use Internet-Drafts as reference material
or to cite them other than as ``work in progress.''

To learn the current status of any Internet-Draft, please check the
"1id-abstracts.txt" listing contained in the Internet- Drafts Shadow
Directories on ds.internic.net (US East Coast), nic.nordu.net (Europe),
ftp.isi.edu (US West Coast), or munnari.oz.au (Pacific Rim).

Distribution of this memo is unlimited.  Please send comments to the
author.

2.  Abstract

This document describes various problems which have arisen in attempts
to deliver negotiated content using HTTP and examines several
scenarios by which improvements in delivery might be accomplished.
This document does not discuss how HTTP might be used to negotiate the
use of other protocols to deliver content.

3. Problems with content negotiation in HTTP

Content providers on the World Wide Web would like to present their
material in the form most appropriate to each individual user.  In
order to do this effectively, the content provider and content user
must be able negotiate among variants when more than one
representation of a resource is available.

The Hypertext Transport Protocol has since version 1.0 [1] used
certain request headers to indicate a list of the responses preferred
by the user agent making the request.  The Accept request header is
used to indicate media types, Accept-Charset a list of preferred
character sets, Accept-Encoding supported content encodings, and
Accept-Language the set of natural languages preferred.  In theory, a
server receiving these headers can use them to determine which of the
available variants of a particular item is most appropriate to the
requesting user agent.

Problems have emerged with this approach.  First, the explosion of
content types in use on the World Wide Web has meant that Accept
headers would have to be extraordinarily long in order to adequately
state the user's Accept preferences.  Increasing the length of the
Accept headers in order to negotiate consumes bandwidth for every
request, whether the resource has variants or not.  Increasing their
complexity in order to account for all of the possible axes of
variation can also consume excessive amounts of server processing
power.  Second, even if using a full list of Accept preferences did
not present resource problems, it could create privacy problems.  A
full list of Accept preferences creates a user profile; in some
situations it could be combined with other information to track a
user's interaction with a server or set of servers even if the user
has taken steps to eliminate that possiblity.

Many content providers have attempted to circumvent the problems
associated with negotiation based on Accept headers.  The most popular
method has been to examine the User-Agent header and to create content
tailored to the typical capabilities of that user agent.  Because many
web user agents are both configurable and extensible, this method has
serious drawbacks.  A user agent capable of using graphics but
configured to display only text may be presented with completely
unusable content.  Conversely, a user agent which has been extended to
handle new media types may never be presented with them.  The
usefulness of this method has been further degraded by some client
authors, who have chosen to use the User-Agent strings associated
with other clients in order to indicate compatibility; commonly, this
has been done even when the two clients are not, in fact, fully
compatible.

Content negotiation based on User-agent strings also creates
difficulties for caching proxies, as they have no way to determine
when the form of content delivered to two different user-agents would
be the same.  The delivery of inappropriate content can be prevented
with the appropriate use of expiration times and cache control
directives, but only at the cost of serious losses in cache
efficiency.


4. Requirements for effective content negotiation

Content negotiation methods should improve the likelihood that the
best available resource variant is served to each user.  If no
available resource variant is appropriate for a particular user, they
should furnish a method for the content provider to indicate that
lack.

Content negotiation methods should also accomplish their task without
substantially increasing the bandwidth required to deliver the
negotiated variant over that required to deliver similar invariant
content.  Some increase must be expected, but the smallest possible
increase is preferred, and any overhead should be limited to those
resources for which variants are available.  As a rule of thumb, any
method that increases bandwidth requirements by more than 50% is
unacceptable.  For example, a method that delivered all available
variants to a client and expected the client to discard those that
were not used might be very successful in providing the best available
resource and in limiting overhead to those resources for which
variants existed; it would, nonetheless, be unacceptable for
deployment on the global Internet because of the wastage associated
with each delivery.

In addition, content negotiation methods should interact well with
caching proxies.  They must prevent the delivery of inappropriate
content to proxy users.  If possible, they should allow caches to
store sufficient information to participate in the negotiation
process.


5. Scenarios

5.1 Improvements to the current system

Content negotiation based on Accept headers or User-Agent strings is
undertaken by the server based on information present in all requests
from clients.  The current problems with this method could be
ameliorated in a number of ways.  Registration of feature bundles
could, for example, allow a client to indicate its capabilities
without the increase in packet size that individual enumeration of
capabilities requires.  The use of registered feature bundles does,
however, require a fair amount of maintenance for both server and
clients, especially given their dynamic extendability.

Content providers could also improve the cachability of the variants
served by issuing redirects to unique URLs specific to each variant;
proxy users with equivalent feature bundles or user agents would then
be able to retrieve the cached variant once they received a redirect.
Some content providers already use this method to good effect.

5.2 Client-based negotiation

Improvements in the current method tend, however, to increase demands
on the processing power of the server, which raises serious problems
of scalability.  Shifting all or part of the processing involved in
the negotiation to the client may improve scalability and shifts the
locus of negotiation to the party most aware of both the current and
potential possibilities for presentation to the user.  

Client-based content negotiation can take two basic forms: elective
negotiation and "active content" negotiation.

5.2.1. Elective negotiation

Elective negotiation relies on the idea that the first response to a
request for a resource which has variants should be a statement naming
the choices and/or the axes along which they vary.  The simplest form
of elective negotiation occurs when a content provider creates a
meta-resource with links to several variants and explanations of the
features needed to present each variant.  This basic form presumes a
least common denominator for presentation of the links and an active
user selecting among the presentations.  "Transport Content
Negotiation in HTTP" [2] provides for a more complex version of
elective negotiation.  It describes a standard method for delineating
the axes along which a resource varies and a set of methods by which
caches can participate in the negotiation process.

Elective negotiation limits the process of selecting among variants to
the axes along which they vary.  This eliminates the need for clients'
enumerating their capabilities and user preferences.  It also provides
both a bandwidth savings and an increase in privacy. Many times,
however, this process requires a user to actively select among the
resources provided, which reduces perceived efficiency and increases
perceived latency.

5.2.2 "Active content" negotiation

"Active content" negotiation takes place when the request for a
resource which has variants returns content for execution in the
client context rather than for presentation to the user.  This active
content determines the capabilities of the user agent and the
preferences of the user and then retrieves the best available variant
or provides an error status if no variants are appropriate.  This
method has several advantages: the negotiation takes place without the
need for user intervention; the processing for selection of variants
is moved completely off the server; and the active content itself
should be cachable.

There are, however, several limitations to active content negotiation.
The most severe problem is that there is not yet a cross-platform
standard for active content.  Without such a standard, servers may
have to engage in Accept header or User-Agent negotiation prior to
using active content in order to identify the correct form of active
content to send in response to a particular request.  Some user agents
will also be unable to use active content, either because they have
limited processing power or because of security considerations (see
Section 7).  For these user agents, a completely different method must
be provided.  As with server-side negotiation, each variant will
require a unique URL to ensure cache correctness.

6.  Summary

Content negotiation using HTTP currently presents problems in
accuracy, scalability, and privacy.  Amelioration of the problems in
the current design is possible, but other content negotiation methods
may provide significant improvements.  The type of content negotiation
most appropriate to a specific context will depend on the type of
variable content being served and the capabilities of the parties to
the negotiation.  Before deployment on the global Internet, however,
any client-based content negotiation method MUST meet the following
tests:

1) It MUST provide at least as good a content negotiation facility as
the current Accept header and User-Agent methods, and it SHOULD
increase the likelihood that the best available variant is served to
each client.

2) It MUST NOT significantly increase the bandwidth required to
deliver the content.

3) It MUST NOT allow inappropriate content to be delivered to the
users of caching proxies.  It SHOULD reduce overall bandwidth
requirements by providing cachable content and allowing proxies to
participate in the negotiation process.

Experimentation with various content negotiation methods may be
necessary to determine their usefulness for different types of
resources and contexts.  Interoperability will be best maintained,
however, if a single method of each type is made standard as soon as
experience permits.


7. Security Considerations

"Active content" negotiation presumes that clients will execute code
provided by servers and that those clients will allow that code to
examine user preferences and user agent capabilities in order to
select among variants.  Any such system would have to be tightly
bounded to prevent the active content from executing arbitrary code on
the client.  The examination of user preferences and user agent
capabilities also presents a possible privacy problem, especially as
the selection of a particular variant provides an opportunity for the
active content to communicate its discoveries.


8. References
 
   [1] Tim Berners-Lee, Roy Fielding, Henrik Frystk.  Hypertext
   Transfer Protocol -- HTTp/1.0.  RFC RFC1945.  May 1996.

   [2] Koen Holtman and Andrew Mutz.  Transport Content Negotiation in
   HTTP.  Internet-Draft draft-ietf-http-negotiation-02.txt, HTTP Working
   Group, May 26, 1997.


9.  Author's Address

   Edward Hardie
   NASA Network Information Center
   MS 204-14
   Moffett Field, CA 94035-1000 USA
   +1 415 604  0134
   hardie@nasa.gov


1	INTERNET-DRAFT Edward Hardie
2	Expires: December, 1997 NASA NIC
3	<draft-ietf-http-negotiate-scenario-00.txt>
4
5
6
7
8	Scenarios for the Delivery of Negotiated Content using HTTP
9
10
11
12	1. Status of this Memo
13
14	This document is an Internet-Draft. Internet-Drafts are working
15	documents of the Internet Engineering Task Force (IETF), its areas, and
16	its working groups. Note that other groups may also distribute working
17	documents as Internet-Drafts.
18
19	Internet-Drafts are draft documents valid for a maximum of six months
20	and may be updated, replaced, or obsoleted by other documents at any
21	time. It is inappropriate to use Internet-Drafts as reference material
22	or to cite them other than as ``work in progress.''
23
24	To learn the current status of any Internet-Draft, please check the
25	"1id-abstracts.txt" listing contained in the Internet- Drafts Shadow
26	Directories on ds.internic.net (US East Coast), nic.nordu.net (Europe),
27	ftp.isi.edu (US West Coast), or munnari.oz.au (Pacific Rim).
28
29	Distribution of this memo is unlimited. Please send comments to the
30	author.
31
32	2. Abstract
33
34	This document describes various problems which have arisen in attempts
35	to deliver negotiated content using HTTP and examines several
36	scenarios by which improvements in delivery might be accomplished.
37	This document does not discuss how HTTP might be used to negotiate the
38	use of other protocols to deliver content.
39
40	3. Problems with content negotiation in HTTP
41
42	Content providers on the World Wide Web would like to present their
43	material in the form most appropriate to each individual user. In
44	order to do this effectively, the content provider and content user
45	must be able negotiate among variants when more than one
46	representation of a resource is available.
47
48	The Hypertext Transport Protocol has since version 1.0 [1] used
49	certain request headers to indicate a list of the responses preferred
50	by the user agent making the request. The Accept request header is
51	used to indicate media types, Accept-Charset a list of preferred
52	character sets, Accept-Encoding supported content encodings, and
53	Accept-Language the set of natural languages preferred. In theory, a
54	server receiving these headers can use them to determine which of the
55	available variants of a particular item is most appropriate to the
56	requesting user agent.
57
58	Problems have emerged with this approach. First, the explosion of
59	content types in use on the World Wide Web has meant that Accept
60	headers would have to be extraordinarily long in order to adequately
61	state the user's Accept preferences. Increasing the length of the
62	Accept headers in order to negotiate consumes bandwidth for every
63	request, whether the resource has variants or not. Increasing their
64	complexity in order to account for all of the possible axes of
65	variation can also consume excessive amounts of server processing
66	power. Second, even if using a full list of Accept preferences did
67	not present resource problems, it could create privacy problems. A
68	full list of Accept preferences creates a user profile; in some
69	situations it could be combined with other information to track a
70	user's interaction with a server or set of servers even if the user
71	has taken steps to eliminate that possiblity.
72
73	Many content providers have attempted to circumvent the problems
74	associated with negotiation based on Accept headers. The most popular
75	method has been to examine the User-Agent header and to create content
76	tailored to the typical capabilities of that user agent. Because many
77	web user agents are both configurable and extensible, this method has
78	serious drawbacks. A user agent capable of using graphics but
79	configured to display only text may be presented with completely
80	unusable content. Conversely, a user agent which has been extended to
81	handle new media types may never be presented with them. The
82	usefulness of this method has been further degraded by some client
83	authors, who have chosen to use the User-Agent strings associated
84	with other clients in order to indicate compatibility; commonly, this
85	has been done even when the two clients are not, in fact, fully
86	compatible.
87
88	Content negotiation based on User-agent strings also creates
89	difficulties for caching proxies, as they have no way to determine
90	when the form of content delivered to two different user-agents would
91	be the same. The delivery of inappropriate content can be prevented
92	with the appropriate use of expiration times and cache control
93	directives, but only at the cost of serious losses in cache
94	efficiency.
95
96
97	4. Requirements for effective content negotiation
98
99	Content negotiation methods should improve the likelihood that the
100	best available resource variant is served to each user. If no
101	available resource variant is appropriate for a particular user, they
102	should furnish a method for the content provider to indicate that
103	lack.
104
105	Content negotiation methods should also accomplish their task without
106	substantially increasing the bandwidth required to deliver the
107	negotiated variant over that required to deliver similar invariant
108	content. Some increase must be expected, but the smallest possible
109	increase is preferred, and any overhead should be limited to those
110	resources for which variants are available. As a rule of thumb, any
111	method that increases bandwidth requirements by more than 50% is
112	unacceptable. For example, a method that delivered all available
113	variants to a client and expected the client to discard those that
114	were not used might be very successful in providing the best available
115	resource and in limiting overhead to those resources for which
116	variants existed; it would, nonetheless, be unacceptable for
117	deployment on the global Internet because of the wastage associated
118	with each delivery.
119
120	In addition, content negotiation methods should interact well with
121	caching proxies. They must prevent the delivery of inappropriate
122	content to proxy users. If possible, they should allow caches to
123	store sufficient information to participate in the negotiation
124	process.
125
126
127	5. Scenarios
128
129	5.1 Improvements to the current system
130
131	Content negotiation based on Accept headers or User-Agent strings is
132	undertaken by the server based on information present in all requests
133	from clients. The current problems with this method could be
134	ameliorated in a number of ways. Registration of feature bundles
135	could, for example, allow a client to indicate its capabilities
136	without the increase in packet size that individual enumeration of
137	capabilities requires. The use of registered feature bundles does,
138	however, require a fair amount of maintenance for both server and
139	clients, especially given their dynamic extendability.
140
141	Content providers could also improve the cachability of the variants
142	served by issuing redirects to unique URLs specific to each variant;
143	proxy users with equivalent feature bundles or user agents would then
144	be able to retrieve the cached variant once they received a redirect.
145	Some content providers already use this method to good effect.
146
147	5.2 Client-based negotiation
148
149	Improvements in the current method tend, however, to increase demands
150	on the processing power of the server, which raises serious problems
151	of scalability. Shifting all or part of the processing involved in
152	the negotiation to the client may improve scalability and shifts the
153	locus of negotiation to the party most aware of both the current and
154	potential possibilities for presentation to the user.
155
156	Client-based content negotiation can take two basic forms: elective
157	negotiation and "active content" negotiation.
158
159	5.2.1. Elective negotiation
160
161	Elective negotiation relies on the idea that the first response to a
162	request for a resource which has variants should be a statement naming
163	the choices and/or the axes along which they vary. The simplest form
164	of elective negotiation occurs when a content provider creates a
165	meta-resource with links to several variants and explanations of the
166	features needed to present each variant. This basic form presumes a
167	least common denominator for presentation of the links and an active
168	user selecting among the presentations. "Transport Content
169	Negotiation in HTTP" [2] provides for a more complex version of
170	elective negotiation. It describes a standard method for delineating
171	the axes along which a resource varies and a set of methods by which
172	caches can participate in the negotiation process.
173
174	Elective negotiation limits the process of selecting among variants to
175	the axes along which they vary. This eliminates the need for clients'
176	enumerating their capabilities and user preferences. It also provides
177	both a bandwidth savings and an increase in privacy. Many times,
178	however, this process requires a user to actively select among the
179	resources provided, which reduces perceived efficiency and increases
180	perceived latency.
181
182	5.2.2 "Active content" negotiation
183
184	"Active content" negotiation takes place when the request for a
185	resource which has variants returns content for execution in the
186	client context rather than for presentation to the user. This active
187	content determines the capabilities of the user agent and the
188	preferences of the user and then retrieves the best available variant
189	or provides an error status if no variants are appropriate. This
190	method has several advantages: the negotiation takes place without the
191	need for user intervention; the processing for selection of variants
192	is moved completely off the server; and the active content itself
193	should be cachable.
194
195	There are, however, several limitations to active content negotiation.
196	The most severe problem is that there is not yet a cross-platform
197	standard for active content. Without such a standard, servers may
198	have to engage in Accept header or User-Agent negotiation prior to
199	using active content in order to identify the correct form of active
200	content to send in response to a particular request. Some user agents
201	will also be unable to use active content, either because they have
202	limited processing power or because of security considerations (see
203	Section 7). For these user agents, a completely different method must
204	be provided. As with server-side negotiation, each variant will
205	require a unique URL to ensure cache correctness.
206
207	6. Summary
208
209	Content negotiation using HTTP currently presents problems in
210	accuracy, scalability, and privacy. Amelioration of the problems in
211	the current design is possible, but other content negotiation methods
212	may provide significant improvements. The type of content negotiation
213	most appropriate to a specific context will depend on the type of
214	variable content being served and the capabilities of the parties to
215	the negotiation. Before deployment on the global Internet, however,
216	any client-based content negotiation method MUST meet the following
217	tests:
218
219	1) It MUST provide at least as good a content negotiation facility as
220	the current Accept header and User-Agent methods, and it SHOULD
221	increase the likelihood that the best available variant is served to
222	each client.
223
224	2) It MUST NOT significantly increase the bandwidth required to
225	deliver the content.
226
227	3) It MUST NOT allow inappropriate content to be delivered to the
228	users of caching proxies. It SHOULD reduce overall bandwidth
229	requirements by providing cachable content and allowing proxies to
230	participate in the negotiation process.
231
232	Experimentation with various content negotiation methods may be
233	necessary to determine their usefulness for different types of
234	resources and contexts. Interoperability will be best maintained,
235	however, if a single method of each type is made standard as soon as
236	experience permits.
237
238
239	7. Security Considerations
240
241	"Active content" negotiation presumes that clients will execute code
242	provided by servers and that those clients will allow that code to
243	examine user preferences and user agent capabilities in order to
244	select among variants. Any such system would have to be tightly
245	bounded to prevent the active content from executing arbitrary code on
246	the client. The examination of user preferences and user agent
247	capabilities also presents a possible privacy problem, especially as
248	the selection of a particular variant provides an opportunity for the
249	active content to communicate its discoveries.
250
251
252	8. References
253
254	[1] Tim Berners-Lee, Roy Fielding, Henrik Frystk. Hypertext
255	Transfer Protocol -- HTTp/1.0. RFC RFC1945. May 1996.
256
257	[2] Koen Holtman and Andrew Mutz. Transport Content Negotiation in
258	HTTP. Internet-Draft draft-ietf-http-negotiation-02.txt, HTTP Working
259	Group, May 26, 1997.
260
261
262	9. Author's Address
263
264	Edward Hardie
265	NASA Network Information Center
266	MS 204-14
267	Moffett Field, CA 94035-1000 USA
268	+1 415 604 0134
269	hardie@nasa.gov
270
271
272