1 |
|
2 |
|
3 |
HTTP Working Group Jeffrey Mogul, DECWRL |
4 |
Internet-Draft Paul J. Leach, Microsoft |
5 |
Expires: 3 January 1998 3 July 1997 |
6 |
|
7 |
|
8 |
Simple Hit-Metering and Usage-Limiting for HTTP |
9 |
|
10 |
draft-ietf-http-hit-metering-03.txt |
11 |
|
12 |
|
13 |
STATUS OF THIS MEMO |
14 |
|
15 |
This document is an Internet-Draft. Internet-Drafts are |
16 |
working documents of the Internet Engineering Task Force |
17 |
(IETF), its areas, and its working groups. Note that other |
18 |
groups may also distribute working documents as |
19 |
Internet-Drafts. |
20 |
|
21 |
Internet-Drafts are draft documents valid for a maximum of |
22 |
six months and may be updated, replaced, or obsoleted by |
23 |
other documents at any time. It is inappropriate to use |
24 |
Internet-Drafts as reference material or to cite them other |
25 |
than as "work in progress." |
26 |
|
27 |
To learn the current status of any Internet-Draft, please |
28 |
check the "1id-abstracts.txt" listing contained in the |
29 |
Internet-Drafts Shadow Directories on ftp.is.co.za |
30 |
(Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific |
31 |
Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US |
32 |
West Coast). |
33 |
|
34 |
Distribution of this document is unlimited. Please send |
35 |
comments to the HTTP working group at |
36 |
<http-wg@cuckoo.hpl.hp.com>. Discussions of the working |
37 |
group are archived at |
38 |
<URL:http://www.ics.uci.edu/pub/ietf/http/>. General |
39 |
discussions about HTTP and the applications which use HTTP |
40 |
should take place on the <www-talk@w3.org> mailing list. |
41 |
|
42 |
|
43 |
ABSTRACT |
44 |
|
45 |
This document proposes a simple extension to HTTP, using a |
46 |
new ``Meter'' header, which permits a limited form of |
47 |
demographic information (colloquially called |
48 |
``hit-counts'') to be reported by caches to origin servers, |
49 |
in a more efficient manner than the ``cache-busting'' |
50 |
techniques currently used. It also permits an origin |
51 |
server to control the number of times a cache uses a cached |
52 |
response, and outlines a technique that origin servers can |
53 |
use to capture referral information without |
54 |
``cache-busting.'' |
55 |
|
56 |
|
57 |
|
58 |
Mogul, Leach [Page 1] |
59 |
|
60 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
61 |
|
62 |
|
63 |
TABLE OF CONTENTS |
64 |
|
65 |
1 Introduction 2 |
66 |
1.1 Goals, non-goals, and limitations 3 |
67 |
1.2 Brief summary of the design 4 |
68 |
1.3 Terminology 5 |
69 |
2 Overview 5 |
70 |
2.1 Discussion 7 |
71 |
3 Design concepts 7 |
72 |
3.1 Implementation of the "metering subtree" 8 |
73 |
3.2 Format of the Meter header 9 |
74 |
3.3 Negotiation of hit-metering and usage-limiting 10 |
75 |
3.4 Transmission of usage reports 13 |
76 |
3.5 When to send usage reports 14 |
77 |
3.6 Subdivision of usage-limits 16 |
78 |
4 Analysis 17 |
79 |
4.1 Approximation accuracy for counting users 17 |
80 |
4.2 What about "Network Computers"? 18 |
81 |
4.3 Critical-path delay analysis 18 |
82 |
5 Specification 19 |
83 |
5.1 Specification of Meter header and directives 19 |
84 |
5.2 Abbreviations for Meter directives 22 |
85 |
5.3 Counting rules 22 |
86 |
5.3.1 Counting rules for hit-metering 23 |
87 |
5.3.2 Counting rules for usage-limiting 23 |
88 |
5.3.3 Equivalent algorithms are allowed 24 |
89 |
5.4 Counting rules: interaction with Range requests 25 |
90 |
5.5 Implementation by non-caching proxies 26 |
91 |
5.6 Implementation by cooperating caches 26 |
92 |
6 Examples 26 |
93 |
6.1 Example of a complete set of exchanges 26 |
94 |
6.2 Protecting against HTTP/1.0 proxies 28 |
95 |
6.3 More elaborate examples 29 |
96 |
7 Interactions with content negotiation 29 |
97 |
7.1 Treatment of responses carrying a Vary header 30 |
98 |
7.2 Interaction with Transparent Content Negotiation 31 |
99 |
8 A Note on Capturing Referrals 31 |
100 |
9 Alternative proposals 32 |
101 |
10 Security Considerations 32 |
102 |
11 Acknowledgements 33 |
103 |
12 References 33 |
104 |
13 Authors' addresses 34 |
105 |
|
106 |
|
107 |
1 Introduction |
108 |
|
109 |
For a variety of reasons, content providers want to be able to |
110 |
collect information on the frequency with which their content is |
111 |
accessed. This desire leads to some of the "cache-busting" done by |
112 |
existing servers. ("Cache-busting" is the use by servers of |
113 |
techniques intended to prevent caching of responses; it is unknown |
114 |
|
115 |
Mogul, Leach [Page 2] |
116 |
|
117 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
118 |
|
119 |
|
120 |
exactly how common this is.) This kind of cache-busting is done not |
121 |
for the purpose of maintaining transparency or security properties, |
122 |
but simply to collect demographic information. Some cache-busting is |
123 |
also done to provide different advertising images to appear on the |
124 |
same page (i.e., each retrieval of the page sees a different ad). |
125 |
|
126 |
This proposal supports a model similar to that of publishers of |
127 |
hard-copy publications: such publishers (try to) report to their |
128 |
advertisers how many people read an issue of a publication at least |
129 |
once; they don't (try to) report how many times a reader re-reads an |
130 |
issue. They do this by counting copies published, and then try to |
131 |
estimate, for their publication, on average how many people read a |
132 |
single copy at least once. The key point is that the results aren't |
133 |
exact, but are still useful. Another model is that of coding |
134 |
inquiries in such a way that the advertiser can tell which |
135 |
publication produced the inquiry. |
136 |
|
137 |
1.1 Goals, non-goals, and limitations |
138 |
HTTP/1.1 already allows origin servers to prevent caching of |
139 |
responses, and evidence exists [8] that at least some of the time, |
140 |
this is being done for the sole purpose of collecting counts of the |
141 |
number of accesses of specific pages. Some of this evidence is |
142 |
inferred from the study of proxy traces; some is based on explicit |
143 |
statements of the intention of the operators of Web servers. |
144 |
Information collected this way might or might not be of actual use to |
145 |
the people who collect it; the fact is that they want to collect it, |
146 |
or already do so. |
147 |
|
148 |
The goal of this proposal is to provide an optional performance |
149 |
optimization for this use of HTTP/1.1. |
150 |
|
151 |
This specification is: |
152 |
|
153 |
- Optional: no server or proxy is required to implement it. |
154 |
|
155 |
- Proxy-centered: there is no involvement on the part of |
156 |
end-client implementations. |
157 |
|
158 |
- Solely a performance optimization: it provides no |
159 |
information or functionality that is not already available |
160 |
in HTTP/1.1. The intent is to improve performance overall, |
161 |
and reduce latency for almost all interactions; latency |
162 |
might be increased for a small fraction of HTTP |
163 |
interactions. |
164 |
|
165 |
- Best-efforts: it does not guarantee the accuracy of the |
166 |
reported information, although it does provide accurate |
167 |
results in the absence of persistent network failures or |
168 |
host crashes. |
169 |
|
170 |
- Neutral with respect to privacy: it reveals to servers no |
171 |
|
172 |
Mogul, Leach [Page 3] |
173 |
|
174 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
175 |
|
176 |
|
177 |
information about clients that is not already available |
178 |
through the existing features of HTTP/1.1. |
179 |
|
180 |
The goals of this specification do not include: |
181 |
|
182 |
- Solving the entire problem of efficiently obtaining |
183 |
extensive information about requests made via proxies. |
184 |
|
185 |
- Improving the protection of user privacy (although our |
186 |
proposal may reduce the transfer of user-specific |
187 |
information to servers, it does not prevent it). |
188 |
|
189 |
- Preventing or encouraging the use of log-exchange |
190 |
mechanisms. |
191 |
|
192 |
- Avoiding all forms of "cache-busting", or even all |
193 |
cache-busting done for gathering counts. |
194 |
|
195 |
This design has certain potential limitations: |
196 |
|
197 |
- If it is not deployed widely in both proxies and servers, |
198 |
it will provide little benefit. |
199 |
|
200 |
- It may, by partially solving the hit-counting problem, |
201 |
reduce the pressure to adopt more complete solutions, if |
202 |
any become available. |
203 |
|
204 |
- Even if widely deployed, it might not be widely used, and |
205 |
so might not significantly improve performance. |
206 |
|
207 |
These potential limitations might not be problems in actual practice. |
208 |
|
209 |
1.2 Brief summary of the design |
210 |
This section is included for people not wishing to read the entire |
211 |
document; it is not a specification for the proposed design, and |
212 |
over-simplifies many aspects of the design. |
213 |
|
214 |
The goal of this design is to eliminate the need for origin servers |
215 |
to use "cache-busting" techniques, when this is done just for the |
216 |
purpose of counting the number of users of a resource. |
217 |
(Cache-busting includes techniques such as setting immediate |
218 |
Expiration dates, or sending "Cache-control: private" in each |
219 |
response.) |
220 |
|
221 |
The design adds a new "Meter" header to HTTP; the header is always |
222 |
protected by the "Connection" header, and so is always hop-by-hop. |
223 |
This mechanism allows the construction of a "metering subtree", which |
224 |
is a connected subtree of proxies, rooted at an origin server. Only |
225 |
those proxies that explicitly volunteer to join in the metering |
226 |
subtree for a resource participate in hit-metering, but those proxies |
227 |
that do volunteer are required to make their best effort to provide |
228 |
|
229 |
Mogul, Leach [Page 4] |
230 |
|
231 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
232 |
|
233 |
|
234 |
accurate counts. When a hit-metered response is forwarded outside of |
235 |
the metering subtree, the forwarding proxy adds "Cache-control: |
236 |
s-maxage=0", so that other proxies (outside the metering subtree) are |
237 |
forced to forward all requests to a server in the metering subtree. |
238 |
|
239 |
--------- |
240 |
NOTE: the HTTP/1.1 specification does not currently define a |
241 |
"s-maxage" Cache-control directive. The HTTP working group has |
242 |
decided to add such a directive to the next revision of the |
243 |
HTTP/1.1 specification [6]. |
244 |
--------- |
245 |
|
246 |
The Meter header carries zero or more directives, similar to the way |
247 |
that the Cache-control header carries directives. Proxies may use |
248 |
certain Meter directives to volunteer to do hit-metering for a |
249 |
resource. If a proxy does volunteer, the server may use certain |
250 |
directives to require that a response be hit-metered. Finally, |
251 |
proxies use a "count" Meter directive to report the accumulated hit |
252 |
counts. |
253 |
|
254 |
The Meter mechanism can also be used by a server to limit the number |
255 |
of uses that a cache may make of a cached response, before |
256 |
revalidating it. |
257 |
|
258 |
The full specification includes complete rules for counting "uses" of |
259 |
a response (e.g., non-conditional GETs) and "reuses" (conditional |
260 |
GETs). These rules ensure that the results are entirely consistent |
261 |
in all cases, except when systems or networks fail. |
262 |
|
263 |
1.3 Terminology |
264 |
This document uses terms defined and explained in the HTTP/1.1 |
265 |
specification [3], including ``origin server,'' ``resource,'' |
266 |
``hop-by-hop,'' ``unconditional GET,'' and ``conditional GET.'' The |
267 |
reader is expected to be familiar with the HTTP/1.1 specification and |
268 |
its terminology. |
269 |
|
270 |
|
271 |
2 Overview |
272 |
|
273 |
The design described in this document introduces several new features |
274 |
to HTTP: |
275 |
|
276 |
- Hit-metering: allows an origin server to obtain reasonably |
277 |
accurate counts of the number of clients using a resource |
278 |
instance via a proxy cache, or a hierarchy of proxy caches. |
279 |
|
280 |
- Usage-limiting: allows an origin server to control the |
281 |
number of times a cached response may be used by a proxy |
282 |
cache, or a hierarchy of proxy caches, before revalidation |
283 |
with the origin server. |
284 |
|
285 |
|
286 |
Mogul, Leach [Page 5] |
287 |
|
288 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
289 |
|
290 |
|
291 |
These new non-mandatory features require minimal new protocol |
292 |
support, no change in protocol version, relatively little overhead in |
293 |
message headers. The design adds no additional network round-trips |
294 |
in any critical path that directly affects user-perceived latency |
295 |
(see section 4.3 for an analysis). |
296 |
|
297 |
The primary goal of hit-metering and usage-limiting is to obviate the |
298 |
need for an origin server to send "Cache-control: s-maxage=0" with |
299 |
responses for resources whose value is not likely to change |
300 |
immediately. In other words, in cases where the only reason for |
301 |
contacting the origin server on every request that might otherwise be |
302 |
satisfied by a proxy cache entry is to allow the server to collect |
303 |
demographic information or to control the number of times a cache |
304 |
entry is used, the extension proposed here will avoid a significant |
305 |
amount of unnecessary network traffic and latency. |
306 |
|
307 |
This design introduces one new ``Meter'' header, which is used both |
308 |
in HTTP request messages and HTTP response messages. The Meter |
309 |
header is used to transmit a number of directives and reports. In |
310 |
particular, all negotiation of the use of hit-metering and usage |
311 |
limits is done using this header. No other changes to the existing |
312 |
HTTP/1.1 specification [3] are proposed in this document. |
313 |
|
314 |
This design also introduces several new concepts: |
315 |
|
316 |
1. The concepts of a "use" of a cache entry, which is when a |
317 |
proxy returns its entity-body in response to a conditional |
318 |
or non-conditional request, and the "reuse" of a cache |
319 |
entry, which is when a proxy returns a 304 (Not Modified) |
320 |
response to a conditional request which is satisfied by |
321 |
that cache entry. |
322 |
|
323 |
2. The concept of a hit-metered resource, for which proxy |
324 |
caches make a best-effort attempt to report accurate |
325 |
counts of uses and/or reuses to the origin server. |
326 |
|
327 |
3. The concept of a usage-limited resource, for which the |
328 |
origin server expects proxy caches to limit the number of |
329 |
uses and/or reuses. |
330 |
|
331 |
The new Meter directives and reports interact to allow proxy caches |
332 |
and servers to cooperate in the collection of demographic data. The |
333 |
goal is a best-efforts approximation of the true number of uses |
334 |
and/or reuses, not a guaranteed exact count. |
335 |
|
336 |
The new Meter directives also allow a server to bound the inaccuracy |
337 |
of a particular hit-count, by bounding the number of uses between |
338 |
reports. It can also, for example, bound the number of times the |
339 |
same ad is shown because of caching. |
340 |
|
341 |
Section 7.1 describes a way to use server-driven content negotiation |
342 |
|
343 |
Mogul, Leach [Page 6] |
344 |
|
345 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
346 |
|
347 |
|
348 |
(the Vary header) that allows an HTTP origin server to flexibly |
349 |
separate requests into categories and count requests by category. |
350 |
Implementation of such a categorized hit counting is likely to be a |
351 |
very small modification to most implementations of Vary; some |
352 |
implementations may not require any modification at all. |
353 |
|
354 |
2.1 Discussion |
355 |
Mapping this onto the publishing model, a proxy cache would increment |
356 |
the use-count for a cache entry once for each unconditional GET done |
357 |
for the entry, and once for each conditional GET that results in |
358 |
sending a copy of the entry to update a client's invalid cached copy. |
359 |
Conditional GETs that result in 304 (Not Modified) are not included |
360 |
in the use-count, because they do not result in a new user seeing the |
361 |
page, but instead signify a repeat view by a user that had seen it |
362 |
before. However, 304 responses are counted in the reuse-count. |
363 |
HEADs are not counted at all, because their responses do not contain |
364 |
an entity-body. |
365 |
|
366 |
The Meter directives apply only to shared proxy caches, not to |
367 |
end-client (or other single-user) caches. Single user caches should |
368 |
not use Meter, because their hits will be automatically counted as a |
369 |
result of the unconditional GET with which they first fetch the page, |
370 |
from either the origin-server or from a proxy cache. Their |
371 |
subsequent conditional GETs do not result in a new user seeing the |
372 |
page. |
373 |
|
374 |
The mechanism specified here counts GETs; other methods either do not |
375 |
result in a page for the user to read, aren't cached, or are |
376 |
"written-through" and so can be directly counted by the origin |
377 |
server. (If, in the future, a "cachable POST" came into existence, |
378 |
whereby the entity-body in the POST request was used to select a |
379 |
cached response, then such POSTs would have to be treated just like |
380 |
GETs.) The applicability of hit-metering to any new HTTP methods |
381 |
that might be defined in the future is currently unspecifiable. |
382 |
|
383 |
In the case of multiple caches along a path, a proxy cache does the |
384 |
obvious summation when it receives a use-count or reuse-count in a |
385 |
request from another cache. |
386 |
|
387 |
|
388 |
3 Design concepts |
389 |
|
390 |
In order to allow the introduction of hit-metering and usage-limiting |
391 |
without requiring a protocol revision, and to ensure a reasonably |
392 |
close approximation of accurate counts, the negotiation of metering |
393 |
and usage-limiting is done hop-by-hop, not end-to-end. If one |
394 |
considers the "tree" of proxies that receive, store, and forward a |
395 |
specific response, the intent of this design is that within some |
396 |
(possibly null) "metering subtree", rooted at the origin server, all |
397 |
proxies are using the hit-metering and/or usage-limiting requested by |
398 |
the origin server. |
399 |
|
400 |
Mogul, Leach [Page 7] |
401 |
|
402 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
403 |
|
404 |
|
405 |
Proxies at the leaves of this subtree will insert a "Cache-control: |
406 |
s-maxage=0" directive, which forces all other proxies (below this |
407 |
subtree) to check with a leaf of the metering subtree on every |
408 |
request. However, it does not prevent them from storing and using |
409 |
the response, if the revalidation succeeds. |
410 |
|
411 |
No proxy is required to implement hit-metering or usage-limiting. |
412 |
However, any proxy that transmits the Meter header in a request MUST |
413 |
implement every unconditional requirement of this specification, |
414 |
without exception or amendment. |
415 |
|
416 |
This is a conservative design, which may sometimes fail to take |
417 |
advantage of hit-metering support in proxies outside the metering |
418 |
subtree. However, it is likely that without the reliability offered |
419 |
by a conservative design, managers of origin servers with |
420 |
requirements for accurate approximations will not take advantage of |
421 |
any hit-metering proposal. |
422 |
|
423 |
The hit-metering/usage-limiting mechanism is designed to avoid any |
424 |
extra network round-trips in the critical path of any client request, |
425 |
and (as much as possible) to avoid excessively lengthening HTTP |
426 |
messages. |
427 |
|
428 |
The Meter header is used to transmit both negotiation information and |
429 |
numeric information. |
430 |
|
431 |
A formal specification for the Meter header appears in section 5; the |
432 |
following discussion uses an informal approach to improve clarity. |
433 |
|
434 |
3.1 Implementation of the "metering subtree" |
435 |
The "metering subtree" approach is implemented in a simple, |
436 |
straightforward way by defining the new "Meter" header as one that |
437 |
MUST always be protected by a Connection header in every request or |
438 |
response. I.e., if the Meter header is present in an HTTP message, |
439 |
that message: |
440 |
|
441 |
1. MUST contain "Connection: meter", and MUST be handled |
442 |
according to the HTTP/1.1 specification of the Connection |
443 |
header. |
444 |
|
445 |
2. MUST NOT be sent in response to a request from a client |
446 |
whose version number is less than HTTP/1.1. |
447 |
|
448 |
3. MUST NOT be accepted from a client whose version number is |
449 |
less than HTTP/1.1. |
450 |
|
451 |
The reason for the latter two restrictions is to protect against |
452 |
proxies that might not properly implement the Connection header. |
453 |
Otherwise, a subtree that includes an HTTP/1.0 proxy might |
454 |
erroneously appear to be a metering subtree. |
455 |
|
456 |
|
457 |
Mogul, Leach [Page 8] |
458 |
|
459 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
460 |
|
461 |
|
462 |
--------- |
463 |
Note: It appears that for the Connection header mechanism to |
464 |
function correctly, a system receiving an HTTP/1.0 (or |
465 |
lower-version) message that includes a Connection header must |
466 |
act as if this header, and all of the headers it protects, |
467 |
ought to have been removed from the message by an intermediate |
468 |
proxy. |
469 |
|
470 |
Although RFC2068 does not specifically require this behavior, |
471 |
it appears to be implied. Otherwise, one could not depend on |
472 |
the stated property (section 14.10) that the protected options |
473 |
``MUST NOT be communicated by proxies over further |
474 |
connections.'' This should probably be clarified in a |
475 |
subsequent draft of the HTTP/1.1 specification. |
476 |
|
477 |
This specification does not, in any way, propose a modification |
478 |
of the specification of the Connection header. |
479 |
--------- |
480 |
|
481 |
From the point of view of an origin server, the proxies in a metering |
482 |
subtree work together to obey usage limits and to maintain accurate |
483 |
usage counts. When an origin server specifies a usage limit, a proxy |
484 |
in the metering subtree may subdivide this limit among its children |
485 |
in the subtree as it sees fit. Similarly, when a proxy in the |
486 |
subtree receives a usage report, it ensures that the hits represented |
487 |
by this report are summed properly and reported to the origin server. |
488 |
|
489 |
When a proxy forwards a hit-metered or usage-limited response to a |
490 |
client (proxy or end-client) not in the metering subtree, it MUST |
491 |
omit the Meter header, and it MUST add "Cache-control: s-maxage=0" to |
492 |
the response. |
493 |
|
494 |
3.2 Format of the Meter header |
495 |
The Meter header is used to carry zero or more directives. Multiple |
496 |
Meter headers may occur in an HTTP message, but according to the |
497 |
rules in section 4.2 of the HTTP/1.1 specification [3], they may be |
498 |
combined into a single header (and should be so combined, to reduce |
499 |
overhead). |
500 |
|
501 |
For example, the following sequence of Meter headers |
502 |
|
503 |
Meter: max-uses=3 |
504 |
Meter: max-reuses=10 |
505 |
Meter: do-report |
506 |
|
507 |
may be expressed as |
508 |
|
509 |
Meter: max-uses=3, max-reuses=10, do-report |
510 |
|
511 |
|
512 |
|
513 |
|
514 |
Mogul, Leach [Page 9] |
515 |
|
516 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
517 |
|
518 |
|
519 |
3.3 Negotiation of hit-metering and usage-limiting |
520 |
An origin server that wants to collect hit counts for a resource, by |
521 |
simply forcing all requests to bypass any proxy caches, would respond |
522 |
to requests on the resource with "Cache-control: s-maxage=0". (An |
523 |
origin server wishing to prevent HTTP/1.0 proxies from improperly |
524 |
caching the response could also send both "Expires: <now>", to |
525 |
prevent such caching, and "Cache-control: max-age=NNNN", to allow |
526 |
newer proxies to cache the response). |
527 |
|
528 |
The purpose of the Meter header is to obviate the need for |
529 |
"Cache-control: s-maxage=0" within a metering subtree. Thus, any |
530 |
proxy may negotiate the use of hit-metering and/or usage-limiting |
531 |
with the next-hop server. If this server is the origin server, or is |
532 |
already part of a metering subtree (rooted at the origin server), |
533 |
then it may complete the negotiation, thereby extending the metering |
534 |
subtree to include the new proxy. |
535 |
|
536 |
To start the negotiation, a proxy sends its request with one of the |
537 |
following Meter directives: |
538 |
|
539 |
will-report-and-limit |
540 |
indicates that the proxy is willing and able to |
541 |
return usage reports and will obey any usage-limits. |
542 |
|
543 |
wont-report indicates that the proxy will obey usage-limits but |
544 |
will not send usage reports. |
545 |
|
546 |
wont-limit indicates that the proxy will not obey usage-limits |
547 |
but will send usage reports. |
548 |
|
549 |
A proxy willing to neither obey usage-limits nor send usage reports |
550 |
MUST NOT transmit a Meter header in the request. |
551 |
|
552 |
By definition, an empty Meter header: |
553 |
|
554 |
Meter: |
555 |
|
556 |
is equivalent to "Meter: will-report-and-limit", and so, by the |
557 |
definition of the Connection header (see section 14.10 of the |
558 |
HTTP/1.1 specification [3]), a request that contains |
559 |
|
560 |
Connection: Meter |
561 |
|
562 |
and no explicit Meter header is equivalent to a request that contains |
563 |
|
564 |
Connection: Meter |
565 |
Meter: will-report-and-limit |
566 |
|
567 |
This makes the default case more efficient. |
568 |
|
569 |
An origin server that is not interested in metering or usage-limiting |
570 |
the requested resource simply ignores the Meter header. |
571 |
Mogul, Leach [Page 10] |
572 |
|
573 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
574 |
|
575 |
|
576 |
If the server wants the proxy to do hit-metering and/or |
577 |
usage-limiting, its response should include one or more of the |
578 |
following Meter directives: |
579 |
|
580 |
For hit-metering: |
581 |
|
582 |
do-report specifies that the proxy MUST send usage reports to |
583 |
the server. |
584 |
|
585 |
dont-report specifies that the proxy SHOULD NOT send usage |
586 |
reports to the server. |
587 |
|
588 |
timeout=NNN sets a metering timeout of NNN minutes, from the time |
589 |
that this response was originated, for the reporting |
590 |
of a hit-count. If the proxy has a non-zero hit |
591 |
count for this response when the timeout expires, it |
592 |
MUST send a report to the server at or before that |
593 |
time. Implies "do-report". |
594 |
|
595 |
By definition, an empty Meter header in a response, or any Meter |
596 |
header that does not contain "dont-report", means "Meter: do-report"; |
597 |
this makes a common case more efficient. |
598 |
|
599 |
--------- |
600 |
Note: an origin server using the metering timeout mechanism to |
601 |
bound the collection period over which hit-counts are obtained |
602 |
should adjust the timeout values in the responses it sends so |
603 |
that all responses generated within that period reach their |
604 |
metering timeouts at or before the end of that period. |
605 |
|
606 |
If the origin server simply sends a constant metering timeout T |
607 |
with each response for a resource, the reports that it receives |
608 |
will reflect activity over a period whose duration is between T |
609 |
and N*T (in the worst case), where N is the maximum depth of |
610 |
the metering subtree. |
611 |
--------- |
612 |
|
613 |
For usage-limiting |
614 |
|
615 |
max-uses=NNN sets an upper limit of NNN "uses" of the response, |
616 |
not counting its immediate forwarding to the |
617 |
requesting end-client, for all proxies in the |
618 |
following subtree taken together. |
619 |
|
620 |
max-reuses=NNN sets an upper limit of NNN "reuses" of the response |
621 |
for all proxies in the following subtree taken |
622 |
together. |
623 |
|
624 |
When a proxy has exhausted its allocation of "uses" or "reuses" for a |
625 |
cache entry, it MUST revalidate the cache entry (using a conditional |
626 |
request) before returning it in a response. (The proxy SHOULD use |
627 |
|
628 |
Mogul, Leach [Page 11] |
629 |
|
630 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
631 |
|
632 |
|
633 |
this revalidation message to send a usage report, if one was |
634 |
requested and it is time to send it. See sections 3.4 and 3.5.) |
635 |
|
636 |
These Meter response-directives apply only to the specific response |
637 |
that they are attached to. |
638 |
|
639 |
--------- |
640 |
Note that the limit on "uses" set by the max-uses directive |
641 |
does not include the use of the response to satisfy the |
642 |
end-client request that caused the proxy's request to the |
643 |
server. This counting rule supports the notion of a |
644 |
cache-initiated prefetch: a cache may issue a prefetch request, |
645 |
receive a max-uses=0 response, store that response, and then |
646 |
return that response (without revalidation) when a client makes |
647 |
an actual request for the resource. However, each such |
648 |
response may be used at most once in this way, so the origin |
649 |
server maintains precise control over the number of actual |
650 |
uses. |
651 |
--------- |
652 |
|
653 |
A server MUST NOT send a Meter header that would require a proxy to |
654 |
do something that it has not yet offered to do. A proxy receiving a |
655 |
Meter response-directive asking the proxy to do something it did not |
656 |
volunteer to do SHOULD ignore that directive. |
657 |
|
658 |
A proxy receiving a Meter header in a response MUST either obey it, |
659 |
or it MUST revalidate the corresponding cache entry on every access. |
660 |
(I.e., if it chooses not to obey the Meter header in a response, it |
661 |
MUST act as if the response included "Cache-control: s-maxage=0".) |
662 |
|
663 |
--------- |
664 |
Note: a proxy that has not sent the Meter header in a request |
665 |
for the given resource, and which has therefore not volunteered |
666 |
to honor Meter directives in a response, is not required to |
667 |
honor them. If, in this situation, the server does send a |
668 |
Meter header in a response, this is a protocol error. However, |
669 |
based on the robustness principle, the proxy may choose to |
670 |
interpret the Meter header as an implicit request to include |
671 |
"Cache-control: s-maxage=0" when it forwards the response, |
672 |
since this preserves the apparent intention of the server. |
673 |
--------- |
674 |
|
675 |
A proxy that receives the Meter header in a request may ignore it |
676 |
only to the extent that this is consistent with its own duty to the |
677 |
next-hop server. If the received Meter request header is |
678 |
inconsistent with that duty, or if no Meter request header is |
679 |
received and the response from the next-hop server requests any form |
680 |
of metering or limiting, then the proxy MUST add "Cache-control: |
681 |
s-maxage=0" to any response it forwards for that request. (A proxy |
682 |
SHOULD NOT add or change the Expires header or max-age Cache-control |
683 |
directive.) |
684 |
|
685 |
Mogul, Leach [Page 12] |
686 |
|
687 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
688 |
|
689 |
|
690 |
--------- |
691 |
For example, if proxy A receives a GET request from proxy B for |
692 |
URL X with "Connection: Meter", but proxy A's cached response |
693 |
for URL does not include any Meter directives, then proxy A may |
694 |
ignore the metering offer from proxy B. |
695 |
|
696 |
However, if proxy A has previously told the origin server |
697 |
"Meter: wont-limit" (implying will-report), and the cached |
698 |
response contains "Meter: do-report", and proxy B's request |
699 |
includes "Meter: wont-report", then proxy B's offer is |
700 |
inconsistent with proxy A's duty to the origin server. |
701 |
Therefore, in this case proxy A must add "Cache-control: |
702 |
s-maxage=0" when it returns the cached response to proxy B, and |
703 |
must not include a Meter header in this response. |
704 |
--------- |
705 |
|
706 |
If a server does not want to use the Meter mechanism, and will not |
707 |
want to use it any time soon, it may send this directive: |
708 |
|
709 |
wont-ask recommends that the proxy SHOULD NOT send any Meter |
710 |
directives to this server. |
711 |
|
712 |
The proxy SHOULD remember this fact for up to 24 hours. This avoids |
713 |
virtually all unnecessary overheads for servers that do not wish to |
714 |
use or support the Meter header. (This directive also implies |
715 |
``dont-report''.) |
716 |
|
717 |
3.4 Transmission of usage reports |
718 |
To transmit a usage report, a proxy sends the following Meter header |
719 |
in a request on the appropriate resource: |
720 |
|
721 |
Meter: count=NNN/MMM |
722 |
|
723 |
The first integer indicates the count of uses of the cache entry |
724 |
since the last report; the second integer indicates the count of |
725 |
reuses of the entry (see section 5.3 for rules on counting uses and |
726 |
reuses). The transmission of a "count" directive in a request with |
727 |
no other Meter directive is also defined as an implicit transmission |
728 |
of a "will-report-and-limit" directive, to optimize the common case. |
729 |
(A proxy not willing to honor usage-limits would send "Meter: |
730 |
count=NNN/MMM, wont-limit" for its reports.) |
731 |
|
732 |
Note that when a proxy forwards a client's request and receives a |
733 |
response, the response that the proxy sends immediately to the |
734 |
requesting client is not counted as a "use". I.e., the reported |
735 |
count is the number of times the cache entry was used, and not the |
736 |
number of times that the response was used. |
737 |
|
738 |
A proxy SHOULD NOT transmit "Meter: count=0/0", since this conveys no |
739 |
useful information. |
740 |
|
741 |
|
742 |
Mogul, Leach [Page 13] |
743 |
|
744 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
745 |
|
746 |
|
747 |
Usage reports MUST always be transmitted as part of a conditional |
748 |
request (such as a GET or HEAD), since the information in the |
749 |
conditional header (e.g., If-Modified-Since or If-None-Match) is |
750 |
required for the origin server to know which instance of a resource |
751 |
is being counted. Proxys forwarding usage reports up the metering |
752 |
subtree MUST NOT change the contents of the conditional header, since |
753 |
otherwise this would result in incorrect counting. |
754 |
|
755 |
A usage report MUST NOT be transmitted as part of a forwarded request |
756 |
that includes multiple entity tags in an If-None-Match or If-Match |
757 |
header. |
758 |
|
759 |
--------- |
760 |
Note: a proxy that offers its willingness to do hit-metering |
761 |
(report usage) must count both uses and reuses. It is not |
762 |
possible to negotiate the reporting of one but not the other. |
763 |
--------- |
764 |
|
765 |
3.5 When to send usage reports |
766 |
A proxy that has offered to send usage reports to its parent in the |
767 |
metering subtree MUST send a usage report in each of these |
768 |
situations: |
769 |
|
770 |
1. When it forwards a conditional GET on the resource |
771 |
instance on behalf of one of its clients (if the GET is |
772 |
conditional on at most one entity-tag). |
773 |
|
774 |
2. When it forwards a conditional HEAD on the resource |
775 |
instance on behalf of one of its clients. |
776 |
|
777 |
3. When it must generate a conditional GET to satisfy a |
778 |
client request because the max-uses limit has been |
779 |
exceeded. |
780 |
|
781 |
4. Upon expiration of a metering timeout associated with a |
782 |
cache entry that has a non-zero hit-count. |
783 |
|
784 |
5. When it removes the corresponding non-zero hit-count entry |
785 |
from its storage for any reason including: |
786 |
|
787 |
- the proxy needs the storage space for another |
788 |
hit-count entry. |
789 |
|
790 |
- the proxy is not able to store more than one response |
791 |
per resource, and a request forwarded on behalf of a |
792 |
client has resulted in the receipt of a new response |
793 |
(one with a different entity-tag or last-modified |
794 |
time). |
795 |
|
796 |
Note that a cache might continue to store hit-count |
797 |
information even after having deleted the body of the |
798 |
|
799 |
Mogul, Leach [Page 14] |
800 |
|
801 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
802 |
|
803 |
|
804 |
response, so it is not necessary to report the hit-count |
805 |
when deleting the body; it is only necessary to report it |
806 |
if the proxy is about to "forget" a non-zero value. |
807 |
|
808 |
(Section 5.3 explains how hit-counts become zero or non-zero.) |
809 |
|
810 |
If the usage report is being sent because the proxy is about to |
811 |
remove the hit-count entry from its storage, or because of an expired |
812 |
metering timeout: |
813 |
|
814 |
- The proxy MUST send the report as part of a conditional |
815 |
HEAD request on the resource instance. |
816 |
|
817 |
- The proxy is not required to retry the HEAD request if it |
818 |
fails (this is a best-efforts design). To improve |
819 |
accuracy, however, the proxy SHOULD retry failed HEAD |
820 |
requests, subject to resource constraints. |
821 |
|
822 |
- The proxy is not required to serialize any other operation |
823 |
on the completion of this request. |
824 |
|
825 |
--------- |
826 |
Note: proxy implementors are strongly encouraged to batch |
827 |
several HEAD-based reports to the same server, when possible, |
828 |
over a single persistent connection, to reduce network overhead |
829 |
as much as possible. This may involve a non-naive algorithm |
830 |
for scheduling the deletion of hit-count entries. |
831 |
--------- |
832 |
|
833 |
If the usage count is sent because of an arriving request that also |
834 |
carries a "count" directive, the proxy MUST combine its own (possibly |
835 |
zero) use and reuse counts with the arriving counts, and then attempt |
836 |
to forward the request. |
837 |
|
838 |
However, the proxy is not required to forward an arriving request |
839 |
with a "count" directive, provided that: |
840 |
|
841 |
- it can reply to the request using a cached response, in |
842 |
compliance with other requirements of the HTTP |
843 |
specification. |
844 |
|
845 |
- such a response does not exceed a max-uses limit. |
846 |
|
847 |
- it is not required to forward the request because of an |
848 |
expired metering timeout. |
849 |
|
850 |
If an arriving request carries a "count" directive, and the proxy no |
851 |
longer has a cache entry for the resource, the proxy MUST forward the |
852 |
"count" directive. (This is, in any case, what a proxy without a |
853 |
suitable cache entry would normally do for any valid request it |
854 |
receives.) |
855 |
|
856 |
Mogul, Leach [Page 15] |
857 |
|
858 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
859 |
|
860 |
|
861 |
3.6 Subdivision of usage-limits |
862 |
When an origin server specifies a usage limit, a proxy in the |
863 |
metering subtree may subdivide this limit among its children in the |
864 |
subtree as it sees fit. |
865 |
|
866 |
For example, consider the situation with two proxies P1 and P2, each |
867 |
of which uses proxy P3 as a way to reach origin server S. Imagine |
868 |
that S sends P3 a response with |
869 |
|
870 |
Meter: max-uses=10 |
871 |
|
872 |
The proxies use that response to satisfy the current requesting |
873 |
end-client. The max-uses directive in this example allows the |
874 |
combination of P1, P2, and P3 together to satisfy 10 additional |
875 |
end-client uses (unconditional GETs) for the resource. |
876 |
|
877 |
This specification does not constrain how P3 divides up that |
878 |
allocation among itself and the other proxies. For example, P3 could |
879 |
retain all of max-use allocation for itself. In that case, it would |
880 |
forward the response to P1 and/or P2 with |
881 |
|
882 |
Meter: max-uses=0 |
883 |
|
884 |
P3 might also divide the allocation equally among P1 and P2, |
885 |
retaining none for itself (which may be the right choice if P3 has |
886 |
few or no other clients). In this case, it could send |
887 |
|
888 |
Meter: max-uses=5 |
889 |
|
890 |
to the proxy (P1 or P2) that made the initial request, and then |
891 |
record in some internal data structure that it "owes" the other proxy |
892 |
the rest of the allocation. |
893 |
|
894 |
Note that this freedom to choose the max-uses value applies to the |
895 |
origin server, as well. There is no requirement that an origin |
896 |
server send the same max-uses value to all caches. For example, it |
897 |
might make sense to send "max-uses=2" the first time one hears from a |
898 |
cache, and then double the value (up to some maximum limit) each time |
899 |
one gets a "use-count" from that cache. The idea is that the faster |
900 |
a cache is using up its max-use quota, the more likely it will be to |
901 |
report a use-count value before removing the cache entry. Also, high |
902 |
and frequent use-counts imply a corresponding high efficiency benefit |
903 |
from allowing caching. |
904 |
|
905 |
Again, the details of such heuristics would be outside the scope of |
906 |
this specification. |
907 |
|
908 |
|
909 |
|
910 |
|
911 |
|
912 |
|
913 |
Mogul, Leach [Page 16] |
914 |
|
915 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
916 |
|
917 |
|
918 |
4 Analysis |
919 |
|
920 |
This section includes informal analyses of several aspects of |
921 |
hit-metering: |
922 |
|
923 |
1. the accuracy of results when applied to counting users |
924 |
(section 4.1). |
925 |
|
926 |
2. the problem of counting users whose browsers do not |
927 |
include caches, such as Network Computers (section 4.2). |
928 |
|
929 |
3. delays imposed on ``critical paths'' for HTTP operations |
930 |
(section 4.3). |
931 |
|
932 |
4.1 Approximation accuracy for counting users |
933 |
For many (but not all) service operators, the single most important |
934 |
aspect of the request stream is the number of distinct users who have |
935 |
retrieved a particular entity within a given period (e.g., during a |
936 |
given day). The hit-metering mechanism is designed to provide an |
937 |
origin server with an approximation of the number of users that |
938 |
reference a given resource. The intent of the design is that the |
939 |
precision of this approximation is consistent with the goals of |
940 |
simplicity and optional implementation. |
941 |
|
942 |
Almost all Web users use client software that maintains local caches, |
943 |
and the state of the art of local-caching technology is quite |
944 |
effective. (Section 4.2 discusses the case where end-client caches |
945 |
are small or non-existent.) Therefore, assuming an effective and |
946 |
persistent end-client cache, each individual user who retrieves an |
947 |
entity does exactly one GET request that results in a 200 or 203 |
948 |
response, or a 206 response that includes the first byte of the |
949 |
entity. If a proxy cache maintains and reports an accurate use-count |
950 |
of such retrievals, then its reported use-count will closely |
951 |
approximate the number of distinct users who have retrieved the |
952 |
entity. |
953 |
|
954 |
There are some circumstances under which this approximation can break |
955 |
down. For example, if an entity stays in a proxy cache for much |
956 |
longer than it persists in the typical client cache, and users often |
957 |
re-reference the entity, then this scheme will tend to over-count the |
958 |
number of users. Or, if the cache-management policy implemented in |
959 |
typical client caches is biased against retaining certain kinds of |
960 |
frequently re-referenced entities (such as very large images), the |
961 |
use-counts reported will tend to overestimate the user-counts for |
962 |
such entities. |
963 |
|
964 |
Browser log analysis has shown that when a user revisits a resource, |
965 |
this is almost always done very soon after the previous visit, almost |
966 |
always with fewer than eight intervening references [10]. Although |
967 |
this result might not apply universally, it implies that almost all |
968 |
reuses will hit in the end-client cache, and will not be seen as |
969 |
unconditional GETs by a proxy cache. |
970 |
Mogul, Leach [Page 17] |
971 |
|
972 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
973 |
|
974 |
|
975 |
The existing (HTTP/1.0) "cache-busting" mechanisms for counting |
976 |
distinct users will certainly overestimate the number of users behind |
977 |
a proxy, since it provides no reliable way to distinguish between a |
978 |
user's initial request and subsequent repeat requests that might have |
979 |
been conditional GETs, had not cache-busting been employed. The |
980 |
"Cache-control: s-maxage=0" feature of HTTP/1.1 does allow the |
981 |
separation of use-counts and reuse-counts, provided that no HTTP/1.0 |
982 |
proxy caches intervene. |
983 |
|
984 |
Note that if there is doubt about the validity of the results of |
985 |
hit-metering a given set of resources, the server can employ |
986 |
cache-busting techniques for short periods, to establish a baseline |
987 |
for validating the hit-metering results. Various approaches to this |
988 |
problem are discussed in a paper by James Pitkow [8]. |
989 |
|
990 |
4.2 What about "Network Computers"? |
991 |
The analysis in section 4.1 assumed that "almost all Web users" have |
992 |
client caches. If the Network Computers (NC) model becomes popular, |
993 |
however, then this assumption may be faulty: most proposed NCs have |
994 |
no disk storage, and relatively little RAM. Many Personal Digital |
995 |
Assistants (PDAs), which sometimes have network access, have similar |
996 |
constraints. Such client systems may do little or no caching of HTTP |
997 |
responses. This means that a single user might well generate many |
998 |
unconditional GETs that yield the same response from a proxy cache. |
999 |
|
1000 |
First note that the hit-metering design in this document, even with |
1001 |
such clients, provides an approximation no worse than available with |
1002 |
unmodified HTTP/1.1: the counts that a proxy would return to an |
1003 |
origin server would represent exactly the number of requests that the |
1004 |
proxy would forward to the server, if the server simply specifies |
1005 |
"Cache-control: s-maxage=0". |
1006 |
|
1007 |
However, it may be possible to improve the accuracy of these |
1008 |
hit-counts by use of some heuristics at the proxy. For example, the |
1009 |
proxy might note the IP address of the client, and count only one GET |
1010 |
per client address per response. This is not perfect: for example, |
1011 |
it fails to distinguish between NCs and certain other kinds of hosts. |
1012 |
The proxy might also use the heuristic that only those clients that |
1013 |
never send a conditional GET should be treated this way, although we |
1014 |
are not at all certain that NCs will never send conditional GETs. |
1015 |
|
1016 |
Since the solution to this problem appears to require heuristics |
1017 |
based on the actual behavior of NCs (or perhaps a new HTTP protocol |
1018 |
feature that allows unambiguous detection of cacheless clients), it |
1019 |
appears to be premature to specify a solution. |
1020 |
|
1021 |
4.3 Critical-path delay analysis |
1022 |
In systems (such as the Web) where latency is at issue, there is |
1023 |
usually a tree of steps which depend on one another, in such a way |
1024 |
that the final result cannot be accomplished until all of its |
1025 |
predecessors have been. Since the tree structure admits some |
1026 |
|
1027 |
Mogul, Leach [Page 18] |
1028 |
|
1029 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1030 |
|
1031 |
|
1032 |
parallelism, it is not necessary to add up the timings for each step |
1033 |
to discover the latency for the entire process. But any single path |
1034 |
through this dependency tree cannot be parallelized, and the longest |
1035 |
such path is the one whose length (in units of seconds) determines |
1036 |
the overall latency. This is the ``critical path,'' because no |
1037 |
matter how much shorter one makes any other path, that cannot change |
1038 |
the overall latency for the final result. |
1039 |
|
1040 |
If one views the final result, for a Web request, as rendering a page |
1041 |
at a browser, or otherwise acting on the result of a request, clearly |
1042 |
some network round trips (e.g., exchanging TCP SYN packets if the |
1043 |
connection doesn't already exist) are on the critical path. This |
1044 |
hit-metering design does add some round-trips for reporting non-zero |
1045 |
counts when a cache entry is removed, but, by design, these are off |
1046 |
any critical path: they may be done in parallel with any other |
1047 |
operation, and require only ``best efforts,'' so a proxy does not |
1048 |
have to serialize other operations with their success or failure. |
1049 |
|
1050 |
Clearly, anything that changes network utilization (either increasing |
1051 |
or decreasing it) can indirectly affect user-perceived latency. Our |
1052 |
expectation is that hit-metering, on average, will reduce loading and |
1053 |
so even its indirect effects should not add network round-trips in |
1054 |
any critical path. But there might be a few specific instances where |
1055 |
the added non-critical-path operations (specifically, usage reports |
1056 |
upon cache-entry removal) delay an operation on a critical path. |
1057 |
This is an unavoidable problem in datagram networks. |
1058 |
|
1059 |
|
1060 |
5 Specification |
1061 |
|
1062 |
5.1 Specification of Meter header and directives |
1063 |
The Meter general-header field is used to: |
1064 |
|
1065 |
- Negotiate the use of hit-metering and usage-limiting among |
1066 |
origin servers and proxy caches. |
1067 |
|
1068 |
- Report use counts and reuse counts. |
1069 |
|
1070 |
Implementation of the Meter header is optional for both proxies and |
1071 |
origin servers. However, any proxy that transmits the Meter header |
1072 |
in a request MUST implement every requirement of this specification, |
1073 |
without exception or amendment. |
1074 |
|
1075 |
The Meter header MUST always be protected by a Connection header. A |
1076 |
proxy that does not implement the Meter header MUST NOT pass it |
1077 |
through to another system (see section 5.5 for how a non-caching |
1078 |
proxy may comply with this specification). If a Meter header is |
1079 |
received in a message whose version is less than HTTP/1.1, it MUST be |
1080 |
ignored (because it has clearly flowed through a proxy that does not |
1081 |
implement Meter). |
1082 |
|
1083 |
|
1084 |
Mogul, Leach [Page 19] |
1085 |
|
1086 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1087 |
|
1088 |
|
1089 |
A proxy that has received a response with a version less than |
1090 |
HTTP/1.1, and therefore from a server (or another proxy) that does |
1091 |
not implement the Meter header, SHOULD NOT send Meter request |
1092 |
directives to that server, because these would simply waste |
1093 |
bandwidth. This recommendation does not apply if the proxy is |
1094 |
currently hit-metering or usage-limiting any responses from that |
1095 |
server. If the proxy receives a HTTP/1.1 or higher response from |
1096 |
such a server, it should cease its suppression of the Meter |
1097 |
directives. |
1098 |
|
1099 |
All proxies sending the Meter header MUST adhere to the "metering |
1100 |
subtree" design described in section 3. |
1101 |
|
1102 |
Meter = "Meter" ":" 0#meter-directive |
1103 |
|
1104 |
meter-directive = meter-request-directive |
1105 |
| meter-response-directive |
1106 |
| meter-report-directive |
1107 |
|
1108 |
meter-request-directive = |
1109 |
"will-report-and-limit" |
1110 |
| "wont-report" |
1111 |
| "wont-limit" |
1112 |
|
1113 |
meter-report-directive = |
1114 |
| "count" "=" 1*DIGIT "/" 1*DIGIT |
1115 |
|
1116 |
meter-response-directive = |
1117 |
"max-uses" "=" 1*DIGIT |
1118 |
| "max-reuses" "=" 1*DIGIT |
1119 |
| "do-report" |
1120 |
| "dont-report" |
1121 |
| "timeout" "=" 1*DIGIT |
1122 |
| "wont-ask" |
1123 |
|
1124 |
|
1125 |
A meter-request-directive or meter-report-directive may only appear |
1126 |
in an HTTP request message. A meter-response-directive may only |
1127 |
appear in an HTTP response directive. |
1128 |
|
1129 |
An empty Meter header in a request means "Meter: |
1130 |
will-report-and-limit". An empty Meter header in a response, or any |
1131 |
other response including one or more Meter headers without the |
1132 |
"dont-report" or "wont-ask" directive, implies "Meter: do-report". |
1133 |
|
1134 |
The meaning of the meter-request-directives are as follows: |
1135 |
|
1136 |
will-report-and-limit |
1137 |
indicates that the proxy is willing and able to |
1138 |
return usage reports and will obey any usage-limits. |
1139 |
|
1140 |
|
1141 |
Mogul, Leach [Page 20] |
1142 |
|
1143 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1144 |
|
1145 |
|
1146 |
wont-report indicates that the proxy will obey usage-limits but |
1147 |
will not send usage reports. |
1148 |
|
1149 |
wont-limit indicates that the proxy will not obey usage-limits |
1150 |
but will send usage reports. |
1151 |
|
1152 |
A proxy willing neither to obey usage-limits nor to send usage |
1153 |
reports MUST NOT transmit a Meter header in the request. |
1154 |
|
1155 |
The meaning of the meter-report-directives are as follows: |
1156 |
|
1157 |
count "=" 1*DIGIT "/" 1*DIGIT |
1158 |
Both digit strings encode decimal integers. The |
1159 |
first integer indicates the count of uses of the |
1160 |
cache entry since the last report; the second integer |
1161 |
indicates the count of reuses of the entry. |
1162 |
|
1163 |
Section 5.3 specifies the counting rules. |
1164 |
|
1165 |
The meaning of the meter-response-directives are as follows: |
1166 |
|
1167 |
max-uses "=" 1*DIGIT |
1168 |
sets an upper limit on the number of "uses" of the |
1169 |
response, not counting its immediate forwarding to |
1170 |
the requesting end-client, for all proxies in the |
1171 |
following subtree taken together. |
1172 |
|
1173 |
max-reuses "=" 1*DIGIT |
1174 |
sets an upper limit on the number of "reuses" of the |
1175 |
response for all proxies in the following subtree |
1176 |
taken together. |
1177 |
|
1178 |
do-report specifies that the proxy MUST send usage reports to |
1179 |
the server. |
1180 |
|
1181 |
dont-report specifies that the proxy SHOULD NOT send usage |
1182 |
reports to the server. |
1183 |
|
1184 |
timeout "=" 1*DIGIT |
1185 |
sets a metering timeout of the specified number of |
1186 |
minutes (not seconds) after the origination of this |
1187 |
response (as indicated by its "Date" header). If the |
1188 |
proxy has a non-zero hit count for this response when |
1189 |
the timeout expires, it MUST send a report to the |
1190 |
server at or before that time. Timeouts should be |
1191 |
implemented with an accuracy of plus or minus one |
1192 |
minute. Implies "do-report". |
1193 |
|
1194 |
wont-ask specifies that the proxy SHOULD NOT send any Meter |
1195 |
headers to the server. The proxy should forget this |
1196 |
advice after a period of no more than 24 hours. |
1197 |
|
1198 |
Mogul, Leach [Page 21] |
1199 |
|
1200 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1201 |
|
1202 |
|
1203 |
Section 5.3 specifies the counting rules, and in particular specifies |
1204 |
a somewhat non-obvious interpretation of the max-uses value. |
1205 |
|
1206 |
5.2 Abbreviations for Meter directives |
1207 |
To allow for the most efficient possible encoding of Meter headers, |
1208 |
we define abbreviated forms of all Meter directives. These are |
1209 |
exactly semantically equivalent to their non-abbreviated |
1210 |
counterparts. All systems implementing the Meter header MUST |
1211 |
implement both the abbreviated and non-abbreviated forms. |
1212 |
Implementations SHOULD use the abbreviated forms in normal use. |
1213 |
|
1214 |
The abbreviated forms of Meter directive are shown below, with the |
1215 |
corresponding non-abbreviated literals in the comments: |
1216 |
|
1217 |
Abb-Meter = "Meter" ":" 0#abb-meter-directive |
1218 |
|
1219 |
abb-meter-directive = abb-meter-request-directive |
1220 |
| abb-meter-response-directive |
1221 |
| abb-meter-report-directive |
1222 |
|
1223 |
abb-meter-request-directive = |
1224 |
"w" ; "will-report-and-limit" |
1225 |
| "x" ; "wont-report" |
1226 |
| "y" ; "wont-limit" |
1227 |
|
1228 |
abb-meter-report-directive = |
1229 |
| "c" "=" 1*DIGIT "/" 1*DIGIT ; "count" |
1230 |
|
1231 |
abb-meter-response-directive = |
1232 |
"u" "=" 1*DIGIT ; "max-uses" |
1233 |
| "r" "=" 1*DIGIT ; "max-reuses" |
1234 |
| "d" ; "do-report" |
1235 |
| "e" ; "dont-report" |
1236 |
| "t" "=" 1*DIGIT ; "timeout" |
1237 |
| "n" ; "wont-ask" |
1238 |
|
1239 |
|
1240 |
--------- |
1241 |
Note: although the Abb-Meter BNF rule is defined separately |
1242 |
from the Meter rule, one may freely mix abbreviated and |
1243 |
non-abbreviated Meter directives in the same header. |
1244 |
--------- |
1245 |
|
1246 |
5.3 Counting rules |
1247 |
|
1248 |
--------- |
1249 |
Note: please remember that hit-counts and usage-counts are |
1250 |
associated with individual responses, not with resources. A |
1251 |
cache entry that, over its lifetime, holds more than one |
1252 |
response is also not a "response", in this particular sense. |
1253 |
--------- |
1254 |
|
1255 |
Mogul, Leach [Page 22] |
1256 |
|
1257 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1258 |
|
1259 |
|
1260 |
Let R be a cached response, and V be the value of the Request-URI and |
1261 |
selecting request-headers (if any, see section 14.43 of the HTTP/1.1 |
1262 |
specification [3]) that would select R if contained in a request. We |
1263 |
define a "use" of R as occurring when the proxy returns its stored |
1264 |
copy of R in a response with any of the following status codes: a 200 |
1265 |
(OK) status; a 203 (Non-Authoritative Information) status; or a 206 |
1266 |
(Partial Content) status when the response contains byte #0 of the |
1267 |
entity (see section 5.4 for a discussion of Range requests). |
1268 |
|
1269 |
--------- |
1270 |
Note: when a proxy forwards a client's request and receives a |
1271 |
response, the response that the proxy sends immediately to the |
1272 |
requesting client is not counted as a "use". I.e., the |
1273 |
reported count is the number of times the cache entry was used, |
1274 |
and not the number of times that the response was used. |
1275 |
--------- |
1276 |
|
1277 |
We define a "reuse" of R as as occurring when the proxy responds to a |
1278 |
request selecting R with a 304 (Not Modified) status, unless that |
1279 |
request is a Range request that does not specify byte #0 of the |
1280 |
entity. |
1281 |
|
1282 |
5.3.1 Counting rules for hit-metering |
1283 |
A proxy participating in hit-metering for a cache response R |
1284 |
maintains two counters, CU and CR, associated with R. When a proxy |
1285 |
first stores R in its cache, it sets both CU and CR to 0 (zero). |
1286 |
When a subsequent client request results in a "use" of R, the proxy |
1287 |
increments CU. When a subsequent client request results in a "reuse" |
1288 |
of R, the proxy increments CR. When a subsequent client request |
1289 |
selecting R (i.e., including V) includes a "count" Meter directive, |
1290 |
the proxy increments CU and CR using the corresponding values in the |
1291 |
directive. |
1292 |
|
1293 |
When the proxy sends a request selecting R (i.e., including V) to the |
1294 |
inbound server, it includes a "count" Meter directive with the |
1295 |
current CU and CR as the parameter values. If this request was |
1296 |
caused by the proxy's receipt of a request from a client, upon |
1297 |
receipt of the server's response, the proxy sets CU and CR to the |
1298 |
number of uses and reuses, respectively, that may have occurred while |
1299 |
the request was in progress. (These numbers are likely, but not |
1300 |
certain, to be zero.) If the proxy's request was a final HEAD-based |
1301 |
report, it need no longer maintain the CU and CR values, but it may |
1302 |
also set them to the number of intervening uses and reuses and retain |
1303 |
them. |
1304 |
|
1305 |
5.3.2 Counting rules for usage-limiting |
1306 |
A proxy participating in usage-limiting for a response R maintains |
1307 |
either or both of two counters TU and TR, as appropriate, for that |
1308 |
resource. TU and TR are incremented in just the same way as CU and |
1309 |
CR, respectively. However, TU is zeroed only upon receipt of a |
1310 |
"max-uses" Meter directive for that response (including the initial |
1311 |
|
1312 |
Mogul, Leach [Page 23] |
1313 |
|
1314 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1315 |
|
1316 |
|
1317 |
receipt). Similarly, TR is zeroed only upon receipt of a |
1318 |
"max-reuses" Meter directive for that response. |
1319 |
|
1320 |
A proxy participating in usage-limiting for a response R also stores |
1321 |
values MU and/or MR associated with R. When it receives a response |
1322 |
including only a max-uses value, it sets MU to that value and MR to |
1323 |
infinity. When it receives a response including only a max-reuses |
1324 |
value, it sets MR to that value and MU to infinity. When it receives |
1325 |
a response including both max-reuses and max-reuses values, it sets |
1326 |
MU and MR to those values, respectively. When it receives a |
1327 |
subsequent response including neither max-reuses nor max-reuses |
1328 |
values, it sets both MU and MR to infinity. |
1329 |
|
1330 |
If a proxy participating in usage-limiting for a response R receives |
1331 |
a request that would cause a "use" of R, and TU >= MU, it MUST |
1332 |
forward the request to the server. If it receives a request that |
1333 |
would cause a "reuse" of R, and TR >= MR, it MUST forward the request |
1334 |
to the server. If (in either case) the proxy has already forwarded a |
1335 |
previous request to the server and is waiting for the response, it |
1336 |
should delay further handling of the new request until the response |
1337 |
arrives (or times out); it SHOULD NOT have two revalidation requests |
1338 |
pending at once that select the same response, unless these are Range |
1339 |
requests selecting different subranges. |
1340 |
|
1341 |
There is a special case of this rule for the "max-uses" directive: if |
1342 |
the proxy receives a response with "max-uses=0" and does not forward |
1343 |
it to a requesting client, the proxy should set a flag PF associated |
1344 |
with R. If R is true, then when a request arrives while if TU >= MU, |
1345 |
if the PF flag is set, then the request need not be forwarded to the |
1346 |
server (provided that this is not required by other caching rules). |
1347 |
However, the PF flag MUST be cleared on any use of the response. |
1348 |
|
1349 |
--------- |
1350 |
Note: the "PF" flag is so named because this feature is useful |
1351 |
only for caches that could issue a "prefetch" request before an |
1352 |
actual client request for the response. A proxy not |
1353 |
implementing prefetching need not implement the PF flag. |
1354 |
--------- |
1355 |
|
1356 |
5.3.3 Equivalent algorithms are allowed |
1357 |
Any other algorithm that exhibits the same external behavior (i.e., |
1358 |
generates exactly the same requests from the proxy to the server) as |
1359 |
the one in this section is explicitly allowed. |
1360 |
|
1361 |
--------- |
1362 |
Note: in most cases, TU will be equal to CU, and TR will be |
1363 |
equal to CR. The only two cases where they could differ are: |
1364 |
|
1365 |
1. The proxy issues a non-conditional request for the |
1366 |
resource using V, while TU and/or TR are non-zero, and |
1367 |
the server's response includes a new "max-uses" and/or |
1368 |
|
1369 |
Mogul, Leach [Page 24] |
1370 |
|
1371 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1372 |
|
1373 |
|
1374 |
"max-reuses" directive (thus zeroing TU and/or TR, but |
1375 |
not CU and CR). |
1376 |
|
1377 |
2. The proxy issues a conditional request reporting the |
1378 |
hit-counts (and thus zeroing CU and CR, but not TU or |
1379 |
TR), but the server's response does not include a new |
1380 |
"max-uses" and/or "max-reuses" directive. |
1381 |
|
1382 |
To solve the first case, the proxy has several implementation |
1383 |
options |
1384 |
|
1385 |
- Always store TU and TR separately from CU and CR. |
1386 |
|
1387 |
- Create "shadow" copies of TU and TR when this situation |
1388 |
arises (analogous to "copy on write"). |
1389 |
|
1390 |
- Generate a HEAD-based usage report when the |
1391 |
non-conditional request is sent (or when the |
1392 |
"max-uses=0" is received), causing CU and CR to be |
1393 |
zeroed (analogous in some ways to a "memory barrier" |
1394 |
instruction). |
1395 |
|
1396 |
In the second case, the server implicitly has removed the |
1397 |
usage-limit(s) on the response (by setting MU and/or MR to |
1398 |
infinity), and so the fact that, say, TU is different from CU |
1399 |
is not significant. |
1400 |
--------- |
1401 |
|
1402 |
--------- |
1403 |
Note: It may also be possible to eliminate the PF flag by |
1404 |
sending extra HEAD-based usage-report requests, but we |
1405 |
recommend against this; it is better to allocate an extra bit |
1406 |
per entry than to transmit extra requests. |
1407 |
--------- |
1408 |
|
1409 |
5.4 Counting rules: interaction with Range requests |
1410 |
HTTP/1.1 allows a client to request sub-ranges of a resource. A |
1411 |
client might end up issuing several requests with the net effect of |
1412 |
receiving one copy of the resource. For uniformity of the results |
1413 |
seen by origin servers, proxies need to observe a rule for counting |
1414 |
these references, although it is not clear that one rule generates |
1415 |
accurate results in every case. |
1416 |
|
1417 |
The rule established in this specification is that proxies count as a |
1418 |
"use" or "reuse" only those Range requests that result in the return |
1419 |
of byte #0 of the resource. The rationale for this rule is that in |
1420 |
almost every case, an end-client will retrieve the beginning of any |
1421 |
resource that it references at all, and that it will seldom retrieve |
1422 |
any portion more than once. Therefore, this rule appears to meet the |
1423 |
goal of a "best-efforts" approximation. |
1424 |
|
1425 |
|
1426 |
Mogul, Leach [Page 25] |
1427 |
|
1428 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1429 |
|
1430 |
|
1431 |
5.5 Implementation by non-caching proxies |
1432 |
A non-caching proxy may participate in the metering subtree; this is |
1433 |
strongly recommended. |
1434 |
|
1435 |
A non-caching proxy (HTTP/1.1 or higher) that participates in the |
1436 |
metering subtree SHOULD forward Meter headers on both requests and |
1437 |
responses, with the appropriate Connection headers. |
1438 |
|
1439 |
If a non-caching proxy forwards Meter headers, it MUST comply with |
1440 |
these restrictions: |
1441 |
|
1442 |
1. If the proxy forwards Meter headers in responses, such a |
1443 |
response MUST NOT be returned to any request except the |
1444 |
one that elicited it. |
1445 |
|
1446 |
2. Once a non-caching proxy starts forwarding Meter headers, |
1447 |
it should not arbitrarily stop forwarding them (or else |
1448 |
reports may be lost). |
1449 |
|
1450 |
A proxy that caches some responses and not others, for whatever |
1451 |
reason, may choose to implement the Meter header as a caching proxy |
1452 |
for the responses that it caches, and as a non-caching proxy for the |
1453 |
responses that it does not cache, as long as its external behavior |
1454 |
with respect to any particularly response is fully consistent with |
1455 |
this specification. |
1456 |
|
1457 |
5.6 Implementation by cooperating caches |
1458 |
Several HTTP cache implementations, most notably the Harvest/Squid |
1459 |
cache [1], create cooperative arrangements between several caches. |
1460 |
If such caches use a protocol other than HTTP to communicate between |
1461 |
themselves, such as the Internet Cache Protocol (ICP) [11], and if |
1462 |
they implement the Meter header, then they MUST act to ensure that |
1463 |
their cooperation does not violate the intention of this |
1464 |
specification. |
1465 |
|
1466 |
In particular, if one member of a group of cooperating caches agrees |
1467 |
with a server to hit-meter a particular response, and then passes |
1468 |
this response via a non-HTTP protocol to a second cache in the group, |
1469 |
the caches MUST ensure that the server which requested the metering |
1470 |
receives reports that appropriately account for any uses or resues |
1471 |
made by the second cache. Similarly, if the first cache agreed to |
1472 |
usage-limit the response, the total number of uses by the group of |
1473 |
caches MUST be limited to the agreed-upon number. |
1474 |
|
1475 |
|
1476 |
6 Examples |
1477 |
|
1478 |
6.1 Example of a complete set of exchanges |
1479 |
This example shows how the protocol is intended to be used most of |
1480 |
the time: for hit-metering without usage-limiting. Entity bodies are |
1481 |
omitted. |
1482 |
|
1483 |
Mogul, Leach [Page 26] |
1484 |
|
1485 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1486 |
|
1487 |
|
1488 |
A client sends request to a proxy: |
1489 |
|
1490 |
GET http://foo.com/bar.html HTTP/1.1 |
1491 |
|
1492 |
The proxy forwards request to the origin server: |
1493 |
|
1494 |
GET /bar.html HTTP/1.1 |
1495 |
Host: foo.com |
1496 |
Connection: Meter |
1497 |
|
1498 |
thus offering (implicitly) "will-report-and-limit". |
1499 |
|
1500 |
The server responds to the proxy: |
1501 |
|
1502 |
HTTP/1.1 200 OK |
1503 |
Date: Fri, 06 Dec 1996 18:44:29 GMT |
1504 |
Cache-control: max-age=3600 |
1505 |
Connection: meter |
1506 |
Etag: "abcde" |
1507 |
|
1508 |
thus (implicitly) requiring "do-report" (but not requiring |
1509 |
usage-limiting). |
1510 |
|
1511 |
The proxy responds to the client: |
1512 |
|
1513 |
HTTP/1.1 200 OK |
1514 |
Date: Fri, 06 Dec 1996 18:44:29 GMT |
1515 |
Etag: "abcde" |
1516 |
Cache-control: max-age=3600, proxy-mustcheck |
1517 |
Age: 1 |
1518 |
|
1519 |
Since the proxy does not know if its client is an end-system, or a |
1520 |
proxy that doesn't do metering, it adds the "proxy-mustcheck" |
1521 |
directive. |
1522 |
|
1523 |
Another client soon asks for the resource: |
1524 |
|
1525 |
GET http://foo.com/bar.html HTTP/1.1 |
1526 |
|
1527 |
and the proxy sends the same response as it sent to the other client, |
1528 |
except (perhaps) for the Age value. |
1529 |
|
1530 |
After an hour has passed, a third client asks for the response: |
1531 |
|
1532 |
GET http://foo.com/bar.html HTTP/1.1 |
1533 |
|
1534 |
But now the response's max-age has been exceeded, so the proxy |
1535 |
revalidates the response with the origin server: |
1536 |
|
1537 |
|
1538 |
|
1539 |
|
1540 |
Mogul, Leach [Page 27] |
1541 |
|
1542 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1543 |
|
1544 |
|
1545 |
GET /bar.html HTTP/1.1 |
1546 |
If-None-Match: "abcde" |
1547 |
Host: foo.com |
1548 |
Connection: Meter |
1549 |
Meter: count=1/0 |
1550 |
|
1551 |
thus simultaneously fulfilling its duties to validate the response |
1552 |
and to report the one "use" that wasn't forwarded. |
1553 |
|
1554 |
The origin server responds: |
1555 |
|
1556 |
HTTP/1.1 304 Not Modified |
1557 |
Date: Fri, 06 Dec 1996 19:44:29 GMT |
1558 |
Cache-control: max-age=3600 |
1559 |
Etag: "abcde" |
1560 |
|
1561 |
so the proxy can use the original response to reply to the new |
1562 |
client; the proxy also zeros the use-count it associates with that |
1563 |
response. |
1564 |
|
1565 |
Another client soon asks for the resource: |
1566 |
|
1567 |
GET http://foo.com/bar.html HTTP/1.1 |
1568 |
|
1569 |
and the proxy sends the appropriate response. |
1570 |
|
1571 |
After another few hours, the proxy decides to remove the cache entry. |
1572 |
When it does so, it sends to the origin server: |
1573 |
|
1574 |
HEAD /bar.html HTTP/1.1 |
1575 |
If-None-Match: "abcde" |
1576 |
Host: foo.com |
1577 |
Connection: Meter |
1578 |
Meter: count=1/0 |
1579 |
|
1580 |
reporting that one more use of the response was satisfied from the |
1581 |
cache. |
1582 |
|
1583 |
6.2 Protecting against HTTP/1.0 proxies |
1584 |
An origin server that does not want HTTP/1.0 caches to store the |
1585 |
response at all, and is willing to have HTTP/1.0 end-system clients |
1586 |
generate excess GETs (which will be forwarded by HTTP/1.0 proxies) |
1587 |
could send this for its reply: |
1588 |
|
1589 |
HTTP/1.1 200 OK |
1590 |
Cache-control: max-age=3600 |
1591 |
Connection: meter |
1592 |
Etag: "abcde" |
1593 |
Expires: Sun, 06 Nov 1994 08:49:37 GMT |
1594 |
|
1595 |
HTTP/1.0 caches will see the ancient Expires header, but HTTP/1.1 |
1596 |
caches will see the max-age directive and will ignore Expires. |
1597 |
Mogul, Leach [Page 28] |
1598 |
|
1599 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1600 |
|
1601 |
|
1602 |
--------- |
1603 |
Note: although most major HTTP/1.0 proxy implementations |
1604 |
observe the Expires header, it is possible that some are in use |
1605 |
that do not. Use of the Expires header to prevent caching by |
1606 |
HTTP/1.0 proxies might not be entirely reliable. |
1607 |
--------- |
1608 |
|
1609 |
6.3 More elaborate examples |
1610 |
Here is a request from a proxy that is willing to hit-meter but is |
1611 |
not willing to usage-limit: |
1612 |
|
1613 |
GET /bar.html HTTP/1.1 |
1614 |
Host: foo.com |
1615 |
Connection: Meter |
1616 |
Meter: wont-limit |
1617 |
|
1618 |
Here is a response from an origin server that does not want hit |
1619 |
counting, but does want "uses" limited to 3, and "reuses" limited to |
1620 |
6: |
1621 |
|
1622 |
HTTP/1.1 200 OK |
1623 |
Cache-control: max-age=3600 |
1624 |
Connection: meter |
1625 |
Etag: "abcde" |
1626 |
Expires: Sun, 06 Nov 1994 08:49:37 GMT |
1627 |
Meter: max-uses=3, max-reuses=6, dont-report |
1628 |
|
1629 |
Here is the same example with abbreviated Meter directive names: |
1630 |
|
1631 |
HTTP/1.1 200 OK |
1632 |
Cache-control: max-age=3600 |
1633 |
Connection: meter |
1634 |
Etag: "abcde" |
1635 |
Expires: Sun, 06 Nov 1994 08:49:37 GMT |
1636 |
Meter:u=3,r=6,e |
1637 |
|
1638 |
|
1639 |
7 Interactions with content negotiation |
1640 |
|
1641 |
This section describes two aspects of the interaction between |
1642 |
hit-metering and``content-negotiated'' resources: |
1643 |
|
1644 |
1. treatment of responses carrying a Vary header (section |
1645 |
7.1). |
1646 |
|
1647 |
2. treatment of responses that use the proposed Transparent |
1648 |
Content Negotiation mechanism (section 7.2). |
1649 |
|
1650 |
|
1651 |
|
1652 |
|
1653 |
|
1654 |
Mogul, Leach [Page 29] |
1655 |
|
1656 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1657 |
|
1658 |
|
1659 |
7.1 Treatment of responses carrying a Vary header |
1660 |
Separate counts should be kept for each combination of the headers |
1661 |
named in the Vary header for the Request-URI (what [3] calls "the |
1662 |
selecting request-headers"), even if they map to the same entity-tag. |
1663 |
This rule has the effect of counting hits on each variant, if there |
1664 |
are multiple variants of a page available. |
1665 |
|
1666 |
--------- |
1667 |
Note: This interaction between Vary and the hit-counting |
1668 |
directives allows the origin server a lot of flexibility in |
1669 |
specifying how hits should be counted. In essence, the origin |
1670 |
server uses the Vary mechanism to divide the requests for a |
1671 |
resource into arbitrary categories, based on the request- |
1672 |
headers. (We will call these categories "request-patterns".) |
1673 |
Since a proxy keeps its hit-counts for each request-pattern, |
1674 |
rather than for each resource, the origin server can obtain |
1675 |
separate statistics for many aspects of an HTTP request. |
1676 |
--------- |
1677 |
|
1678 |
For example, if a page varied based on the value of the User-Agent |
1679 |
header in the requests, then hit counts would be kept for each |
1680 |
different flavor of browser. But it is in fact more general than |
1681 |
that; because multiple header combinations can map to the same |
1682 |
variant, it also enables the origin server to count the number of |
1683 |
times (e.g.) the Swahili version of a page was requested, even though |
1684 |
it is only available in English. |
1685 |
|
1686 |
If a proxy does not support the Vary mechanism, then [3] says that it |
1687 |
MUST NOT cache any response that carries a Vary header, and hence |
1688 |
need not implement any aspect of this hit-counting or usage-limiting |
1689 |
design for varying resources. |
1690 |
|
1691 |
--------- |
1692 |
Note: this also implies that if a proxy supports the Vary |
1693 |
mechanism but is not willing to maintain independent hit-counts |
1694 |
for each variant response in its cache, then it must follow at |
1695 |
least one of these rules: |
1696 |
|
1697 |
1. It must not use the Meter header in a request to offer |
1698 |
to hit-meter or usage-limit responses. |
1699 |
|
1700 |
2. If it does offer to hit-meter or usage-limit responses, |
1701 |
and then receives a response that includes both a Vary |
1702 |
header and a Meter header with a directive that it |
1703 |
cannot satisfy, then the proxy must not cache the |
1704 |
response. |
1705 |
|
1706 |
In other words, a proxy is allowed to partially implement the |
1707 |
Vary mechanism with respect to hit-metering, as long as this |
1708 |
has no externally visible effect on its ability to comply with |
1709 |
the Meter specification. |
1710 |
--------- |
1711 |
Mogul, Leach [Page 30] |
1712 |
|
1713 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1714 |
|
1715 |
|
1716 |
This approach works for counting almost any aspect of the request |
1717 |
stream, without embedding any specific list of countable aspects in |
1718 |
the specification or proxy implementation. |
1719 |
|
1720 |
7.2 Interaction with Transparent Content Negotiation |
1721 |
[A description of the interaction between this design and the |
1722 |
proposed Transparent Content Negotiation (TCN) design [5] will be |
1723 |
made available in a later document.] |
1724 |
|
1725 |
|
1726 |
8 A Note on Capturing Referrals |
1727 |
|
1728 |
It is alleged that some advertisers want to pay content providers, |
1729 |
not by the "hit", but by the "nibble" -- the number of people who |
1730 |
actually click on the ad to get more information. |
1731 |
|
1732 |
Now, HTTP already has a mechanism for doing this: the "Referer" |
1733 |
header. However, perhaps it ought to be disabled for privacy reasons |
1734 |
-- according the HTTP/1.1 spec: |
1735 |
|
1736 |
"Because the source of the link may be private information |
1737 |
or may reveal an otherwise private information source, it is |
1738 |
strongly recommended that the user be able to select whether |
1739 |
or not the Referer field is sent." |
1740 |
|
1741 |
However, in the case of ads, the source of the link actually wants to |
1742 |
let the referred-to page know where the reference came from. |
1743 |
|
1744 |
This does not require the addition of any extra mechanism, but rather |
1745 |
can use schemes that embed the referrer in the URI in a manner |
1746 |
similar to this: |
1747 |
|
1748 |
http://www.blah.com/ad-reference?from=site1 |
1749 |
|
1750 |
Such a URI should point to a resource (perhaps a CGI script) which |
1751 |
returns a 302 redirect to the real page |
1752 |
|
1753 |
http://www.blah.com/ad-reference.html |
1754 |
|
1755 |
Proxies which do not cache 302s will cause one hit on the redirection |
1756 |
page per use, but the real page will get cached. Proxies which do |
1757 |
cache 302s and report hits on the cached 302s will behave optimally. |
1758 |
|
1759 |
This approach has the advantage that it works whether or not the |
1760 |
end-client has disabled the use of Referer. Combined with the rest |
1761 |
of the hit-metering proposal in this design, this approach allows, |
1762 |
for example, an advertiser to know how often a reference to an |
1763 |
advertisement was made from a particular page. |
1764 |
|
1765 |
|
1766 |
|
1767 |
|
1768 |
Mogul, Leach [Page 31] |
1769 |
|
1770 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1771 |
|
1772 |
|
1773 |
9 Alternative proposals |
1774 |
|
1775 |
There might be a number of other ways of gathering demographic and |
1776 |
usage information; other mechanisms might respond to a different set |
1777 |
of needs than this proposal does. This proposal certainly does not |
1778 |
preclude the proposal or deployment of other such mechanisms, and |
1779 |
many of them may be complementary to and compatible with the |
1780 |
mechanism proposed here. |
1781 |
|
1782 |
There has been some speculation that statistical sampling methods |
1783 |
might be used to gather reasonably accurate data. One such proposal |
1784 |
is to manipulate cache expiration times so that selected resources |
1785 |
are uncachable for carefully chosen periods, allowing servers to |
1786 |
accurately count accesses during those periods. The hit-metering |
1787 |
mechanism proposed here is entirely complementary to that approach, |
1788 |
since it could be used to reduce the cost of gathering those counts. |
1789 |
James Pitkow has written a paper comparing an earlier draft of this |
1790 |
hit-metering proposal with sampling approaches [8]. |
1791 |
|
1792 |
Phillip Hallam-Baker has proposed using a log-exchange protocol [4], |
1793 |
by which a server could request a proxy's logs by making an HTTP |
1794 |
request to the proxy. This proposal asserts that it is ``believed to |
1795 |
operate correctly in configurations involving multiple proxies,'' but |
1796 |
it is not clear that this is true if an outer proxy is used as a |
1797 |
(one-way) firewall. The proposal also leaves a number of open |
1798 |
issues, such as how an origin server can be sure that all of the |
1799 |
proxies in the request subtree actually support log-exchange. It is |
1800 |
also not clear how this proposal couples a proxy's support of |
1801 |
log-exchange to a server's permission to cache a response. |
1802 |
|
1803 |
For general background on the topic of Web measurement standards, see |
1804 |
the discussion by Thomas P. Novak and Donna L. Hoffman [7]. Also see |
1805 |
the ``Privacy and Demographics Overview'' page maintained by by the |
1806 |
World Wide Web Consortium [9], which includes a pointer to some |
1807 |
tentative proposals for gathering consumer demographics (not just |
1808 |
counting references) [2]. |
1809 |
|
1810 |
|
1811 |
10 Security Considerations |
1812 |
|
1813 |
Which outbound clients should a server (proxy or origin) trust to |
1814 |
report hit counts? A malicious proxy could easily report a large |
1815 |
number of hits on some page, and thus perhaps cause a large payment |
1816 |
to a content provider from an advertiser. To help avoid this |
1817 |
possibility, a proxy may choose to only relay usage counts received |
1818 |
from its outbound proxies to its inbound servers when the proxies |
1819 |
have authenticated themselves using Proxy-Authorization and/or they |
1820 |
are on a list of approved proxies. |
1821 |
|
1822 |
It is not possible to enforce usage limits if a proxy is willing to |
1823 |
cheat (i.e., it offers to limit usage but then ignores a server's |
1824 |
Meter directive). |
1825 |
Mogul, Leach [Page 32] |
1826 |
|
1827 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1828 |
|
1829 |
|
1830 |
Regarding privacy: it appears that the design in this document does |
1831 |
not reveal any more information about individual users than would |
1832 |
already be revealed by implementation of the existing HTTP/1.1 |
1833 |
support for "Cache-control: max-age=0, proxy-revalidate" or |
1834 |
"Cache-control: s-maxage=0". It may, in fact, help to conceal |
1835 |
certain aspects of the organizational structure on the outbound side |
1836 |
of a proxy. In any case, the conflict between user requirements for |
1837 |
anonymity and origin server requirements for demographic information |
1838 |
cannot be resolved by purely technical means. |
1839 |
|
1840 |
|
1841 |
11 Acknowledgements |
1842 |
|
1843 |
We gratefully acknowledge the constructive comments received from |
1844 |
Anselm Baird-Smith, Ted Hardie, Koen Holtman (who suggested the |
1845 |
technique described in section 8), Dave Kristol, Ari Luotonen, |
1846 |
Patrick R. McManus, Ingrid Melve, and James Pitkow. |
1847 |
|
1848 |
|
1849 |
12 References |
1850 |
|
1851 |
1. Anwat Chankhunthod, Peter B. Danzig, Chuck Neerdaels, Michael |
1852 |
F. Schwartz, and Kurt J. Worrell. A Hierarchical Internet Object |
1853 |
Cache. Proc. 1996 USENIX Technical Conf., San Diego, January, 1996, |
1854 |
pp. 153-163. |
1855 |
|
1856 |
2. Daniel W. Connolly. Proposals for Gathering Consumer |
1857 |
Demographics. http://www.w3.org/pub/WWW/Demographics/Proposals.html. |
1858 |
|
1859 |
3. Roy T. Fielding, Jim Gettys, Jeffrey C. Mogul, Henrik Frystyk |
1860 |
Nielsen, and Tim Berners-Lee. Hypertext Transfer Protocol -- |
1861 |
HTTP/1.1. RFC 2068, HTTP Working Group, January, 1997. |
1862 |
|
1863 |
4. Phillip M. Hallam-Baker. Notification for Proxy Caches. W3C |
1864 |
Working Draft WD-proxy-960221, World Wide Web Consortium, February, |
1865 |
1996. http://www.w3.org/pub/WWW/TR/WD-proxy.html. |
1866 |
|
1867 |
5. Koen Holtman and Andrew Mutz. Transparent Content Negotiation in |
1868 |
HTTP. Internet-Draft draft-ietf-http-negotiation-01.txt, HTTP |
1869 |
Working Group, March, 1997. This is a work in progress. |
1870 |
|
1871 |
6. J. Mogul. Forcing HTTP/1.1 proxies to revalidate responses. |
1872 |
Internet Draft draft-mogul-http-revalidate-01.txt, HTTP Working |
1873 |
Group, May, 1997. This is a work in progress. |
1874 |
|
1875 |
7. Thomas P. Novak and Donna L. Hoffman. New Metrics for New Media: |
1876 |
Toward the Development of Web Measurement Standards. This is a draft |
1877 |
paper, currently available at |
1878 |
http://www2000.ogsm.vanderbilt.edu/novak/web.standards/webstand.html. |
1879 |
Cited by permission of the author; do not quote or cite without |
1880 |
permission. |
1881 |
|
1882 |
Mogul, Leach [Page 33] |
1883 |
|
1884 |
Internet-Draft Hit-Metering for HTTP 3 July 1997 15:35 |
1885 |
|
1886 |
|
1887 |
8. James Pitkow. In search of reliable usage data on the WWW. |
1888 |
Proc. Sixth International World Wide Web Conference, Santa Clara, CA, |
1889 |
April, 1997, pp. ??-??. (To appear). |
1890 |
|
1891 |
9. Joseph Reagle, Rohit Khare, Dan Connolly, and Tim Berners-Lee. |
1892 |
Privacy and Demographics Overview. |
1893 |
http://www.w3.org/pub/WWW/Demographics/. |
1894 |
|
1895 |
10. Linda Tauscher and Saul Greenberg. Revisitation Patterns in |
1896 |
World Wide Web Navigation. Research Report 96/587/07, Department of |
1897 |
Computer Science, University of Calgary, March, 1996. |
1898 |
http://www.cpsc.ucalgary.ca/projects/grouplab/ |
1899 |
papers/96WebReuse/TechReport96.html. |
1900 |
|
1901 |
11. Duane Wessels and K Claffy. Internet Cache Protocol (ICP), |
1902 |
version 2. Internet Draft draft-wessels-icp-v2-00.txt, IETF, |
1903 |
November, 1996. This is a work in progress. |
1904 |
|
1905 |
|
1906 |
13 Authors' addresses |
1907 |
|
1908 |
Jeffrey C. Mogul |
1909 |
Western Research Laboratory |
1910 |
Digital Equipment Corporation |
1911 |
250 University Avenue |
1912 |
Palo Alto, California, 94305, U.S.A. |
1913 |
Email: mogul@wrl.dec.com |
1914 |
Phone: 1 415 617 3304 (email preferred) |
1915 |
|
1916 |
Paul J. Leach |
1917 |
Microsoft |
1918 |
1 Microsoft Way |
1919 |
Redmond, Washington, 98052, U.S.A. |
1920 |
Email: paulle@microsoft.com |
1921 |
|
1922 |
|
1923 |
|
1924 |
|
1925 |
|
1926 |
|
1927 |
|
1928 |
|
1929 |
|
1930 |
|
1931 |
|
1932 |
|
1933 |
|
1934 |
|
1935 |
|
1936 |
|
1937 |
|
1938 |
|
1939 |
Mogul, Leach [Page 34] |