IETF IIIR Working Group Chris Weider INTERNET--DRAFT Merit Network, Inc. October, 1993 Resource Transponders Status of this Memo In this paper, the author describes a mechanism for automatically maintaining resource location systems which contain pointers to resources. This document is an Internet Draft. Internet Drafts are working documents of the Internet Engineering Task Force (IETF), its Areas, and its Working Groups. Note that other groups may also distribute working documents as Internet Drafts. Internet Drafts are draft documents valid for a maximum of six months. Internet Drafts may be updated, replaced, or obsoleted by other documents at any time. It is not appropriate to use Internet Drafts as reference material or to cite them other than as a "working draft" or "work in progress." Please check the I-D abstract listing contained in each Internet Draft directory to learn the current status of this or any other Internet Draft. This Internet Draft expires April 22, 1993. Abstract Although a number of systems have been created in the last several years to provide resource location and navigation on the Internet, the information contained in these systems must be maintained and updated by hand. This paper describes an automatic mechanism, the resource transponder, for maintaining resource location information. Author's note: This document is being circulated as a research paper; consequently there are no protocol specifications or anything of the sort. I hope that we can go from here and actually get some implementation experience. 1. Introduction In the past few years, we've seen the invention and growth of a number of information location systems on the Internet, e.g. archie, Gopher, and WAIS. However, as these systems have become widely deployed, a number of maintenance and security problems have arisen with them. Some of the major ones INTERNET--DRAFT Resource Transponders Weider 1) Out of necessity, most of these systems contain pointers to the desired resources rather than the resources themselves. Therefore, if a resource becomes obsolete, is modified, or is moved, the location system must be updated by hand. Some systems, such as Archie and Veronica, proactively create updated indexes by contacting every resource on a certain time schedule. However, this means that the system can be wildly out of date, depending on the interval between polls of the resources. This process can be highly inefficient depending on the percentage of information that has changed. 2) Conversely, anyone who maintains a resource that they wish indexed must keep track of every directory which contains a pointer to that resource, so that if it is modified, all the directories can be updated. This obviously is an optimistic scenario. 3) Many organizations which have installed these systems do not have the available resources or expertise to maintain the information in the systems. Thus we have long periods where the information drifts, then a short period when the information is updated again. 4) Even though these systems are almost always out of date today, this problem will become increasingly harder for humans to manage by hand as everyone on the net becomes their own publisher. Also, as the net speeds up and people rely more and more on accurate information, human-induced delays in updates of these systems will become increasingly intolerable. 5) Most, if not all, of these systems provide no security whatsoever; if a pointer to a resource appears in a locator system, then it is assumed to be meant for public consumption. There are many potential information providers who would like to use publicly deployed information systems to publish to a very selected clientele, and do not wish to allow the whole net access to their resources. 2: Requirements for a solution There are several objectives which must be met by any proposed solution to these problems: 1) We need to decrease the personnel resources needed for indexing and pointer maintenance. 2) We need to increase the reliability and accuracy of the information held in resource location systems. 3) We need to provide some mechanisms for security, particularly by mediating access to the resources INTERNET--DRAFT Resource Transponders Weider 4) We need to make it easy for non-experts, such as librarians, archivists, and database maintainers, to announce their new resources to the various resource location services. Many of these problems can be solved by a 'resource transponder' mechanism. 3: Resource Transponders The resource transponder system works by adding two new layers to every resource: metainformation and an agent to update a resource location system (RLS) with that metainformation. The metainformation layer is physically attached to every resource, so that when the resource is moved or altered, the metainformation is immediately available to update the RLS. The agent layer may also be attached to the resource or may not be; the implications of both of these options are discussed in detail below. 3.1: Metainformation The metainformation layer of a given resource contains any information which might be required to create a pointer to this resource, and any information which may be useful for indicating how to catalog or index the resource. For example, the metainformation layer of a text document might contain such things as the Uniform Resource Name (URN) of the document [Weider 1993], the title of the document, a Uniform Resource Locator (URL) for the document [Berners-Lee 1993], the size of the document, etc. Thus the metainformation layer contains data _about_ the resource to which it is attached. This metainformation is expected to be modifiable. For example, the metainformation layer may contain a history of where this particular copy of a resource has been. Let's say that a resource/transponder pair has been moved. When it gets to its new location, the agent can then attempt to contact the resource at its old location to determine whether the resource is still there (in which case the agent will simply cause the new location to be added to the RLS) or whether the resource is not there (in which case the agent can tell the RLS to add the current pointer and delete the old one). A number of other possibilities for the contents of the metainformation level are contained in section 4.1. 3.2: Agents The agent layer of a given resource contains an executable program which is responsible for reading the metainformation attached to the resource and using that information to update a RLS. It is also responsible for INTERNET--DRAFT Resource Transponders Weider updating the metainformation where necessary and for running any indexing programs required by the RLS it is attempting to update. When the tools required to build agents are constructed and deployed, the author expects the agents to begin mediating access to the resource, particularly for agents attached to resources which are not currently considered active processes, such as text files and digitized images. In this futuristic model, someone wishing to read a given document would have to first negotiate access to the data with the agent; the agent would then be responsible for delivering the data to the client. However, it is expected that this type of agent will not be widely deployed for some time. Different ways of implementing agents are discussed in section 4.2. 4: Models for implementations of resource transponders 4.1: Models for implementations of the metainformation layer The metainformation layer can be implemented in a number of ways, depending on the resource with which it is associated. For an 'active' resource, such as an on-line catalog or a mail-based service, the metainformation can be stored in a file with a well-known name in the software distribution. Alternatively, the metainformation could be stored as a record in the data which the resource serves. For a text document, the metainformation could be stored as the first or last N bytes of the document (which would break a number of editors and file display techniques, but would guarantee that the metainformation is moved with the resource), or perhaps as a file with a logically associated name (paper2.meta associated with paper2.txt, for example). The problem with this second approach is that the user must know that they have to move the metainformation with the file itself, or things will start breaking. If an agent is explicitly attached to the resource, the agent could contain the metainformation internally. In any case, the resource transponder system must be able to guarantee that the metainformation is moved when the resource is moved. 4.2: Models for implementations of the agents The agent layer can also be implemented in a number of ways, depending on such things as system loads, desired sizes of resources, multitasking capabilities, etc. The easiest and for many unitasking systems the cleanest way of implementing an agent is to have one agent per computer. Then when a resource is moved onto that computer, the agent is explicitly activated and notified where the new resource is. For example, let's say that someone wishes to download a copy of a resource and then let the RLS know that that resource is available for public consumption. She would download the resource and then run the agent, which would then notify INTERNET--DRAFT Resource Transponders Weider the RLS and update the metainformation attached to the resource. This model could also be used to track files on a LAN, or to provide local location services with no need to run a larger RLS. Another model for implementation of the agent is to have one agent per resource. In this model, the agent would be moved along with the resource and the metainformation. The agent could be implemented in a file which would be associated with the resource; in that case the agent would have to be explicitly activated when the resource was moved. Alternatively, the agent/metainformation/resource system could be implemented as one system, or in one file. In this case, the agent itself would always be active, and would be responsible for mediating access to the resource. When one did a 'telnet' to a resource with an active agent, the agent would accept the telnet connection and be responsible for providing security and translation for the data. This could provide great security for resources while still allowing pointers to them to be placed in public RLS's; the data in the resource could be encrypted, with the agent responsible for decrypting it. This option is explored more fully in Section 5. 5: Transponders and Objects Section 4.2 details several methods of implementing resource transponder agents. The technique of encapsulating the metainformation *and* the data for the resource 'inside' the agent brings up several interesting possibilities. As noted, once the data is encapsulated in the agent, all actions on the resource are mediated by the agent. In essence, encapsulating the resource this way allows the resource to be treated as an active, 'animate' object on the network. With the appropriate functionality in the agent, the resource could also maintain additional information about where it has been, and can also provide large subsets of the general maintenance functionality required by a rapidly changing information mesh. 5.1: Possible Functionality of Active Resources This is an exploration of several possible functions which may be available on resource objects; this is not intended to be an exhaustive list. 5.1.1: Replicate Sending this command to a resource object causes it to attempt to create a copy of itself at a given location in the network. To achieve this, the resource will have to negotiate permissions with the target system. This functionality would allow: Automatic mirroring of resource objects. If a transponder is attached to a resource which is being updated frequently, and that resource is being mirrored in a number of other places, the transponder agent could use this function to maintain the other copies INTERNET--DRAFT Resource Transponders Weider Load balancing. In the resource object model, access to a resource is gained by negotiation with the attached transponder agent. Thus, as a part of the metainformation, the agent can maintain load information, such as a history of access information, elapsed time for given operations, etc. The agent can then determine if there is an access or load problem because of heavy demand for the data in the resource. If there is a load problem, it can then replicate itself and notify the resource location system that a new copy has been created. 5.1.2: Annotate In the resource object model, every resource exists on the net as an independent entity, moving around, copying itself, or perhaps just sitting still. Many of these resources will be academic papers, books, journal articles, even interorganizational memos. Discovery of a given resource will require a sophisticated resource location system, which we are just now starting to build with the various information delivery tools available. However, once we've found an interesting resource, it will be a much more difficult task to find the body of commentary and annotation on this resource. The resource object can alleviate this problem by maintaining a pointer to a body of commentary in the metainformation of the resource. When someone wishes to annotate a given resource, or to retrieve annotations created by other people, they would then send an 'annotate' command to the resource object, and the resource object would then return the pointer to the resource which contains the annotations. Since the resource object which contains the annotations would also have a transponder mechanism, annotations could then be added and the resource location system would be updated. 5.1.3: Destroy This function would cause the resource object to be deleted after the resource location system was updated. The agent would be responsible for verifying that the requester had the proper permissions to delete the resource; in addition, the resource object could determine how many other copies of itself existed and could inform the requester, asking for a confirmation before deleting itself. 6: Conclusion As shown in sections 3 and 4, the resource transponder or something like it could be enormously valuable no matter how it is implemented. In addition, active resource objects allow for additional functionality which will be very helpful for the next phases of the Internet Information Infrastructure. Therefore, I wish to encourage the reader to look at implementation strategies and help build this. INTERNET--DRAFT Resource Transponders Weider 7: References [Berners-Lee 1993] Berners-Lee, Tim, 'Uniform Resource Locators', Internet Draft, July 1993. Available as [Weider 1993] Weider, Chris, and Peter Deutsch, 'Uniform Resource Names', Internet Draft, October 1993. Available as 8: Author's address Chris Weider, clw@merit.edu Merit Network, Inc. 2901 Hubbard, Pod G Ann Arbor, MI 48105 Phone: (313) 747-2730 Fax: (313) 747-3185