Legal knowledge based systems

Preface Legal knowledge based systems JURIX 92 Information Technology and Law The Foundation for Legal Knowledge Systems Editors: C.A.F.M. Grütters...
Author: Griffin Stokes
1 downloads 0 Views 39KB Size
Preface

Legal knowledge based systems JURIX 92 Information Technology and Law

The Foundation for Legal Knowledge Systems Editors:

C.A.F.M. Grütters J.A.P.J. Breuker H.J. Van den Herik A.H.J. Schmidt C.N.J. De Vey Mestdagh

C.A.M. Wildemast and R.V. de Mulder, Some Designs Considerations for a Conceptual Legal Information Retrieval System, in: C.A.F.M. Grütters, J.A.P.J. Breuker, H.J. Van den Herik, A.H.J. Schmidt, C.N.J. De Vey Mestdagh (eds.), Legal knowledge based systems JURIX 92: Information Technology and Law , The Foundation for Legal Knowledge Systems, Lelystad: Koninklijke Vermande, pp. 81-92, 1994 ISBN 90 5458 031 3.

More information about the JURIX foundation and its activities can be obtained by contacting the JURIX secretariat: Mr. C.N.J. de Vey Mestdagh University of Groningen, Faculty of Law Oude Kijk in 't Jatstraat 26 P.O. Box 716 9700 AS Groningen Tel: +31 50 3635790/5433 Fax: +31 50 3635603 Email: [email protected]

1992 JURIX The Foundation for Legal Knowledge Systems

http://jurix.bsk.utwente.nl/

1

SOME DESIGN CONSIDERATIONS FOR A CONCEPTUAL LEGAL INFORMATION RETRIEVAL SYSTEM C.A.M. WILDEMAST and R.V. DE MULDER Centre for Computers and Law, Erasmus University, Rotterdam, The Netherlands

Summary Concept formation in law is not based on empirirical observation nor on explicit and unequivocal conventions. Therefore, concepts in law are not objective data, but rather interpretations determined by social circumstances. Consequently, attempts to design conceptual legal information retrieval systems in which predetermined legal concepts are fixed into the system will always be unsuccesful. In this article, a brief review of these attempts is given and an outline is presented of a "learning concept processor" which enables the user to define his own concepts and enter these into the system. 1.

Introduction

There is now a large consensus of opinion that conceptual retrieval methods are preferable to word-based search procedures. In this article, attention will be paid to design considerations for conceptual legal information retrieval systems. These design considerations must take into account, not only general information retrieval theories, but also the specific demands placed upon such systems by lawyers. A brief examination is made of the present theories of retrieval, concentrating on three aspects: the interface, representation techniques and search methods. An outline will be given of the problems particularly associated with developing legal information retrieval systems. Various solutions have been advanced to deal with the specific problems in this area. The solutions are directed towards modifying the interface and the means of representing legal documents. Design considerations regarding the interface are aimed at allowing the user to describe his questions more satisfactorily which in term will make the search procedure more efficient. Proposals include a thesaurus, hypertext and the development of intelligent interfaces. This article will show that the main obstacle remains the limitation placed on the user of having to work within a preordained set of opinions. Various techniques for document representation have been proposed in recent literature. These approaches may be characterized as manual or automatic, employing either a reducing technique or an interpreting technique. It is argued here that manual techniques are expensive; reducing techniques may involve the loss of potential information while the danger of interpreting techniques is that the user may not be able to modify the concepts presented so that they correspond more closely to his own interpretation of legal notions. Finally, we will outline the "learning concept processor" which has been developed by the Centre for Computers and Law of the Erasmus University. This is a conceptual legal information retrieval system which incorporates a number of considerations detailed in this article.

81

Wildemast, C.A.M. and R.V. de Mulder

2.

The significance of "conceptual retrieval methods"

Information of importance to lawyers is increasingly being stored in, and accessed by, computers [Franken et al., 1992]. What has been stored must also be capable of being located again. Search procedures are usually based on comparing words in the search question with words in the stored documents or representations of those documents. This comparison of words is usually referred to as "string matching". If a question pertains to more words, word combinations can be formulated with the help of logical (boolean) operations. A search can then be carried out on the basis of these word combinations. Recent research has [Blair & Maron, 1985] shown, however, that this search method remains unsatisfactory. Not all relevant documents are found and of those documents which are found, not all are relevant. These shortcomings are connected to the fact that, on the one hand, (a logical combination of) words is still not suitable to describe sufficiently the user's information question while, on the other hand, not enough is known of the relationship between the words used in the documents and the subjects which can be read in the texts. Extensive research has been, and still is being, carried out to find more effective search methods. Within the legal sphere, there is a fairly high level of consensus concerning a rejection of word-based searches, preference being given to search procedures based on the (legal) interpretation of the search question and of the documents required. Search methods in which this concept is applied are referred to by the term "conceptual retrieval methods". Although there is agreement that conceptual search methods will be more effective, there are differing views as to the way in which this can best be realized. Several proposals have been presented in the literature on the subject. A summary of these will be given below. 3.

The retrieval process

The search for relevant documents with the help of a computer can be presented schematically as shown below:

question presentation of documents

command

command interpretation

search operation

Figure 1

82

representation

store

Some Design Considerations for a Conceptual Legal Information Retrieval System

The user has a question. This question has to be converted by the user into a command for the system. The interpretation of the command within the system provides for the translation of the command into an actual search instruction for the system. This results in a comparison between words, numbers and other symbols in the search instruction and those in the representations of the original documents. When texts are stored in a databank, a list is usually (automatically) drawn up of all the words which appear in the texts (apart from extremely frequent words such as "the", "and" etc.). For each word a reference is made of the document(s) in which it appears. This list is often called an "inverted file". This is a means of representing the original texts. If the search instruction corresponds to one or more elements of the representation, the texts (documents) pertaining to it will be presented to the user. The search result can be printed out or used to reformulate the search question or command. The latter is sometimes referred to as "relevance feedback". The methods proposed in the literature for conceptual retrieval are aimed at three elements in the diagram: * The interface with the users * The representation of documents * The search operation. 4.

A summary of conceptual retrieval methods

The following summary presents the approaches to the three elements specified above. First of all, the interface. Methods which try to realize conceptual retrieval by helping the user in the formulation of the question will be discussed. Under the title of representation, the next section discusses those methods based on the assumption that conceptual retrieval can be realized if the representations of the original texts are based on the legal importance or the legal meaning of a text. Decisions with respect to the design of the interface and the method of representation are related to each other. For example, if the interface provides the user with alternative search terms, these have to be part of the representation of the documents in order to be at all useful. In this article, relatively little attention will be paid to the third element of the retrieval process, the search procedure. Given the representation methods and the possibilities of the interface, all sorts of search methods could be applied. These could vary from simple (boolean) string matching via statistical techniques to neural networks. Furthermore, as far as design considerations are concerned this aspect is not crucial, as supplements and/or alterations to a search method will, in general, not alter the design of a retrieval method. It is also usually possible to incorporate a number of search procedures within a retrieval system without changing the design. 4.1.

The interface

It is the interface which makes communication between the user and the computer possible. It assists in the translation of the user's question into an actual search instruction for the computer. When the search instruction has been carried out, the interface is responsible for the reproduction of the results. On the basis of these results it is then possible to assess the relevance of the documents which have been found and to reformulate the question (or the actual command) if necessary. An interface can also assist the user in the formulation of the question [Vries et al., 1991].

83

Wildemast, C.A.M. and R.V. de Mulder

question

Documents or suggestions for reformulation of question or command

command

command interpretation

search operation

representation

store

Figure 2

4 . 1 . 1 . Thesaurus As it is often difficult for the user to find the right words for the description of the question, use can be made of a thesaurus. A thesaurus is a list of synonyms. The advantage of this is that the system is able to offer the user alternatives to the words being looked for. It has been pointed out [Bing, 1987, p.44] that while a thesaurus can be useful, it is not able to solve all retrieval problems. In particular, it is not able to solve the problem that the meaning of a term is dependent on its context. This means that the interface should also contain information on the context. 4 . 1 . 2 . Intelligent interfaces Interfaces which give information on the relationships between legal concepts in a particular field have been described in [Bing, 1987] and [Guidotti et al., 1990]. Bing based the hierarchical structures which he proposed on the applicable legal regulations. The texts of the legal regulations were converted by hand into IF-THEN lines. All the legal regulations which have been rewritten in this way together determine the hierarchical structure. This is shown to the users graphically by means of arrow diagrams. The whole field can be reviewed with the help of these diagrams. Words from the diagrams which are relevant to the question can then be selected. When a choice has been made, synonyms are also shown in order that a search can be made with these. It is for the user to decide. The system itself then generates a (boolean) question to the databank. In [Guidotti et al., 1990], a semantic network is proposed to reflect such a hierarchical conceptual structure. The concepts are connected to each other on the basis of selected (legal) semantic relationships. The user is shown a graphic representation of the network. The user can choose terms which refer to the concepts in order to describe his question. The selected terms are connected to others in the network (for example, because they are synonymous). These terms are also incorporated in the search question which the system automatically generates.

84

Some Design Considerations for a Conceptual Legal Information Retrieval System

4 . 1 . 3 . Hypertext Hypertext is a way of linking pieces of information that are stored in separate locations. This makes it possible to connect related information stored in different documents [Wiley, 1989, p. 223]. Greenleaf states that legal materials are particularly suited to hypertext presentation because of the fact that legal texts are densely cross-related [Greenleaf et al., 1991, p. 216]. A possible use of hypertext in the field of text retrieval is a hypertext presentation of the retrieved documents [Greenleaf et al., 1991][Merkl et al., 1990]. The documents are not presented as separate, individual items, but as a network in which interconnections between the documents are visible through linking terms. This enables the user to browse through the documents following his own line of interest. 4 . 1 . 4 . Conclusion on interfaces One of the characteristics of intelligent interfaces is that the user must work with the concepts which have been programmed into the interface. As Leith [1990] and others have convincingly argued, these concepts are not objective data but rather interpretations determined by social circumstances. De Mulder emphasises the fact that, unlike the physical sciences, the concepts used are not derived from empirical observation nor, as in mathematics, are they based on explicit and univocal conventions [Mulder, 1984] [Mulder et al., 1989]. Concepts therefore could change from user to user, from time to time and even within one search question.1 The interfaces described above, however, provide only one set of opinions. The user should be able to change the legal knowledge that is incorporated in the interface. This means that the interface should be flexible and be able to "learn" from, possibly a number of, users. Each individual user should be able to access the information through his own concepts. 4.2.

Representation

The representation methods which have been proposed fall into two categories: those which use (only) manual methods and those which use automatic methods. Each of these categories can be further subdivided between "mainly reducing" and "mainly interpreting" approaches. The reducing approach means that the original texts are not represented by all the words which appear in them and the order in which they appear, but only by those words or groups of words which reflect most adequately the (legal) contents of the text. The interpreting approach uses various methods which represent documents with the help of (legal) knowledge. reducing

indexing

interpreting

knowledge representation

manual methods

representation reducing

indexing

automatic methods document profile interpreting

clustering

Figure 3

85

Wildemast, C.A.M. and R.V. de Mulder

4 . 2 . 1 . Manual, reducing Intellectual indexing To search directly in untreated texts is not expedient for practising lawyers given the present state of computer technology and that which can be expected in the near future. The data files from which lawyers must select their texts are simply too extensive. Documents must, therefore, still be adapted before search instructions can be carried out. Texts may be, for example, represented by lists of words. One of the methods for producing such lists of words is to let an "expert" allot words to a text. An advantage of this method is that a word can be allotted to a text regardless of whether that word actually appears in the text. This approach also often leads to a considerable reduction in the files which would be consulted by the search program. This reduction, however, may entail a loss of potential information. Possibly the most significant disadvantage of this kind of representation is that every text has to be worked through by hand which is labour intensive and, therefore, expensive. 4 . 2 . 2 . Manual, interpreting Knowledge representation This approach is based on the assumption that conceptual retrieval can only be realized if the search is carried out on the basis of legal "knowledge". This means that legal "knowledge" has to be described in a way which can be represented by a computer. In order to do this, various knowledge representation techniques have been developed. It is beyond the scope of this article to examine these in any depth [Oskamp, 1990]. The choice of representation technique does not answer the question, however, of what knowledge should be represented. Hafner has proposed that designs for computerized search mechanisms should reflect the way in which a (human) legal expert remembers and classifies judgements [Hafner, 1978][Hafner, 1987]. In order to do this, a model of the legal field required is necessary. Dick proposes a somewhat different approach [Dick, 1987][Dick, 1991]. She proposes that lawyers search for information in the form of arguments which will support a solution to a concrete legal problem. The original texts must, therefore, be represented in terms of the arguments which appear in them. Manual adaptation of the original texts is, in both cases, necessary in order to obtain the information required for the representation. A consequence of this is that the application is restricted to a relatively small legal field. Once again, these approaches are labour intensive and hence rather expensive. 4 . 2 . 3 . Automatic, reducing Indexing Texts can be represented by all the words which appear in them (except for semantically irrelevant words such as the indefinite article). The disadvantage of this is, however, that if a word search is carried out many irrelevant texts are presented because that word appears in many texts [Davis, 1986][Mulder & Oskamp, 1979]. This problem is even more blatant in the case of legal texts. Research has shown that in these texts fewer different words are used than in "ordinary" texts. One solution would be to represent texts by words which are capable of distinguishing (several) texts from the rest of the collection. Words which would be capable of being used as distinguishers are those which appear with a certain frequency in only a few texts. Davis has carried out research concerning obtaining these words automatically [Davis, 1986]. This tested the following hypothesis: words which have a poor distinguishing 86

Some Design Considerations for a Conceptual Legal Information Retrieval System

capacity show a poisson distribution, words which have a good distinguishing capacity show a distribution other than poisson. Words from statute law and case law were examined to ascertain whether there was a poisson distribution. Those words which did not show a poisson distribution were compared to words allotted to the text by hand. As regards case law, too few words were found this way which meant it was pointless to carry out a comparison with words allotted by hand. Statute law, on the other hand, produced a satisfactory number of words which meant that the comparison could be carried out with the aim of determining which words were the same in both methods. It was discovered that only a small section of the words were common to both methods. Although the number of corresponding words was small, the researchers expect that those words will semantically represent the text the best. It will be necessary to test this assumption further. A weakness that is inherent in this method is that the interpretation of the terms which have or have not been incorporated in the extraction will vary from user to user and during time. 4 . 2 . 4 . Automatic, interpreting Document profiles In [Gelbart & Smith, 1990] and [Gelbart & Smith, 1991] it is proposed that documents should be represented by document profiles. A profile can be derived from each document. This profile consists of four parts: concepts, cases, legislation and facts. Each part is described by so-called profile key words. Concepts are described by statements of applicable law and resolutions to the issues when the law has been applied to the facts. Cases are described by citations of related issues, legislation by citations of applicable legislation and facts by description of the factual situation. These profile key words are automatically generated from the original text by means of a method which combines legal knowledge with linguistic methods. This makes it possible to recognize legal word combinations as well as words. Each profile key word has a weight ascribed to it. The key words are then ordered according to weight. The user's question is also described in the form of a profile. The user provides a profile sketch of the documents he requires by describing which concepts, cases, legislation and facts are considered to be important. Relevant documents are found on the basis of comparing the question profile with the documents' profiles. Documents with profiles which correspond most closely to the question profile are ordered according to the level of similarity and are presented to the user. It is also possible, however, to describe only one component, for example the facts. In this way cases are found in which there is a similarity in the factual situation. This method may be considered to be a compromise between an approach which advocates the desirability of a concept structure and one which advocates the desirability of variable interpretation depending on the user and the course of time. The concept structure used here (the fourfold distinction) is kept extremely general. It cannot in itself, however, be changed by the user. Document classification The underlying principle of this method is that documents which deal with "the same" matters should be grouped together. This is also referred to as the clustering of documents [Salton, 1989, p. 326-345]. The process of clustering is based on the similarity between documents. As concerns the legal application of this method, the following has been proposed. Merkl calculates the similarity between pairs of documents at four levels [Merkl et al., 1990]: 87

Wildemast, C.A.M. and R.V. de Mulder

1 2 3 4

co-occurrent terms, co-occurrent law citations, co-occurrent terms and law citations which appear together and co-occurrent law citations which refer to a subject domain.

Level 1 contributes the least and level 4 the most to the similarity. The idea behind this is that documents which deal with the same legal subject will be more similar than documents which deal with differing legal subjects. Levels 1, 2 and 3 are calculated on the basis of statistical information which is acquired in the indexing phase such as term frequencies, document frequencies and the weight of terms. Level 4 is determined according to information from knowledge files. A certain legal field is then divided into subject fields. The knowledge file is a hierarchical reproduction of legal subjects which can be distinguished in a particular legal domain plus the corresponding legal regulations. A document is considered to belong to a certain subject domain if the corresponding legislative article appears [Merkl et al., 1992].2 After all the pairs of documents have been compared, a hierarchical structure of clusters is automatically generated. Documents concerned with the same legal subject appear in each cluster. Certain clusters are chosen from the hierarchy to serve as so-called super documents. A super document is represented by terms and law citations which appear in the documents belonging to the super document. A user's question is compared with the representations of the super documents. Documents belonging to the super document which corresponds most closely to the question are presented to the user. The advantage of this method is that it is possible to search more directly for a set of documents, in which set the documents have an equal information value. 4 . 2 . 5 . Conclusion on representation All the reducing methods have the disadvantage of the potential loss of information. Their main advantage is the lower cost of computer storage and a decrease in the time that a search operation takes. On the basis of technical developments, which have made a marked decrease in the cost of processing and storage possible, the expectation is that these advantages will not compensate for the disadvantage. Reducing methods of representation, however, are useful in a full text system in order to achieve a quick response time when such is desired. Given the fast growth of the amount of documents that are available in the legal field, it seems that methods which are based on manual representation will increasingly become too expensive to be feasible. Furthermore, in addition to the problem in the legal field that there is no established empirical or conventional scientific basis for indexing, achieving an appropriate level of objectivity will be costly, if not impossible. Knowledge representation could be useful in certain specific domains, but has, apart from its apparent cost, the disadvantage of representing only the opinion of one expert, or group of experts, at a certain time. Different users may hold different opinions, and they may wish to change the knowledge brought into the representation. If they would be enabled to do so, this could easily lead to errors and inconsistencies given the complexity of a domain of minimal sophistication. Automatic interpretative methods would also have to be based on opinions of one expert or group of experts and would also be hard to modify by the user. The representation could possibly be partly based upon linguistic techniques. This in itself would not compensate for the empirically unfounded legal notions. It would seem that providing end-users with the option of making non-trivial changes to a vast and complex set of automatically or manually interpreted data cannot be expected in the next decade.

88

Some Design Considerations for a Conceptual Legal Information Retrieval System

The conclusion is, that legal documents will have to be electronically stored in the foreseeable future by a full text representation, with additional automatically generated non-interpretative indexes. 4.3.

Search operations

By search operation is understood the function which ensures that the concrete search instruction (whether or not already reworked in the interface) is carried out on the documents represented in the system. Even in more complicated techniques than the pure boolean search, the search operation will in principle consist of the comparison of words, numbers and/or other symbols or combinations of these in the search instruction with those in the representations. These terms will typically only be searched for in terms of occurrence, rather then, for example, their frequency in the documents. The result of the boolean search operation is the answering of a yes/no question for each document as to whether the document satisfies the search instruction. In other search operations, what is looked for is a measurement which indicates the extent to which the document satisfies the search instruction. This may possibly be expressed in the form of an estimation of probability. In the case of such techniques, it is usually possible to order the documents found according to their expected relevancy [Salton, 1989][Bookstein & Klein, 1990]. This allows the user to devote his attention firstly to those documents at the top of the list. If he finds a large number of these to be relevant he can then work his way through the list until he has exhausted those of use to him. Ordinary statistical methods are applied in such search techniques. A similar result is achieved by search techniques which make use of "neural networks". Neural networks Conceptual retrieval with the help of neural networks was proposed in [Belew, 1987] and [Rose & Belew, 1989]. These networks make a certain level of associative searching possible. A simple example will illustrate how this is realized. Neural networks consist of nodes and links between the nodes. Imagine that there are two layers: term nodes and document nodes. Each term node is connected to the document node in which the term appears. The retrieval mechanism works as follows. If there is a search for a term, there is activity in the network. This activity can best be imagined as a current which flows from the term to all the documents in which the term appears. The documents are activated and pass on the activity to all the connected terms. The activity is then passed on to the documents again via the terms and so on. In the course of time, the spreading of the activity can be stopped. It now appears that certain documents and terms are more activated than others. Those documents which are the most activated are presumed to be relevant to the question. The terms which have been most activated can be presented to the user. These terms are probably ones which correspond to the term in the question. With the help of these terms, the question can be adapted for a new search operation. 4 . 3 . 1 . Conclusion on search operations An advantage of the neural network technique is that no legally fixed, and therefore limited, representation of the incorporated texts is chosen. The user can keep trying to improve his command description so that it corresponds as well as possible to his search requirements. The techniques of neural networks are still in the developing stage. It would seem that this field could contribute substantially to improving conceptual legal information retrieval. Ordinary statistical techniques, however, should not be ruled out prematurely.

89

Wildemast, C.A.M. and R.V. de Mulder

As stated above, from the point of view of designing a retrieval system, unlike the decisions about the interface and the representation method, the choice of a search method is not critical. Given the adequacy of the interface and the representation technique, even a multitude of search techniques could be implemented. In many cases, newly developed techniques could be added to a retrieval system in a later stage. 5.

Conclusion

The above analysis of the advantages and disadvantages of the techniques presented in current literature leads to the conclusion that it would be desirable if both the method of text representation and the interface would allow the user to define his own concepts. These concepts can then be more precisely re-defined on the basis of the results of search operations or interpretations by the interface. The system should store the concepts of the user: it would have to be "learning". The representation technique must, therefore, not only be objective but also be complete while ensuring that the search time does not become unpractically long. A full text storage supplemented with a combination of, on the one hand, a complete word list plus the corresponding word frequency in each document and, on the other hand, a document list including the frequency of each word of every document would seem to be a workable choice. As regards the interface, it is especially important that the user can bring into the system and modify his own concepts. We would argue that the quality of the interface is therefore the constraint factor in conceptual legal information retrieval at present. Research efforts should be concentrated on this area. A lot more can be done. For example, in the available literature there is little mention of an obvious method of allowing the user to make his ideas explicit: that the user can tell the system examples of clearly relevant documents which are known to him [Bookstein, & Klein, 1990][Gelbart & Smith, 1991, p. 229]. The choice of search technique is not a crucial design decision as, given the design choices for interface and document representation, various search techniques can be used as alternatives or supplements to each other. Concept processor A prototype of a system, containing a very large collection of legal cases and formal legislation and operating with techniques in which the considerations formulated here have been realized, is now nearing its completion at the Centre for Computers and Law of the Erasmus University, Rotterdam. We like to refer to it as a learning concept processor. Via the interface the documents can be looked up and given a relevance score through statistical techniques and are indicated by the user to be relevant or not. Concepts are roughly speaking - stored in terms of sets of relevant documents, with concept names, user name and date and time. Relationships between concepts can be traced and/or indicated by the users. In our opinion, such a concept processor is a necessary part of a legal conceptual retrieval system because in law, concepts do not have a fixed and objective content, but can vary from user to user, from problem to problem and from time to time. 6.

Notes

1

For Example, if the user is involved in litigation which concerns two competing legal theories. The accusing party and teh defendant might wish to produce different documents on thier common issue. A refinement of the knowledge file in the sense that an attempt is made to determine the importance of a legislative regulation for a document.

2

90

Some Design Considerations for a Conceptual Legal Information Retrieval System

7.

References

This article may only be cited as: Wildemast, C.A.M. and R.V. de Mulder, Some Design Considerations for a Conceptual Legal Information Retrieval System. In: Grütters, C.A.F.M., J.A.P.J. Breuker, H.J. van den Herik, A.H.J. Schmidt, C.N.J. de Vey Mestdagh (eds), Legal Knowledge Based Systems: Information Technology & Law, JURIX'92, Koninklijke Vermande, Lelystad, NL, 1992. [Belew, 1987] Belew, R.K., a connectionist approach to conceptual information retrieval. In: Proceedings of the First International Conference on AI and Law, Boston, 1987, pp. 116-126. [Bing, 1987] Bing, J., Designing Text Retrieval Systems for Conceptual Searching. In: Proceedings of the First International Conference on AI and Law, Boston, 1987, pp. 43-51. [Blair & Maron, 1985] Blair, D.C. and M.E. Maron, An evaluation of Retrieval effectiveness for a fulltext document-retrieval system. In: Communications of the ACM, vol. 28, 1985, no. 3, pp. 289-299. [Bookstein & Klein, 1990] Bookstein, A. and S.T. Klein, Information Retrieval Tools for Literary Analysis. In: Database and Expert Systems Applications (DEXA'90), Vienna, Austria, 1990, pp. 1-7. [Davis, 1986] Davis D., Semantic Analysis in Legal Text Information Retrieval. In: Automatic Analysis of Legal Text: Logic, Informatics and Law, Elsevier Science Publishers, North-Holland, 1986, pp. 473-481. [Dick, 1987] Dick, J.P., Conceptual Retrieval and Case Law. In: Proceedings of the First International Conference on AI and Law, Boston, 1987, pp. 106-114. [Dick, 1991] Dick, J.P., Representation of Legal Text for conceptual Retrieval. In: Proceedings of the Third International Conference on AI and Law, Oxford, 1991, pp. 244-253. [Franken et al., 1992] Franken H., H.W.K. Kaspersen and A.H. de Wild (eds), Recht en computer, Kluwer, Deventer, 1992. [Gelbart & Smith, 1990] Gelbart, D. and J.C. Smith, Toward A Comprehensive Legal Information Retrieval System. In: Tjoa A.M. and R. Wagner (eds), Database and Expertsystems Applications (DEXA'90), Vienna, Austria, 1990, pp. 121-125. [Gelbart & Smith, 1991] Gelbart, D.and J.C. Smith, FLEXICON, A Legal Text-Based Intelligent System. In: Proceedings of the Third International Conference on AI and Law, Oxford, 1991, pp. 225-233. [Greenleaf et al., 1991] Greenleaf, G., A. Mowbray and A. Tyree, The DataLex Legal Workstation Integrating Tools for Lawyers. In: Proceedings of the Third International Conference on AI and Law, Oxford, 1991, pp. 215-224. [Guidotti et al., 1990] Guidotti, P., L. Lucchesi, P. Mariani, M. Ragona and D. Ticornia, A Simple Intelligent Interface to Data Bases on Environmental Law. In: Tjoa A.M. and R. Wagner (eds), Database and Expertsystems Applications (DEXA'90), Vienna, Austria, 1990, pp. 285-289. [Hafner, 1978] Hafner, C.D., An Information Retrieval System, Based on a Computer Model of Legal Knowledge. UMI research press, Michigan, 1978. [Hafner, 1987] Hafner, C.D., Conceptual Organization of Case Law Knowledge Bases. In: Proceedings of the First International Conference on AI and Law, Boston, 1987, pp. 35-42. [Leith, 1990] Leith, Ph., Formalism in AI and Computer Science. Ellis Horwood, Simon and Schuster, 1990. [Merkl et al., 1992] Merkl, D., A.M. Tjoa and S. Vieweg, BRAND - An Approach for Knowledge Based Document Classification in the Information Retrieval Domain. In: Tjoa A.M., and Ramos I. (eds), Database and Expertsystem Applications (DEXA'92), Vienna, Austria, 1992, pp. 245259. [Merkl et al., 1990] Merkl, D., S. Vieweg and A. Karapetjan, KELP: A Hypertext oriented User-Interface for an Intelligent Legal Fulltext Information Retrieval System. In: Tjoa A.M. and R. Wagner Database and Expertsystem Applications (DEXA'90), Vienna, Austria, 1990, pp. 399-404. [Mulder, 1984] Mulder, R.V. de, Een model voor juridische informatica. Koninklijke Vermande, Lelystad, NL, 1984. [Mulder & Oskamp, 1979] Mulder, R.V. de, and A. Oskamp, Het woordgebruik van het Wetboek van Strafrecht en het Wetboek van Strafvordering. Faculteit der Rechtsgeleerdheid, Erasmus Universiteit,. Rotterdam, 1979, p. 75. [Mulder et al., 1989] Mulder, R.V. de, C. van Noordwijk and H.O. Kerkmeester, Knowledge Systems and Law-The JURICAS Project. In: Martino A.A. (ed.), Pre-Proceedings of the Third International Conference on Logica, Informatica, Diritto, vol. 1, Florence, 1989, pp. 205-216. [Oskamp, 1990] Oskamp, A., Het ontwikkelen van juridische expertsystemen . Kluwer, Deventer, 1990.

91

Wildemast, C.A.M. and R.V. de Mulder [Rose & Belew, 1989] Rose D.E. and R.K. Belew, Legal information retrieval: a hybrid approach. In: Proceedings of the Second International Conference on AI and Law, Vancouver, 1989, pp. 138146. [Salton, 1989] Salton, G., Automatic Text Processing: The transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley, US, 1989. [Vries et al., 1991] Vries, W.S. de, H.J. van den Herik and A.H.J. Schmidt, Seperate Modelling of UserSystem Cooperation. In: Breuker, J.A.P.J., R.V. de Mulder and J.C. Hage, Legal Knowledge Based Systems: Model-Based Legal Reasoning JURIX'91, Koninklijke Vermande, Lelystad, 1991, pp. 28-39. [Wiley, 1989] Wiley, Intelligent Databases, Object Oriented, Deductive Hypermedia Technologies, 1989.

92

Suggest Documents