INFORMATION FILTERING : SOME PROBLEMS AND CHALLENGES
By Ding Choo Ming
SEACOM '96 Malaysia: 2nd International Conference on Communications and Computer Networks - New Applications and Trends for High Speed Networks. Petaling Jaya: Gabungan Komputer Nasional Malaysia, 1996. pp.77-89
Abstract: Information is power; but the irony is that an abundance of it can give rise to massive and unwieldy problems in accessing it. To assist users cope with the barrage of information, many libraries and information centres have generated such in-house current awareness services as information filtering, selective dissemination of information, news clippings and journal routing for different user groups. This shows that getting the right information at the right time is not getting any easier despite rapid developments in communications technology and IT. Thus, information filtering for example, will continue to be a major concern in information management. Its basic premise is customisation. It is a myth that it can be automated fully or reduced to simple logic. But, with the help of a powerful and friendly technology, cooperation from clients and some ingenuity, information professionals can improve this service.
1.0 INTRODUCTION
Rapid developments in communications technology have added new pathways and highways to the global information network, resulting in a deluge of information. This phenomenon of information overflow or overloading, used interchangeably with such other terms as ‘information explosion’, ‘information overabundance’ and ‘information pollution’ has threatened the retrieval of relevant information with the possibility of affecting negatively the original intent of making global information sharing more efficient. The reason is that quite often only a small fraction of the information available is actually relevant. It is here that lies the importance of such information alerting and retrieval services as information filtering (hereafter as IF), selective dissemination of information (SDI), news clippings, and journal monitoring for different user groups. Clearly, once they have become extensively established, they may complement the commercially available abstracting and indexing journals. With time, they will become increasingly important tools in information management.
1.1 Although we always filter information in the sense of choosing the best information that suits our needs in whatever we do, be it shopping, reading, eating, choosing friends, etc. the concept of information filtering is relatively new in information management. It is different from the conventional library reference service which is aimed at keeping the interested individuals or groups informed of the latest developments in their fields of interest, in the sense that researchers under the pressure of information overload and the stress of workload, expect the information retrieval systems and information professionals to filter the right information in the right amount and at the right moment (Rosenfeld & Holland l994:28). In short, users want the maximum amount of current and relevant information with minimum effort. Their main concern is both currency and relevancy.
2.0 CURRENT STATUS IN COMPUTERISED AND MANUAL SYSTEMS
Though information filtering has undergone radical changes following rapid developments in IT, it is still labour intensive. Some of the radical changes in IT referred to include the development of automated filtering systems to replace the manual ones. The systems are also user-friendly because they allow natural language indexing, as opposed to the pre co-ordinated terms, or controlled vocabulary practised in the past. But, can these processes which combine advanced information retrieval tools with high speed computing mitigate the problem of information overloading? Rosenfeld and Holland are doubtful that automatic filtering alone can be the practical answer to this problem because of the inherent characteristics of data as well as the unique user behaviour and expectations (l994:28). Being a very personalised service, it is thus expensive to operate. Regardless of whether it is done manually or electronically, it is generally complex to operate, not only because it involves a differentiated array of products and services in a variety of forms, but also because it involves a combination of processes: locating, reviewing, matching and filtering.
2.1 Before discussing the various factors influencing information filtering performance, it is understood that the basic mechanism used in IF is the keyword search and matching using the Boolean, vector space and probabilistic retrieval techniques. In discussing their respective effectiveness in terms of recall and precision, Belkin and Croft (l992) explained that the first technique is based on matching keywords in the interest profiles and the texts, while the second is on weighting computed terms on the basis of statistical distribution of the terms in the database and that in the texts, and the last one on Probability Ranking Principle, that is ranking the texts in the database in the order of their probability of relevance to request. Generally, information filtering, whether manual or computerised, is done by some kind of vocabulary control and matching. Major activities are illustrated in the following diagram taken from Belkin and Croft (1992:31).
The relevancy of documents is usually determined by the occurrence of the pre-determined and selected keywords in both the interest profiles and the texts. The difference lies in the methodology. In the past, the search and matching was done manually in batch modes on a weekly, or some other basis regularly; while today such operations can be done automatically based on computer-matching.
2.2 The terms to be searched can include authors’ names, journals, journal codes (and also molecular formulas and registry numbers in chemistry), besides keywords in titles and texts. It is undoubtedly true that the greater the variety of approaches used, the better and more productive the search results. This is also true in the traditional card catalogue. It is the additional cards filed in the subject catalogue that help in more fruitful searches, although some degree of redundancy and irrelevancy is inevitable. The advantage of the electronic system over the manual lies in the fact that it is more interactive in nature allowing information seekers (i.e. both the end-users and information professionals) to manipulate the search terms used. They can combine keyword descriptors with journal titles or authors’ names or other terms they wish to search. An extension of this technique is the method called the clustering technique, which is the grouping of documents with related terms and phrases. The reason for developing such a technique is the belief that documents sharing common words are always judged to be related, and if so, they are also held to be relevant. Foltz develops this argument further by introducing Latent Semantic Indexing, ‘a technique that captures deeper associative structure than simple term-to-term correlation’ (l992:53). He finds that this technique can improve prediction performance over keyword matching by about 13%. Generally the recall and precision capabilities are governed by specificity and exhaustivity in both indexing language and search strategies as illustrated in the following figure taken from F. Wilfrid Lancaster Information Retrieval Systems. New York: John Wiley, 1968 (p.71).
2.3 Following the recent upsurge in interest in natural language searching, indexing using word-based representation or lexical scanning to identify the relevancy of documents has proved cost-effective in well defined subjects with relatively stable terms, especially in the technical fields. Attracting great interest from end-users, it certainly holds great promise for the eventual improvement in the effectiveness of the information filtering system. As there is no single technique that can effectively be used in IF, different methods should be employed in various combinations. Certainly, matching interest profiles against the words used in respective texts is rather straightforward. It has, however, been proven to be ineffective in respect of not only interdisciplinary subjects but also of texts or titles with ambiguous words. Here, the role of trained and experienced information professionals is crucial. Their skills may be viewed as the institution’s intellectual capital. They can identify, specify and interpret texts which do not use familiar terms, or use them in a fixed pattern, but rather use terms or words of the authors’ own choice to uniquely describe their subject of interest. This is true of literature. The following are examples of books on autobiography.
In predicting and speculating the contents of some documents, the latest edition of classification, indexing and abstracting tools or journals would certainly be useful. In the above examples, subtitles if any, would be of great help. Nevertheless, due to subject complexity, there is a growing appreciation of the importance of professional tasks. Consequently, a promising alternative will be the human mediation in computerised systems or a configuration of computerised and manual systems.
2.4 IF shares many features with other information alerting and retrieval services. Like SDI, it matches a vast amount of incoming data with an interest profile, thus leading users to the relevant documents. Belkin and Croft have, however, identified three major differences between IF and information retrieval (IR). Firstly, the user profiles used in IF work on a long term basis, while those in IR are short term; secondly, requests in IF are for the latest documents, while requests in IR are for both current and retrospective ones; and, thirdly, filtering means the removing of irrelevant documents, while retrieval means searching for the relevant ones (1992). Nevertheless, McCleary, thinks of IF in the context of a virtual library where the summarised information is provided electronically through online or wireless connections, such as DIALOG, to clients before it appears in printed or electronic form (1994:40). Now, to manage and filter the ever increasing volume of information for the varied needs of the clients, many institutions have resorted to computerised filtering systems. This type of filtering service as exemplified in DIALOG, MEDLINE, LEXIS and other online systems would certainly herald a significant revaluation of the role of information alerting services, particularly in academic and research institutions in the way that the conventional current awareness service had once benefited from Swetcan, (developed by Swets Subscription Services of the Netherlands) in the past. Despite the differences of opinions concerning the roles and functions of various forms of IF, there is certainly a consensus on the provision of the latest and relevant information. With the information superhighway and the accompanying sophisticated hardware and software within our easy reach, customer demands for customisation have increased tremendously. Customisation, meaning meeting clients’ specific needs, is the ultimate goal in IF. Customised information will continue to be an intriguing topic for information professionals and information seekers worldwide. But, many institutions are just unable to exploit their present staff capabilities and expertise as well as the technology to provide the services their clients expect due to many problems beyond their grasp.
2.5 Though a new concept, IF as discussed here is not as revolutionary an idea as defined by McCleary. He justifies his definition of IF by claiming that the traditional information management service contributes to information overload, while IF minimises it by extracting highly relevant documents electronically to targeted clients (l994:33). Examples quoted by him include CompuServe Clipping Service and other major information providers such as Reuters, Associated Press, Wall Street Journal, CNN, Dow Jones and Financial Times. The service as envisaged by McCleary may change our perception of the value of information and so revolutionise the present information filtering and alerting services in years to come, but most information centres, other than those that took part in the trial tests, do not accept this scenario. The reason is that not only are many libraries not in the position of providing this service, but also that most researchers do not require information as current as when it is despatched to news agencies or concerning Presidents, Prime Ministers or other heads of state who need to make instant key decisions. Researchers and scholars would continue to need relevant and reliable information from both the current and retrospective files, in their own time and at their own pace.
2.6 Following the argument that automation is the most appropriate step in improving our ability to meet clients’ demands for fast service, many online SDI, news clippings and other IF related services were launched with great enthusiasm, but few have survived. As they are mainly in the experimental infancy, most online information filtering systems are not only impractical for most information seekers, but also too expensive. Their impracticality and high cost are due to the fact that the volume of customised products is not only small, but also difficult to deal with. Examples are Pinpoint Information Corp , VA, USA; Individual Inc of Cambridge, MA, USA and also Bellcore Customized Information Delivery System discussed by Hawkins (l990:117-120). In addition, Rosenfeld and Holland have also demonstrated several drawbacks in automatic filtering using Cognitive Science and Machine Intelligence Laboratory (CSMIL) at the University of Michigan from 1991 to 1992 (l994). They discover that the quality of filtered products is proportional to labour the users have invested in them. Certainly, no busy manager or casual user can afford that. Most information customisation services are still conducted manually as information in various formats are not as easily automated as the technical operations incorporated in the assembly operations. There is also no software or hardware possessing the ability to perform specific and fast information filtering.
2.7 The computerised filtering system also encounters many of the problems common to the manual system: obvious information can be filtered easily but not the obscure ones. Both systems have yet to succeed in translating clients’ language into search terms. Until they do, Rosenfeld & Holland also maintain that automatic or computerised filtering systems will perform no better than the traditional manual system (1994:29). Certainly, electronic information can be filtered using the newly created technology, but not those in printed form or on microform. The inevitable outcome of this is that clients may miss the opportunity to benefit from an increasing amount of unfiltered and probably relevant documents. In other words, technology alone cannot sell information. In all probability, only efficient information management that combines professional knowledge with IT can achieve this important objective. Efforts focused on customisation still defy automation and standardisation especially in the very personalised IF service where operations are not so easily automated. In stressing that not all relevant information can be filtered using the currently available IT systems, Foltz and Dumais too are of the opinion that there is no one system which can be used (efficiently) by all the users, or even the same user at all times (1992).
3.0 PROBLEMS IN DESIGNING INTEREST PROFILES
Some of the problems raised in the foregoing section are due to the inherent difficulties in expressing the information needs straight into the information filtering system. This makes the task of designing a high quality interest profile a particularly difficult one. Due to the problems in computerised searching, users should discuss their information needs with the professional staff who will translate them into a searchable profile. In the past, such a profile would contain terms taken from a thesaurus or any other controlled vocabulary. Although, it is now possible to use a combination of keyword descriptors in formulating a search strategy, it is best for end users who have a fairly wide range of interests to submit and modify their requests from time to time. Updating of interest profiles is necessary because even though researchers are known to work in well defined areas, their needs may change following a new assignment which is likely to happen in this era of globalisation and diversification.
3.1 Whenever and wherever possible, clients should provide only the relevant search terms (regardless of whether it is a single term, a group of alternative terms and specific term combinations) that could be used for searching and filtering information. This is because terms and concepts that are too broad will generate a lot of materials. This will not only consume more time, but could also produce too many irrelevant documents. On the other hand, it could yield too few alerts or none at all if the terms are too specific. Anyway, high quality profiles usually come from well organised and literature-conscious scholars and researchers working in well-defined fields. For example, the most likely index and search terms used in sanitary engineering will be:
Air pollution
Environmental pollution
Industrial and factory sanitation
Industrial and factory wastes
Low temperature sanitary engineering
Municipal engineering
Municipal refuse
Rural sanitary engineering
Sewage collection and disposal systems
Sewerage
Solid wastes
Street cleaning. Litter and its removal
Water distribution systems
Water pollution
Water purification
Water treatment and conditioning
Water supply (for domestic and industrial purposes)
A good quality interest profile is certainly a problem solving device, unlike profiles developed by people who depend on ‘grapevine’, face-to-face contacts or by ‘hit-and-run’ researchers. The latter are generated by those who move quickly from one topic to the next and seldom possess a stable interest profile. Without a stable and high quality profile, it is difficult for information professionals to specify and identify ‘filters’ or ‘alerts’ and so select and filter the relevant documents from those scanned. Clearly, only quality profiles can help information professionals refine the search in order to reduce, if not entirely eliminate overlapping and ambiguity.
3.2 Since, as argued earlier, designing an interest profile is a continuous process, it is obvious that there will always be room for improvement on both sides. Originally, the interest profile is created based on a few familiar, important or novel words that form the core of the profile for a specific subject. Over time, it may become necessary to modify it in view of the urgency of a new assignment, new sources of information and so on. Nevertheless, the core of the file should still remain the same: keywords and concepts in the profile are used not only to match but also to predict which other incoming documents scanned from time to time are relevant. Though this procedure is found to be successful in predicting user preferences for general topics, it sometimes fails as a matching technique. One of the reasons for the failure is that the words writers use do not unambiguously reflect the contents. Also, a single word may carry many meanings. A typical example is the article ‘Dua orang pengarang, tiga suara gelap dan suatu sub genre’ by Muhammad Haji Salleh, published in Archipel (Paris) 42(1991):155-176. It is about two Malay writers in the late 19th century. Similarly, a concept can be expressed using different words. Seen in this light, word stems are not very reliable as an alerting device. Therefore, professional expertise, judgement, interpretation and prediction of what documents clients might prefer is unavoidable.
3.3 Another reason why the computerised information filtering system cannot perform better than the manual system is that many users do not bother about the quality of their interest profiles. Many readers are more concerned about the availability of the right documents at the right moment and in the right format. Many of them develop their interest profiles with terms taken randomly from common subject headings or from indexing and abstracting journals. Worse still, they do not bother to revise, refine and modify the terms to keep pace with the latest developments.
3.4 The trend towards interdisciplinary work, combining specialisation with multidisciplinary research is likely to cause some problems in information filtering too. The trend is likely to grow as it shows an increasingly practical orientation. As a result, in addition to core journals in a particular field of research, there are many interdisciplinary journals published to cater for the needs of researchers in various related fields. On the whole, however, current research in any field ( be it medicine, chemistry, biology, history or IT), seems to be moving towards integrating specialised knowledge from many disciplines. An example of such a development is the linking of what were previously unrelated concepts, such as mathematical analysis in literary criticism, or IT in business management or medicine. Once popularised and used widely as index terms, such conceptual combinations would pose no problem. In the absence of any feedback from clients, (i.e. without the necessary quality filters) information professionals would be unable to search for new documents hitherto unused or unindexed. In other words, users should play their part in assisting information professionals to ensure that they get good quality service. Without informing the professional staff of the changes in their profiles, users will not be able to keep up with the latest publications. Still less can they expect to enjoy customised service.
3.5 It is not unusual that some information-privileged and IT conscious executives and researchers are able to retrieve and filter documents for relevant information themselves following the growth in indexing systems using natural language. By combining index terms learned from information professionals, they know how to select individual index terms and combine them to suit their particular needs (Ford, l980:l06). They are certainly proactive and eager to retrieve and filter the documents themselves. But they can only perform relatively straightforward matching and filtering operations similar to the performance of mechanical operations centred around the self-issuing system, CD-ROM searching, OPAC searching and photocopying (Stoker, l993:172). The situation here is similar to the way nurses can conduct minor operations in hospitals with authority, although they cannot perform more complicated surgeries. In the same way, executives and researchers cannot perform more complicated matching involving the manipulation of different permutations of index terms (such as chaining, rotating, cycling, or shunting role indicators, relational operators, phase relationship indicators). Even using the complex combination of concepts and terms makes searching no easier as it is still technically too demanding for casual users. Nevertheless, the trend towards title, keyword and free-text searching - i.e. toward more powerful and flexible searching systems - will to a certain extent be beneficial to information professionals handling subjects that they have no expertise in. Such developments will allow more interactive information filtering which is attributable not only to the increasing availability of free text databases, but also to the increasing computer capabilities. In other words, despite the rise of powerful LANs, effective user interfaces and the other latest developments in IT, there is still nobody better equipped to handle the filtering task than the experienced and trained information professionals. That being so, both clients and information managers need to co-operate to develop a level of multiplicity of possible frames of reference including an open mind, authority dependence and dogmatism, as stressed by Ford (l980:101).
4.0 NEW ROLES OF INFORMATION PROFESSIONALS
The best way to filter information in this transitional stage of IT development (from manual to automated system) is to engage trained and experienced information professionals who possess subject expertise and IT. Without these qualifications, they may be unable to provide a satisfactory service in filtering the right information at the right time and in the right form (Mason, 1990). As effective and efficient intermediaries, information professionals with IT know how to integrate information from various data bases and files (both online and printed) for further processing. It is by filtering and repackaging that they add much more value to the information, hence making it really relevant. However, the danger is that because of time constraints and the level of technology available, many professionals may just choose to be mere generalists busy with their demanding work schedule in the selection of materials and administration. Consequently, they may not only suffer from the of role ambiguity syndrome, but there might also be leakage in information filtering due to much of their time, effort and talent being spent on other tasks. This is particularly so in libraries where the current trend is towards a streamlined staff establishment due to budgetary constraints and the intervention of IT.
4.1 To offer quality service, quality staff is as vital as the technology. The fear that staff will not be needed in information management in the future when IT becomes more developed is baseless. It is a well known fact that IT can help in managing massive data, but it cannot filter the information and documents without human mediation. Through experience or trial and error in working with a large number of databases and files, information professionals have become skilled in filtering information and documents from sources that an inexperienced user would not have thought of or the inexperienced staff would not have the expertise to search efficiently (Ding,1995a; 1995b). As the exploitation (including filtering) of information needs experience, interpretation, judgement and creativity, the position of experienced and trained information professionals should be considered seriously in years to come. In addition, IT-privileged professionals can even handle the sophisticated computer systems and the user-unfriendly databases which are incomprehensible to many casual readers initially. Despite the increasing popularity of do-it-yourself CD-Rom searches, OPAC and other services, more users are in fact approaching the professional staff to obtain help in filtering and retrieving information and documents more quickly. It is clear that the difficulties users encountered in filtering and retrieving relevant information have made them realise the importance of the expertise of the information professionals in formulating and reformulating search strategies, especially in more complex searches. More importantly, the satisfaction users get from the professionals’ expertise has enhanced the professionals’ position, image, authority and status. Similarly, information professionals cannot perform their task successfully without the necessary support from clients, the management and IT.
4.2 The basic premise of information filtering is customisation. Measured in terms of relevancy and timeliness of the information provided, it is inevitably that information professionals should be aware of the information needs of the users. It is here that the real problems arise and the test of quality service begins. As the information needs of the clients may change in an almost infinite number of ways (including combinations of concepts which may be different from the established interest profiles), information professionals have to anticipate the various ways a new concept may be expressed. As no two information filtering situations are the same, Mason argues that to cater for a client’s need is not purely a technical problem (l990:130), but it involves a certain kind of interpretation, speculation and judgement of high calibre such as that expected of managers in business. This is likely to be particularly true of a ‘volatile’ and creative field such as literature as illustrated earlier. The lesson learned from this is that combinations of different matching techniques should be employed to aid information seekers to retrieve the few relevant alerts they need; otherwise, the sheer amount of documents generated will be overwhelming.
4.3 In providing quality information, information professionals should have the certain qualities. They should (l) be familiar with the existing collection and databases, (2) know where to look and how to look for documents and information, (3) possess analytical and predictive abilities as well as diagnostic skills to be able to exercise sound judgements, (4) be highly experienced, (5) possess expertise in specific subject areas, and (6) constantly keep up to date with information and IT developments (Duke, l985:157). Apart from these qualities, Ford (l980:101) adds that they should also be open-minded, besides being authoritative. All these qualities are important to minimise complaints (from professional staff) that users are ‘expecting too much’, or are ‘unreasonable’ or ‘do not understand’. These complaints tend to stem from the users’ protest that they have been getting too many irrelevant documents despite the fact that there is an interest profile. The professionals may be experts in one field, but not in others. Again they may be able to pick up the fast developing technology, but not the expert subject knowledge in unrelated fields. It is here that much can be learned from secondary services employed in indexing and abstracting journals. In these reference sources, the current articles are not only classified, but are also provided with cross-references. All these linkages provide vital clues that could be of immense benefit to information seekers (embracing both the clients and the professional staff) in accessing the body of literature. By all standards, many biologists, physicists, pharmacologists and others are also interested in chemistry, for example, because chemistry is a major discipline in the physical sciences, thus penetrating into areas outside their own field.
4.4 Following the growth in interdisciplinary studies, information of interest to researchers is scattered in various journals: some considered as core, while others as peripheral in a particular discipline. One way of evaluating documents of potential value to biologists, for example, is to locate them through the Chemical Abstracts, Science Citation Index and many others. The problem here is how information professionals play their roles in reconciling ‘conflicts’ when different users with different goals pursue the same documents. Here lies another test of good quality staff, because it is here that their judgement, experience, knowledge and management skill are tested in filtering the right information to the right client in the right amount. Experience and lessons gained from previous events and various abstracting and indexing journals, could help in the development of diagnostic and predictive abilities in new situations (Duke, 1985:157). Due to the complexity of matching, only the trained and experienced information professionals (with the help of clients and the necessary support from the management) are capable of handling such diverse situations as those involving, on the one hand, a lot of interrelationship of unrelated subject disciplines; and on the other, many disciplines areas which are becoming more focused in scope. In the former case, it is difficult to establish a clear demarcation of IT in medicine and in the latter, it is equally hard to distinguish between physical chemistry and chemical physics, or to know where chemistry ends and physics begins. If these distinctions cannot be clearly determined, a normal matching will naturally generate virtually everything including information peripheral to the subject. The question of ‘perfect’ or ‘imperfect’ matching is dependent on the weightage or emphasis ascribed to the intellectual contents of the documents. It is an anathema to the information professionals that they have to worry about the relevancy of their service: that is, providing the right texts in the right form at the right time.
5.0 FUTURE CHALLENGES
Information filtering will in future be both the fundamental and ultimate function of information management. In the next few years we will see an accelerated improvement in information filtering systems developed by data base and network suppliers and vendors. Their effectiveness would certainly depend on how well they can be customised to meet the varied needs of users. The tremendous pressure on libraries, for instance, to improve information filtering and retrieval services to match the expectations of clients has prompted the library administrations to adopt more user friendly databases and systems. But, libraries can only offer what the information professionals, trained and possessing expertise in certain subject areas, know what they do with or without IT. Thus, effective information filtering is much more demanding than simply responding passively to demands placed upon libraries. It requires effective team work among users, professionals and technology. Though it is possible now for end users to access and manipulate information undreamed of in the past through the powerful packages such as Microsoft Access and Lotus Approach, it is still far from the case that we can easily access relevant information through INTERNET, CD-ROM or Networking. On the other hand, it is very much the case that professionals who play various irreplaceable intermediary roles, can add value to information, thus making it relevant for users. They are not looking for more materials, but ways to cut down on the amount of irrelevant documents (Gotlieb, 1994:20). The future trend in IF will be the provision of the summarised gist, the core, or the kernel of documents as discussed by McCleary (1994). But for the foreseeable future, this customised service can only be given within a predetermined range of staff capabilities and subject to the organisational structure in each library.
REFERENCES CONSULTED