61st IFLA General Conference - Conference Proceedings - August 20-25, 1995

The role of assessment of services in planning future developments in Parliamentary Libraries

Jennifer Tanfield, Librarian of the House of Commons, Westminster, United Kingdom

PAPER

Given the pressure on all parliamentary libraries to make the best use of theresources available to them, it is increasingly important for these libraries to be confident that they understand their users' needs and priorities. They may therefore need to consider methods of assessing and evaluating the services they provide. This paper lists and discusses some of the feedback techniques used by parliamentary libraries, with a view to building up a reference resource of reports, questionnaires, feedback forms, etcetera, which could be use d by any parliamentary library to initiate or modify assessment procedures.

1. Observation

Probably every legislative library considers information obtained by observation of its users when making decisions about the development of services. Observation can provide simple data such as the level of use of reader spaces.

2. Committees of Members

At Westminster five domestic select committees have been set up to "representMembers' interests as consumers in determining the provision of services to the House." Library services are conside red by the Information Committee, which is consulted about new services, changes to existing services, or any proposal to stop a service. Some Members of Parliament have always channelled their sugg estions or complaints about the Library through the chairman of the Information Committee or its predecessor, the Library Sub Committee.

3. Satisfaction surveys

A number of legislatures conduct annual or regular satisfaction surveys of the whole range of services to Members; this is the case, for example, in New Zealand and Queensland. In other parliaments, such as the Federal Parliament in Australia, the Library conducts its own survey. The surveys are usually undertaken using questionnaires and may be conducted by a n independent agency.

A high level of general satisfaction with the services of parliamentary libraries is usually revealed by these regular surveys. The 1993 94 Report of the New Zealand Parliamentary Service says that "Library based research services received quality ratings of very good or better from 95 percent of the respondents and overall library services were rated very good or better by just over 80 percent of the respondents. In terms of amount of services a rating of adequate or better was achieved from 100 percent of respondents." In Queensland's 1994 survey more than 84 percent of respondents gav e an average score of "high" or "very high" to the services offered by the Library. 98 percent of the respondents to the 1993 confidential survey of the services of the Parliamentary Library in Canb erra rated overall services as either "very good" or "good". This high level of overall satisfaction might cast doubt on the value of such surveys but they do allow comparison to be made over time a nd, provided that the parliamentary library aims to achieve a level of satisfaction approaching 100 percent, can be a useful indicator of change.

These surveys also present the opportunity to question users in more depth about particular services or proposed new ones. Queensland's 1994 questionnaire sought users' views on whether a service pr oviding typed transcripts of recorded material should be instituted, as well as asking for assessments of the quality of certain aspects of the Research and Reference Section and satisfaction with the Current Awareness Service.

4. Other questionnaires

Audit or value for money studies often involve surveys of user satisfaction and the needs which users perceive; this was the case in the Canadian Library of Parliament in 1991 and in the House of Com mons Library in Westminster in 1994. Parliamentary libraries themselves also commission such studies, one of the best known of which was conducted by Ed Parr in 1990 91 for the Department of the Par liamentary Library in Australia. Other detailed studies include the 1990 survey in Chile, and surveys of theResearch Branch of the Parliamentary Library in Ottawa in 1988 and 1991

Questionnaires may be used to test Members' responses to a new service as was done in Westminster in connection with a proposal to store press material electronically. The findings helped to reach t he decision to include the project in the Library's development plan, but almost as useful was evidence (as in the Australian Ed Parr study) that a number of Members' staff were not aware of existing Library services.

5. Interviews with individual Members and their staff

Interviews allow a more targeted, in depth approach than is possible by use of questionnaires alone and are often used in conjunction with surveys based on questionnaires. The Congressional Research Service (CRS) reports that it obtains good feedback from interviews. They may be the best way to learn, for example, why a parliamentarian does not use some or all of the parliamentary library's se rvices. Recently the Head of the Canberra Parliamentary Research Service "instituted a program of visits to Senators and Members...to seek feedback and suggestions on future directions." It will be interesting to learn about the value of this initiative.

There can be some difficulties in obtaining representative feedback using the interview method. Legislators are very busy people who are reluctant to devote time to interviews. They also tend to ha ve individual ways of working, which makes essential the selection of a sample reflecting a wide spectrum of approaches to the job.

6. Focus groups

A focus group is a group of users with common characteristics who are interviewed as a group. CRS uses focus groups with success, obtaining honest and useful assessment. However, when the House of commons Library considered the use of focus groups it was warned, both by interviewers with experience of Westminster parliamentarians and by the Members of the Information Committee, that MPs could not be relied upon to keep such appointments. The benefits to be derived from carefully selected focus groups could be mitigated by failure to attend on any significant scale.

7. Feedback forms attached to written reports

Returnable feedback forms with boxes to tick and space for comments have been attached to House of Commons Library Research Papers since January 1994. Only a small number of forms are returned (on a verage less than one per paper). Rather surprisingly, more of the forms have been returned by Members than by their staff. To date 63 percent of forms returned have given the paper maximum box mark s, and comments are constructive and may include criticism. This response is in contrast to the experience of CRS which dropped the use of returnable forms more than ten years ago because those who tended to send back questionnaires generally had very high opinions of CRS work. The Research Branch of the Canadian Library of Parliament includes evaluation forms with each response, apparently with a si milar level of satisfaction to that experienced by CRS.

8. Telephone follow up of responses

The concept of selecting a sample of responses and telephoning parliamentarians to seek feedback shortly after they have received the information is an attractive one. It was used by the value for m oney team who investigated the research services of the House of Commons Library. They reported on 31 responses but did not say on how many occasions they had failed to contact the Member. Poor res ponse rate is likely to be a problem with this method of obtaining feedback bearing in mind the pressures on parliamentarians and the difficulty of contacting them by telephone.

9. Monitoring of complaints or suggestions

Given the high level of overall satisfaction with the services of parliamentary libraries, the number of complaints is likely to be small, but can provide useful management information. Westminster library staff are now required to record all complaints, comments, or suggestions, whether these are made orally or in writing. Results are collated and consider ed twice a year.

10. Analyzing information about use of services

Information about the extent to which Members use particular parts of the parliamentary library service is important in allocating resources to maximum effect. With the coming of computers this sort of analysis has become much easier to conduct. Requests put to the research service in Westminster are now logged into an computerised ENQuiries database and considerably more detail is readily ava ilable about, for example, which Members use which part of the service and the deadlines imposed.

Most computer databases routinely produce information about usage data on the number of times particular passwords have been used, time of use, and the part of the system interrogated, as well as pe rformance data like speed of response. Modern telephone systems can also provide data on the time taken to answer a call and the number of occasions on which a caller hangs up. The problem is to de fine the user reports so that they show important trends. It is easy for managers to be overwhelmed by reports that are too numerous and too detailed. Future methods of evaluation

It is clear that parliamentary libraries are rapidly adopting electronic methods of accessing and delivering information and in this area the emerging legislatures are often ahead because they have g one directly to information technology in the absence of alternative systems. To avoid the danger the managers will be overwhelmed with usage information generated by library systems it is important that specifications provide for reports that contribute to performance measurement and assessment. Westminster's ENQuiries database allows for the calculation of elapsed time between an enquiry bei ng placed and its completion so that performance indicators can be monitored.

A "help desk" is a common feature of computer systems made available to library users. Requests placed by legislators and their staff, both to help desks and to those providing user training, often do not seek technical help with hardware and software, but related to problems with documentation and indexing. These are areas where the parliamentary library could make improvements by amending pr ocedures or developing more useful menus and explanations.

It is technically possible to put feedback forms or questionnaires addressed to users on computer networks, perhaps linked to particular databases, but users might be irritated by this method of seek ing feedback. Speed is one of the great advantages of using information technology to obtain information and users may be resistant to anything that makes it slower for them to access, or exit from computer databases.

Conclusion

The House of Commons Library is in the process of moving from reliance on observation and assumptions about Members' needs to more systematic collection offeedback. Our 1994 questionnaire to Members and their staff on use of press articles has given us confidence to seek money and staffing to introduce a networked electronic storage system in 1997/ 98. In the early summer of 1995 a firm of con sultants will be asking a sample of our Members and their staff about how well the Library's services meet their needs. We expect to follow up this survey in future years and will use the informatio n gained to prioritise the distribution of resources.