European Commission
Directorate General XIII

Electronic publishing and libraries 


Telematics for Libraries

Annual Peer Review, 1998




1. Introduction and Background to this review

2. Workplan and objectives

3 Review

4 Review Meeting

4.1 Presentation of the project
4.2 Discussion
5 Summary of Reviewers' Conclusions 5.1 Compliance with objectives and workplan, including usage of resources and impact of previous review recommendations (if applicable)

5.2 Approach, methods and results

5.3 Management, dissemination and concertation

6 Reviewers' Recommendations 6.1 Overall recommendation

6.2 Recommendations for future work.


Annex 1: Report of Prof. M. Casey

Annex 2: Report of Mr P. Holm Christensen

Annex 3: Review meeting agenda

Annex 4: Review criteria

Annex 5: Annual peer review report of the project

lb-4086/a-chilias: Children in libraries: improving multimedia virtual library access and information skills

second and final review report

30 september 1998


1 Introduction and Background to this Review

CHILIAS is a libraries sector project funded under the 1995 Call for Proposals of the Telematics Applications Programme. This was the first Call for Proposals for the libraries sector under FP4. CHILIAS addresses Action Line A: Network-oriented Internal Library Systems, Call Topic 1 - Integration tools and interfaces for library systems in the local network.

This was the second and final review and was held in Luxembourg on the morning of 30 September 1998. The review team comprised: Professor Mike Casey, Infosolve Ltd, Ireland; Mr Peter Holm Christensen, Randersegnens Biblioteker, Denmark; Ms Pat Manson, DGXIII-E/4 (chair).

Present from the project were: Ms Ingrid Bussman, Stadt Stuttgart; Mr Tiago Carvalho, ISEGI, Universidade nova de Lisboa; Ms Kirsti Tuominen, IT Centre for Schools, University of Helsinki.

The materials circulated in advance of the meeting comprised:

Ref Description Type
D 4: Prototype 1:
Functional and acceptable demonstrator of the virtual library with modules "act" and "net" in Stuttgart and documentation
D 5: Prototype of skills: Demonstrator and documentation Integrated in all URLs see below as "Infoton"
D6: 2nd Prototype:
Functional and acceptable demonstrator of the CHILIAS service with modules "act" and "net" in all partner libraries and documentation
Software + documentation
(+ all partner sites)
D 7: Verification Report. Final version Report
D 8: Demonstration plan. final Report
D 9: Demonstration report. Final version Report
D 10: Validation report. Final version Report
D 12: Exploitation plan. Final version Report
D 17: Final Report on project results Draft only

2 Workplan and Objectives

The consortium comprises the following full partners:

Stadt Stuttgart- Kulturamt Stadtbücherei (coordinating contractor ), DE

Hellenic American Educational Foundation, Athens College Library,GR;

Diputació de Barcelona Servei de Biblioteques; ES;

Universidade Nova de Lisboa Instituto superior de Estatistica e Gestão de Informação, PT;

Borough Council of Gateshead Libraries and Arts Service; UK

City of Vantaa, Vantaa City Library - Culture and Education, FI

University of Helsinki - Vantaa Institute for Continuing Education - Information Technology Centre for Schools, FI

The following partners are associated to the full partners in their respective countries:

Costeas Gitonas School Ltd, GR

University of Sunderland Information Services, UK;

Association of Finnish Local Authorities, Education, Cultural Services and Leisure Activities Unit; Akateeminen Tietopalvelu ATP Oy; University Turku, Department of Education -Centre for Learning Research; FI.

Ravensburger Interactive Media GmbH; and IBM Deutschland were sponsors.
The main objectives of CHILIAS are: - to develop a new concept for European children’s libraries of the future as a stimulating environment for innovative learning and creative use of multimedia and networked technology with links to traditional library services.

- to enhance the competence of children using interactive multimedia and networked information and communication systems and to improve their information seeking skills in new learning environments.

To meet these, the project envisaged 4 complementary and integrated applications, in a WWW environment, namely: LIB (virtual library module); ACT (interactive application for creative input from children); NET (discussion and feedback tool for children); and SKILLS (information skills tool). In development, these were integrated into a single application environment named the InfoPlanet.

The workplan is structured in 3 main stages, each concluding with a major milestone.

  3 Review

The first review covered stage 1 - user needs analysis, content and functional specification - and some of stage 2 - development of first prototype. This second and final review of the project covered the completion of the prototype and stage 3 - the optimisation of the prototype and service in each partner country, verification, demonstration and validation, as well as dissemination and exploitation. The review took place at the end of month 29, on completion of the project work and just prior to finalisation of the final report.

4 Review Meeting

4.1 Presentation of the project

The project presented how it had met its objectives through the development of the InfoPlanet. This integrates interactive Web pages for children, as a new environment for the learning, reading and playing activities of the children's library and includes a guestbook function (for structured communications between children and with the library), a storyboard function (for creativity), and a skills function (Infoton for learning exercises in understanding and accessing information).

The project had completed its development and demonstration activities on schedule and according to the workplan, with the extension of 4 months covering only the final project workshop on 24/25 September 1998, the proceedings and the finalisation of the final report.

Key issues during this review period were the demonstration and validation, development of the exploitation plan and the final report. Demonstration varied between countries according to the local infrastructures and organisation of schools and libraries. Training was carried out with library staff in the use of the administration tools and in creating simple Web content. Validation questionnaires were sent to librarians, teachers, children, parents and included those not previously involved with the project. The overall results were very positive, with the new service seen as beneficial and interesting. Findings seemed to indicate that InfoPlanet led to increased membership and use of children's libraries and of the physical materials.

Exploitation strategies had been explored at length by the project which had identified a clear difference between internal exploitation (ie continued use and development of the service by members of the consortium) and external exploitation (commercialisation by a third party). All participants have their plans for continuing the service and integrating it into their operational environments, according to the different country models. No suitable strategies or candidate organisations to support external exploitation have been identified.

Demonstration: the demonstration covered the different types of content on the InfoPlanet sites, as illustrations also of the different creative activities set up during the project, the guestbook and storyboard facilities, and the skills module.

4.2 Discussion

This covered the question of intuitive versus structured approaches to learning - the project felt it maintained a balance of both, but there was general agreement that new non-linear approaches to learning were more encouraged in the approach adopted for InfoPlanet. Also raised was the management and quality of the content - children require immediacy of response and to see their input immediately available but this has to be weighed against mis-use of the system and creating inappropriate or meaningless information, especially via the guestbook. CHILIAS has developed administrative tools which allow inappropriate content to be removed. The final discussion point concerned validation, firstly on whether the questionnaire methodology could realistically get valid responses from children and secondly on the different levels of validation activity carried out in the different partner countries. This was explained as a natural consequence of the library/school situations in each.

5 Summary of Reviewers' Conclusions

5.1 Compliance with objectives and workplan, including usage of resources and impact of previous review recommendations (if applicable)

The project fully meets the objectives of the libraries sector and of Action Line A, providing a strongly user-oriented example of an innovative library service for children. It is extremely timely, coinciding with developments in multimedia resources on the Web, on their use and impact on skills and learning. The project has taken cognisance of these developments and fostered an important and closer relationship between librarians and other actors. The empirical results of this project will serve as very useful input to the theoretical work needed in the field of pedagogical skills and learning in this new resource environment. It has adhered to timescales and to targets, with no deviations in either. The use of resources is balanced across the consortium and tasks, and the reviewers noted and commended the considerable additional uncosted effort contributed by the partners.

No major modifications were requested at the previous review and the exploitation and validation issues noted have been addressed.

5.2 Approach, methods and results

The approach is strongly user-oriented, involving interactions with teachers and school libraries, authors, and with children as active content creators. The methodologies and technical solutions are appropriate overall and responsive to the changes in the wider environment. The project has successfully avoided remaining static in approach or objectives. The integration of the original modules into the InfoPlanet is admirable, including the decision to integrate the skills component also as a dynamic tool. In the management of the content, a good balance has been struck between the need for immediacy in encouraging the creativity of children and the need to maintain quality control of the information and resource. The results of the validation are indicative and qualitative rather than quantitative but this is acceptable and realistic in the context of such a broad-based project. The quality of technical deliverables is outstanding.

5.3 Management, dissemination and concertation

The management is clearly very effective, retaining the dynamism and proactive approaches noted in the preceding review. The use of the external project manager has been beneficial. The standard and range of dissemination activities is impressive - there is good balance and input from all participating partners and countries, outreach to other countries, use of different dissemination media and channels (web, press, radio, television, video, video-conferencing as well as professional library journals).

5.4 Exploitation potential

There is a clear exploitation path - partners have all committed to continue InfoPlanet in their operational services, and have taken appropriate steps to accommodate the necessary organisational changes needed to do so. The consortium has concluded an informal agreement to continue to communicate and exchange information. The impact for replicability, given support for training and know-how transfer, is extremely high . Establishing InfoPlanets through key actors in other EU countries should be encouraged.

The project encapsulates a number of key and emerging issues in terms of education, educational curricula, and the information society and the results of CHILIAS and awareness of its activities should be filtered through to policy makers in these areas.

6 Reviewers' Recommendations

6.1 Overall recommendation
Successful completion

Project: LB-4086/A-CHILIAS
Reviewer: Prof. Michael Casey
Review: Final
Review date: 30 September 1998
Location: CEC-DG XIII, Euroform Building, Gasperich, Luxembourg
Report date: 05 October 1998

A. Objectives, Work plan and Resources

The project aimed to develop a new concept for European children’s libraries of the future as a stimulating environment for innovative learning and creative use by children of multimedia and networked technologies with links to traditional library services. The outcome has been the InfoPlanet, an integrated package incorporating a virtual library or magazine, a Guestbook (a tool for discussion and feedback), a Storybuilder (an interactive tool for creative input) and Infoton (a dynamic information skills module).

The objectives have been fully met and this is a genuine library focused project. Its timeliness is admirable in that it encapsulates a wide spectrum of current policy issues that must be addressed by the library world i.e. the changing role of the library in the new global information environment; the need to shift attention from locally (library) based print resources to networked multimedia resources; the need for interaction between libraries and other key actors such as schools, authors, and publishers in preparing children for the Information Society. The project advances and updates much of the important ground work undertaken in the 1980s in the UK, Scandinavia and the USA in areas such as print based information skills teaching and library user education and training. It also addresses pedagogical issues at first level education - a key strategic investment in information users of the future. The relatively modest empirical data supplied by this study will serve as a useful starter/catalyst for more theoretical studies in pedagogics and information user behaviour.

The consortium has adhered well to targets and timescales and the use of resources has been appropriate and well balanced. In fact it is noted that, such has been the commitment and enthusiasm of the partners, a considerable amount of uncosted effort by way of additional work has been invested in the project.

B. Approach, Methods and Results The approach is dynamic, technically sound and highly user-oriented. The design is based on an excellent user need study and the system has been developed with input from all the relevant players - children, teachers, librarians, authors and publishers. The end product, InfoPlanet, is an extremely attractive and intelligently designed package incorporating and integrating all the specified modules. The overall approach has been open and dynamic and has taken account of changing developments in media and networking resources as the project has progressed. The model is generic and has also accommodated local environments and requirements at the partner sites. A good balance of flexibility and control has been achieved in relation to content by allowing unrestricted input and filtering output as appropriate.

The approach to validation has been objective and systematic across the project and it is acknowledged that results are, and can only be, indicative and qualitative at this stage. However, the findings will serve as a useful input for further development and expansion of the system and will also provide useful baseline data for more theoretical pedagogical studies of children’s information needs and behaviour in the context of the emerging global information environment. The project also has significant relevance for curriculum development studies at primary level education.

The quality of the technical deliverables, both demonstrator and printed reports, is exce

C. Management, Dissemination and Concertation The project has been managed competently and efficiently and has operated as a cohesive and productive entity within the allocated timescales and resources. Dissemination has been extensive, both in range and format, and has reached well beyond the confines of the project. The quality of dissemination content, formats and delivery is excellent. All partners have pulled weight and made appropriate contributions to the dissemination programme.

Concertation has also been good. The consortium has maintained contacts with CONCORDIA, ESPRIT/PROSOMA, ILIERS and PUBLICA and has also participated in the European Digital Cities project and the Global Bangemann Challenge.

D. Modifications and Developments since last Review There were no major modifications recommended at the last interim review but those made in relation to validation and exploitation have been taken on board in the work period covered by this (final) review. E. View on Project Status This has been a model project focusing on key policy areas of immediate and important relevance to librarians and educators. It has been a model in terms of topics addressed, execution of the work programme, results, dissemination and concertation. The consortium has radiated an unusual degree of energy and enthusiasm, possibly instilled by exposure to the creativity and unabated adventurism of children, and a realisation that the project represents a quantum leap beyond the conservative shackles of traditional library service towards exciting leading edge research on newly emerging interactive roles of libraries in the global Information Society.

The project has achieved all its objectives and is clearly set to continue under its own momentum, refining and expanding its product and taking on new partners and players within and beyond Europe.

F. Exploitation Potential All partners have made internal organisational changes to continue InfoPlanet within their current operations as part of their services. This is most encouraging and bodes well for future exploitation. The consortium has also agreed to continue co-operation and liaison in maintaining and developing InfoPlanet and has taken steps to take other participants on board. The generic model, shown to be capable of supporting local input to match local needs, and the prospect of establishing multi-site linkages on the Web, demonstrates high potential for replicability and global networking. Such development is, however, subject to know-how transfer, which the partners are prepared to provide, and support for local training. G. Recommendations for Future Work The establishment of InfoPlanets through actors in other EU regions should be promoted and supported.

Because of the policy issues implicit in CHILIAS, it is recommended that the results of the project be brought to the attention of policy makers involved in the Information Society, librarianship, pedagogics, education and curriculum development.

The relevance and potential of CHILIAS for the Information Society Project Office (ISPO) programme and regional programmes such as the Regional Information Society Initiative (RISI) Digital Cities project should be examined.

H. Overall Recommendation The project has been successfully completed.  
Reviewer’s signature: ______________________ Date: _______________

Telematics Application Programme

1994 – 1998

Telematics for Libraries



Review Report

Peter Holm Christensen

Director, Randersegnens Biblioteker

Review meeting: The 30th of September 1998, Luxembourg


A. Objectives, work plan and resources

1. Contribution to the objectives of the libraries sector

The project has reference to Action Line A and meets especially the objectives in Call Topic 1 - Integration tools and interfaces for library systems in local network.

The aim is to develop a strongly user oriented concept for European children’s libraries - a stimulating environment for innovative learning and creative use of multimedia and networked technology. Furthermore the project is meant to enhance the competence of children by improving their information seeking skills in new learning environments.

2. Adherence to specific targets and timescales

The development of the targets is timely and in agreement with the workplan. The project has created a virtual multimedia simulation of the children’s library on the Internet.

Children are able to browse through a multi-media based library from everywhere – their homes, their schools or the libraries. The virtual library has been developed

The consortium has digitised excerpts from different library materials and has made very good examples of the content of an advanced and attractive virtual library. Especially the tool for development of information skills (INFOTON – formerly SKILLS) has improved and is today an exemplary pedagogical tool for bridging between the virtual and the traditional library.

The partners have avoided copyright problems with the digitisation of public printed materials by making agreements with special publishers and by utilising different possibilities in the legislation.

A major part of the content has been produced by children in co-operation with the partners. Many features are made by children for other children and have ended in very user-relevant results (e.g. the Author Gallery and The Internet Reporters).

3. Deviations from or modifications to the work plan and the justification for these

The project has changed the original plan making four different modules – LIB, NET, ACT and SKILLS, and merged these into one single application named the INFOPLANET. The original planned contents of the four modules have not been changed significantly. The deviation is very user-friendly when you look at the demonstrator. The children don’t need to differ between several functions but feel as if they are browsing around in coherent user-interfaces.

It seems as if the former deviation from WP4 into a cyclic development process has succeeded.

4. Balance between the work done and the usage of resources overall and by each of the project participants.

The usage of resources has been dynamic and successful. The partners have exploited the professional forces in each site and have utilised different local possibilities such as schools, sport-clubs, teachers, authors etc.

The cyclic development process has released new efforts and the project has got appreciable powers from many local enthusiastic contributors.

Organisational changes and their consequences.

There have not been any organisational changes. The co-operation is well organised.

    1. B. Approach, methods and results
1. Are appropriate library and end-user needs properly reflected in the approach taken by the project?

The partners have co-operated quite a bit with "informal partners" such as. schools, authors etc. This approach has certainly improved the results. The project has respectively reflected the needs shown in the results of the user-analysis by making creative applications within the children’s favourites – music, pets and sports. The partners have also made new successfully efforts with other subjects like an electronic newspaper (The Internet Reporters), games etc.

The approach is described as "glocal", which is relevant in the sense that the project has brought together general European ideas and local originalities and distinctive marks.

The interfaces of the INFOPLANET seem to be in harmony with children’s fascination of the virtual element. The metaphor is space and the design is done with happy colours and funny drawings.

The INFOTON-component is creatively made and has a lot of qualities that could benefit adults as well as children when it comes to learn how to use the traditional library.

2. Are the methodologies used appropriate?

The methodology is very user-oriented and responsible to the changes of the use and the content that might happen in the future.

The fact that informal partners have joined the project enriches the approach and the development of services, and will be maintained beyond the projects lifetime. The methodology is very much a self-perpetuating process that makes it possible to create a rational synergy effect.

3. Is the technical basis of the project valid and timely?

The technology used in the project is based on relevant standards and tools and can be implemented in many-sided environments. The occurred problems were due to more general technological aspects or local situations (e.g. early generation browsers, screen size etc.).

The documentation of the used technology (Deliverable 3) is very good and proves that it will be possible to implement the application in all sorts of libraries. Even libraries that do not have access to the Internet will benefit from a CD-ROM with the INFOTON (SKILLS)-application. This publication will soon released in English, Swedish and Finnish languages.

4. Have project tasks been executed in a competent manner, based on sound knowledge to the issues involved?

All partners have demonstrated their competence in how to create new library services for children. Especially, the partners have been very professional in the virtual promotion of library facilities and relevant skills for using the library.

5. Does the project contribute to and/or apply relevant standards, and to what extent do the developed products/services fulfil the requirement of interoperability?

All developments are based on Internet standards and therefore the use of - and access to -the applications will be possible as long as the user is connected to the Internet.

6. Are validation and demonstration activities and plans appropriate for what the project aims to achieve?

The validation part of the project is not very quantitative. The project has made three questionnaires: one for librarians who have been involved in the project, one for librarians and adults who have not been involved and finally one for children. The results are very positive. However the answers from librarians and adults who have not been involved are generally more critical to the interfaces and the applications. It seems strange that these criticisms are not reflected in the conclusions of the Validation Report (WP6).

However the user-oriented approach guarantees that there has been a current validation during the project. Several end-users have assessed the results and have made remarks and proposals and the partners have currently adjusted the applications.

7. Is the quality of the key technical deliverables sufficient in view of the overall objectives of the project and the objectives of individual work packages?

The technical deliverables are all sufficient and more refined than originally planned in the objectives.

C. Management, dissemination and concertation

1. Are the management techniques applied and the lines of communication between the partners satisfactory for the type and size of the project? Do they foster effective and goal-oriented co-operation among the partners?

It is clear that some of the partners are well experienced in project managing. The partners have solved smaller problems very professionally during the project, concerning language, culture and library tradition. The "glocal" approach takes the different local interests as well as the general objectives in consideration. The openness for local decisions between the partners is admirable and results in a good project and communications spirit.

The standard of the project manager is very high; this is especially reflected in the quality and the accuracy of the deliverables.

2. Are timescales and resource allocation realistic?

Timescales and resource allocation have been realistic although it is debatable if the project should have spent some more resources in the validation part.

3. Are the extent and quality of dissemination activities acceptable, given the current state of the project?

The range of dissemination activities seems overwhelming. Apart from the formal dissemination activities there has been many informal activities in connection with the current co-operation with different user-groups. The contact with the end-users during the project has intensified the dissemination element and has resulted in several potential users - especially public libraries around Europe – are curious and interested in future co-operation with the CHILIAS-partners.

The demonstrator will undoubtedly be integrated in other countries around Europe and in case there is a need for it, the applications will be adjusted to the different cultures. Even South African libraries have shown interest for the INFOPLANET.

D. Modifications and developments since the last review

After the first review, it was recommended that the partners should try to co-operate with publishers around Europe to increase the content of the publications and applications available in the INFOPLANET. Unfortunately, although the partners have been negotiating these possibilities with several publishers, these efforts have not shown any results. However the range of publications and applications used in the demonstrator is outstanding and satisfying.

E. View on project status

The project is completed.

F. Exploitation potential

All partners are going to exploit the national versions of the INFOPLANET and each partner will integrate the applications in their operational services. The impact of replicability is high and compared with the interest there is for using the INFOPLANET around in EU-countries, the potential exploitation seems extremely positive. The partners ought to search for certain key-actors around Europe and use them as a sort of platform for further exploitation.

G. Recommendations of the future work

It is important that the partners are aware of libraries as well as schools, leisure-time institutions and others in the further exploitation of the INFOPLANET. Although the core subject of the INFOPLANET is to build a bridge between the virtual and the traditional library, the applications will be relevant in non-library organisations with services for children as well.

H. Overall recommendation

Successful completion. No further work required.

Date: 15.10.1998