In order to obtain an overview of the strengths and focus of New Zealand’s research community, ongoing evaluation of the knowledge base must be performed.
Bibliometrics is the quantitative study of research publications3. It can be used (along with measures of patent and innovation activity) to:
- gain an overview of a country’s research output;
- understand the subject distribution of a country’s research effort;
- estimate the impact of a country’s publications by counting the number of times they are cited by other authors; and
- examine collaborative activity both within a country and internationally.
Purpose
The objective of this report is to describe the productivity, impact and intensity of collaboration of knowledge production within New Zealand. This is achieved by:
• determining the number of New Zealand papers in various research fields;
• determining the impact of New Zealand research papers through the analysis of citations to them;
• investigating the contribution of different sectors (tertiary, Crown Research Institutes, government, local government and private sector) to New Zealand’s research output and impact;
• investigating patterns of inter-sector and international collaboration;
• examining changes in New Zealand’s research output, impact and collaboration over time and
• benchmarking results against international findings, wherever possible.
Methodology
The main analysis methodology for the first three objective points above is to:
- count the number of papers published in a certain ‘period’ of time, and to
- count the number of citations to each paper for a certain ‘window’ of time.
Although they may sometimes refer to the same years, the publication ‘period’ and the citation ‘window’ are NOT the same thing. For analyses pertaining to the count of papers, the publication periods vary according to what is most appropriate. Generally 2001-2003 is used as full data was obtained for these years. At other times 2001-2004 is used, as early (incomplete) data was also obtained for 2004. Occasionally a single year is used, e.g. 2003.
For the citation analyses, the publication ‘period’ is usually 2001 with a citation ’window’ of 2001-2004, although sometimes other combinations of years are used.
The basic counting exercise to produce the results is simplistic in itself. However care must be taken when comparing results between different studies. Some useful questions to alert the user to statistical traps that can hinder meaningful comparisons are:
- Is the publication ‘period’ of the same length?
- Is the citation ‘window’ of the same length?
- Are the databases used in the analyses comparable in their coverage?
As with the previous bibliometric studies, the raw data was obtained from the Thomson-ISI New Zealand National Citation Report database. Thomson-ISI records all papers in which at least one author has a New Zealand address, and for the purposes of this study, for the period 2001 to 2004 (due to indexing time lags 2004 data was not complete). The number of citations made to each paper is shown in the database. Also shown for each paper is the mean number of citations a typical article in that journal historically received.
To conduct international comparisons two extra datasets were obtained from Thomson-ISI4. These international datasets counted articles and reviews only and excluded papers in the Arts and Humanities. For the sector and collaboration analyses, all types of publication were included in all subject areas.