Excellence in research - where is it?
31 March 2011, by Prof Alan Pettigrew
Prof Alan Pettigrew is a Professorial Fellow at the LH Martin Institute and was formerly Vice-Chancellor of the University of New England.
The Excellence in Research for Australia (ERA) National Report 2010 (1) has been in the public arena since January this year. Naturally enough the report has attracted considerable attention across the university sector. Some of this attention has arisen from the universities themselves as they describe their place in the perceived hierarchy of the nation’s research institutions. Some individuals in the academic community have also commented upon the nature of the ERA methodology. Relatively little attention appears to have been given yet to the distribution of Australia’s excellence as measured by the ERA across the 180 or so (2 and 4 digit) coded disciplines that were the subject of the report. Similarly, there has been little attention paid to the distribution of research excellence across Australia’s universities, save for the highly publicised rankings produced by The Australian newspaper (2) within what seemed like minutes of the Report being released. It is the case that the ERA was not designed to allow for a simple ranking of universities to be generated, and the responsible Minister, Senator Kim Carr, has expressly commented on this matter.(3)
Nevertheless, there are three major reasons why there needs to be some consideration given to the broad ERA outcomes for the higher education sector. First, Australia’s universities will receive in 2011 almost $1.5B in Block Grants that are designed to support their research activities.(4) It would be prudent then to take some steps to see whether the distribution of those funds to universities across the sector is aligned with a broad measure of the excellence of research in all those institutions. Some alignment might be expected because the types of performance measures and their timing that are used to determine the Block Grant components are all relevant to the 2010 ERA outcomes (including research grant funding, postgraduate student enrolments and completions, and publications arising from research). However we now have the opportunity to see just how well the levels of performance-based Block Grant funding match the excellence of research outcomes in all universities as measured in the ERA.
Second, examination of all the new data will be important in the design and implementation of national strategies directed towards improving research outcomes across the sector. We know, for example, that the ERA data will be used in the development of Compacts between the Government and each university and that some funding will be contingent on these arrangements. These Compact decisions for individual universities need to be based on an understanding of the relative performance of each university within the sector as a whole.
Third, it is important that each university within the sector has the opportunity to know where it lies in the overall ‘shape’ of the sector so that it can respond appropriately to national policy settings as well as to set its own optional, strategic priorities for improvement in research performance.
The total Block Grant funding per university in 2011 is shown in rank order of funding per university in Figure 1. (5)
Figure 1: Total Block Grant funding per university (rank order).
As has been known for some time, the data starkly reveal that the Group of Eight (Go8) universities receive the bulk of the Block Grant funding ($914M, 64%), while 33 institutions receive significantly lesser amounts ($520M, 36%).
The ERA Report 2010 provides information about the number of research disciplines assessed at each university as well as their rating on an excellence scale of 1 to 5. A rating of 3 on this scale is regarded as ‘world standard’. The number of research disciplines per university rated at ERA level 3 or above is shown in Figure 2 against the block funding awarded to each university in 2011.
Figure 2: Number of disciplines at ERA level 3 and above versus Block Grant funding ($) per university.
These data suggest that there are four clusters of universities in Australia characterised by their level of Block Grant funding and their number of ‘world standard’ research disciplines.
The ERA data also show a variation across universities in the proportion of disciplines per university that were assessed at ‘world standard’ or above, as shown in Figure 3. These data show a similar clustering of universities as in Figure 2, although there is some overlap.
Figure 3: Proportion of disciplines assessed at ERA level 3 and above versus Block Grant funding ($) per University.
The data indicate that the Go8 universities (Groups A and B) are a long way ahead on Block Grant funding but they are not so far ahead of some universities in Group C in terms of the proportion of all their disciplines that were assessed as being ‘world standard’ and above. Indeed five universities in Group C are within 7% of three universities in Group B on this measure despite receiving considerably less in Block Grant funding.
It would be wrong to draw strong conclusions from these data (for several reasons) without also taking account of the varying sizes of Australia’s universities. There is a strong positive correlation (not shown here) between the size of Australia’s universities (as measured by FTE staff numbers) (6) and (i) the amount of performance-based Block Grant funding that they receive (which reflects the level of research activity at each university), and (ii) the number of disciplines rated in the ERA at ‘world standard’ and above. In Figure 4 the number of research disciplines at ‘world standard’ and above per 1000 staff in each university (Y axis) has been plotted against the Block Grant funding per 1000 staff for each university.
Figure 4: Number of disciplines assessed at ERA level 3 and above per 1000 staff (Y axis) versus Block Grant funding ($) per 1000 staff (X axis) for each university.
These data indicate that many of our mid-size universities (Group C above) have a number of world standard research disciplines per 1000 staff that is similar to that observed amongst the Go8 universities even though the level of Block Grant funding to the Go8 universities per 1000 staff is much higher (up to 2.5 times).
Further analysis of the ERA Report (not shown here) indicates that the higher levels of Block Grant funding at Go8 universities (Groups A and B above), together with Macquarie University in Group C, are associated with greater numbers of disciplines rated in the ERA as ‘well above world standard’ (level 5). This category of excellence was observed in more than 15 disciplines in each of these nine universities (range 16 to 54), compared with 7 or less at all other universities.
These data suggest that more than half of Australia’s universities have a strong profile of research disciplines at ERA 2010 ‘world standard’ and above relative to their size. Our largest universities can also boast a large number of disciplines at the highest level of excellence as assessed by the ERA process. However, many of our universities have various numbers of disciplines rated at levels ‘below’ or ‘well below world standard’ (see Figure 3). In 18 of the 39 universities in Australia (46%), more than 50% of their research disciplines assessed in the ERA were rated as ‘below’ or ‘well below’ world standard. The implications of this observation from the ERA 2010 data are not entirely clear but it does raise questions about the nature and standards of our higher education system where research is regarded as being fundamental to a quality education.(7) It will be of interest to see if these figures change in ERA 2012.
The ERA 2010 data are no doubt being considered within each university as they prepare for ERA 2012. Each university is probably also considering longer term strategies to address the range of their research disciplines and levels of excellence. Some of these considerations may include maximising resource use and effectiveness by focussing on a more limited range of disciplines within the university (sometimes termed ‘specialisation’ or ‘differentiation’). Other considerations may include greater cooperation and collaboration between universities and or strategies for targeted recruitment. It is to be hoped that all of these strategies, in aggregate, will lead to improved levels of research excellence generally across the sector.
The Government will also no doubt have a view on how best to allocate its resources to maximise research performance and excellence across the sector. The extent to which this might be achieved through the Compact discussions with individual universities will emerge in due course. However, the funding available across the sector for this component of Block Grant funding in 2011 ($82M) is relatively small compared to the total Block Grant ($1.5B). These outcomes will also have no bearing on ERA 2012. Thus, it is largely a challenge for the universities to determine their best outcome within the current resource framework. If the Government determines from the ERA 2010 outcomes that Australia needs a greater number of universities with world standard research disciplines, then it would appear that greater funding will be required in some universities. One way to achieve this would be to increase the availability of funds for the competitive funding agencies so that more staff in more universities have both the incentive and the potential opportunity of securing funding to pursue world class research.(8)
Overall, the current picture emerging from this analysis is that the ERA 2010 has identified research of ‘world standard’ and above across a greater number of Australia’s universities than might otherwise have been expected from simple observation of the rank order of their Block Grant funding. The challenges that lie ahead are to implement strategies at both university and government levels that will ensure that the range and geographic distribution of research disciplines at and above ‘world standard’ across the nation’s universities are available to support a widely distributed tertiary education sector and to drive innovation.
5. Two institutions that receive Block Grant funding (Melbourne College of Divinity and the Batchelor Institute of Indigenous Tertiary Education) are not included in the data presented in this paper. The data also exclude the National Institute’s grant to the ANU.
6. DEEWR figures for 2009, FTE staff excluding casuals, except for Charles Darwin University (Annual Report data used).
- Appeared in the LH Martin Institute March e-Newsletter
* Prof Pettigrew will be leading the Institute's Executive Leadership and Management in Research program, commencing 2 May 2011 in Canberra.