Evaluation of Marsden Fund shows scientific output increased

A study by researchers at Motu Economic and Public Policy Research – a not-for-profit, non-partisan research institute – has found that Marsden Funding does increase the scientific output of the funded researchers.

The Marsden Fund is the premiere funding mechanism for blue skies science research in New Zealand. In 2014, $56 million was awarded to 101 research projects chosen from among 1222 applications from researchers at universities, Crown Research Institutes and independent research organizations.

Dr Adam Jaffe, director of Motu, said the research found a team that is given Marsden funding shows a 6-12 percent increase in their academic publications and a 13-30 percent increase in the papers that cite their work.

“Publications and citations are, of course, only proxies for research output. However, we expect successful research to be highly cited,” said Dr Jaffe.

But the researchers also found no evidence that the selection process could meaningfully predict the likely success of different proposals.

The application process for the Marsden Fund has two stages. An initial one-page proposal is reviewed by a subset of the appropriate panel and given a preliminary score. 71-84 percent of the proposals are rejected.

In the second stage, longer proposals are submitted and sent to external (typically international) anonymous referees for review. Applicants are given the chance to respond to referee comments before the panel scores and ranks the proposals.

“Interestingly, we didn’t find a link between a project’s future success and the rankings given to it by the second-round panel,” said Dr Jaffe.

“This means there is no reason to expect diminishing returns if Marsden funding were increased. It also means the significant resources devoted to the second round evaluation could be reduced without degrading the quality of decision-making.”

This analysis is based on 1,263 Marsden proposals who reached the second stage of review between 2003 and 2008. Overall 41% of the second-round proposals were funded. Around 25% of the proposals were Fast Start (funding for early-career researchers) and slightly more than half of these were funded.

The average researcher on these teams made six proposals and received 1.2 grants between 2000 and 2012. Motu researchers also identified the approximately 1500 New Zealand based researchers named on these proposals and examined their annual publication and citation record between 1996-2012.

“We were very foturnate to have access to all the funded and unfunded proposals, including their evaluation scores. This meant we could control statistically for potential bias driven by the Fund’s efforts to fund projects that are expected to be successful,” said Dr Jaffe. The Marsden Fund and the Royal Society should be commended for making these data availabe, something very few research funding agencies do.”

New Zealand spends less money on research, relative to its size, than three-quarters of the countries in the OECD.

The government is considering expanding public funding to narrow this gap.

But very little has been known about the efficacy of existing funding mechanisms until now, Jaffe said.

The working paper “The effect of public funding on research output: The New Zealand Marsden Fund”, was funded by the Motu Research and Education Foundation, Queensland University of Technology and the Ministry of Business Innovation and Employment.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: