European FP6 nanotechnology research evaluation - little impact, no kidding?
(Nanowerk Spotlight) A new 290-page tome titled "Strategic impact, no revolution" (pdf download, 5.7 MB) is the result of a year-long effort to study the strategic value and impact of NMP in its wider European and international context, with special focus on the ERA dimension, against the general policy objectives of FP6 and against the specific objectives of NMP.
Here is the problem with this report: the expressed goal of FP6 is to benefit industry. How can it then be that when you evaluate this massively expensive program you don't include 'industry'? Rather, the 48 experts interviewed for the evaluation were an illustrious mix of researchers, EU bureaucrats, functionaries from various organizations, and 7 people classified under 'Business'. A closer look at these Business experts reveals that none of them represents industrial companies. The closest to an industry representative is an employee of MagForce, the only industrial-type company on the panel, albeit a very small one, and a representative from Rusnano (seriously? Europe is full of seasoned venture capital firms and you pick someone from a state-funded Russian outfit as the only VC person included in the panel?)
For an honest review on how European industry has benefitted or what needs to be done that they could benefit from programs like FP6 you would expect to see Siemens, Airbus, BAE Systems, Dassault, Fiat and the likes, together with representatives from industrial organizations. But that would probably make for something too honest for the taste of EU Commission bureaucrats...
FP6, which was followed in 2007 by FP7, ran from 2000 to 2006, although some projects are still active. The total cost of FP6 NMP projects is €2.34 billion, of which €1.44 bn are EU contribution, the remainder is borne by the individual EU countries directly.
The overall goal of the FP6 program has been clearly defined: "Based on the Treaty establishing the European Union, the Framework Programme has to serve two main strategic objectives: Strengthening the scientific and technological bases of industry and encourage its international competitiveness while promoting research activities in support of other EU policies. These two objectives are setting the general scene for choosing priorities and instruments."
Note the reference to industry. In other words, the EU proposed to spend €2.4 billion to future-proof its industry, create new technologies, products and potentially entire industries based on nanotechnology and multifunctional materials; i.e., FP6 is seen as an instrument of industrial policy on a large scale.
Here now is the conclusion from the report's authors:
"The title of this report [Strategic impact, no revolution] refers to the general finding that the third thematic priority (NMP) in FP6 strategically affected Europe's competitive position and was an important programme which also influenced Member States' policies and research agendas. However, it cannot be directly linked to a revolution with regard to creating substantial scientific or industrial breakthroughs although these were among the explicitly targeted objectives. The programme strengthened Europe's position as one of the world leaders in the respective scientific and industrial fields, but did not enable Europe to outperform other key actors such as the United States or Japan."
And that probably is a major problem with this kind of efforts: they fall prey to their own hype. Why is it necessary to talk about a 'revolution' – and raise vastly hyped-up expectations – in order to justify basic research programs? Why not sit down at the outset with the intended beneficiaries (industry) and agree on a sensible course of action? Although a close integration of small and medium-sized companies (SME) explicitly was part of the program's design – but somehow it didn't happen.
The authors of the report have to do a lot of tap-dancing to paint their conclusions in the best possible colors (the report, after all, was paid for by the EC). Here is an example:
"Although this strategic evaluation cannot be a fully developed impact assessment on the level of individual projects or measure the actual outcomes of those, it was necessary to analyse the programme outputs and outcomes but on a higher, more aggregated level. However, this cannot be completed without an understanding of the 'physical' impacts, which are in many cases not created yet due to the fact that part of FP6 projects are still running and eventual scientific impact or commercialization of scientific products coming from those projects in most of the cases will be continued in the years to come."
That is quite an interesting statement, given that a bit further down they write that "71% of the projects covered by the survey were finished before the survey was launched in September 2009". How many years should we give them to assess if and how successful commercialization took place?
In all fairness, it is quite difficult to actually evaluate these big European programs since usually they contain lots of nice words and big visions but are short on hard numbers and targets when it comes to putting anything out there against which they can be evaluated. In the authors' own words:
"A large part of this evaluation report is dedicated to the analysis of the objectives and the extent to what they were met or reached. The reason for that lies in the way these objectives were designed and formulated, as no quantifiable target parameters of success were defined at programme, area, or topic level."
Another systemic problem might be this: "In spite of NMP FP6 alignment with the Member States, the USA and Japan in addressing key scientific, technical and industrial challenges in terms of transforming their old industries, a greater market orientation of the research and closer co-operation with the industry stakeholders in Europe is considered to be a key European problem."
On the bright side, though, NMP FP6 appears to have been quite successful with regard to the production and strengthening of new knowledge, critical mass, shifts in research, and education/career chances/mobility of/for researchers.
Some of the recommendations made by the report's authors with regard to improving the rate of commercialization (which shouldn't be difficult since the starting line is close to the zero line) appear quite sensible, although it remains to be seen it they will be implemented.
Can't wait to see the evaluation report on NMP FP7...