In 2016, the Austrian Platform for Research and Technology Policy Evaluation will celebrate its 20st birthday. We took that as an occasion to scrutinise the Platforms usefulness for its members and the impact it has created. This internal assessment consits, firstly, of a compilation of services provided by the Platform since its beginnings in 1996. Secondly, Katharina Warta and Stefan Philipp conducted group interviews with most of the member organisations in October and November 2015, to collect their views on fteval’s activities, and past and present challenges for RTI-evaluation.
The study comes up with key findings along five dimensions: (i) fteval without doubt had a strong impact on the establishment of an evaluation culture in Austria; the publication of common standards but also trainings and evaluation competences are relevant elements of this undertaking. Still, there are varying views on the degree of implementation of these standards, and various interviewees see room for further improvement. (ii) Indeed, after 20 years, there is some need to redefine the role of the platform, either to preserve what has been achieved so far, or to adopt a more critical or provocative position. (iii) A key challenge for the evaluation community as a whole and the fteval as its platform is the increasing importance of and focus on impact evaluation, which is politically requested but technically tricky, both in terms of methodology and timing. (iv) This leads to the first observation of increased complexity of RTI-policy evaluation, due to increased links between policy fields, increased availability of secondary data, which might be used by evaluators, and a related goal overload in terms of references of evaluations. (v) Finally, the actual importance of fteval is seen as communication platform, both for meetings, conferences and publications. The various formats developed so far are strongly appreciated by member organisations, some updating is needed – and on the way – concerning Internet presence and electronic publications.
The full version of the summary of the results (in German language) can be found here.
