GENERAL PRINCIPLES OF RTI EVALUATIONS
These principles represent an integral part of the evaluation standards and define the requirements from and expectations of various participants, such as the client or client organisation, the evaluators, and those affected by the evaluation. These principles are valid for all types of evaluation in the RTI policy area. Evaluations are different from one another in nature, depending on scope, objectives, methodical approach etc. Nevertheless it is expected that the principles and standards will be applied and observed conscientiously (as a code of conduct). All principles described in these standards are of equal importance and interdependent.
All those involved in an evaluation are guided by a shared understanding of evaluation and in the context of the specific evaluation concerned, have a common understanding of the objective and scope of the evaluation. They agree that the interim and final findings, and the evaluation process itself are important. An evaluation should therefore also be regarded as a communication process.
Evaluations have the support and commitment of decision makers in the institutions concerned. One expression of this commitment is to allocate sufficient budgetary resources, appropriate to the scope of the evaluation, and to make the necessary time available. Individual tasks and responsibilities in the context of the evaluation are clearly defined and documented.
An evaluation should focus on the information requirements of the eventual users and/or user groups. Both the process of evaluation and its findings should be useful to them. The benefits of an evaluation are generally enhanced if relevant interest groups are involved in the evaluation process, if specific evaluation questions are formulated and responded to professionally, and if coherent recommendations are communicated as a result of the evaluation. One prerequisite for the usefulness of an evaluation is a timely beginning and end, so that the findings can be integrated promptly into decision-making and improvement processes. The proposed subsequent use and application of data collected is to be made clear in the framework of the evaluation; this should generally be evident in the tender documents and/or the contract for the evaluation, or at the latest on acceptance of the evaluation report.
An evaluation is feasible for all relevant stakeholders within the given time frame. Sufficient time is important not only for the preparation of high quality proposals, but also for the stated purpose of the evaluation, the clearly formulated evaluation questions, the set of methods, the data analysis, the validation of the results, etc. An evaluation is successfully feasible if the nature of the evaluation questions and the methods are compatible with the scope of the evaluation and sufficient resources are made available, the evaluation has been carried out by competent evaluators and no external factors hinder the implementation.
An evaluation is a process with an open outcome. The entire process of evaluation and its findings imply impartial action and positions, and are carried out in a transparent and open manner. Any relevant views held by participants which could possibly influence the evaluation process or its findings, are made explicit. Differing opinions, perspectives and positions are represented in interim and/or concluding findings.
An evaluation is designed using scientific methods and takes account of differing opinions and positions. The evaluation is not materially influenced or manipulated by political interests, the client, programme managers or those affected, nor by any possible bias of the evaluators themselves. “Analyses as a favour” or similar forms of biased process – particularly with a view to generating possible subsequent contracts, or to avoid criticism that could have a negative impact in the wider political context – are not a serious form of evaluation and are inconsistent with the sense and purpose of an evaluation.
Relevant interest groups are identified early in the process of evaluation, for example through stakeholder analysis, and involved in the evaluation. This increases the benefits and learning outcomes of evaluations. It is also important to check the proportion of women and men involved in the evaluation object and affected by the process of evaluation, and to ensure that they are properly represented.
Both the client and the evaluators should have the necessary skills to direct and conduct an evaluation. These include expertise in the topic area and methodology, gender awareness, management skills, social skills and knowledge of different types of procedure for awarding contracts.
Gender aspects of the evaluation object are considered at every step of the evaluation, and by all those involved. In the event that there is no gender dimension apparent in the evaluation object, this should also be indicated and explained. Gender-specific evaluation questions are included in the Terms of Reference (ToR). Gender-specific data are collected, evaluated and interpreted. Contexts (concepts, strategies, policy documents, programme documents, interventions, etc.) and any information gathered are analysed for gender-specific differences and interpreted in this light. Recommendations resulting from an evaluation take account of gender-specific aspects. All documentation prepared for the evaluation, such as contract tender documents, the evaluation report etc., are formulated in a gender-neutral manner. This means that gender-specific language should be avoided, for example by using gender-neutral terms or by including both masculine and feminine forms.
The evaluation process, its findings, and the subsequent recommendations are conducted and completed in a manner that is transparent and accountable for all those involved and affected. This also requires open access to evaluation findings/reports and their publication in the fteval Repository, https://repository.fteval.at/. If differing perceptions, expectations, interests and needs should come to light, it is essential to address these at an early stage, in order to clarify inconsistencies, differing expectations or possible misunderstandings.
Activities undertaken in connection with evaluations are carried out in a manner that is ethically responsible, gender aware, and with openness toward social and cultural diversity (e.g. age, background, language). All those involved in or affected by an evaluation are treated with respect and fairness. Evaluations are planned and conducted in a manner that protects the individual rights, security and dignity of all those affected or involved. Sensitive data may not be used without consent, and should not be traceable to their source. Data collected are subject to data protection law and generally belong to the client. To ensure ethical practice, evaluation activities must not be obstructed and their findings must not be presented or interpreted in a distorted manner. An evaluation may not be misused with the purpose of deliberate advantage or harm to any institutions, programmes or persons.
An evaluation is regarded as credible if it is planned, managed and conducted in an independent, unbiased, transparent and competent manner, and if it has identified and involved relevant interest groups. In methodological terms, this refers to the use of different methods, for instance, as appropriate to the evaluation questions, and that these are “triangulated”. Evidence obtained from analyses, interpretations and the resulting assessments and recommendations are presented in a comprehensible and transparent manner.