{"id":8886,"date":"2026-09-22T08:47:05","date_gmt":"2026-09-22T06:47:05","guid":{"rendered":"https:\/\/fteval.at\/?post_type=podcast&p=8886"},"modified":"2026-09-14T10:47:05","modified_gmt":"2026-09-14T08:47:05","slug":"episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler","status":"publish","type":"podcast","link":"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/","title":{"rendered":"Episode #40 &#8211; Counterfactual Analysis in Evaluation with Jakob Kofler"},"content":{"rendered":"<p>Promoted to be the \u201cgold standard\u201d of evaluation, counterfactual evaluation is on the rise.<\/p>\n<p>This is one of the many findings from the new fteval cooperative study jointly conducted by <a href=\"https:\/\/www.kmuforschung.ac.at\/\" target=\"_blank\" rel=\"noopener\">SME Research Austria<\/a>, <a href=\"https:\/\/www.joanneum.at\/\" target=\"_blank\" rel=\"noopener\">Joanneum Research<\/a>, and <a href=\"https:\/\/www.wpz-research.com\/\" target=\"_blank\" rel=\"noopener\">WPZ Research<\/a>. <\/p>\n<p>Counterfactual evaluations look at points A and B in time to see how policy interventions have had an effect. Their appeal is quantification \u2013 a number in the end to legitimise their findings. <\/p>\n<p>What happens when we ignore what is between A and B? What impact could counterfactual evaluations have? And what wishes do we further have for this method and the evaluation community?  <\/p>\n<p>In this REvaluation podcast episode, we talk with <a href=\"https:\/\/www.kmuforschung.ac.at\/teammembers\/jakob-kofler\/\" target=\"_blank\" rel=\"noopener\">Jakob Kofler<\/a> from KMU Forschung Austria (SME Research Austria) about methods, conclusions, and recommendations.<\/p>\n<p><b>Resources:<\/b><\/p>\n<ul>\n<li>The study (in German): <a href=\"https:\/\/repository.fteval.at\/id\/eprint\/841\/\" target=\"_blank\" rel=\"noopener\">Chancen &#038; Grenzen kontrafaktischer Wirkungsanalysen f\u00fcr die \u00f6sterreichische FTI-Politik \u2013 eine Situationsanalyse &#8211; fteval repository<\/a><\/li>\n<li>Training in Vienna on 23 November 2026 (in German): <a href=\"https:\/\/fteval.at\/event\/trainingsangebot-kontrafaktische-wirkungsanalysen-in-oesterreichischen-fti-evaluationen\/\" target=\"_blank\" rel=\"noopener\">Trainingsangebot: Kontrafaktische Wirkungsanalysen in \u00f6sterreichischen FTI-Evaluationen &#8211; fteval<\/a><\/li>\n<li>SIP talk with the Austrian Council on counterfactual evaluations on 19 October 2026, in German: <a href=\"https:\/\/forwit.at\/sip-talk\/kausalanalysen-in-der-oesterreichischen-fti-evaluierung\/\" target=\"_blank\" rel=\"noopener\">https:\/\/forwit.at\/sip-talk\/kausalanalysen-in-der-oesterreichischen-fti-evaluierung\/<\/a><\/li>\n<li><a href=\"https:\/\/forwit.at\/sip-talk\/kausalanalysen-in-der-oesterreichischen-fti-evaluierung\/\" target=\"_blank\" rel=\"noopener\">Kausalanalysen in der \u00f6sterreichischen FTI-Evaluierung &#8211; FORWIT (in German)<\/a><\/li>\n<li>Related previous podcast episode: <a href=\"https:\/\/fteval.at\/en\/podcast\/episode-8-lets-data-the-austrian-microdata-center\/\" target=\"_blank\" rel=\"noopener\">Folge #8 &#8211; Das AMDC: F\u00fcr verkn\u00fcpfte Mikrodaten ziehen wir alle Register &#8211; fteval<\/a> (in German)<\/li>\n<li>Subscription <a href=\"https:\/\/fteval.at\/en\/newsletter\/\" target=\"_blank\" rel=\"noopener\">Newsletter &#8211; fteval<\/a> (EN) &#038; <a href=\"https:\/\/fteval.at\/en\/newsletter\/\" target=\"_blank\" rel=\"noopener\">Newsletter &#8211; fteval<\/a> (DE)<\/li>\n<\/ul>\n<p><b>Music<\/b>: Urbana-Metronica (wooh-yeah mix) (ft. Morusque, Jeris, CSoul, Alex Beroza) by spinningmerkaba http:\/\/ccmixter.org\/people\/jlbrock44; SouljaUnit Remix of DeadRobot Music\u2019s Surfy via https:\/\/freesound.org\/people\/SouljaUnit\/sounds\/640175\/<br \/>\n<b>Photo<\/b>: Johnny Be Good<\/p>\n","protected":false},"author":3,"featured_media":8885,"template":"","class_list":["post-8886","podcast","type-podcast","status-publish","has-post-thumbnail","hentry"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.6 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Episode #40 - Counterfactual Analysis in Evaluation with Jakob Kofler - fteval<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Episode #40 - Counterfactual Analysis in Evaluation with Jakob Kofler - fteval\" \/>\n<meta property=\"og:description\" content=\"Promoted to be the \u201cgold standard\u201d of evaluation, counterfactual evaluation is on the rise. This is one of the many findings from the new fteval cooperative study jointly conducted by SME Research Austria, Joanneum Research, and WPZ Research. Counterfactual evaluations look at points A and B in time to see how policy interventions have had an effect. Their appeal is quantification \u2013 a number in the end to legitimise their findings. What happens when we ignore what is between A and B? What impact could counterfactual evaluations have? And what wishes do we further have for this method and the evaluation [...]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/\" \/>\n<meta property=\"og:site_name\" content=\"fteval\" \/>\n<meta property=\"og:image\" content=\"https:\/\/fteval.at\/wp-content\/uploads\/2026\/09\/Jakob.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1920\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/fteval.at\\\/en\\\/podcast\\\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\\\/\",\"url\":\"https:\\\/\\\/fteval.at\\\/en\\\/podcast\\\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\\\/\",\"name\":\"Episode #40 - Counterfactual Analysis in Evaluation with Jakob Kofler - fteval\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/fteval.at\\\/en\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/fteval.at\\\/en\\\/podcast\\\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/fteval.at\\\/en\\\/podcast\\\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/fteval.at\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/Jakob.png\",\"datePublished\":\"2026-09-22T06:47:05+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/fteval.at\\\/en\\\/podcast\\\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/fteval.at\\\/en\\\/podcast\\\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/fteval.at\\\/en\\\/podcast\\\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\\\/#primaryimage\",\"url\":\"https:\\\/\\\/fteval.at\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/Jakob.png\",\"contentUrl\":\"https:\\\/\\\/fteval.at\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/Jakob.png\",\"width\":1920,\"height\":1920},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/fteval.at\\\/en\\\/podcast\\\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/fteval.at\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Podcasts\",\"item\":\"https:\\\/\\\/fteval.at\\\/en\\\/podcast\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Episode #40 &#8211; Counterfactual Analysis in Evaluation with Jakob Kofler\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/fteval.at\\\/en\\\/#website\",\"url\":\"https:\\\/\\\/fteval.at\\\/en\\\/\",\"name\":\"fteval\",\"description\":\"Austrian Platform for Research and Technology Policy Evaluation\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/fteval.at\\\/en\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Episode #40 - Counterfactual Analysis in Evaluation with Jakob Kofler - fteval","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/","og_locale":"en_US","og_type":"article","og_title":"Episode #40 - Counterfactual Analysis in Evaluation with Jakob Kofler - fteval","og_description":"Promoted to be the \u201cgold standard\u201d of evaluation, counterfactual evaluation is on the rise. This is one of the many findings from the new fteval cooperative study jointly conducted by SME Research Austria, Joanneum Research, and WPZ Research. Counterfactual evaluations look at points A and B in time to see how policy interventions have had an effect. Their appeal is quantification \u2013 a number in the end to legitimise their findings. What happens when we ignore what is between A and B? What impact could counterfactual evaluations have? And what wishes do we further have for this method and the evaluation [...]","og_url":"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/","og_site_name":"fteval","og_image":[{"width":1920,"height":1920,"url":"https:\/\/fteval.at\/wp-content\/uploads\/2026\/09\/Jakob.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/","url":"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/","name":"Episode #40 - Counterfactual Analysis in Evaluation with Jakob Kofler - fteval","isPartOf":{"@id":"https:\/\/fteval.at\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/#primaryimage"},"image":{"@id":"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/#primaryimage"},"thumbnailUrl":"https:\/\/fteval.at\/wp-content\/uploads\/2026\/09\/Jakob.png","datePublished":"2026-09-22T06:47:05+00:00","breadcrumb":{"@id":"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/#primaryimage","url":"https:\/\/fteval.at\/wp-content\/uploads\/2026\/09\/Jakob.png","contentUrl":"https:\/\/fteval.at\/wp-content\/uploads\/2026\/09\/Jakob.png","width":1920,"height":1920},{"@type":"BreadcrumbList","@id":"https:\/\/fteval.at\/en\/podcast\/episode-40-counterfactual-analysis-in-evaluation-with-jakob-kofler\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/fteval.at\/en\/"},{"@type":"ListItem","position":2,"name":"Podcasts","item":"https:\/\/fteval.at\/en\/podcast\/"},{"@type":"ListItem","position":3,"name":"Episode #40 &#8211; Counterfactual Analysis in Evaluation with Jakob Kofler"}]},{"@type":"WebSite","@id":"https:\/\/fteval.at\/en\/#website","url":"https:\/\/fteval.at\/en\/","name":"fteval","description":"Austrian Platform for Research and Technology Policy Evaluation","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/fteval.at\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/fteval.at\/en\/wp-json\/wp\/v2\/podcast\/8886","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fteval.at\/en\/wp-json\/wp\/v2\/podcast"}],"about":[{"href":"https:\/\/fteval.at\/en\/wp-json\/wp\/v2\/types\/podcast"}],"author":[{"embeddable":true,"href":"https:\/\/fteval.at\/en\/wp-json\/wp\/v2\/users\/3"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/fteval.at\/en\/wp-json\/wp\/v2\/media\/8885"}],"wp:attachment":[{"href":"https:\/\/fteval.at\/en\/wp-json\/wp\/v2\/media?parent=8886"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}