Evaluation of Science, Technology and Innovation Policy in Brazil: high complexity and new methodological frontiers Tatiana de P. A. Maranhão S &T Analyst.

Apresentação em tema: "Evaluation of Science, Technology and Innovation Policy in Brazil: high complexity and new methodological frontiers Tatiana de P. A. Maranhão S &T Analyst."— Transcrição da apresentação:

1 Evaluation of Science, Technology and Innovation Policy in Brazil: high complexity and new methodological frontiers Tatiana de P. A. Maranhão S &T Analyst Ministry of Science, Technology and Innovation Montevideo, 24 de setembro de 2013.

2 Situation of Science, Technology and Innovation in Brazil Prior = focused mainly on human resources formation (post-graduation, master and PhD degrees) and international indicators Present = technological development for innovation, related to health, defense, education and infrastructure More innovation driven efforts than pure and applied research

3 Fact 1 = BR government invests more in ST&I than private sector Dispêndio nacional em ciência e tecnologia (C&T) em relação ao produto interno bruto (PIB) por setor, 2000-2011 Fonte: Coordenação-Geral de Indicadores (CGIN) - ASCAV/SEXEC - Ministério da Ciência, Tecnologia e Inovação (MCTI).

4 Situation of Science, Technology and Innovation in Brazil New policies: -Intensified stimuli for industry R&D associated with national industry strategies; -Funding industry R&D activities, including venture capital - Science without Borders = internationalization of human resources formation

5 Fact 2: Highly complex environment (concentrated investments)

6 Fact 3 = Total amount invested is still low compared to other countries Fonte: Coordenação-Geral de Indicadores (CGIN) - ASCAV/SEXEC - Ministério da Ciência, Tecnologia e Inovação (MCTI). 8.1.1 Dispêndios nacionais em pesquisa e desenvolvimento (P&D) de países selecionados, 2000-2011 Fonte (s): Organisation for Economic Co-operation and Development (OECD), Main Science and Technology Indicators, 2013/1 e Brasil: Coordenação-Geral de Indicadores (CGIN) - ASCAV/SEXEC - Ministério da Ciência, Tecnologia e Inovação (MCTI). Elabo ração : Coordenação-Geral de Indicadores (CGIN) - ASCAV/SEXEC - Ministério da Ciência, Tecnologia e Inovação (MCTI). Nota( s): 1) PPC - Paridade do poder de compra. Atuali zada em: 28/08/2013

7 Fact 4 = We have more doctors, researchers and research groups ( 137% more doctors in 2010 than 2002)

8 Data available: www.cnpq.br and www.capes.gov.brwww.cnpq.brwww.capes.gov.br Mapa de investimentos Plataforma Carlos Chagas CNPq Mapa das bolsas - GeoCAPES CAPES Scholarships (Ministry of ST&I and Ministry of Education)

9 Fact 5 = Researches are publishing more articles than before 2007

10 The National System of Science, Technology and Innovation is highly fragmented, with many stakeholders and different levels of political action; Fragmented evaluation experiences within MCTI

11 Research Infrastructure (MCTI)

12 Coordenação-Geral de Indicadores Assessoria de Acompanhamento e Avaliação Secretaria Executiva Ministério da Ciência, Tecnologia e Inovação Information and Actor Network Fontes nacionais de informações primárias IBGE CENSO, PNAD, PINTEC, Contas Nacionais e CONCLA CNPq Fomento, bolsas e Diretório dos Grupos de Pesquisa CAPES D ados da pós-graduação e bolsas INEP Censo do ensino superior SETEC/MCT Lei do Bem FINEP/MCT Fundos setoriais SESU / MEC Bolsas PET RFB / MF Renúncia fiscal STN / MF SIAFI, balanços dos estados INPI Patentes e balanço tecnológico Banco Central Balanço tecnológico Secretarias estaduais de C&T Investimentos em C&T Ministério do Trabalho RAIS FAPs Bolsas e fomento MCT Rede de Articulação Nacional de Indicadores de C&T CGIN Secretarias estaduais de C&T e FAPs RIEC&T IBGE USPTO Patentes Thomson Reuters SCImago Bibliometria EPO Patentes Fontes internacionais de informações primárias CCT EUROSTAT Comparações internacionais OCDE Comparações Internacionais e manuais Banco Mundial Dados econômicos (PPC) UNESCO Comparações Internacionais e manuais RICyT Comparações internacionais Disseminação Comunidade científica e tecnológica Mídia Setor governamental Setor empresarial Usuários Sites e publicações: MCT RICyT UNESCO OCDE Assessoramento Técnico Consultores Comissão Permanente de Indicadores Ministérios e estatais federais CGEE

13 Previous efforts Ministry of Social Development – specific Secretary focused on monitoring and evaluation (M&E) Bolsa Família; Ministry of Education – system of evaluation of basic education (SAEB) (alumni grades); ENEM and Prova Brasil; evaluation of post-graduate courses Ministry of ST&I – PACTI (2007-2010), ENCTI (2012-2015) and Monitoring and Evaluation Policy (Portaria nº

14 PACTI (2007-2010)

15 Present effort Ministry of ST&I – Monitoring and Evaluation Policy, coordinated by Permanent Committee on M&E; PAMA 2013 = 7 program evaluations and 12 monitoring activities; Currently, there are 3.772 researchers focused on evaluation and 27 groups (Lattes CNPq);

16 Formulation and planning Implementation Results/ Impacts ex-ante evaluation process evaluation ex-post evaluation Monitoring and info systems Results and impact evaluations “Base line” studies (map) EVALUATION OF ST&I POLICIES

17 Results How many patents and innovations have been generated by projects / institutions supported and in which areas? How many publications have been produced by scholars or supported projects?Impact Policies encouraging innovation has improved the innovative performance of firms and sectors benefited? The scholarship policy has contributed to improve the training of professionals in strategic areas for the development of the country? What is the impact of scientific and technological research projects fostered by MCTI? Need for methodologies able to establish causality and to isolate other factors that influence the observed results.

18 Examples PROANTAR = Bibliometric study of scientific production – quality (citation index), cooperation networks, scientific areas; interview with more and less productive and evaluation of program management ;

19 Examples Impact Assessment Subvention (“Lei do Bem”) over private investment in R & D = econometric study with use of control groups: companies that use the LB tend to spend more on R & D because of the use, or just replace a font by another?

20 What has been achieved in the impact assessment? Employing the control group, just as in experiments with drugs, it was observed that companies that make use of the Lei do Bem tend to invest in R & D at least 7% more than companies that do not use this legislation.

21 Examples Evaluation of the Brazilian Space Program Study = qualitative and quantitative impact on technological development of domestic industry, its direct and indirect impacts, and development of international indicators; Evaluation of programs and actions available on International Cooperation CNPq; Evaluation of projects supported by the Sectorial Funds since 2000.

22 New frontiers ST&I Quasi-experimental; Econometric studies with group control; Qualitative methods such as discourse analysis, with webcrawler instruments; In depth interview related to previous data, creating new categories; Rethink planning and implementation problems based on the incorporation of social demands.

23 Our challenges..... how to overcome? Fragmentation of ST&I public policies (many stakeholders, low coordination at national level); Concentration of resources and investments in the Southeast region; High quality evaluation needs time, resources (human, systems and $) and lots of political support!

24 Sites www.mct.gov.br http://lattes.cnpq.br/ www.capes.gov.br www.embrapa.gov.br http://www.ibge.gov.br/biblioteca/visualizacao/periodi cos/2/bn_2013_v21.pdf http://www.ibge.gov.br/biblioteca/visualizacao/periodi cos/2/bn_2013_v21.pdf www.cgee.org.br www.ipea.gov.br www.embrapa.br www.fapesp.br

25 Thank you! Tatiana de P. Albuquerque Maranhão tmaranhao@mct.gov.br