Assessment of the impacts of Innovation Grants in Brazilian States
Érika de Andrade Silva Leal
professoraerikaleal@gmail.com
Federal Institute of Espírito Santo – IFES, Vitória, Espírito Santo, Brazil.
Márcia Elisa Echeveste
echeveste.mar@gmail.com
Federal University of Rio Grande do Sul – UFRGS, Porto Alegre, Rio Grande do Sul, Brazil.
Lia Buarque de Macedo Guimarães
liabmg@gmail.com
Federal University of Rio Grande do Sul – UFRGS, Porto Alegre, Rio Grande do Sul, Brazil.
Aline Cafruni Gularte
alinecgularte@gmail.com
Federal University of Rio Grande do Sul – UFRGS, Porto Alegre, Rio Grande do Sul, Brazil.
ABSTRACT
Goal: This article presents an assessment of the impacts of the “Research in Companies Support Program (PAPPE): Economical Subvention to Innovation”, in the decentralized modality, i.e., projects contracted in the states of the Brazilian federation. Considering the high opportunity cost of the PAPPE program, it was deemed important to evaluate how the financed projects impacted the three dimensions (economic, social and environmental) of sustainable development.
Methodology: The study was based on literature review on innovation grants; interviews with one Finep technician and technicians from Fapes, Fapesc and Sebrae-PR; and documental research and analysis of 53 projects carried out in the states of Espírito Santo, Paraná and Santa Catarina, from 2009 to 2013. Descriptive analysis was conducted for comparing performance of the three states.
Results: Under the economic dimension, the PAPPE program enhanced the ability of funding recipients to compete on a national level. However, there was no evidence of impact on the social and environmental dimensions.
Limitations of the investigation: This research is restricted to three Brazilian states, chosen due to the homogeneity of project types and the interest of the researchers, with a focus on the state of Espírito Santo.
Practical implications: Although both social and environmental dimensions are not explicit in the scope of the PAPPE program, the assessment of these dimensions should be included in the design and evaluation of any policy sponsored by public funding, since they are critical to the country.
Keywords: Grants innovation; assessment of the impacts; sustainable development.
1. INTRODUCTION
Government funding and/or coordination of public programs in Science, Technology and Innovation (S,T&I) showed an increase between 2000 and 2010, leading to the demand of evaluation of the impacts of the implemented programs (Bozeman et Link, 2015; Frank et al., 2016; Dimos et Pugh, 2016). Evaluation is important to understand the effects of technological policies and programs; to create mechanisms to measure the return on innovation programs (projects) investments; to learn about past experiences; to justify policy continuity; to provide objectivity and robustness to public S,T&I budgets; to determine priorities, design programs, organize management and transfer technological strategies; to improve articulation and understanding of the role of the funding agency; to ensure continuity of the agency and verification of its mission fulfillment; to aid public agencies in the development and examination of the effectiveness of policy intervention, since good instruments of measurement allow the creation of inputs for innovation policies (Hud et Hunssinger, 2015; Dimos et Pugh, 2016; Bozeman et Youtie, 2017; Caloffi et al., 2018).
Considering knowledge and innovation as fundamental factors for the competitivity of nations, regions, sectors and companies (Cassiolato et Lastres, 1999; Figueiredo et al., 2017), since 2005, with the regulation of the Brazilian Innovation Law, Brazil launched a support program for research and development of innovative products and processes in companies, via non refundable innovation grants (Costa, 2013). The Brazilian Innovation Agency, Finep, was responsible for this program, contracting projects directly with the companies that attended the public call for proposals. Aiming to give more capillarity to the program, Finep started carrying out the innovation grant in states, in a decentralized way, through local innovation support agencies, namely, the Research Support Foundations (FAPs), the Brazilian Micro and Small Enterprises' Support Service (Sebrae), the Industries Federation, among other institutions (Leal et Souza, 2011).
In 2006, Finep released the Research Support Program for innovation in Companies (PAPPE), the first public call (No. 002/2006) to encourage the submission of proposals from micro and small companies. At the beginning of 2016, Finep initiated the cycle of program evaluation, which was limited to the verification of the fulfillment of the proposed goals, with no analysis of their social or environmental impacts. Therefore, this study aimed to evaluate the impacts of the PAPPE grant, in the decentralized modality, considering the economic, social and environmental dimensions of sustainable development. The evaluation was applied to the projects carried out in the states of Espírito Santo (ES), Paraná (PR) and Santa Catarina (SC) in the period 2009-2013.
Following this introduction, Section 2 presents the context of innovation funding in Brazil; Section 3 brings the methodological procedures; section 4, the results and discussion; section 5 the general considerations; and section 6 presents the conclusion.
2. THE GRANTS IN THE CONTEXT OF BRAZIL’S INNOVATION FUNDING POLICY
In recent years, in order to implement an innovation policy, the Brazilian government took several steps, such as (Carrijo et Botelho, 2013; Rocha, 2015; Frank et al., 2016): the Industrial, Technological and Foreign Trade Policy (ITFTP) in 2003; the Brazilian Innovation Law in 2004 (Law No. 10.973), regulated in 2005 (altered in 2016 by Law No. 13.243 (Brasil, 2016); the Good Law in 2005 (Brazil, 2005); the Growth Acceleration Program for Science, Technology and Innovation (S,T&I GAP) in 2007; the Productive Development Policy (PDP) in 2008; and the Greater Brazil Plan (GBP) in 2011 (Brasil, 2011).
In the context of ITFTP, with the purpose of funding innovation-related actions, the country started encouraging innovative projects via financial non-refundable support directly to companies. According to Andrade (2009), this “innovation grant” was initially instituted by the Brazilian Innovation Law, which established mechanisms to fund research in agricultural industry, healthcare, biotechnology and genetics, aeronautics, as well as in companies that carry out innovation for competitivity.
The grant’s main feature consists of its non-refundable nature, i.e., it is an instrument that does not require the return of the resources to the funding agency. Nevertheless, this does not mean that the funding organ has to deal with all the risks involved in the innovation process: it is shared between the state and the companies which, although have no need to give guarantees in order to receive the grant, must show reciprocity. Furthermore, the grant focuses on micro and small companies (SMCs) (Morais, 2008), since they usually have more difficulty in accessing credit.
It is worth noting that the grant may represent one of the most powerful instruments to induce the innovation process in companies while meeting public interests. If the instrument is used in a discretionary way, through public policy decisions that select areas or themes for eligible projects, the development and innovation of products with high technological content, or of strong interest for the country, may be fostered. Mazzucato (2014) brings several examples of direct governmental action, determining which areas will be considered strategic and encouraged through public policies. This involves a high risk that will certainly be more faced by the government than by the market. The election of strategic areas does not take place with the same intensity in the public modalities of innovation support, such as refundable credit, fiscal incentives and risk capital, because these modalities use non-discretionary mechanisms for project selection (Costa, 2013).
As responsible for the program, Finep released official calls for proposals, inviting companies to come up with Research, Development and Innovation (R,D&I) projects to compete for the resources. Striving for more program capillarity on national territory, Finep celebrated partnerships with the states Research Support Foundations (FAPs) for the regional execution of the PAPPE grant (Leal et Souza, 2011). The program initially released calls in 14 states, with Finep investing R$ 144,000,000,00 in the approval of 547 projects, of which 363 have been concluded.
One aspect of the program that merits criticism refers to its general goal of “promoting a significant increase of innovation activities and the increment of the competitivity of the company and the country’s economy”. The goal’s generic character shows that the program lacks a clear vision in terms of what it hopes to achieve. An analysis of the calls for proposals released since 2006 shows that all of them share a reference to the development of “innovative processes and products” in their scope, making it clear that the focus of the instrument lies on the project, rather than on the company’s innovation strategy (Andrade, 2009).
3. METHOD
This exploratory research has a qualitative and quantitative approach, and was conducted upon the following steps:
a. Collection of Brazilian literature on innovation grants;
b. Definition and characterization of the target audiences: in step a, it was verified that innovation grants in Brazil have been an object of evaluation in several states, but not in the state of Espírito Santo (ES). Considering that the population of projects was of just 14 concluded ones in that state, it was decided to also assess the 52 projects concluded in the state of Santa Catarina (SC) and the 55 projects in the state of Paraná (PR), which, according to the Finep technician, has set the benchmark in the program;
c. Documental research and interviews: the analyzed documents were those available at Finep and the institutions that execute the program in each state: the FAPs from ES (Fapes) and SC (Fapesc), and Sebrae-PR, from PR. The Finep technician and technicians from the three anchor institutions were interviewed in-person and through e-mail;
d. Data collection: it was used an instrument for assessing the program’s impact, elaborated with the Survey Monkey® platform, which was sent to the funding recipients via e-mail. Data collection took place during the months of June, July and August 2017, with a return of 53 projects;
e. Data Analysis: initially, the data analysis was conducted in a general way, considering the following variables: number of supported projects; contemplated sectors; median value funded per project and age of the contemplated companies. Subsequently, data analysis was done for each of the three (economic, social and environmental) sustainable development dimensions.
4. RESULTS AND DISCUSSIONS
4.1 Characterizing the population and the research sample
In this section, the population (N) and the research sample (n) will be characterized. Table 1 represents the number and sector of the contemplated projects in the three selected states, in the period 2009-2013.
Table 1. Number and sector of the contemplated projects by state
Source: Reports from Finep, Fapes, Fapesc and Sebrae-PR.
In the three states, 121 projects were contemplated in the period 2009-2013, out of which 55 from PR, 52 from SC, and 14 from ES. Regarding the main contemplated sectors, it can be observed that Information and Communication Technology (ICT) was the prevalent sector (44 contemplated projects), followed by Electronics (16), Environment (14) and Healthcare (13). This result is in agreement with Carrijo and Botelho (2013) who analyzed the grants in the states of Minas Gerais, Rio de Janeiro and São Paulo. Pavitt (1984) showed that such sectors are science based, i.e., companies produce goods and services in the technological frontier, using internal and external Research and Development (R&D) through partnerships with universities, research centers, institutes, etc.
Regarding the total amount funded, R$ 12 million were used to sign the contracts of the 52 projects in SC, R$ 6 million being from Finep and R$ 6 million by way of counterpart (Finep, 2016). In PR, R$ 16,7 million were used for contracting the 55 projects, R$ 10 million being from Finep and R$ 6,7 million by way of counterpart. In ES, R$ 3,2 million were invested, R$ 2 million coming from Finep and R$ 1,2 million by way of counterpart. The total sum for funding the 121 projects was of R$ 31,9 million, R$ 18 million being from Finep and R$ 13,9 million by way of counterpart from local partners. The median sum funded per project was of R$ 242 thousand: the lowest amount invested in a project was R$ 50,800,00 and the highest was of R$ 300,000,00 (the maximum sum allowed in these calls for projects).
With regards to the age of the contemplated companies, they were, on average, less than 10 years old upon being contemplated – between 2009 and 2013. Thus, such companies were created in the early 2000s.
4.2 Impacts of innovation grants in each dimension of sustainable development
4.2.1 Economic dimension
In this dimension, it was possible to verify the impact of grants upon the development and commercialization of innovative new products, services and processes. Out of the 53 supported projects, 52 funding recipients declared to have developed 109 products and services in total (including softwares), 32 of these new services and products being for the global market, and 91 new products and services for the national market. The data initially reveal that the program has been relevant to encourage the development of products and services for the global and national markets, and especially for innovation at company level. Figure 1 illustrates the number of new products and services developed according to each location.
Paraná stood out in the number of new products developed, confirming that the state has set the benchmark in the program. Another factor worth noting concerns the distance between the largest number of products and services developed for the global market (32) and for the national market (91). This result is in agreement with Carrijo et Botelho (2013), who found that the grant was more relevant to promote innovation in the national market. When it comes to the number of new processes developed, the 52 companies reported the development of 60 new processes due to the grant, seven of them being new processes for the global market and 36 being new for the national market. PR companies also informed to have developed a bigger number of process innovations. The results indicate that the companies were more innovative in products than in processes. Figure 2 illustrates the number of process innovations per location.
As for the program’s ability to increase sales, i.e., to promote the commercialization of new products and services, the companies declared that, out of the 109 developed products, 76 of them got to the market – a commercialization rate of 70%. Link et Scott (2010), upon assessing the data for the Small Business Innovation Research (SBIR) projects – an American program which inspired grant implementation in Brazil – found a rate of commercialization slightly higher than 50%, a proportion similar to the one found by Wessner (2008). Link et Scott (2018) recently showed that SBIR has been fulfilling its goal of stimulating the commercialization of technologies related to healthcare in the private sector. Salles Filho et al. (2011), in turn, found a proportion of almost 67% upon evaluating the rate of commercialization of the projects supported by PIPE, in the state of São Paulo.
An assessment of the program’s ability to promote exportation shows that 23 products were exported, which corresponds to 21% of the total. Under this criterion, PR informed better performance both in terms of number of commercialized products and number of exported goods. Companies from ES, the state with the highest level of openness to foreign trade (IJSN, 2012), i.e., the sum of exports plus imports divided by the GDP, presented a shy performance in terms of products exportation. Only one project resulted in exportation of one product to other countries. The data reveal the program’s weak capacity to help companies entering the global market.
It is also worth highlighting the degree of concentration of products per project. Three projects alone have developed 29 products, i.e., 31,9% of the total number of products for the national market. On the other hand, 50% of the projects resulted in at least one product for the national market. A slightly lower percentage (45,3%) was found for commercialization: out of the 53 contemplated projects, 24 reached the market with at least one product.
Upon evaluating the grant’s impact on the expansion of the median revenue of the funding recipients (Figure 4), it is noted that SC companies declared having obtained an average increase in their median annual revenue (25.8%) higher than the PR companies (19.16%). Such information makes it evident that, although the PR projects have obtained higher performance in terms of revenue increase in all the previously described criteria, SC projects have higher added value. In other words, although SC projects were numerically inferior in comparison to those from PR, the former ones had a higher median value in revenue increase.
Figure 1. Number of new products/services developed
Figure 2. Number of new processes/services developed
Figure 3. Number of new products/services commercialized/exported
Figure 4. Percentage of median annual increment in revenue
4.2.2 Social dimension
An analysis of the social dimension is crucial when it comes to non-refundable public innovation programs, where competitivity for public resources and the opportunity cost are high and, therefore, a relevant social return is expected. As shown by Geels et al. (2010), one of the reasons for the public investment in innovation is to unite efforts with the private sector to solve the challenges related to social demands for sustainability.
Figures 5 and 6 show the relevance of the program for job creation, mainly in the case of SC where companies reported to have created 99 jobs, 23 of them for people with master’s degree and 5 for people with doctoral degree. PR companies reported to have created 88 jobs, while ES companies, 40. In total, the program enabled the creation of 225 jobs in the three states, 47 for people with master’s degrees and 13 for people with doctoral degrees. Together, the 53 projects funded the qualification of 164 employees – especially the funding recipients in PR, which informed to have trained 82 employees. Thus, besides job creation, the analysis reveals that the program was also important for the qualification of human resources for innovation related activities.
Another indicator also used to evaluate the program in the social dimension was the relevance of the contemplated projects to expand access to healthcare, education and safety services. The degree of relevance was measured by the scale Very Relevant; Medium Relevant; Poorly Relevant; and Non-Relevant. From the results presented in Figure 8, the projects were poorly relevant to stimulate these services, which are critical in the country. In the case of ES, no beneficiary projects have aimed at improving violence, for example, although rates are high in the state.
Figure 5. Total number of jobs created
Figure 6. Number of Masters and Doctors hired
Figure 7. Number of personnel trained in the activities
Figure 8. Project’s Degree of Relevance to stimulate innovation
4.2.3 Environmental dimension
As the social dimension, the PAPPE program should take into consideration solutions for socially perceived environmental challenges. Nonetheless, according to the Finep technician, the assessment of the program’s environmental impacts is not encouraged by this primary funding agency because PAPPE does not have that scope and also because each project has a different goal, which makes a global evaluation very difficult. But it was deemed necessary to include this dimension in the analysis and, for that purpose, environmental impact indicators, which are usually used in the three evaluated states, were selected from IBGE (2015). Figure 9 shows the degree of relevance attributed to the environmental indicators for the PAPPE beneficiary projects.
Figure 9. PAPPE’s degree of relevance to stimulate selected environmental activities
As it can be observed in Figure 9, the innovations accomplished in the context of PAPPE were not considered relevant to solve environmental problems. However, the innovation initiative had an impact upon the reduction of pollutant emission in 35,19% of the projects, as well as upon the expansion of selective waste collection in 16,57% of the projects. An environmental problem that has greatly afflicted Brazil in recent years - and especially the sate of Espírito Santo - refers to the water supply. However, only 9,26% of the examined projects considered innovation important to expand the water supply and sewage system.
5. GENERAL CONSIDERATIONS AND REPERCUSSIONS OF PUBLIC POLICIES
The problem identified by public management in Brazil, which was the basis for elaborating the innovation grant program, consists of the low competitivity of national companies, usually due to low investment in innovation. In fact, the Industrial Survey of Technological Innovation (IBGE, 2016) reveals that, in 2014, Brazilian companies spent 2.54% of liquid revenue from sales in innovation activities, with 30% of these resources (0.77% of the revenue) being invested in R&D. Furthermore, it is worth noting the Brazilian government’s low investment in innovation: according to IPEA (2017), while the public investment in relation to the GDP in Germany, the United States and Japan were of 2.83%, 2.74%, 3.48%, respectively, Brazil’s expense was of just 1.24% in 2013.
Thus, the general objective of the program consists of encouraging the development of innovative products and processes, by directly funding innovation projects in companies and sharing the risks involved in innovation with them. The program was conceived in the context of PITCE, which, as shown in section 2, possessed 3 focuses: (1) increase of the productive structure’s efficiency; (2) increase of Brazilian companies’ ability to innovate; (3) expansion of exports. However, considering the high opportunity cost involved in the program, it was expected that it would have an impact on a societal level, besides the corporate one. In the economic dimension, the program was effective to further stimulate the commercialization of new product and service innovations for the national market and for the companies, making them more competitive in the domestic market; however, it was modest in promoting the companies’ international competitivity, and much less effective to expand exportation. Therefore, the program did little to contribute to focus 3 of the policy.
In terms of return to society, it is worth highlighting the grant’s relevance for the creation of jobs and qualification of personnel in innovation activities, corroborating the result of Salles Filho et al. (2011) and Link et Scott (2012). Santa Catarina, which turned out to be the state where the program was more effective in terms of revenue, also generated more qualified jobs in the contemplated companies, especially for people with master’s degrees.
The development of innovation for social challenges, such as healthcare, education and safety was not attested, which is a shame, since, due to the shortage of public resources, public policies should consider the combination of private and social impacts. The same can be said about the environmental dimension, since out of all publicized goals originated from the public call no. 02/2006 (MCT/FINEP, 2006), only SC made it explicit that the projects to benefit from the program should “contribute to the generation and maintenance of jobs and profit and to regional development with environmental preservation”. In times of technological transition to foster sustainability, promoting innovation that solves social hurdles in an environmentally responsible manner becomes crucial, as shown by Perez (2012), and that must be a task faced and led by the states.
6. CONCLUSION
This article presents an assessment of the impacts of public research, development and innovation projects (R,D&I) originated from public grants considering the economic, social and environmental dimensions of sustainable development. A total of 53 projects carried out with the PAPPE grant from 2009 to 2013 were examined; seven of them were from the state of Espírito Santo, 21 from Santa Catarina and 25 from Paraná.
The results revealed that, under the economic dimension, funding recipients have been able to expand their competitivity in the national market. In terms of the returns to society (social and environmental dimensions), it was found that, although the program has been effective in creating jobs, its impacts were modest in terms of encouraging innovation that expands access to healthcare, education and safety services, aside from poorly advancing in terms of environmentally responsible innovation.
Although the beneficiaries from the PAPPE program succeed in developing innovation for the national market, much must be done to push companies to be effectively competitive in the international market, potentializing the private returns. Besides, the Brazilian government did not design its innovation program to contribute to priority areas. Thus, it is recommended that innovation grant programs, with a high opportunity cost for society, are designed in a way as to encourage the beneficiaries of these financially supported innovation initiatives to also attend to both social and environmental issues, which are vulnerabilities faced by society in times of sustainability-centered technological transitions.
REFERENCES
Andrade, A. Z. B. (2009), Estudo Comparativo entre a Subvenção Econômica à Inovação Operada pela Finep e Programas Correlatos de Subsídio em Países Desenvolvidos, Dissertação de Mestrado em Administração Pública, Fundação Getúlio Vargas, Rio de Janeiro.
Bozeman, B; Link, A. (2015), “Toward an assessment of impacts from US technology and innovation policies”, Science and Public Policy, Vol. 42, No. 3, pp. 369–376.
Bozeman, B; Youtie, J. (2017), “Socio-economic impacts and public value of government-funded research: Lessons from four US National Science Foundation initiatives”, Research Policy, Vol. 46, No. 8, pp. 1387–1398.
Brasil (2011), Ministério do Desenvolvimento, Indústria e Comércio Exterior, Plano Brasil Maior 2011/2014. Inovar para competir. Competir para crescer, MDCI, Brasília, DF.
Brasil (2016), Lei nº 13.243, de 11 de janeiro de 2016, Dispõe sobre incentivos à inovação e à pesquisa científica e tecnológica no ambiente produtivo e dá outras providências, Diário Oficial da União, 16 maio 2005, disponível em http://www.planalto.gov.br/ccivil_03/_ato2004-2006/2004/lei/l10.973.htm (acesso em: fev. 2019).
Caloffi, A et al. (2018), “A comparative evaluation of regional subsidies for collaborative and individual R&D in small and medium-sized enterprises”, Research Policy, Vol. 47, No. 8, pp. 1437–1447.
Carrijo, M.C; Botelho, M.R.A. (2013), “Cooperação e inovação: uma análise dos resultados do Programa de Apoio à Pesquisa em Empresas (Pappe)”, Revista Brasileira de Inovação, Vol. 12, No. 2, pp. 417-447.
Cassiolato, J.E., Lastres, H. (1999), Globalização e inovação localizada: experiências de sistemas locais no Mercosul, IBICT, Brasília, DF.
Costa, A. C. (2013), Política de inovação brasileira: análise dos novos instrumentos operados pela FINEP, Tese de Doutorado em Economia da Indústria e da Tecnologia - Programa de Pós-Graduação em Economia, Universidade Federal do Rio de Janeiro, Rio de Janeiro.
Dimos, C; Push, G. (2016), “The effectiveness of R&D subsidies: A meta-regression analysis of the evaluation literature”, Research Policy, Vol. 45, No. 4, pp. 797–815.
Figueiredo et al. (2017), “Innovative capacity dynamics: kibs dimensions”, Brazilian Journal of Operations & Production Management, Vol. 14, No. 4, pp. 438-445.
Fundação de Amparo à Pesquisa do Espírito Santo (FAPES), disponível em: http://www.Fapes.es.gov.br/default.asp (acesso em: set. 2015).
Frank, A. G et al. (2016), “The effect of innovation activities on innovation outputs in the Brazilian industry: Market-orientation vs. technology-acquisition strategies”, Research Policy, Vol.45, No. 3, pp. 577-592.
Geels, F.W. (2010), “Ontologies, socio-technical transitions (to sustainability), and the multi-level perspective”, Research Policy, Vol. 39, No. 4, pp. 495-510.
Hud, M.; Hussinger, K. (2015), “The impact of R&D subsidies during the crisis”, Research Policy, Vol. 44, No. 10, pp. 1844–1855.
IBGE - Instituto Brasileiro de Geografia e Estatística (2015), “Indicadores de Desenvolvimento Sustentável – Brasil 2015”, IBGE, Brasília, DF.
IBGE - Instituto Brasileiro de Geografia e Estatística (2016), “Pesquisa de inovação 2014”, IBGE, Rio de Janeiro, disponível em: https://biblioteca.ibge.gov.br/visualizacao/livros/liv99007.pdf (acesso em: fev. 2019).
IJSN - Instituto Jones dos Santos Neves (2012), Nota técnica 36 - Exportações versus importações no Espírito Santo, IJSN, Vitória, ES, disponível em: http://www.ijsn.es.gov.br/component/attachments/download/1708 (acesso em: set. 2016).
IPEA - Instituto de Pesquisa Econômica e Aplicada (2017), “Brasil gasta menos que outros países com P&D”, disponível em http://www.ipea.gov.br/portal/index.php?option=com_content&view=article&id=29255 (acesso em: fev. 2019).
Leal, E. A. S; Souza, M. A. V. F (2011), “O financiamento à inovação tecnológica no Espírito Santo com recursos não reembolsáveis: o pappe -subvenção econômica -resultados preliminares e desafios”, em XXXI Encontro Nacional de Engenharia de Produção, Belo Horizonte, 2011.
Link, N. A; Scott, T. J. (2010), “Government as entrepreneur: Evaluating the commercialization success of SBIR projects”, Research Policy, Vol. 39, No. 5, pp. 589–601.
Link, N. A; Scott, T. J. (2012), “The Small Business Innovation Research Program”, Science and Public Policy.
Link, N. A; Scott, T. J. (2018), “Toward an assessment of the US Small Business Innovation Research Program at the National Institutes of Health”, Science and Public Policy, Vol. 45, pp. 83–91.
MCT/FINEP – Ministério da Ciência e Tecnologia, Financiadora de Estudos e Projetos (2006). Chamada Pública MCT/FINEP – PAPPE, Subvenção – 02/2006, disponível em http://www.finep.gov.br/arquivos_legados/fundos_setoriais/subvencao_economica/resultados/RESULTADO_PAPPE_SUBVENCAO.pdf (acesso em: fev. 2019).
Mazzucato, M. (2014), O Estado Empreendedor: Desmascarando o mito do setor público vs. setor privado, 1 ed., Portfolio-Penguin, São Paulo.
Morais, J. M. (2008) “Uma avaliação de programas de apoio financeiro à inovação tecnológica com base nos Fundos Setoriais e Lei de Inovação, Políticas de incentivo à inovação tecnológica no Brasil”, Rio de Janeiro: IPEA/ Secretaria de Assuntos Estratégicos da Presidência da República, pp. 67-105, disponível em: http://www.redesist.ie.ufrj.br/p8/ (acesso: fev. 2017).
OECD (1998). Best Practice Guidelines for Evaluation, PUMA Policy Brief. nº 05, 1998, disponível em www.oecd.or/dataoecd//11/56/1902965.pdf (acesso: fev. 2017).
Pavitt, K. (1984), “Sectoral patterns of technical change: toward a taxonomy and a theory”, Research Policy, Vol. 13, No. 6, pp. 343-373.
Perez, C. (2012), “Financial bubbles, crises and role of government in unleashing golden ages”, FINNOV.
Rocha, F. (2015), “Does governmental support to innovation have positive effect on R&D investments? Evidence from Brazil”, Revista Brasileira de Inovação, Vol. 14, No. Esp., pp. 37-60.
Salles Filho, et al. (2011), “Evaluation of ST&I Programs: a methodological approach to the Brazilian Small Business Program and some comparisons with the SBIR program”, Research Evaluation, Vol. 20, No. 2, pp. 159–171.
SBIR – Small Business Innovation Research, disponível em https://www.sbir.gov/about/about-sbir (acesso em: dez. 2017).
Wessner, C W. (2008), “An Assessment of the SBIR Program at the National Science Foundation”. Washington, DC: National Academies Press.
Received: 12 Dec 2018
Approved: 28 Dec 2018
DOI: 10.14488/BJOPM.2019.v16.n1.a10
How to cite: Leal, E. A. S.; Echeveste, M. E. S.; Guimarães, L. B. M. et al. (2019), “Assessment of the impacts of Innovation Grants in Brazilian States”, Brazilian Journal of Operations & Production Management, Vol. 16, No. 1, pp. 104-112, available from: https://bjopm.emnuvens.com.br/bjopm/article/view/731 (access year month day).