How to measure Entrepreneurial Ecosystems? Current debate and challenges for the Brazilian case

Abstract

Objective:

The paper reviews and systematizes the metrics proposed in the literature for measuring the different elements of entrepreneurship ecosystems and discuss the challenges of measuring ecosystems at the regional level and in the Brazilian case. Method: Bibliographic review of the metrics proposed for Entrepreneurship Ecosystems and descriptive statistics to systematize the indicators found in the literature.

Results:

There is still no uniformity in the proposal of metrics. Most of them have a national approach and are related to the dimensions of public policy and regulation, culture, and resources. The effort to apply theory to the Brazilian case is still incipient, and it is necessary to construct adequate metrics for the national reality.

Theoretical/methodological contributions:

The article contributes to the debate on the measurement of entrepreneurship ecosystems by showing the diversity of proposed methodologies and indicators, existing gaps, the need to think about indicators for the subnational level, and the implications for the Brazilian case.

Relevance/Originality:

Despite its potential as a tool for public policies, the concept of EE and its measurement methods are still underexplored by the Brazilian academic community. The article aims to fill this gap and demonstrate that this line of academic research is promising.

Keywords:
Entrepreneurial Ecosystems; Metrics; Regional Development; Public Policies

Resumo

Objetivo:

O artigo realiza uma revisão e uma sistematização das métricas propostas na literatura para a mensuração dos diferentes elementos dos ecossistemas de empreendedorismo e discussão acerca dos desafios para a mensuração de ecossistemas em nível regional e no caso brasileiro.

Método:

Revisão bibliográfica acerca das métricas propostas para Ecossistemas de Empreendedorismo e estatística descritiva para sistematizar os indicadores encontrados na literatura.

Resultados:

Ainda não há uniformidade na proposição de métricas. A maioria delas possui abordagem nacional e estão relacionadas com as dimensões políticas públicas e regulação, cultural e recursos. O esforço de aplicar a teoria sobre o caso brasileiro ainda é incipiente e é necessário construir métricas adequadas à realidade nacional.

Contribuições teóricas/metodológicas:

O artigo contribui com o debate acerca da mensuração de ecossistemas de empreendedorismo ao mostrar a diversidade de metodologias e indicadores propostos, as lacunas existentes, a necessidade de se pensar indicadores para o nível subnacional e as implicações para o caso brasileiro.

Relevância/Originalidade:

Apesar de seu potencial como ferramenta para políticas públicas, o conceito de EE e suas formas de mensuração ainda é pouco explorado pela comunidade acadêmica brasileira. O artigo pretende preencher essa lacuna e demonstrar que essa linha de pesquisa acadêmica é promissora.

Palavras-chave:
Ecossistemas de Empreendedorismo; Métricas; Desenvolvimento Regional; Políticas Públicas

INTRODUCTION

The concept of entrepreneurial ecosystems (EE) gained widespread recognition over the last decade (Spigel et al., 2020) within public policy discussions (Isenberg, 2011; Spigel, 2016; Mack & Qian 2016) due to its evident potential for job and income generation, and consequently, economic growth and development. In spite of its popularity, the concept still faces challenges in achieving theoretical consolidation and grapples with limitations both in academic production and practical application (Cardona & Torres, 2022).

Essentially, the center of the debate on the topic has a theoretical (Isenberg, 2016; Stam & Spigel, 2016) and methodological nature (Alvedalen & Boschma, 2017; Ratten, 2020). In this sense, a lack of consensus in the literature regarding the definition of the concept is reflected in the difficulty to make it an operational concept that can be approached by public policy. In this framework, the primary challenges observed by Spigel et al. (2020) include: the limitations regarding the existing data sources, the need to balance the findings of quantitative and qualitative studies, and the imperative necessity to temper different approaches that use simplified models and more complex systems.

In order to address these multiple challenges, various authors have recently set out to reflect upon and propose indicators and metrics to measure entrepreneurial ecosystems (Stam, 2018). This prolific material allows the conduction of a comparative analysis of these measuring means in terms of their development, while simultaneously enhancing their effectiveness as public policy instruments. These metrics aim to identify the causal mechanisms involved in the dynamics within the ecosystem and serve as a guide to public policy strategies focused on regional development. Therefore, by raising awareness about the elements that directly or indirectly impact entrepreneurship, it is possible to identify the input that derives from public policy and provide entrepreneurs with broader knowledge on the environment in which they interact with peers, namely, other agents.

It is worth highlighting the inherently regional essence of the entrepreneurial phenomenon (Feldman, 2001), and that acknowledging the features that comprise its ecosystem does not necessarily indicates that the development of public policies to foster entrepreneurship is universal, that is, there are no one-size-fits-all protocols (Tödtling & Trippl, 2005). In view of this, depending on the specific characteristics of each region, some aspects may be more relevant than others and, based on the knowledge of different metrics, government officials are able to allocate their efforts and resources strategically to formulate public policies that promote entrepreneurship across multiple analytical levels.

In view of the fact that measuring the different elements of entrepreneurial ecosystems is essential for the design of effective and appropriate public policies that consider the specific features of each region, it is equally essential to find ways to develop instruments for this purpose. Hence, the main objective of the present study is to elaborate a comprehensive review and systematization of the metrics found in the literature available on measuring the different elements of entrepreneurial ecosystems. In addition to its main purpose, the study will also address the challenges of measuring ecosystems at the regional level and within the Brazilian context.

Striving to provide an overview of how metrics are currently proposed in the literature, the present study was conducted so as to identify patterns in the methodologies and approaches adopted by the authors of the studies reviewed and systematized. Thereby, it was possible to outline fundamental features involved in the measuring of EEs such as: the measuring methodologies used, the geographical scope, the dimensions referred to, among others. The findings of this step will serve as the foundation for the analysis of the overarching theme under discussion, that is, the possibilities and challenges inherent to the creation of metrics to measure entrepreneurial ecosystems at the regional level and the Brazilian case.

In addition to this introduction, the paper is divided into five sections. Section 2 presents an overview of the theoretical framework on entrepreneurial ecosystems and of the recent efforts to develop metrics that measure EEs. Section 3 describes the research methodology. Section 4 discusses the study’s key findings. Section 5 offers a brief analysis of the results. The final section, Section 6, is dedicated to the final remarks, which point out the limitations of this study and present future research questions.

THEORETICAL FOUNDATION: ENTREPRENEURIAL ECOSYSTEMS AND RECENT MEASURING EFFORTS

Over the last decade, the concept of Entrepreneurial Ecosystems (EE) has emerged in the literature and gained popularity, not only among scholars, but also among experts on entrepreneurship and government officials. Highlighting the significance of the context for entrepreneurial activity, ecosystems are defined as a collection of actors and other elements such as institutions, culture, social structures and entrepreneurial processes that combine and interact in an organic and complex form to foster productive entrepreneurship within a specific region (Isenberg, 2010; Mason & Brown, 2014; Stam & Spigel, 2016). According to Isenberg (2011), one of the most recognized contributions on the theme, Entrepreneurial Ecosystems are formed by features, dimensions and conditions. The author’s model proposes that six are the dimensions of the Entrepreneurial Ecosystem, namely: Markets, Policy, Finance, Culture, Supports and Human capital.

Despite its popularity among public administrators and entrepreneurs and the theoretical progress made by scholars, the concept of Entrepreneurial Ecosystem has yet to find a consensus among experts in the field. In fact, it has recently been the object of fierce criticism when it comes to the development of its theoretical, empirical and methodological frameworks. A thorough examination of these controversies is beyond the scope of the present article; however, it is worth mentioning some of the empirical and methodological criticism that is, ultimately, an indication of the necessity to create metrics for Entrepreneurial Ecosystems.

As to the empirical approach to entrepreneurial ecosystems, the literature underscores the scarcity of comparative studies that are able to elucidate the different trajectories observed for different EEs (Roundy & Bayer, 2019), as well as a lack of studies that unveil the dynamics of networks within these ecosystems (Alvedalen & Boschma, 2017). In the methodological realm, there is criticism towards the use of static tools to describe EEs, which disregard their evolution (Alvedalen & Boschma, 2017). Simultaneously, an integration of methodologies from diverse disciplines has been proposed, in order to investigate the ways in which connections are formed and cultivated within these ecosystems (Ratten, 2020). In response to the criticism involving the conduction of static analyses, some authors suggest incorporating contributions from alternative approaches to introduce a systemic and evolutionary perspective to the concept of entrepreneurial ecosystems.

In this sense, Roundy et al. (2018), O’Connor et al. (2018) and Fredin and Lidén (2020) use the Complex Adaptive Systems (CAS) approach to understand the dynamics of connections and interactions within an EE. Mack and Mayer (2016) and Cantner et al. (2021), in turn, propose a dynamic life cycle model for entrepreneurial ecosystems so as to investigate the emergence, development and consolidation of these ecosystems, analyzing them as evolutionary phenomena. Gasparoto and Fischer (2019) used the Social Network Analysis to examine entrepreneurial ecosystems. The authors defend that this approach facilitates the investigation of relationships, characteristics and the impact that the connections among heterogeneous actors make. They also assert that the approach provides established procedures for data collection and analysis, as well as it provides a collection of specific metrics tailored for networks that standardize practice and facilitate comparative analyses. Finally, the authors claim this approach consists of constructs that are not limited by scale, enables the establishment of networks at different maturing stages and, lastly, promotes the execution of dynamic studies on networks.

Regarding the regional level of ecosystems, O’Connor and Audretsch (2022) draw inspiration from the field of ecology, specifically the theory of forests, to formulate the concept of Regional Entrepreneurship Ecosystem (REE), since both forests and the REE consist of partitioned elements and conflicts of interest and can be characterized by their structure, function, and composition. Therefore, the same evolutionary traits observed in forests apply to entrepreneurial ecosystems. According to the parallels identified, the key indicators for measuring the trajectory of an EE are structural complexity and social diversity. In turn, Stam and Van de Ven (2021), and Nicotra et al. (2018) present a causal model of entrepreneurial ecosystem, emphasizing the coevolutionary perspective of various organizations and institutions that interact with each other, both cooperatively and competitively. These entities play complementary roles and aim to fulfill their own needs and interests.

In addition to enhancing the concept’s theoretical robustness and assisting public administrators in its practical application by making its use less metaphorical and abstract (Stam, 2015), a growing number of authors have begun to advocate for and contribute to the development of EE measuring tools (O’connor et al., 2018). The proposal of metrics aims to simultaneously draw inspiration from and address the recent criticism directed at the concept of EE.

A pioneer in this effort, Stam (2018) introduced a measuring model divided into ten elements which were related to official indicators and data sources. This effort has not only inspired numerous other authors, including Stam himself, to delve deeper into the subject, but has also enriched the discussion through the proposal of new metrics and indicators. It is worth mentioning that the model proposed by Stam expands that of Isenberg (2011) to new dimensions, and incorporates the following elements: Formal institutions, Entrepreneurial culture, Physical infrastructure, Demand, Networks, Leadership, Talent, Funding, New knowledge and Intermediary services (for a more comprehensive overview, refer to Stam, 2018, p. 179).

In order to address the limitations of traditional indicators and scales created to measure entrepreneurship at the national and regional levels, such as the Global Entrepreneurship Monitor (GEM), Doing Business (DB), and the National Federation of Independent Business’s Poll (NSBP), Liguori et al. (2018) also expand on Isenberg's (2011) model that introduces the six dimensions of EE to develop the Multidimensional Entrepreneurship Ecosystem Scale (MEES), created to assess the EEs cognitively by considering perceptions that influence individual behavior rather than relying on objective metrics. The authors state that the greatest advantages of this method is raising more awareness about the factors that promote entrepreneurship, as well as the ability to contribute practically through precise ecosystem measuring.

In the same vein, the contribution of Corrente et al. (2019) is based on the analysis of the EE measuring instruments developed by the World Bank, the World Economic Forum, and the Organisation for Economic Co-operation and Development (OECD). They propose the adoption of the Stochastic Multicriteria Acceptability Analysis (Lahdelma et al., 1998) technique, a method that offers the advantage of objectively determining the weights assigned to each variable of the index, thereby reducing the arbitrariness of weights and enhancing the comprehensive value derived from the data analysis.

Leenderstse et al. (2020) prioritize the analysis on the regional level, justified by the fact that entrepreneurship is primarily a regional phenomenon (Feldman, 2001), and by the fact that there is a significant variation in entrepreneurial activity across different regions of the same country. In view of this, they developed a methodology consisting of two dimensions: quantification, achieved by measuring key elements using official data sources (Credit et al., 2018), and qualification, carried out through the development of a methodology that provides insights into the interdependence between these elements, the overall quality of the entrepreneurial economy, and the relationship between these elements and the system's output.

In order to address the gap in comparative studies and overcome the obstacle of having to analyze EEs in different stages of the life cycle of ecosystems, Content et al. (2020) complement the approach proposed by Bruns et al. (2017) and introduced a formal model based on empirical analysis that tests and classifies the existing heterogeneity among entrepreneurial ecosystems, aiming to reveal the impact of entrepreneurial activity on growth differentials across regions.

The most comprehensive and elaborate application of measuring tools among the ones that are currently under development is presented in the GSER (2022) report, which categorizes and describes the Startup EEs. The report introduces a linear model of development for ecosystems and employs different methodologies to classify EEs in various stages of development. This approach allows for the use of metrics that are most relevant according to the prevailing dynamics at each maturing stage. The methodology used to classify ecosystems is based on preconceived metrics and weights for measuring factors that are used to calculate the overall performance of ecosystems. The weight vector used was obtained through correlation tests and linear regression modeling, with the factors featuring as independent variables and performance metrics as the dependent variable. The report, therefore, expands the practical application of the theoretical and methodological principles discussed in the literature, demonstrating that a line of action that considers evolutionary aspects, metrics, and the presence of heterogeneity among ecosystems is already underway.

Finally, in order to measure value creation and capture within ecosystems, Cavallo et al. (2021) drew on constructs of strategic networks, value networks and business models to conduct a qualitative survey in the San Francisco Bay Area, California, and developed a more systemic model to assess ecosystems when it comes to relationships and value exchanges, which can be valuable for orienting the design of public policies.

All in all, it is blatant that the theoretical advancement of the Entrepreneurial Ecosystems concept, as well as its increased utility for the formulation and implementation of public policies cannot be dissociated from the development of metrics, indicators, and other measuring methods. However, this effort is still incipient for several reasons, including data scarcity, limitations connected to methodologies and to the theoretical framework, which is indication that this field is full of possibilities for willing researchers.

METHODOLOGY

Considering the predominantly qualitative nature of the present study, a search for articles in the Periódicos CAPES (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior) database was conducted using the following keywords: "Entrepreneurial Ecosystems", "Metrics," "Index", and "Indicators" - in both English and Portuguese, as well as their combinations. After reviewing the abstracts found, the articles selected were the ones that proposed metrics and indicators to measure EEs, and that were published over the last five years. In addition to the studies identified in the CAPES database, we incorporated books from the Innovation Economics Group (GEI) collection, a research group from the Institute of Economics (IE), at the Federal University of Rio de Janeiro (UFRJ). The criterion that oriented the book selection was their relevance to the theme studied.

Following the summarization of each of the seventeen articles selected, the metrics and indicators proposed by the authors were identified, categorized and analyzed. The collected metrics were then identified and organized into a Microsoft Excel® spreadsheet according to various parameters such as Metric, Category, Measuring Object, Determination Method, Original Article, Main Geographic Focus, Source, Year, Nature of the Research (quantitative or qualitative) and, finally whether or not Brazil is somehow part of the analysis carried out in the study. Subsequently, a results table was organized which displays the information and groups the metrics under their corresponding categories. This grouping and categorization indicate the potential elements of the entrepreneurial ecosystem that the metrics refer to. In this regard, the determined categories or dimensions include: Culture, Support networks, Human capital, Markets, Public policy & Regulation, Finance, Output, and General characteristics. These eight dimensions were created based on the six dimensions proposed by the Isenberg model (2011) created to characterize entrepreneurial ecosystems. The choice is justified by the extensive use of this model by the experts who discuss EEs. It is worth noting, however, that the last two categories - Output and General characteristics - are part of the contribution of the present research, as they were considered necessary to categorize the metrics. The Output dimension corresponds to the outcomes of entrepreneurial activity, and the General characteristics dimension is connected to other features of the entrepreneurial ecosystem not covered by the aforementioned dimensions.

RESULTS

In order to systematize the core pieces of information found, we organized Table 1 with the author, the methodology used, and the results achieved in each study analyzed. We have chosen to present articles that share the same methodology on the same row.

It is interesting to note the wide range of different methods used, the number of indicators, the dimensions considered, and the fact that only a couple of articles share the same methodology. This particular finding is indication that the efforts and techniques used for measuring entrepreneurial ecosystems have not yet been standardized by experts in related studies and approaches.

Subsequently, all the metrics proposed in this set of articles were identified, totaling 349. It is essential to highlight that these metrics are not meant to individually characterize the ecosystem, but are rather indicators that correspond to each of the chosen dimensions or, at least, a part of them. Therefore, the metrics were very much connected to the 6 dimensions proposed by Isenberg (2011), and to the two additional dimensions suggested by the authors of this study. The 8 dimensions are the following: Culture, Support networks, Human capital, Markets, Public policy & Regulation, Finance, Output, and General characteristics.

Each dimension encompasses a set of related features and or actors considered accordingly in the conduction of the present analysis. The Culture dimension corresponds to the social norms and principles that impact entrepreneurship. The Support networks dimension includes the entities and the support systems responsible for assisting entrepreneurs, such as physical infrastructure, non-governmental organizations, and professional guidance and mentorship (lawyers, consultants). The Human capital dimension involves the availability of top talent. The Markets dimension consists of the networks and channels through which goods and services provided by firm formation reach end consumers. The Public policy & Regulation dimension refers to the institutional environment and affirmative actions taken by governments to foster entrepreneurship. Finance encompasses the access to various forms of credit. The Output dimension corresponds to the products and services generated by the ecosystem. Finally, the General characteristics dimension includes information, mainly demographic, beyond the scope of the previously mentioned dimensions. It is important to note that the metrics were essentially assessed based on intrinsic characteristics, and their effectiveness was not evaluated.

In the subsequent stage, we aimed to observe the relevant characteristics that would facilitate the process of applying these metrics to the Brazilian case. Our initial concern was to identify metrics that include Brazil, in order to determine the extent to which data about the country is available. If a metric that did not include Brazil provided relevant data, it makes sense to come up with alternative methods to obtain the same information using the available data for the country.

Another criterion used was the level of analysis of the metrics, especially regarding the existence of data at subnational levels. Regional-level metrics are less frequent than metrics that consider the national level, due to the different availability of data at these levels, which makes the examination of entrepreneurial ecosystems more challenging. However, while ecosystem analyses at national levels are extremely useful for comparisons with other national ecosystems and for the identification of potential shortcomings, the understanding of the causal relationships between local dimensions and the eco-product, as well as the formulation of regional development policies cannot be dissociated from ecosystem analyses at the regional level.

The results of analysis of the spreadsheet revealed that out of the 349 metrics mapped, 71.63% include some information about Brazil. Based on the data sources that originated these metrics, it was possible to conclude that the dominant approach was the one that considers national levels, with a recurrence of 57.88%, while the subnational level approach accounted for 24.07%, and the multilevel approach for 18.05% (see Figure 1).

Figure 1
Metrics by Geographic Level (%)

The categorization of these metrics was conducted according to the data collection methodology, which could either be qualitative or quantitative in nature. Qualitative metrics provide insights on how agents perceive what they want to measure, making them more subjective. They are typically obtained through questionnaires and interviews and are valuable for understanding the context in which the data are generated. In contrast, quantitative metrics are instruments of comparison obtained by counting the actual occurrence of the phenomenon observed. Among the analyzed metrics, 55.3% were considered quantitative, 27.79% were considered qualitative, and 16.91% were considered undetermined due to a lack of clarity regarding the qualitative or quantitative nature of their data collection methodology.

The dimension classification we carried out revealed that the metrics analyzed are well distributed across the eight dimensions. Public policy & Regulation, Culture, and Human capital are the most observed dimensions, while General characteristics is the least frequent, accounting for only 6.30% of the metrics mapped (see Figure 2).

Table 1
Main Data Collected from the Articles Reviewed, Including Author, Methodology and Results

Author	Methodology	Results
Ács et al. (2014)	The approach is grounded in both theoretical and empirical foundations. The index comprises variables related to individual behavior and the institutional context, and these variables are weighted using the novel index methodology to mitigate issues related to arbitrariness and normalization. Institutional variables are incorporated as weights for individual-related variables, leading to differing impacts of attitude, skills, and ambition on entrepreneurship, according to the changes in the institutional environment across countries.	The article presents 30 metrics, which encompass various geographical approaches and the dimension categories proposed by Isenberg. In terms of geographical approach, the article incorporates metrics at the national, international, and subnational levels. As for dimension segmentation, the metrics span across dimensions such as Culture, Finance, Output, Markets, Support networks, Human capital, Public policy & Regulation, and General characteristics.
Arruda et al. (2013)	The Isenberg EE model served as a guide for the conduction of the qualitative research conducted, while the OECD model formed the basis for the quantitative research.	The article introduces 91 metrics spanning across the dimension categories proposed by Isenberg. Regarding the geographical approach, the article includes national-level metrics. In terms of dimension segmentation, the metrics encompass the Culture, Finance, Markets, Human capital, Public policy & Regulation, Support networks, and General characteristics dimensions.
Inácio Jr et al. (2016)	The Global Entrepreneurship Index (GEI) methodology was used to analyze the Brazilian entrepreneurial ecosystem in the context of the National Systems of Entrepreneurship (NSE) theory.	The article provides 13 metrics, spanning across the dimensions proposed by Isenberg. Concerning the geographical approach, it encompasses national-level metrics. In terms of dimension segmentation, the metrics cover the Culture, Finance, Output, Markets, Support networks, Human capital, and Public policy & Regulation dimensions.
Souza et al. (2016)	Primary data collected from a survey was the foundation of the quantitative study.	The article introduces 31 metrics, which vary within Isenberg's proposed dimension categories. Regarding the geographical approach, the article includes national-level metrics. In terms of dimension categories, the metrics encompass dimensions such as Culture, Finance, Support networks, Human capital, Public policy & Regulation, and General characteristics.
Bruns et al. (2017) and Content et al. (2020)	A formal model created through empirical analysis that tests and classifies heterogeneity among entrepreneurial ecosystems with the aim of revealing the impact of entrepreneurial activity on growth differentials across multiple regions.	The articles introduce 8 metrics that correspond to Isenberg's proposed dimension categories. Concerning geographical focus, the articles encompass subnational-level metrics. As for dimension categories, the metrics span across the Culture, Output, Human capital, and General characteristics dimensions.
Credit et al. (2018)	Systematic research aimed at identifying studies that use secondary data, followed by a categorization of the sources and types of data used. Additionally, the detection of the trends and gaps presented by these data was conducted.	The article provides 32 metrics varying across geographical approaches and Isenberg's proposed dimension model. In terms of geographical coverage, the article encompasses national and subnational-level metrics. Regarding dimension categories, the metrics span across Culture, Output, Markets, Infrastructure, Human capital, and General characteristics dimensions.
Liguori et al. (2018)	Proposed a Multidimensional Entrepreneurship Ecosystem Scale (MEES).	The article provides 22 metrics spanning across different dimensions proposed by Isenberg. Regarding the geographical approach, the article incorporates metrics at multiple levels. In terms of dimension categorization, the metrics encompass aspects of the Culture, Finance, Markets, Support networks, Human capital, and Public policy & Regulation dimensions.
Nicotra et al. (2018)	Causal model of an entrepreneurial ecosystem, emphasizing the coevolutionary perspective of different organizations and institutions that interact with each other in a cooperative and competitive manner, playing complementary roles and aiming to meet their own needs and interests.	The article introduces 26 metrics, spanning across geographical approaches and the dimensions proposed by Isenberg. In terms of geographical scope, the article includes both national and subnational-level metrics. Regarding dimension segmentation, the metrics assessed span across the Culture, Finance, and Output dimensions.
O’connor et al. (2018)	The Complex Adaptive Systems (CAS) approach was used.	The study presents 16 metrics, which correspond to the dimensions proposed by Isenberg. In terms of geographical scope, the article includes subnational-level metrics. Regarding dimension categories, the metrics cover a range of dimensions, including Culture, Finance, Output, Markets, Support networks, Human capital, and Public policy & Regulation.
Stam (2018)	The creation of an Entrepreneurial Ecosystem Index was proposed using input elements measured with indicators from official databases. Results are correlated with some of the Output elements.	Seventeen empirical indicators representing ten ecosystem dimensions were identified, including Formal institutions, Entrepreneurship culture, Physical infrastructure, Demand, Networks, Leadership, Talent, Finance, New knowledge, and Intermediate services.
Alves et al. (2019)	The Fuzzy-Set Qualitative Comparative Analysis (fsQCA) method was used.	The article presents 12 metrics that mirror the dimension model proposed by Isenberg. Concerning the geographical approach, the article includes subnational and regional-level metrics (Knowledge-intensive EE). In terms of dimension segmentation, the metrics encompass dimensions such as Finance, Output, Markets, Support networks, Human capital and General characteristics.
Corrente et al. (2019)	The Stochastic Multicriteria Acceptability Analysis technique was used.	The article introduces 12 metrics, which vary across the dimensions proposed by the Isenberg EE model. In terms of the geographical approach, the article includes national-level metrics. Regarding the dimension segmentation, the metrics covered span across the Cultural, Finance, Markets, Infrastructure, Human capital, and Public policy & Regulation dimensions.
Gasparoto and Fischer (2019)	The Social Network Analysis (SNA) was used to analyze entrepreneurial ecosystems.	The network formed during data collection consisted of 540 nodes representing various institutions, and a total of 1,184 connections between them. Among the 51 participating institutions, 35 were businesses, while the remaining 16 were entities supporting entrepreneurship. Within the business category, a significant portion (31) were spin-offs originating from Unicamp. There are a total of 5 distinct networks, each representing institutions, with the key difference being in how the nodes are presented. Networks A, B, C and D depict institutions individually, where each node corresponds to a specific institution. In contrast, network E organizes nodes into groups based on the categories of the organizations. Network B is a subnetwork of network A, and networks C and D are subnetworks of network B.
Maysami et al. (2019)	The meta-synthesis method was applied to propose the main components of the technological entrepreneurship ecosystem, and twelve dimensions were identified in the ETE approach in addition to the outlining of the principles that govern the development of a measurement framework for EEs. Results include the definition of six criteria. Finally, eighteen measuring frameworks associated with the ETE were reviewed.	The 12 dimensions identified in the conduction of the ETE approach are: Governance, Capital, Culture, Support Services, Infrastructure, Talent, Education & Research, Customers & Markets, Networks & Relations, Special conditions, Organizations, and Tech Entrepreneurs. Meanwhile, the 6 criteria established for ecosystem measurement are: Comprehensiveness/Complexity, Types of Measures, Designing Method, Data Gathering Method, Assessment Level and Implementation Period.
Leenderstse et al. (2020)	The methodology consists of two axes: quantification, achieved by measuring key elements through official data sources, and qualification, carried out through the development of a methodology that provides an insightful take on the interdependence between elements, the overall quality of the entrepreneurial economy, and the relationship between elements and the system's output.	The article introduces 41 metrics, spanning across geographical approaches and the dimensions proposed by Isenberg. Regarding the geographical approach, the article considers both national and subnational-level metrics. As to dimension segmentation, the metrics assessed span across the Culture, Finance, Output, Markets, Human capital, Public policy & Regulation, Support networks, and General categories dimensions.
Beneli et al. (2022)	The procedures employed included a review of bibliographic materials, the adaptation of the methodology proposed by the European Innovation Scoreboard, and the compilation of available databases within the Brazilian statistical system. The report suggests outlining the theoretical model that underpins the selection of indicators linked to the measured phenomena and applying statistical procedures to replace the arbitrary nature of indicators.	The article introduces 15 metrics, which correspond to the dimensions proposed by the Isenberg EE model. Regarding the geographical approach, the article includes subnational-level metrics (SRI). In terms of dimension segmentation, the metrics assessed span across the Culture, Output, Human capital, and Public Policy & Regulation dimensions.

• Note: The authors’ own elaboration (2023).

Figure 2
Metrics by Dimension (%)

As to the data sources, 80 different origins were identified. The five most relevant sources were the Global Entrepreneurship Monitor (GEM) with 17.48% recurrence, the World Bank with 10.89%, Eurostat with 10.03%, and the studies conducted by Souza et al. (2016) and Liguori et al. (2018) with 8.88% and 6.30%, respectively. However, as the last two are research papers, they have lower potential for comparison and replication. In addition, it was also observed that the origins are not only diverse but also scattered, as the average recurrence did not exceed 1.25% among all sources, which are mainly international statistical research agencies and developed countries.

As previously noted, the Brazilian case is addressed to some extent in most of the studies reviewed. However, even in the cases in which the country is not considered at all, it was observed that it is possible to apply the metrics to the Brazilian context with a high degree of methodological proximity. For instance, the Eurostat database primarily covers European Union countries with two main focuses: entrepreneurial ecosystem outputs and macroeconomic aggregates. Similar output-related data can be obtained from reports such as the 'Mapa de Empresas' from the Brazilian Ministry of Finance, and the access to the Brazilian macroeconomic data is provided by the Brazilian Institute of Geography and Statistics (IBGE).

IMPLICATIONS OF THE MEASURING OF EES AT THE REGIONAL LEVEL AND THE CHALLENGES OBSERVED IN THE BRAZILIAN CASE

As previously mentioned, most of the metrics proposed in the analyzed literature covering Entrepreneurial Ecosystems focus on the national level. However, this level is not comprehensive enough to expand our understanding of the concept. This is because the local determinants are influenced by geographic dimensions and impact the performance of the EEs, which require subnational-level analysis to be properly understood. In this sense, designing public policies to support EEs that will ultimately foster local development and growth, income generation and job creation requires the identification of limits, weaknesses, and possible improvements within the entrepreneurial dynamics of that particular region.

In order to conduct this diagnosis, it is important to collect data available to create an appropriate framework of metrics tailored to the local level to be analyzed. Unlike the national level, which is defined by the country's geographical borders, the subnational level requires segmentation, as it can encompass different areas and perimeters. For example, when defining a region to be analyzed, the researcher must establish the criteria for including entire municipalities or only portions of them in the study. A reasonable strategy for defining this criteria might involve choosing a geographical center and adopting the scope of the causal mechanisms of the ecosystem as the radius for the area analyzed. However, defining this radius can be challenging, as the mechanisms can have varying scopes within reach (Leenderstse et al., 2020). For instance, if a consulting service provided by a freelance professional reaches entrepreneurs within a 100-kilometer radius and a financial institution can extend credit within a 300-kilometer radius, it becomes impractical to establish a single radius for the area based on a specific number of kilometers.

Another issue observed frequently is spatial nesting, where the influence of federative entities at different levels overlaps. To address this challenge, it is necessary to acknowledge the imperfection of the analysis unit, conceive ecosystems as being delimited by open borders that allow interaction with other units, and define the area based on a common criterion (Leenderstse et al., 2020). For example, the European geographical system named Nomenclature of Territorial Units for Statistics (NUTS), the NUTS classification, uses socio-economic data to define areas that can be geographic units that correspond to entrepreneurial ecosystems. However, the Eurostat, the statistical office of the European Union, only provides Europe-wide data. European researchers also benefit from the abundance of regional-level data, which, unfortunately, is not the case of other regions across the globe.

Despite a significant number of metrics are made available for the Brazilian case, the measurement efforts in the country are still in early stages. Similarly to researchers that analyze international cases, researchers interested in measuring the Brazilian entrepreneurial ecosystem make use of various measuring methodologies.

An example of these incipient measuring efforts is found in Arruda et al. (2013), who analyzed the Brazilian entrepreneurial ecosystem (at the national level), by conducting interviews to collect perceptions. The data collection, oriented by the Isenberg model (2010), was used to gather qualitative information. Quantitative data was obtained by adapting the variables proposed by the OECD, according to the corresponding data available for Brazil. In turn, Gasparoto and Fischer (2019) gathered qualitative data from questionnaires to draw conclusions on the entrepreneurial ecosystem of the metropolitan region of Campinas, in the Brazilian state of São Paulo. Finally, Autio et al. (2016) developed an original metric named GEI, by using institutional and individual variables. Institutional variables were obtained directly from sources such as the Global Competitiveness Report, the Index of Economic Freedom, the World Bank's Ease of Doing Business, the United Nations, UNESCO, and the KOF Index of Globalization. Individual variables were obtained directly from the Global Entrepreneurship Monitor (GEM). These studies highlight the scarcity of databases and dedicated surveys capable of providing metrics to explore the Brazilian case and, more so, subnational contexts.

FINAL REMARKS

Due to their role in job creation, income generation, innovation, and as drivers of technological change, entrepreneurial ecosystems have become increasingly popular among scholars and policymakers. The latter aim to create, develop, and consolidate ecosystems in their countries, regions, and cities. To do so, reliable diagnosis is required, which, in turn, is the outcome of effective metric definition. This necessity of diagnosis drives researchers from various fields to develop conceptual frameworks and theoretical foundations capable of supporting the construction of indicators and metrics for EEs. This effort has the potential to create a virtuous cycle between epistemic and practical communities, contributing to the popularization and dissemination of the concept of entrepreneurial ecosystems and its measuring approaches. However, the findings of the present article indicate that there is no established standard in the literature for measuring ecosystems. Instead, a myriad of methodologies, data sources, indicators and dimensions were observed in the studies.

The present study also shed light on the fact that the spatial dimension, inherent to entrepreneurial ecosystems, adds another layer of complexity to measuring EEs: the need to generate indicators at subnational levels, often extending beyond traditional administrative boundaries. Much of the effort observed has mostly remained at the national level, mainly due to the greater availability of official data sources. Furthermore, the key reference studies are still predominantly Europe-wide and rely on databases provided by official organizations in the continent. This severely limits the applicability of these findings to other social and economic contexts, especially in developing countries, due to the significant impact of the institutional specificities of each entrepreneurial ecosystem on its performance.

In addition, a discrepancy was noted among the metrics analyzed when it comes to the dimensions identified within EEs, often resulting from variations in data collection and availability. This accounts for the prevalence of quantitative metrics over qualitative ones, even though the latter are valuable for capturing significant aspects of entrepreneurial ecosystems.

In Brazil, the effort to measure EEs is recent and still is in its early stages. Similarly to what was observed in the international literature reviewed, most of the indicators proposed by Brazilian authors are limited to the national level (La Rovere et al., 2021). Nonetheless, efforts to measure regional entrepreneurial ecosystems were observed. These efforts involve the use of both qualitative data such as the information obtained from surveys, and the adaptation of international indicators to create new local metrics. However, it is noting that, while international references serve as a valuable guide for creating a measurement framework that is applicable to the Brazilian case, it is essential to emphasize that ecosystems are not homogeneous structures. Therefore, it is extremely important to prioritize the local specificities responsible for the uniqueness of each entrepreneurial ecosystem.

Despite these limitations, it was possible to find international cases that resemble the Brazilian context or specific regions within the country. This similarity enables a comparison in terms of performance and structure of the national or regional EEs with their counterparts. Consequently, it allows for the identification of underused strengths and the weaknesses that hinder the ecosystem, and the design of potential strategies to address such limitations.

As intended in its main objective, the article expands the literature on entrepreneurial ecosystems by offering a thorough review and systematization of recent efforts of measuring EEs. The study identified significant gaps in the field and delved into the challenges specific to the Brazilian case and subnational cases. Lastly, opportunities to develop appropriate metrics were pointed out, both for the national EEs and for the characterization of local and regional ecosystems.

Acknowledgments

We thank research colleague and advisor Renata Lèbre La Rovere for her partnership over the years.

REFERÊNCIAS

• Ács, Z. J., Autio, E., & Szerb, L. (2014). National Systems of Entrepreneurship: Measurement issue and policy implications. Research Policy, 43, 476-494. https://doi.org/10.1016/j.respol.2013.08.016
  » https://doi.org/10.1016/j.respol.2013.08.016
• Alvedalen, J., & Boschma, R. (2017). A critical review of entrepreneurial ecosystems research: towards a future research agenda. European Planning Studies, 25(6), 887-903. https://doi.org/10.1080/09654313.2017.1299694
  » https://doi.org/10.1080/09654313.2017.1299694
• Alves, A. C., Fischer, B., Vonortas, N. S., & Queiroz, S. R. R. (2019). Configurações de Ecossistemas de Empreendedorismo Intensivo em Conhecimento. Revista de Administração de Empresas, 59(4). https://doi.org/10.1590/S0034-759020190403
  » https://doi.org/10.1590/S0034-759020190403
• Arruda, C., Cozzi, A., Nogueira, V., & Costa, V. (2013). O Ecossistema Empreendedor Brasileiro de Start-ups: Uma análise dos determinantes do empreendedorismo no Brasil a partir dos pilares da OCDE. Fundação Dom Cabral.
• Autio, E. I. J. E., Morini, C., Gimenez, F. A. P., & Dionisio, E. A. (2016). Analysis of the Brazilian Entrepreneurial Ecosystem. Empreendedorismo e Inovação, 14 (37), 5-36. https://doi.org/10.21527/2237-6453.2016.37.5-36
  » https://doi.org/10.21527/2237-6453.2016.37.5-36
• Beneli, D. S., Carvalho, S. A. D. de, & Furtado, A. T. (2022). Indicador composto estadual de inovação (ICEI): uma metodologia para avaliação de sistemas regionais de inovação. Nova Economia, [S. l.], 32(2), 359-395. https://doi.org/10.1590/0103-6351/6982
  » https://doi.org/10.1590/0103-6351/6982
• Bruns, K., Bosma, N., Sanders, M., & Schramm, M. (2017). Searching for the existence of entrepreneurial ecosystems: A regional cross-section growth regression approach. Small Business Economics, 49, 31-54. https://doi.org/10.1007/s11187-017-9866-6
  » https://doi.org/10.1007/s11187-017-9866-6
• Cantner, U., Cunningham, J. A., Lehmann, E. E., & Menter, M. (2021). Entrepreneurial ecosystems: a dynamic lifecycle model. Small Bussiness Economy, 57, 407-423. https://doi.org/10.1007/s11187-020-00316-0
  » https://doi.org/10.1007/s11187-020-00316-0
• Cardona, P. F. M., & Torres, C. A. C. (2022). Ecosistemas de emprendimiento: hacia una reflexión práctica y conceptual. Universidad & Empresa, 24(43), 1-29. https://doi.org/10.12804/revistas.urosario.edu.co/empresa/a.12044
  » https://doi.org/10.12804/revistas.urosario.edu.co/empresa/a.12044
• Cavallo, A., Ghezzi, A., & Sanasi, S. (2021). Assessing entrepreneurial ecosystems through a strategic value network approach: evidence from the San Francisco Area. Journal of Small Business and Enterprise Development, 28(2), 261-276. https://doi.org/10.1108/JSBED-05-2019-0148
  » https://doi.org/10.1108/JSBED-05-2019-0148
• Content, J., Bosma, N., Jordaan, J., & Sanders, M. (2020). Entrepreneurial ecosystems, entrepreneurial activity and economic growth: new evidence from European regions. Regional Studies 54, 1007-1019. https://doi.org/10.1080/00343404.2019.1680827
  » https://doi.org/10.1080/00343404.2019.1680827
• Corrente, S., Greco, S., Nicotra, M., Romano, M., & Schilaci, C. E. (2019). Evaluating and comparing entrepreneurial ecosystems using SMAA and SMAA-S. The Journal of Technology Transfer 44, 485-519. https://doi.org/10.1007/s10961-018-9684-2
  » https://doi.org/10.1007/s10961-018-9684-2
• Credit, K., Mack, M. A., & Mayer, H. (2018). State of the field: Data and metrics for geographic analyses of entrepreneurial ecosystems. Geography Compass 12(9). https://doi.org/10.1111/gec3.12380
  » https://doi.org/10.1111/gec3.12380
• Feldman, M. P. (2001) The Entrepreneurial Event Revisited: Firm Formation in a Regional Context. Industrial and Corporate Change, 10, 861-891. https://doi.org/10.1093/icc/10.4.861
  » https://doi.org/10.1093/icc/10.4.861
• Fredin, S., & Lidén, A. (2020). Entrepreneurial ecosystems: towards a systemic approach to entrepreneurship?. Geografisk Tidsskrift - Danish Journal of Geography, 120(2), 87-97. https://doi.org/10.1080/00167223.2020.1769491
  » https://doi.org/10.1080/00167223.2020.1769491
• Gasparoto, M. R. M., & Fischer, B. (2019). A Universidade e o Ecossistema de Empreendedorismo: Um Estudo Utilizando Análise de Redes Sociais. XLIII Encontro da ANPAD, São Paulo.
• Inácio Júnior, E., Autio, E., Morini, C., Gimenez, F. A. P., & Dionisio, E. A. (2016). Analysis of the Brazilian Entrepreneurial Ecosystem. Desenvolvimento em Questão, 14, 5. https://doi.org/10.21527/2237-6453.2016.37.5-36
  » https://doi.org/10.21527/2237-6453.2016.37.5-36
• Isenberg, D. J. (2010). The big idea: How to start an entrepreneurial revolution. Harvard Business Review, 88(6), 40-50.
• Isenberg, D. J. (2011). The Entrepreneurship Ecosystem Strategy as a New Paradigm for Economic Policy: Principles for Cultivating Entrepreneurship. Paper Presented at the Institute of International European Affairs, Dublin, Ireland.
• Isenberg, D. J. (2016). Applying the Ecosystem Metaphor to Entrepreneurship: Uses and Abuses. The Antitrust Bulletin, 61(4) 564-573. https://doi.org/10.1177/0003603X16676162
  » https://doi.org/10.1177/0003603X16676162
• La Rovere, R. L., Santos, G. O., & Vasconcellos, B. L. X. (2021). Desafios para a mensuração de Ecossistemas de Inovação e de Ecossistemas de Empreendedorismo no Brasil. Revista de Empreendedorismo, Gestão e Pequenas Empresas, 10(1). https://doi.org/10.14211/regepe.v10i1.1971
  » https://doi.org/10.14211/regepe.v10i1.1971
• Lahdelma, R., Hokkanen, J., & Salminen, P. (1998). SMAA - Stochastic multiobjective acceptability analysis. European Journal of Operational Research, 106(1), 137-143. https://doi.org/10.1016/S0377-2217(97)00163-X
  » https://doi.org/10.1016/S0377-2217(97)00163-X
• Leenderstse, J., Schrijvers, M., & Stam, E. (2020). Measure Twice, Cut Once. Entrepreneurial Ecosystem Metrics. Conference: Utrecht University School of Economics Working Paper Series.
• Liguori, E., Bendickson, J., Solomon, S., & McDowell, W. C. (2018). Development of a multi-dimensional measure for assessing entrepreneurial ecosystems. Entrepreneurship & Regional Development, 31, 7-21. https://doi.org/10.1080/08985626.2018.1537144
  » https://doi.org/10.1080/08985626.2018.1537144
• Mack, E., & Mayer, H. (2016). The evolutionary dynamics of entrepreneurial ecosystems. Urban Studies Journal, 53(10), 2118-2133. https://doi.org/10.1177/0042098015586547
  » https://doi.org/10.1177/0042098015586547
• Mack, E. A., & Qian, H. (Eds.) (2016). Geographies of Entrepreneurship. Routledge, London.
• Mason, C., & Brown, R. (2014). Entrepreneurial ecosystems and growth oriented entrepreneurship. Background paper prepared for the Workshop organized by the OECD LEED Programme and the Dutch Ministry of Economic Affairs on Entrepreneurial Ecosystems and Growth Oriented Entrepreneurship, The Hague, Netherlands.
• Maysami, A. M., Elvasi, G. M., Dehkordi, A. M., & Hejazi, S. R. (2019). Toward the Measurement Framework of Technological Entrepreneurship Ecosystem. Journal of Enterprising Culture, 27, 419-444. https://doi.org/10.1142/S0218495819500158
  » https://doi.org/10.1142/S0218495819500158
• Nicotra, M., Romano, M., Del Giudice, M., & Carmela, C. E. (2018). The causal relation between entrepreneurial ecosystem and productive entrepreneurship: a measurement framework. Journal of Technology Transfer 43, 640-673. https://doi.org/10.1007/s10961-017-9628-2
  » https://doi.org/10.1007/s10961-017-9628-2
• O’Connor, A., & Audretsch, D. V. (2022). Regional entrepreneurial ecosystems: learning from forest ecosystems. Small Business Economics. https://doi.org/10.1007/s11187-022-00623-8
  » https://doi.org/10.1007/s11187-022-00623-8
• O’Connor, A., Stam, E., Sussan, F., & Audretsch, D. V. (2018). Entrepreneurial Ecosystems: Place Based Transformations and Transitions. International Studies in Entrepreneurship, 38, 173-196. https://doi.org/10.1007/978-3-319-63531-6
  » https://doi.org/10.1007/978-3-319-63531-6
• Ratten, V. (2020). Entrepreneurial ecosystems: Future research trends. Thunderbird International Business Review, 1-6. https://doi.org/10.1002/tie.22163
  » https://doi.org/10.1002/tie.22163
• Roundy, P. T., & Bayer, M. A. (2019). Entrepreneurial ecosystem narratives and the micro-foundations of regional entrepreneurship. The International Journal of Entrepreneurship and Innovation, 20(3), 194-208. https://doi.org/10.1177/1465750318808426
  » https://doi.org/10.1177/1465750318808426
• Roundy, P. T., Bradshaw, M., & Brockman, B. K. (2018). The emergence of entrepreneurial ecosystems: A complex adaptive systems approach. Journal of Business Research, 86, 1-10. https://doi.org/10.1016/j.jbusres.2018.01.032
  » https://doi.org/10.1016/j.jbusres.2018.01.032
• Souza, L., Gerhard, F., La Rovere, R. L., & Câmara, S. (2016). Entrepreneurship and creation of new business: key factors of Brazilian entrepreneurial ecosystem. Revista de Negócios, 20(4), 30-43. http://dx.doi.org/10.7867/1980-4431.2015v20n4p30-43
  » http://dx.doi.org/10.7867/1980-4431.2015v20n4p30-43
• Spigel, B. (2016). Developing and governing entrepreneurial ecosystems. 8th Annual International Conference for Entrepreneurship, Innovation, and Regional Development, Sheffield.
• Spigel, B., Kitigawa, F., & Mason, C. (2020). A manifesto for researching entrepreneurial ecosystems. Local Economy, 0(0). https://doi.org/10.1177/0269094220959052
  » https://doi.org/10.1177/0269094220959052
• Stam, E. (2015). Entrepreneurial Ecosystems and Regional Policy: A Sympathetic Critique. European Planning Studies, 23(9), 1759-1769. https://doi.org/10.1080/09654313.2015.1061484
  » https://doi.org/10.1080/09654313.2015.1061484
• Stam, E. (2018). Measuring Entrepreneurial Ecosystems. In O’Connor, A., Stam, E., Sussan, F., & Audrestch, D. (Eds.). Entrepreneurial Ecosystems: Place-based Transformations and Transitions. Cham, Switzerland: Springer International Publishing. https://doi.org/10.1007/978-3-319-63531-6
  » https://doi.org/10.1007/978-3-319-63531-6
• Stam, E., & Spigel, B. (2016). Entrepreneurial Ecosystems. Discussion Paper Series nº 16-13, Utrecht.
• Stam, E., & Van de Ven, A. (2021). Entrepreneurial ecosystem elements. Small Business Economics, 56, 809-832. https://doi.org/10.1007/s11187-019-00270-6
  » https://doi.org/10.1007/s11187-019-00270-6
• Startup Genome. (2022). The Global Startup Ecosystem Report (GSER): Blue Economy Edition. https://startupgenome.com/reports/the-global-startup-ecosystem-report-blue-economy-edition
  » https://startupgenome.com/reports/the-global-startup-ecosystem-report-blue-economy-edition
• Tödtling, F., & Trippl, M. (2005). One size fits all ? Towards a differentiated regional innovation policy approach. Research Policy, 34, 1203-1219. https://doi.org/10.1016/j.respol.2005.01.018
  » https://doi.org/10.1016/j.respol.2005.01.018

• JEL classification:

  O30, L26

• Financing:

  FAPERJ #E-26/203.800/2021

• Related item (isTranslationOf):

  https://doi.org/10.14211/regepe.esbj.e2330

Edited by

Editor-in-Chef¹ or Adjunct²:

¹ Dr. Edmundo Inácio Júnior https://orcid.org/0000-0003-0137-0778

Univ. Estadual de Campinas, UNICAMP

Associate Editor:

Fernando Antonio Prado Gimenez https://orcid.org/0000-0002-5143-9553

Univ. Federal do Paraná, UFPR

Executive¹ or Assistant² Editor:

¹ M. Eng. Patrícia Trindade de Araújo

² Camille Guedes Melo

Publication Dates

• Publication in this collection
  29 July 2024
• Date of issue
  Jan-Apr 2024

History

• Received
  10 Oct 2022
• Reviewed
  03 June 2023
• Accepted
  11 Mar 2024