Traditional Research versus Advanced Algorithms: “Is Survey in the Last Days?”

ABSTRACT

Objective:  to examine the contemporary challenges faced by the survey method in the administration field, particularly in marketing, due to the emergence of new technologies and changes in respondent behavior.

Provocations:   with the rise of artificial intelligence, the traditional survey method is increasingly being questioned. Issues such as response validity, respondent fatigue, and proliferation of behavioral data obtained through automated means cast doubt on the survey’s effectiveness in capturing actual consumer behavior. Additionally, new legislation may introduce restrictions that could impact data collection via surveys.

Conclusions:  although not obsolete, the survey method must reinvent itself to remain relevant. Integrating new technologies, such as artificial intelligence, and combining them with qualitative methods are suggested paths to improve research effectiveness in an environment heavily influenced by technological advancements. The future of the survey depends on its ability to adapt and complement other emerging approaches.

Keywords: survey; artificial intelligence; silicon sample

RESUMO

Objetivo:  examinar os desafios contemporâneos enfrentados pelo método survey no campo da administração, em especial no marketing, devido ao surgimento de novas tecnologias e mudanças nos comportamentos dos respondentes.

Provocações:  com a ascensão da inteligência artificial, o método survey tradicional começa a ser questionado. Questões como a validade das respostas, a fadiga dos respondentes e a proliferação de dados comportamentais obtidos por meios automatizados questionam a eficácia do survey na captura de comportamentos reais dos consumidores. Além disso, novas legislações podem trazer restrições capazes de afetar a coleta de dados via survey.

Conclusões:  apesar de não estar obsoleto, o método survey deve se reinventar para permanecer relevante. A integração com novas tecnologias, como a inteligência artificial, e a combinação com métodos qualitativos são caminhos sugeridos para melhorar a eficácia das pesquisas em um ambiente com forte avanço tecnológico. O futuro do survey depende da capacidade de adaptação e da complementaridade com outras abordagens emergentes.

Palavras-chave: survey; inteligência artificial; dados sintéticos

INTRODUCTION

The survey method is a cornerstone of applied social sciences because it gathers information directly from individuals, allowing for the systematic analysis of behaviors, attitudes, and opinions across various social contexts. Widely used globally, it is essential to capture representative data from large populations, analyze complex social phenomena, and develop theories based on empirical evidence (Nardi, 2018). In marketing, its importance is even greater, as it provides crucial insights into consumer perceptions, preferences, and motivations, which are fundamental to formulating effective marketing strategies (Goffin et al., 2010).

In developed markets such as the USA and Europe, surveys are employed to monitor economic trends, evaluate public policies, and understand consumer behavior (Sudbury-Riley & Kohlbacher, 2016), which is essential for shaping business policies and strategies. Moreover, the rise of digital technologies has facilitated the growth of online surveys, making them more accessible and scalable (Fricker & Schonlau, 2002), thereby reinforcing their importance in gathering insights for decision-making in various international contexts

This method is also significant in Brazil, both in academia and the business sector. Research on survey use (Mazzon & Hernandez, 2013; Sampaio et al., 2012) has shown that approximately three-quarters of the scientific production in marketing in Brazil is based on data collected from surveys. In the business environment, its relevance is demonstrated by establishing associations dedicated to market research (Meyer et al., 2015) which play a fundamental role in professionalizing and standardizing research practices. As a result, surveys not only provide insights into consumer behavior but also support strategic decisions in business.

However, surveys have faced criticism (Meyer et al., 2015), mainly due to the challenges respondents face in accurately answering questionnaires (Faria, 2024). Common issues include respondents’ lack of time, which can lead to superficial or rushed responses (Evans & Mathur, 2018), and concerns about the validity of the data collected (Meyer et al., 2015), which may be compromised by a lack of engagement or misunderstanding of the questions posed (Evans & Mathur, 2018). Additionally, there are concerns about the survey’s limited ability to capture the complexity of consumer behavior, as standardized responses may not reflect actual behavior but rather intentions, desires, or socially expected answers (Tourangeau & Yan, 2007).

Given the evolution of research practices and emerging criticisms regarding their validity, it is crucial to reassess the role of surveys in the field of marketing. In this paper, we explore the reasons behind the decline in survey research, emphasizing factors such as changes in respondent behavior, the impact of technology, and the rise of new research methods. We also discuss how advancements in artificial intelligence (AI) have prompted a reevaluation of traditional methodologies (Sturgis & Luff, 2020). Finally, we address the challenges in predicting behavior, highlighting the limitations of current approaches and the complexities of effectively capturing the dynamics of human behavior in diverse contexts.

SURVEY METHOD IN CRISIS: REFLECTIONS ON ITS LIMITATIONS

One of the main reasons for the decline in surveys is the change in respondents’ behavior, driven by information overload (Meyer et al., 2015). With the advancement of technology and the increasing pace of life, individuals have become more resistant to participating in surveys due to lack of time, disinterest, insecurity, or distrust regarding the use of collected data. This resistance is even more pronounced in face-to-face surveys, particularly in household surveys in urban centers.

The rise of digital survey channels, such as email, social media, websites, and Internet of Things (IoT) devices, has transformed data collection methods. While this multiplicity offers an opportunity to reach a broader and more diverse audience, enabling surveys to be distributed more efficiently and at scale, it also introduces challenges such as ‘survey fatigue.’ Additionally, it increases the complexity of managing and analyzing the collected data, requiring more sophisticated approaches to ensure the consistency of results. Thus, although the expansion of digital channels has extended the reach and accessibility of surveys, it has also burdened respondents, leading to low response rates in internet-based surveys, which are often perceived as having little value by respondents.

The proliferation of consumer panels raises additional issues. Online data collection faces challenges such as statistical non-response and ‘unintentional’ biases, including social desirability, expectation, and confirmation, which can distort results (Tourangeau & Yan, 2007). Furthermore, the ‘professional’ respondent bias, common in consumer panels, adds another layer of complexity to data interpretation and impacts response quality; ‘professional’ respondents may ‘learn’ how to answer widely used scales in marketing.

Current challenges of the survey method

In marketing research, there has long been debate about how accurately surveys can describe consumer behavior. Although, among other uses, surveys can help access intentions, they are limited in addressing the customer experience (Hulland et al., 2018). This new paradigm recognizes that while purchase intention is an important indicator, it does not always translate into actual consumer behavior.

While surveys offer a structured and quantitative view of consumer motivations, beliefs, and attitudes, qualitative methods such as in-depth interviews and focus groups are often preferred because they provide richer insights. These qualitative methods capture the nuances and complexities of consumer perceptions - something that standardized survey responses often fail to achieve. In addition, ethnographic techniques, which involve direct observation of behavior in real contexts, are used when seeking a deeper and more contextualized understanding of consumer interactions with brands and products.

Digital tracking techniques, such as cookies, pixel tracking, and digital fingerprinting, combined with machine learning (ML) and AI models, have revolutionized how companies and researchers collect and analyze data, contributing to the decline of the survey method as the predominant approach. Continuous monitoring of clicks, social media interactions, and devices such as smartphones and wearables allows real-time collection of attitudinal, behavioral, and geo-referenced data, overcoming the speed, depth, and spatial limitations of traditional surveys.

With the individual’s consent, this data can be collected without the need for specific questions, as in surveys, offering a more authentic and spontaneous view of consumer behavior and eliminating response bias in declarative surveys. In addition, combining transactional data with online interactions makes it possible to create advanced predictive models that identify consumer behavior patterns and preferences with greater precision. This ability to anticipate wants and needs enables content and recommendation personalization on a massive scale, which traditional surveys cannot achieve.

Among the changes challenging the survey method is the new Brazilian legislation (Lei nº 14.874, 2024) which establishes strict guidelines for research involving human subjects across all fields of knowledge, including the creation of the National System of Ethics in Research with Human Beings. One key point is Article 20, which prohibits the remuneration of participants or granting any advantage except for reimbursement of transportation or food expenses. This could directly impact participation rates in Brazilian surveys, as the motivation to participate may decrease without financial incentives.

Surveys in the AI Era

With the rise of AI and its growing influence in marketing research, new opportunities to address the challenges of the survey method have emerged. AI can enhance various stages of the research process, from the design and generation of synthetic data via GANs or LLMs to the distribution of questionnaires and the analysis of collected data (Ikegwu et al., 2022; Li et al. 2024; Ruiz-Real et al., 2021). AI can help personalize questions based on respondents’ profiles, making surveys more relevant and increasing response rates in online surveys. Additionally, machine learning (ML) algorithms can identify hidden response patterns, providing more actionable insights (Haleem et al., 2022; Huang & Rust, 2021).

However, the implementation of AI also introduces ethical challenges, such as ensuring data privacy (Franzke et al., 2020), addressing potential algorithmic biases that may compromise the representativeness of results (Kordzadeh & Ghasemaghaei, 2021) and improving the interpretability of black-box algorithms (e.g., deep learning), where researchers may struggle to justify or even understand the reasons behind certain responses (Molnar, 2018).

Data synthesis using ML algorithms, such as neural networks, has emerged as a solution to privacy and security challenges in research involving sensitive data. Proposed by Rubin (1993) and Little (1993), the technique gained prominence with advancing technologies and regulations, such as the GDPR in Europe and the LGPD in Brazil, which demand alternatives that ensure confidentiality.

Recent research in marketing (e.g., Li et al. 2024) explores the use of silicon sampling as a substitute for human respondents, preserving the essential characteristics of the original data without exposing personal information. This makes it a viable option when real data usage is restricted. However, the representativeness and accuracy of synthetic data are critical challenges, as they depend on the algorithms’ ability to capture the nuances of real data, which is not always guaranteed. Studies such as those by Savage (2023) e Jenkins (2023) indicate that, despite the advantages in terms of security and accessibility, the robustness of synthetic data as a replacement for real data in complex analyses requires caution.

Another trend is integrating survey data with other sources, such as social networks and transactional data. This combination allows researchers to triangulate different types of information, enriching and deepening analyses (Groves & Harris-Kojetin, 2017). For example, while a survey might capture a consumer’s purchase intention, transactional data can reveal their actual purchasing behavior, enabling comparative analysis that highlights discrepancies between intention and action. The merging of these sources provides a holistic view of consumer behavior, contributing to more accurate and effective marketing strategies (Kamakura & Wedel, 1996). However, this approach requires advanced data analysis skills and a critical understanding of the limitations of each source to avoid misleading conclusions.

FINAL CONSIDERATIONS

Despite criticism and challenges, the survey method is not obsolete but rather evolving. In the digital age, surveys are transforming into hybrid tools that combine traditional techniques with technological innovations. One example is the use of short, dynamic surveys sent via mobile devices to capture feedback at key moments in the consumer journey. Additionally, integration with social media platforms enables more effective, real-time targeting of specific consumer segments. These innovations suggest that, while surveys may be declining as a stand-alone tool, they remain an integral part of the marketing research ecosystem when used in conjunction with other methodologies (Mikalef et al., 2019; Venkatesh et al., 2013).

Consequently, changes in the teaching of survey methods in universities are inevitable. Ideally, business administration programs should include courses focused on machine learning and AI, teaching students how to collect, process, and analyze large volumes of data and how this data can be used to predict behaviors and customize marketing strategies. Training should emphasize the combination of quantitative and qualitative methods, enabling students to integrate survey data and experimental results with qualitative insights obtained through in-depth interviews, focus groups, and ethnographies, thus understanding how these approaches complement each other (Venkatesh et al., 2013). Moreover, preparing students to identify and mitigate biases in algorithms and data analysis is essential, ensuring that the insights generated are valid and reliable (Kordzadeh & Ghasemaghaei, 2021; Martin & Murphy, 2017).

The real question is not whether surveys are obsolete but how they can be reinvented and integrated into a broader, more dynamic research ecosystem. By adapting and complementing other approaches, surveys continue to be valuable for data collection, offering insights that, when well interpreted, can be extremely powerful for strategic decision-making in marketing and other areas. Therefore, the future of surveys lies in their ability to evolve and coexist with emerging methodologies, ensuring they remain relevant and effective in an increasingly complex, digital, and data-driven world.

REFERÊNCIAS

• Evans, J. R., & Mathur, A. (2018). The value of online surveys: A look back and a look ahead. Internet Research, 28(5), 854-887. https://doi.org/10.1108/IntR-03-2018-0089
  » https://doi.org/10.1108/IntR-03-2018-0089
• Faria, F. P. (2024, janeiro 9). O aumento da taxa de não resposta: As pesquisas diante de seu maior desafio? - Parte I. Blog do IBRE. https://blogdoibre.fgv.br/posts/o-aumento-da-taxa-de-nao-resposta-pesquisas-diante-de-seu-maior-desafio-parte-i
  » https://blogdoibre.fgv.br/posts/o-aumento-da-taxa-de-nao-resposta-pesquisas-diante-de-seu-maior-desafio-parte-i
• Franzke, A., Bechmann, A., Zimmer, M., & Ess, C. M. (2020). Internet Research: Ethical Guidelines 3.0. The Association of Internet Researchers. https://aoir.org/reports/ethics3.pdf
  » https://aoir.org/reports/ethics3.pdf
• Fricker, R. D., & Schonlau, M. (2002). Advantages and disadvantages of internet research surveys: Evidence from the literature. Field Methods, 14(4), 347-367. https://doi.org/10.1177/152582202237725
  » https://doi.org/10.1177/152582202237725
• Goffin, K., Lemke, F., & Koners, U. (2010). Surveys and interviews. In K. Goffin, F. Lemke, & U. Koners (Eds.), Identifying hidden needs (pp. 27-51). Palgrave Macmillan. https://doi.org/10.1057/9780230294486_2
  » https://doi.org/10.1057/9780230294486_2
• Groves, R. M., & Harris-Kojetin, B. A. (2017). Using multiple data sources and state-of-the-art estimation methods in federal statistics: Frameworks, methods, and assessment. The National Academies Press. https://www.nationalacademies.org/our-work/using-multiple-data-sources-and-state-of-the-art-estimation-methods-in-federal-statistics-frameworks-methods-and-assessment
  » https://www.nationalacademies.org/our-work/using-multiple-data-sources-and-state-of-the-art-estimation-methods-in-federal-statistics-frameworks-methods-and-assessment
• Haleem, A., Javaid, M., Qadri, M. A., Singh, R. P., & Suman, R. (2022). Artificial intelligence (AI) applications for marketing: A literature-based study. International Journal of Intelligent Networks, 3(1), 119-132. https://doi.org/10.1016/j.ijin.2022.08.005
  » https://doi.org/10.1016/j.ijin.2022.08.005
• Huang, M. H., & Rust, R. T. (2021). A strategic framework for artificial intelligence in marketing. Journal of the Academy of Marketing Science, 49, 30-50. https://doi.org/10.1007/s11747-020-00749-9
  » https://doi.org/10.1007/s11747-020-00749-9
• Hulland, J., Baumgartner, H., & Smith, K. M. (2018). Marketing survey research best practices: Evidence and recommendations from a review of JAMS articles. Journal of the Academy of Marketing Science, 46(1), 92-108. https://doi.org/10.1007/s11747-017-0532-y
  » https://doi.org/10.1007/s11747-017-0532-y
• Jenkins, K. (2023). Synthetic data and public policy: Supporting real-world policymakers with algorithmically generated data. Policy Quarterly, 19(2). https://doi.org/10.26686/pq.v19i2.8234
  » https://doi.org/10.26686/pq.v19i2.8234
• Ikegwu, A., Nweke, H. F., Anikwe, C. V., Alo, U. R., & Okonkwo, O. R. (2022). Big data analytics for data-driven industry: A review of data sources, tools, challenges, solutions, and research directions. Cluster Computing, 25(2), 3343-3387. https://doi.org/10.1007/s10586-022-03568-5
  » https://doi.org/10.1007/s10586-022-03568-5
• Kamakura, W. A., & Wedel, M. (1996). Statistical data-fusion for cross-tabulation. Working Paper. University of Groningen.
• Kordzadeh, N., & Ghasemaghaei, M. (2021). Algorithmic bias: Review, synthesis, and future research directions. European Journal of Information Systems, 31(3), 388-409, https://doi.org/10.1080/0960085X.2021.1927212
  » https://doi.org/10.1080/0960085X.2021.1927212
• Lei nº 14.874 (2024). Lei nº 14.874, de 28 de maio de 2024, que dispõe sobre a pesquisa com seres humanos e institui o Sistema Nacional de Ética em Pesquisa com Seres Humanos. Diário Oficial da União. https://www.planalto.gov.br/ccivil_03/_ato2023-2026/2024/lei/l14874.htm
  » https://www.planalto.gov.br/ccivil_03/_ato2023-2026/2024/lei/l14874.htm
• Li, P., Castelo, N., Katona, Z., & Sarvary, M. (2024). Determining the validity of large language models for automated perceptual analysis. Marketing Science, 43(2), 239-468. https://doi.org/10.1287/mksc.2023.0454
  » https://doi.org/10.1287/mksc.2023.0454
• Little, R. J. A. (1993). Statistical analysis of masked data. Journal of Official Statistics, 9, 407-426.
• Martin, K. D., & Murphy, P. E. (2017). The role of data privacy in marketing. Journal of the Academy of Marketing Science, 45(2), 135-155. https://doi.org/10.1007/s11747-016-0495-4
  » https://doi.org/10.1007/s11747-016-0495-4
• Mazzon, J. A., & Hernandez, J. M. D. C. (2013). Brazilian scientific production in marketing in the period 2000-2009. Revista de Administração de Empresas, 53(1), 67-80. https://doi.org/10.1590/S0034-75902013000100007
  » https://doi.org/10.1590/S0034-75902013000100007
• Meyer, B., Mok, W., & Sullivan, J. (2015). Household surveys in crisis. Journal of Economic Perspectives, 29(4), 199-226. https://doi.org/10.1257/jep.29.4.199
  » https://doi.org/10.1257/jep.29.4.199
• Mikalef, P., Boura, M., Lekakos, G., & Krogstie, J. (2019). Big data analytics and firm performance: Findings from a mixed-method approach. Journal of Business Research, 98, 261-276. https://doi.org/10.1016/j.jbusres.2019.01.044
  » https://doi.org/10.1016/j.jbusres.2019.01.044
• Molnar, C. (2018). Interpretable machine learning (2^nd ed.). Leanpub. https://leanpub.next/interpretable-machine-learning
  » https://leanpub.next/interpretable-machine-learning
• Nardi, P. M. (2018). Doing survey research: A guide to quantitative methods. Routledge.
• Rubin, D. B. (1993). Discussion: Statistical disclosure limitation. Journal of Official Statistics, 9, 462-468.
• Ruiz-Real, J. L., Uribe-Toril, J., Torres, J. A., & De Pablo, J. (2021). Artificial intelligence in business and economics research: trends and future. Journal of Business Economics and Management, 22(1), 98-117. https://doi.org/10.3846/jbem.2020.13641
  » https://doi.org/10.3846/jbem.2020.13641
• Sampaio, C. H., Perin, M. G., Luce, F. B., Santos, M. J. D., Santini, F. D. O., Oliveira, M. O. R. D., & Lenz, G. D. S. (2012). Pesquisa científica da área de marketing no Brasil: Uma revisão da primeira década do século 21. Revista de Administração Contemporânea, 16, 459-478. https://doi.org/10.1590/S1415-65552012000300008
  » https://doi.org/10.1590/S1415-65552012000300008
• Savage, N. (2023, April 23). Synthetic data could be better than real data. Nature. https://doi.org/10.1038/d41586-023-01445-8
  » https://doi.org/10.1038/d41586-023-01445-8
• Sturgis, P., & Luff, R. (2020). The demise of the survey? A research note on trends in the use of survey data in the social sciences, 1939 to 2015. International Journal of Social Research Methodology, 24, 691-696.https://doi.org/10.1080/13645579.2020.1844896
  » https://doi.org/10.1080/13645579.2020.1844896
• Sudbury-Riley, L., & Kohlbacher, F. (2016). Ethically minded consumer behavior: Scale review, development, and validation. Journal of Business Research, 69(8), 2697-2710. https://doi.org/10.1016/j.jbusres.2015.11.005
  » https://doi.org/10.1016/j.jbusres.2015.11.005
• Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5), 859. http://dx.doi.org/10.1037/0033-2909.133.5.859
  » http://dx.doi.org/10.1037/0033-2909.133.5.859
• Venkatesh, V., Brown, S. A., & Bala, H. (2013). Bridging the qualitative-quantitative divide: Guidelines for conducting mixed methods research in information systems. MIS Quarterly, 37(1), 21-54. https://doi.org/10.25300/MISQ/2013/3
  » https://doi.org/10.25300/MISQ/2013/3

• JEL Code:
  M3, M100.
• Plagiarism Check
  RAC maintains the practice of submitting all documents approved for publication to the plagiarism check, using specific tools, e.g.: iThenticate.
• Funding
  The authors reported that there was no funding for the research in this article.
• Peer Review Method
  This is an invited article, so it has not been peer-reviewed.
• Data Availability
  RAC encourages data sharing but, in compliance with ethical principles, it does not demand the disclosure of any means of identifying research subjects, preserving the privacy of research subjects. The practice of open data is to enable the reproducibility of results, and to ensure the unrestricted transparency of the results of the published research, without requiring the identity of research subjects.

Edited by

• Editor-in-chief:
  Paula Chimenti (Universidade Federal do Rio de Janeiro, COPPEAD, Brazil) https://orcid.org/0000-0002-6492-4072

Data availability

RAC encourages data sharing but, in compliance with ethical principles, it does not demand the disclosure of any means of identifying research subjects, preserving the privacy of research subjects. The practice of open data is to enable the reproducibility of results, and to ensure the unrestricted transparency of the results of the published research, without requiring the identity of research subjects.

Publication Dates

• Publication in this collection
  11 Nov 2024
• Date of issue
  2024

History

• Published
  19 Sept 2024