The statistics interpretation as production of truths: ethical insights

Abstract

This essay aims to raise critical awareness about the bioethical risk of the understanding of statistics, as the production of scientific truths, within the practical life of the health professional. Statistics give us a probabilistic view of the natural events observed systematically, with innumerable limits intrinsic to their methods. But the scientific milieu also has its degree of non scientific perception, which created in the collective imagination of the scientific milieu the idea that statistics obtain truths about the complex systems of nature, truths as solid as the scientist’s predictions about the orbits of the planets. Thus, the construction of expectations of success and failure in interventions, as well as the careful evaluation of the risks and benefits of the application of experimental results in human life, can be dangerously compromised in the day to day of the biomedical practices.

Data interpretation, statistical; Risk management; Hope; Culture

Resumo

Este ensaio lança um olhar crítico sobre as consequências, na prática do profissional de saúde, da produção de verdades científicas com base em modelos estatísticos. A estatística nos oferece visão probabilística de eventos naturais observados sistematicamente, e vários são os limites intrínsecos a seus métodos. Mas, no senso comum do meio científico, criou-se o imaginário da estatística como meio para obter verdades a respeito de sistemas complexos, da mesma forma que cientistas postulam princípios para a órbita dos planetas. Nesse contexto, as expectativas de sucesso ou fracasso em intervenções e a avaliação dos riscos e benefícios da aplicação de resultados experimentais podem ser perigosamente comprometidas no dia a dia da prática biomédica.

Interpretação estatística de dados; Gestão de riscos; Esperança; Cultura

Resumen

Este ensayo tiene el objetivo de suscitar la conciencia crítica acerca del riesgo bioético de la comprensión de la estadística como producción de verdades científicas en la vida práctica del profesional de salud. La estadística nos ofrece una visión probabilística de los eventos naturales observados sistemáticamente, con innumerables límites intrínsecos a sus métodos. Pero el medio científico también tiene su sentido común. Se creó en el imaginario colectivo del medio científico la idea de que la estadística obtiene verdades respecto de los sistemas complejos de la naturaleza, verdades tan sólidas como las predicciones del científico acerca de las órbitas de los planetas. Así, la construcción de expectativas de éxito y fracaso en intervenciones, así como la evaluación criteriosa de los riesgos y beneficios de la aplicación de resultados experimentales en la vida humana, pueden verse peligrosamente comprometidos en el día a día de las prácticas biomédicas.

Interpretación estadística de datos; Gestión de riesgos; Esperanza; Cultura

For at least 200 years biomedical practices have been based on scientific knowledge that results from the systematic observation of nature, which in turn is composed of regular patterns and chaotic processes. Some of these patterns, such as the orbit of planets, are quite evident and can be measured simply by the observation of events. However, other patterns are lost in the midst of non-linearities, that is, they present a stochastic behavior ¹. The more complex a natural system is, with is biological, psychic and social organisms, the more difficult it will be to determine regularities only on the basis of systematic observation. Science developed specific mathematical tools, which became the basis of statistics, in order to seize on these regular patterns.

Statistics was part of the scientific revolution of the first decades of the twentieth century. Mathematicians developed the fundamentals of the area at the same time that Einstein developed the theory of relativity and quantum mechanics flourished. Although statistics were formally systematised only at this point in History, human beings have always thought statistically, even in pre-mathematical times, prior to cuneiform writing. The perception of the average is inherent in our cognition - we have always been attentive to hidden patterns behind irregularities in natural processes.

Winter is a pattern, just like the size of a fruit. The Prehistoric man (and perhaps other animals) knew whether a fruit was large or small, for in their life experience they spontaneously and naturally established an average for the size of that fruit. The brain itself is a bayesian processor ^2-4: our perceptions about the world are chaotic and nonlinear, but collectively they fit into a frequency distribution, so that the brain infers statistics about what its senses encode, forming the representation of the reality that surrounds us. Therefore, ideas or emotions that arise as responses to stimuli fit statistical models, and can be estimated probabilistically.

Statistics are always present, from physics to market studies. When our object of scientific observation is a set of beings, there is no thesis, descriptive or analytical, that dispenses with statistics. But most researchers do not grasp statistics’ intrinsic mechanisms despite using it, which is understandable given the great complexity of most of statistics’ procedures. Consequently, at all academic levels, users of statistics do not understand the philosophical and epistemological foundations that give meaning to it.

This creates mistaken scientific ways of thinking, with ethical consequences for the professional act, which affect, for example, the decision-making and patient’s expectations. The aim of this essay is to discuss the risk of reaching erroneous conclusions in bioethics based on scientific information obtained through statistical methods and which are taken as absolute truths. For this, empirical sciences and statistics will be approached conceptually, in search of a critical understanding of statistics’ nature and its field of application.

It is necessary to delimit principles for a universal normative ethics that establishes biomedical responsibilities in consonance with the humanistic values of civilisation ⁵. This ethics should have as its axis the benefit of the patient over all other things - a principle that is achieved through the minimisation of iatrogeny and the respect for patient’s rights, including the right to “truth”. Thus, any practice that is based on this normative ethic is considered “good”.

On the other hand, practices unrelated to knowledge and techniques incur bioethical risk, and can harm the patient. Therefore, formal rigor when interpreting scientific information should be part of professional practice. These questions are addressed in this essay and, from the perspective of bioethics, good practices of interpretation of statistical data are proposed, considering what they really have to offer: probabilities.

Statistics and the truth

Every scientific discussion of quality presupposes reasonable doubts about the results of experiments, observations or qualitative interpretations. Uncertainty is part of science. However, such discussions are not always appreciated with reasonable doubt in mind. Many times, for example, research results are disseminated by the media using verbal tenses and expressions that denote certainties.

The scientific medium, be it academic or technical, is made up of people and imagery. Thus, there is a common perception that takes results as definitive truths as it happens in all areas of human action. This common perception is permeated by inadequate ideas about the contributions of statistics to science. When a Ph.D student compares measurements of samples or when a physician reads an article about a new drug, primordial concepts - such as “chance,” “probability,” “sampling bias,” “predictive modeling,” or “inference” - can be theoretically known but not understood.

The first question is: what is statistics purpose? Reveal reality? This is definitely not its purpose. Reality is an intersubjective and circumstantial construct. The size of the fruit is an estimate we consider as reality. The brain, as a Bayesian system, constructs a model of reality based on endogenous inferences and often makes mistakes, hence the sensorial illusions ⁶.

As non-linear beings, we live a paradox in establishing a culture that needs polar certainties, defining things that contrast significantly from one another, and constructing representations based on the idea that something “is this and it is not that.” Our way of thinking shows the deep anguish of the continuous and the uncertain, and the consequent ambivalence of knowledge. The human being does not coexist well with probabilities. Statistics then offers arbitrary realities to analytical and reductionist thinking, taken as scientific objects and then superimposed on the world.

The Universe is now considered a non-linear system ¹. Although the orbit of a planet is measurable, its revolution period will never be absolutely regular. Even if it borders on insignificance, unpredictable and non-modifiable variation will always exist. So there is no absolute period in the revolution of Mercury around the Sun, as there is no reality to be revealed. Thus, statistics only offer probabilities that a given event expresses a natural pattern or unpredictable irregularities which we understand as “random”.

All statistics are based on the idea of probability, a mathematical entity that does not exist in nature and is nothing more than a measure: the measure of uncertainty. Therefore, statistics and their conclusions are abstractions of the natural world. The rejection or acceptance of the “null hypothesis” is the main conclusion of a statistical inference test, based on the p-value, which defines whether the probability is significant.

At this very moment, subjectivity overlaps with mathematics and its accuracy, for significance has always been and will be an attribute of value, a construct at the same time personal, affective, cultural and historical. Money, for example, has diverse values for different people and cultures. Moreover, what about the 5% normally attributed to alpha error? For many, one chance out of twenty is too risky. Would anyone risk a Russian roulette with odds of one against twenty? But a 5% error is admitted in the significance of the effect of a cancer drug, presented by the market as the definitive solution to the disease. It is, therefore, a judgment of value, which, in a Statistics Congress, could change after a new consensus on the ideal alpha error.

Infinitely small probabilities can arise from the Pearson’s chi-squared test when we correlate, for example, spider bites in Brazil and the consumption of chocolate in France in the same period. Spurious correlations such as these, proposed as anecdotal curiosities, feed criticism to statistics as oracle of reality ⁶. In a million of completely random correlations, 50,000 will be within the p = 0.05 criterion. About ten correlations with a p = 10 criterion ⁵, which means a correlation coefficient r = 0.9, should appear in a sample with 25 observations - number worthy of the best deterministic tests in chemistry or physics.

To estimate the relevance of these correlations is to enter deep epistemological discussions about aspects of the universe that are inaccessible to the human deductive intellect given the imponderable human ignorance that comes from a brain that is too limited in relation to its environment. Thus, being a resource of empiricism, statistics became a tool of an inductive approach to know nature.

The inductive method and the production of truths

Since the eighteenth century, with David Hume ⁷, the problems inherent in the inductive method have been demonstrated. Indeed, such a method is logically inconsistent and epistemologically unsustainable, since it is based on the future behaviour of events, and ignores that it is impossible to predict them with absolute certainty. In other words, you can not assign truth value to an event that has not yet happened, because nothing guarantees, with absolute certainty, whether such an event will happen.

As an example, one can cite the sunrise which, although observed every day, will not necessarily continue until the end of time. In fact, as we see the sun rising and setting, we believe that this event will occur the next day. Therefore, it is a belief, not a discovery of the truth.

Our mind works with predictions of the future backed by past events, just as the Bayesian statistical method does, estimating probabilities by taking into account previous occurrences of the event in question ³. Common sense, coming from this Bayesian mind, remains, therefore, the final judge on all measures, and statistics only enriches it with more information.

The discussion about the inductive method is highly profitable and relevant because it determines behaviours that succeed the judgment of significance. In practice, the world seems to work well with human judgment based on statistical results. With statistics, the science of not-so-complex systems can put satellites in orbit, predict the power of a nuclear bomb or maintain the quality of the steel produced. However, when we talk about complex systems, such as the action of HIV on CD4 + cells, the biochemistry of schizophrenia, the behaviour of beta-blockers on cardiovascular function or the causes of chemical dependence, science goes into prickly ground. And statistics are like a blunt scythe that can not clear this land.

Complex systems are multidimensional, thus they can only be described by many variables. For example, to describe a complex system such as the brain of a child with Attention Deficit Hyperactivity Disorder (ADHD), we would need to collect hundreds of variables of different types. Imagine a database with up to 800 variables, as in Abramov and collaborators ⁸: they allow up to 320,000 correlations, in addition to the 800 difference inferences between the ADHD and control groups.

In such cases, given the probabilistic nature of the tests, many spurious relations and redundancies (different variables describing the same dimension) will appear. Moreover, it is impossible to understand the set of inferences and probabilities resulting wit the naked eye: it would be like looking at a crowd of 100,000 people in a football stadium and describing the affective relationships between them. So we try to look for global patterns that cross this multiplicity of dimensions, modulating their interpersonal relationships (for example, what triggers a stadium crowd to perform a wave).

We presume (rationally) that complex systems have latent and comprehensive dynamic standards, fewer in number than than dimensions. They are not described by any particular variable, but must be embodied in many of them, in different quantities. Multivariate Analysis (MVA) were developed to unveil these few presumed patterns, called components or complex system factors ⁹, synthesised in new variables which are considered more objective and comprehensible descriptors of the system as a whole. The MVA comprises a set of very sophisticated theories and techniques that allow statistical access to quantitative studies in psychology, sociology, economics, marketing and neurosciences.

However, the MVA is still speculative, since it presumes the existence of latent variables that are totally virtual (that is, they can not be observed directly). The theory presupposes finding these factors and their characteristics by means of complex analyses of the multidimensional set of data, based on the many interpretations for the similarities and / or differences between the variables. Therefore, we should consider MVA, present in most studies of the human mind, as a critical climax.

However, this critique does not refer to statistics per se, but to the mathematical description of probability-based nature, that is, in levels of uncertainty. MVA is the climax of the responsible application of statistics because it is fully aware that its results are nothing more than information to be judged by the human mind, with its rational deductive mechanisms intertwined with desires and affections completely foreign to mathematical objectivity. And consciousness of the limits of reality as a concept or object of observation is what separates science from belief and good medical practice from quackery, allowing knowledge to be applied in an ethically responsible way.

Production of truth and its consequences: good practices for the interpretation of scientific knowledge

Two main bioethical consequences of the construction of realities based on statistical probabilities are considered in this article. The first refers to the expectation created by procedures or interventions based on scientific research and the way this expectation works in the professional and patient imaginary. The second consequence is in assessing the risks and benefits of a particular procedure based on the context of the person who will be submitted to it. Awareness of the probabilistic and non-linear nature of the observed efficacy of a drug as well as the adverse events associated with it may completely change the judgment of the practitioner and the patient about its use. These are issues to be discussed not only in scientific publications, but in conversations among colleagues, colloquiums, lectures and dissemination of texts, confronting the advertisements of laboratories distorted by the desire to produce certainties and truths that benefit certain interests.

It is worth remembering that ethical judgment is linked both to the rational / objective dimension and to the affective/subjective dimension of human relations. Vulnerable to induced truths, the use of pharmaceutical drugs is far from being just logical and rational. Therefore, from the bioethical point of view, we believe that publications which have a direct impact on human life should not consider alpha and beta errors nor convey pre-established hypotheses. Thus, the conclusions and discussions of the articles should be based on the literal description of the statistics and their probabilistic results, without conclusively determining the effectiveness or safety of drugs and procedures.

This way we would avoid pre established truths, originating from the human imagination, indisputably subjective even among scholars and technicians. As evidenced so far, the production of these truths is inherent in cognition itself, the reductionist nature of human thought, and extrapolates academic formalisms. From this it follows that humanity must assume its own complexity, probing cautiously the relevance of a probability in each particular circumstance.

In some cases, errors of 5% or up to 10% are perfectly acceptable. In others, they are inadmissible. Let’s always remember the Russian roulette with a chance in twenty of you firing at your head: would you consider this a negligible chance? Probably not! Likewise, it should not be forgotten that it is the reader who must draw conclusions from scientific essays, making value judgments about the merits and limitations of research.

Awareness of the issues addressed here can create anguish but also call for accountability, raising criticism of the context of conflicting interests in which knowledge is generated and applied. Giving up the illusion of scientific truth requires effort, and looking at the world of statistics, looking at its probabilistic character and models to which nature must fit (not the other way around), can be a difficult task.

As health professionals, we are distressed by the doubts of our patients, who question us about the future, about the results of a particular treatment, and so on. And while the doubt stirs even more anguish, as good scientists we answer (or we should answer) that such results can not be determined. Based on our experience, and considering the limits of human understanding, one can only infer probabilities.

We know that most patients prefer the fantasy of certainties, although they can not be achieved by human cognition, immersed in the chaotic processes of the universe. Common perception demands from academics and technicians these certainties that no one can have, not even using the most sophisticated statistical model. The anguish of uncertainty may be great, but that is where responsibilities arise.

Final considerations

Complex systems are interpreted by the observation of nature and by inductive processes, subject, therefore, to statistical approximations, which are dependent on subjective validations established by our critical sense and personal values. Thus, in the biomedical sciences, we believe that caution must be exercised in reaching conclusions based on relative truths. From the bioethical point of view, it seems to be more prudent to present results within a probabilistic context, and it is up to the interlocutors, based on their reality, to judge the relevance of the research findings.

Referências

• ¹
  Boeing G. Visual analysis of nonlinear dynamical systems: chaos, fractals, self-similarity and the limits of prediction. Systems [Internet]. 2016 [acesso 3 jul 2018];4(4):37-54. Disponível: https://bit.ly/2DaMIvR
  » https://bit.ly/2DaMIvR
• ²
  Knill DC, Pouget A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci [Internet]. 2004 [acesso 3 jul 2018];27(12):712-9. Disponível: https://bit.ly/2DtJVie
  » https://bit.ly/2DtJVie
• ³
  Clark A. Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behav Brain Sci [Internet]. 2013 [acesso 3 jul 2018];36(3):181-204. Disponível: https://bit.ly/2z89odx
  » https://bit.ly/2z89odx
• ⁴
  Nour MM, Nour JM. Perception, illusions and Bayesian inference. Psychopathology [Internet]. 2015 [acesso 3 jul 2018];48(4):217-21. Disponível: https://bit.ly/2FbZNrs
  » https://bit.ly/2FbZNrs
• ⁵
  Conselho Federal de Medicina. Código de ética médica: Resolução CFM nº 1.931, de 17 de setembro de 2009 (versão de bolso) [Internet]. Brasília: CFM; 2010. [acesso 8 nov 2018]. Disponível: https://bit.ly/2OyVgi9
  » https://bit.ly/2OyVgi9
• ⁶
  Freedman D. From association to causation: some remarks on the history of statistics. Stat Sci [Internet]. 1999 [acesso 3 jul 2018];14(3):243-58. Disponível: https://bit.ly/2PLgm1q
  » https://bit.ly/2PLgm1q
• ⁷
  Hume D. A treatise of human nature. Oxford: Oxford University Press; 2007.
• ⁸
  Abramov DM, Gomes-Junior SC, Mourão-Junior CA, Pontes AT, Rodrigues CQC, Pontes MC et al. Estimating DSM validity for attention deficit/hyperactivity disorder based on neurophysiological, psychological, and behavioral correlates. bioRxiv [Internet]. 2017 [acesso 3 jul 2018]. DOI: 10.1101/126433
  » https://doi.org/10.1101/126433
• ⁹
  Tinsley H, Brown S. Handbook of applied multivariate statistics and mathematical modeling. Cambridge: Academic Press; 2000.

Publication Dates

• Publication in this collection
  Oct-Dec 2018

History

• Received
  9 July 2017
• Reviewed
  30 May 2018
• Accepted
  4 July 2018