Assessing transformation methods for group comparisons under violated assumptions: type I error rate and test power

Ergin, M.; Koşkan, Ö.

doi:10.1590/1678-4162-13026

Acessibilidade / Reportar erro

Brasil

Arquivo Brasileiro de Medicina Veterinária e Zootecnia

Español English

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

Veterinary Medicine and Technology and Inspection of Animal Products • Arq. Bras. Med. Vet. Zootec. 75 (05) • Sep-Oct 2023 • https://doi.org/10.1590/1678-4162-13026 copy

Assessing transformation methods for group comparisons under violated assumptions: type I error rate and test power

[Avaliação de métodos de transformação para comparações de grupos sobre pressupostos violados: taxa de erro tipo I e poder do teste]

Authorship SCIMAGO INSTITUTIONS RANKINGS

ABSTRACT

In this study, some transformation methods that are applied when the assumptions of analysis of variance are not met are evaluated in terms of type I error rate and the test power, under circumstances with different distributions, number of groups, number of observations, variance ratios, and different standard deviation differences. The data set used in the study consisted of random numbers generated from N (0,1), and χ²(3) distributions using the random function of the Numpy library in the Python programming language. The logarithmic, square root and root transformations were evaluated on ANOVA based on simulation combinations. It was observed that the transformation techniques of taking the square root after adding 0.5 and 0.375 to the data were relatively more reliable compared to other transformations in terms of type I error rate. However, in every case, type I error rate determined at the beginning of the experiment increased both before and after the transformation was applied. In particular, interestingly, the third and fourth degree root transformations gave better results of test power in the right skewed distribution. In addition, we compared the transformation techniques in question to determine the normality of the data and the homogeneity of variances by a real data.

Keywords:
data transformation; square root transformation; logarithmic transformation; analysis of variance; type I error rate; test power

Distributions	N (0,1)	χ2(3)
k	N:N:N	χ²(3):χ²(3):χ²(3)
n	3, 5, 10, 15, 30, 50
$σ^{2}$	1, 3, 5, 10
Δ	0.5, 0.75, 1, 1.5, 2

n	Variance Ratios	Original	$\sqrt{x}$	$\sqrt{x + 0.5}$	$\sqrt{x + (\frac{3}{8})}$	$\sqrt[3]{x}$	$\sqrt[4]{x}$	Log
3:3:3 5:5:5 10:10:10 15:15:15 30:30:30 50:50:50	1:1:1	4.93	4.74	4.82	4.81	4.62	4.53	4.93
		4.47	4.95	4.87	4.90	5.10	5.16	4.73
		5.08	4.87	4.94	4.93	4.74	4.68	4.99
		4.31	4.84	4.76	4.77	4.96	5.01	4.58
		5.10	5.02	5.04	5.04	4.97	4.94	5.06
		4.45	4.80	4.75	4.76	4.89	4.92	4.63
		4.91	4.86	4.88	4.88	4.85	4.82	4.91
		4.66	4.89	4.87	4.89	4.91	4.94	4.83
		5.13	5.13	5.13	5.12	5.09	5.06	5.11
		4.84	4.98	4.96	4.97	4.99	5.00	4.95
		5.06	5.01	5.03	5.03	4.96	4.94	5.05
		4.86	4.93	4.91	4.91	4.96	4.97	4.90
3:3:3 5:5:5 10:10:10 15:15:15 30:30:30 50:50:50	1:1:3	6.10	5.77	5.90	5.87	5.57	5.47	6.01
		6.15	7.1	6.97	7.00	7.37	7.50	6.73
		6.05	5.79	5.89	5.88	5.59	5.49	5.97
		6.32	7.38	7.18	7.22	7.67	7.82	6.89
		5.75	5.68	5.72	5.71	5.67	5.67	5.80
		6.32	7.54	7.32	7.35	8.15	8.45	6.95
		5.83	5.91	5.88	5.90	6.08	6.17	6.02
		6.06	7.78	7.40	7.50	8.60	9.15	6.87
		5.61	6.36	6.22	6.25	6.99	7.42	6.43
		5.93	8.94	8.25	8.37	10.87	12.06	7.26
		5.75	7.13	6.82	6.87	8.39	9.14	7.14
		5.77	10.19	9.08	9.32	13.39	15.36	7.65
3:3:3 5:5:5 10:10:10 15:15:15 30:30:30 50:50:50	1:1:5	7.44	6.98	7.11	7.08	6.64	6.48	7.22
		8.32	9.55	9.39	9.43	9.85	9.95	9.07
		7.36	7.06	7.16	7.13	6.83	6.70	7.22
		8.56	9.84	9.65	9.70	10.2	10.39	9.38
		6.68	6.67	6.67	6.67	6.71	6.73	6.90
		7.87	10.06	9.63	9.73	11.06	11.61	9.13
		6.36	6.61	6.58	6.58	6.90	7.12	6.92
		7.57	10.68	10.07	10.18	12.36	13.36	9.37
		6.52	7.67	7.39	7.46	8.97	9.76	8.34
		7.06	12.68	11.50	11.75	16.38	18.57	10.21
		6.36	8.96	8.46	8.57	11.21	12.75	10.25
		6.67	15.50	13.49	13.90	21.72	25.39	11.52
3:3:3 5:5:5 10:10:10 15:15:15 30:30:30 50:50:50	1:1:10	9.39	8.76	8.94	8.92	8.36	8.1	8.93
		12.15	12.99	12.95	12.95	12.98	12.94	12.88
		8.84	8.39	8.49	8.48	8.09	7.93	8.49
		11.50	12.71	12.56	12.59	13.06	13.23	12.46
		7.91	7.67	7.75	7.73	7.72	7.79	8.23
		10.13	13.27	12.77	12.89	14.88	15.82	12.63
		7.73	7.92	7.88	7.90	8.45	8.80	8.99
		9.20	14.23	13.38	13.60	17.05	18.73	13.26
		7.37	9.34	9.06	9.12	11.39	12.62	12.68
		8.32	17.75	16.04	16.40	23.80	27.36	16.22
		7.36	11.65	11.00	11.13	15.51	17.89	18.99
		7.93	22.52	19.93	20.51	32.46	38.27	20.6

$µ_{1} : µ_{2} : µ_{3}$	1:1:1							1:1:3
0.5	Original	$\sqrt{x}$	$\sqrt{x + 0.5}$	$\sqrt{x + (\frac{3}{8})}$	$\sqrt[3]{x}$	$\sqrt[4]{x}$	Log	Original	$\sqrt{x}$	$\sqrt{x + 0.5}$	$\sqrt{x + (\frac{3}{8})}$	$\sqrt[3]{x}$	$\sqrt[4]{x}$	Log
	7.31 7.61	6.90 8.55	7.01 8.4	6.98 8.42	6.65 8.85	6.53 8.96	7.16 8.06	7.65 9.47	7.05 10.74	7.18 10.56	7.17 10.59	6.75 11.02	6.61 11.14	7.37 10.21
	10.19 10.84	9.56 12.37	9.72 12.15	9.69 12.20	9.27 12.78	9.09 12.98	9.94 11.61	8.83 12.23	8.45 13.99	8.55 13.75	8.52 13.80	8.20 14.46	8.06 14.70	8.62 13.26
	17.73 19.10	17.02 22.35	17.17 21.8	17.15 21.92	16.58 23.32	16.32 23.83	17.39 20.72	12.52 16.36	12.92 20.31	12.96 19.66	12.96 19.79	12.94 21.85	12.94 22.62	13.03 18.58
	25.66 27.19	24.70 32.32	24.87 31.42	24.85 31.60	24.15 34.13	23.85 34.95	25.15 29.68	16.68 20.26	18.11 27.12	18.00 26.03	18.01 26.30	18.45 29.98	18.61 31.40	18.03 24.28
	48.91 49.81	47.66 59.85	47.94 58.15	47.87 58.52	46.94 63.03	46.55 64.65	48.25 54.93	30.82 33.27	35.41 48.27	34.95 46.03	35.05 46.53	37.05 54.01	37.88 56.85	35.21 42.66
	72.47 73.19	71.44 83.52	71.69 81.94	71.62 82.29	70.82 86.24	70.45 87.47	71.91 78.75	50.57 51.11	57.55 71.67	56.80 69.01	56.97 69.61	60.04 78.02	61.22 80.90	57.43 65.17
0.75	10.87 11.87	10.01 13.23	10.24 13.05	10.19 13.08	9.59 13.65	9.34 13.78	10.59 12.61	9.56 13.08	8.80 14.27	8.98 14.16	8.93 14.19	8.39 14.54	8.15 14.60	9.20 13.88
	17.52 19.85	16.26 22.27	16.60 21.94	16.54 22.02	15.60 22.89	15.20 23.14	17.02 21.15	13.03 18.26	12.26 20.22	12.48 19.95	12.44 20.00	11.78 20.71	11.48 20.92	12.68 19.48
	35.41 38.63	33.62 44.36	34.05 43.45	33.97 43.65	32.76 46.08	32.21 46.89	34.59 41.52	22.72 28.11	22.77 33.83	22.92 32.99	22.89 33.17	22.44 35.86	22.28 36.92	23.13 31.61
	52.43 54.82	50.59 63.05	50.97 61.70	50.92 61.96	49.65 65.55	49.06 66.78	51.46 59.00	33.49 37.63	34.77 47.42	34.77 46.06	34.77 46.37	34.83 50.95	34.85 52.72	34.98 43.98
	84.96 85.10	83.90 92.21	84.12 91.23	84.06 91.44	83.25 93.90	82.84 94.63	84.41 89.13	64.60 64.03	67.94 79.17	67.74 77.43	67.80 77.85	68.85 83.46	69.24 85.38	68.10 74.89
	97.64 97.69	97.39 99.44	97.45 99.28	97.43 99.32	97.21 99.66	97.11 99.74	97.52 98.88	88.63 87.09	91.08 96.24	90.91 95.53	90.94 95.71	91.77 97.78	92.05 98.28	91.17 94.30
1.0	15.70 18.29	14.19 19.68	14.55 19.58	14.47 19.61	13.49 19.97	13.06 20.04	15.17 19.14	12.62 17.72	11.28 18.70	11.58 18.72	11.51 18.71	10.69 18.76	10.37 18.74	11.92 18.51
	28.34 33.00	26.09 36.08	26.61 35.68	26.48 35.76	24.96 36.68	24.28 36.94	27.40 34.72	19.12 26.39	17.58 28.33	17.95 28.16	17.86 28.21	16.74 28.66	16.28 28.79	18.31 27.72
	58.08 61.05	55.76 67.77	56.26 66.77	56.18 67.00	54.46 69.76	53.72 70.73	56.92 64.70	38.05 43.21	37.33 50.32	37.71 49.41	37.64 49.59	36.55 52.60	36.02 53.78	38.10 47.91
	78.51 79.49	76.77 86.38	77.15 85.39	77.06 85.62	75.78 88.23	75.20 89.01	77.65 83.36	56.81 58.42	57.29 69.17	57.45 67.91	57.40 68.20	56.88 72.47	56.59 74.03	57.77 66.04
	98.21 98.04	97.96 99.49	98.02 99.35	98.01 99.39	97.79 99.69	97.70 99.76	98.10 99.06	90.25 88.06	91.46 95.74	91.42 95.15	91.43 95.26	91.66 97.15	91.68 97.70	91.57 94.27
	99.97 99.96	99.96 100	99.96 100	99.96 100	99.96 100	99.95 100	99.96 99.99	99.33 98.74	99.49 99.87	99.49 99.82	99.49 99.84	99.53 99.95	99.54 99.97	99.50 99.74
1.5	29.82 36.58	26.19 37.27	27.07 37.42	26.90 37.40	24.47 37.06	23.61 36.77	28.34 37.43	21.18 29.58	18.27 29.29	18.88 29.58	18.75 29.51	16.97 28.74	16.32 28.39	19.69 29.87
	57.03 62.14	52.89 64.86	53.83 64.64	53.65 64.69	50.67 65.21	49.42 65.25	55.15 64.01	38.37 45.98	34.57 47.56	35.37 47.55	35.22 47.56	32.58 47.56	31.48 47.50	36.22 47.39
	91.56 91.13	90.24 94.56	90.54 94.17	90.48 94.26	89.50 95.39	88.99 95.72	90.92 93.33	74.69 73.66	72.81 80.69	73.38 80.03	73.26 80.17	71.34 82.63	70.45 83.48	73.93 79.03
	98.88 98.37	98.63 99.50	98.69 99.40	98.68 99.43	98.46 99.64	98.36 99.70	98.76 99.16	92.50 89.51	92.17 95.02	92.35 94.60	92.31 94.71	91.63 96.19	91.26 96.68	92.52 94.04
	100 100	100 100	100 100	100 100	100 100	100 100	100 100	99.94 99.73	99.94 99.98	99.95 99.97	99.95 99.97	99.94 99.99	99.94 99.99	99.95 99.96
	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100
2.0	48.54 56.14	42.33 55.14	43.74 55.69	43.44 55.60	39.38 53.89	37.81 53.12	45.92 56.41	33.97 43.75	28.30 41.81	29.38 42.47	29.15 42.35	25.89 40.43	24.71 39.63	30.93 43.41
	82.50 83.34	78.65 85.11	79.53 85.04	79.36 85.06	76.39 85.18	75.02 85.20	80.71 84.75	62.30 65.60	56.19 66.75	57.48 66.86	57.21 66.86	53.08 66.56	51.26 66.32	58.79 66.96
	99.50 98.90	99.33 99.59	99.38 99.53	99.37 99.55	99.19 99.71	99.10 99.75	99.43 99.41	95.33 91.68	94.32 95.40	94.64 95.15	94.57 95.21	93.45 96.17	92.85 96.50	94.81 94.87
	99.99 99.94	99.98 99.99	99.98 99.99	99.98 99.99	99.98 100	99.98 100	99.98 99.98	99.68 98.66	99.63 99.70	99.65 99.66	99.64 99.67	99.56 99.82	99.52 99.86	99.66 99.62
	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100
	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 1000	100 100	100 100

$µ_{1} : µ_{2} : µ_{3}$	1:1:5							1:1:10
0.5	Original	$\sqrt{x}$	$\sqrt{x + 0.5}$	$\sqrt{x + (\frac{3}{8})}$	$\sqrt[3]{x}$	$\sqrt[4]{x}$	Log	$\sqrt{x}$	$\sqrt{x + 0.5}$	$\sqrt{x + (\frac{3}{8})}$	$\sqrt[3]{x}$	$\sqrt[4]{x}$	Log	$\sqrt{x}$
	8.59 11.76	7.93 13.00	8.09 12.90	8.07 12.92	7.56 13.17	7.35 13.25	8.25 12.65	10.13 14.79	9.30 15.31	9.50 15.36	9.48 15.36	8.80 15.20	8.51 15.10	9.43 15.36
	9.15 13.34	8.80 14.92	8.91 14.68	8.89 14.74	8.52 15.38	8.36 15.63	8.97 14.38	9.83 15.23	9.34 16.60	9.47 16.46	9.44 16.50	9.00 16.99	8.80 17.16	9.36 16.36
	10.98 15.82	11.81 20.13	11.76 19.45	11.78 19.61	11.97 21.88	12.04 22.85	11.94 18.56	10.04 15.13	10.72 20.01	10.73 19.34	10.73 19.50	11.04 22.34	11.22 23.55	11.17 19.22
	13.29 17.91	15.31 25.53	15.12 24.28	15.17 24.56	16.12 28.96	16.54 30.75	15.58 22.95	10.71 15.56	13.21 23.95	13.03 22.79	13.07 23.05	14.38 28.15	14.99 30.39	14.49 22.82
	21.85 25.56	28.18 42.41	27.53 39.91	27.67 40.45	30.98 49.45	32.35 52.97	29.26 37.66	14.24 18.33	21.70 36.80	20.99 34.42	21.15 34.97	25.41 45.58	27.28 50.24	26.68 35.38
	35.70 37.77	46.56 63.32	45.54 60.13	45.77 60.80	50.76 72.07	52.90 76.06	48.94 57.37	20.24 23.70	34.05 53.46	32.81 49.87	33.10 50.70	40.19 65.25	43.46 70.92	44.27 52.19
0.75	9.94 14.53	9.11 15.57	9.28 15.51	9.24 15.54	8.69 15.65	8.45 15.65	9.43 15.37	10.98 16.80	9.88 17.05	10.14 17.12	10.08 17.11	9.32 16.79	9.03 16.59	10.06 17.21
	11.75 18.11	11.09 19.75	11.25 19.61	11.21 19.65	10.70 20.19	10.48 20.36	11.35 19.28	11.51 18.13	10.91 19.45	11.08 19.34	11.05 19.36	10.52 19.75	10.30 19.94	10.92 19.27
	17.22 23.75	17.98 29.30	18.02 28.52	18.01 28.72	18.00 31.56	17.98 32.75	18.21 27.56	13.22 19.97	14.18 25.71	14.23 24.99	14.22 25.17	14.54 28.13	14.67 29.40	14.78 24.95
	24.35 29.76	26.61 39.79	26.51 38.49	26.54 38.77	27.45 43.86	27.75 45.93	27.13 37.14	15.85 22.07	19.08 32.26	18.86 31.00	18.92 31.28	20.34 36.77	20.91 39.27	20.49 31.36
	48.04 49.20	54.10 68.52	53.71 66.39	53.80 66.89	56.17 74.68	57.13 77.50	55.38 64.79	26.92 31.11	35.94 53.38	35.27 50.98	35.43 51.53	39.73 61.95	41.63 66.23	41.11 52.80
	75.79 73.12	81.64 90.64	81.30 89.25	81.37 89.57	83.48 94.19	84.27 95.54	83.06 88.32	46.22 46.92	59.32 76.72	58.55 74.15	58.75 74.74	64.21 84.97	66.61 88.26	67.31 77.04
1.0	12.07 18.40	10.87 18.95	11.14 19.04	11.07 19.01	10.23 18.83	9.93 18.74	11.36 19.04	12.32 19.17	11.01 18.88	11.26 19.11	11.22 19.08	10.36 18.52	10.00 18.31	11.18 19.30
	16.01 23.91	14.73 25.46	15.06 25.33	14.99 25.36	14.09 25.77	13.75 25.89	15.22 25.07	13.45 21.34	12.63 22.43	12.82 22.38	12.78 22.39	12.11 22.70	11.80 22.77	12.65 22.41
	28.05 34.39	28.16 40.96	28.38 40.17	28.36 40.36	27.80 43.50	27.53 44.75	28.74 39.36	18.02 25.87	19.07 32.19	19.19 31.52	19.18 31.68	19.32 34.91	19.34 36.31	19.57 31.65
	41.84 45.41	43.57 57.17	43.66 55.92	43.64 56.21	43.74 61.21	43.74 63.24	44.33 54.85	24.52 31.00	27.74 42.79	27.67 41.48	27.69 41.78	28.81 47.57	29.27 50.06	29.15 42.33
	78.28 75.19	81.18 89.08	81.13 87.97	81.14 88.20	81.96 92.18	82.2 93.48	82.1 87.41	49.28 49.94	56.92 72.24	56.57 70.39	56.65 70.83	59.45 78.96	60.59 82.03	60.86 72.83
	96.83 94.61	97.74 99.14	97.71 98.93	97.72 98.98	97.95 99.59	98.01 99.72	97.97 98.86	79.91 75.60	85.62 93.53	85.43 92.51	85.47 92.77	87.25 96.60	88.01 97.52	88.45 94.30
1.5	18.35 27.50	15.77 26.68	16.28 26.97	16.17 26.89	14.70 26.03	14.14 25.62	16.78 27.45	16.06 25.28	14.00 24.09	14.35 24.46	14.29 24.37	13.09 23.29	12.62 22.81	14.30 24.80
	29.50 38.54	26.44 39.50	27.06 39.57	26.93 39.55	24.88 39.47	24.06 39.39	27.47 39.56	20.85 30.56	18.85 31.08	19.25 31.18	19.17 31.16	17.83 31.05	17.30 30.95	18.92 31.39
	59.25 60.70	57.32 68.25	57.99 67.64	57.83 67.80	55.82 70.60	54.83 71.69	58.49 67.23	35.90 42.45	35.51 49.79	36.01 49.21	35.90 49.35	34.79 52.49	34.23 53.78	36.16 50.09
	81.64 77.93	81.16 87.22	81.62 86.56	81.53 86.71	80.20 89.53	79.56 90.52	81.93 86.43	54.53 55.16	55.24 67.85	55.69 66.93	55.58 67.16	54.77 71.97	54.39 73.90	56.16 68.85
	99.44 98.16	99.51 99.74	99.52 99.69	99.52 99.70	99.48 99.87	99.45 99.91	99.52 99.69	92.49 87.00	93.17 96.34	93.37 95.99	93.33 96.08	93.00 97.88	92.81 98.42	92.86 97.04
	100 99.98	100 100	100 100	100 100	100 100	100 100	99.99 100	99.83 99.17	99.87 99.98	99.88 99.97	99.88 99.97	99.87 99.99	99.86 99.99	99.70 99.98
2.0	27.79 38.33	23.13 35.91	24.04 36.52	23.83 36.40	21.20 34.49	20.21 33.68	24.90 37.43	21.82 32.79	18.38 30.06	18.96 30.64	18.83 30.53	16.96 28.81	16.21 28.08	18.91 31.24
	48.38 54.76	42.65 55.10	43.78 55.38	43.55 55.35	39.89 54.68	38.35 54.37	44.54 55.78	31.80 41.33	27.85 41.15	28.60 41.27	28.45 41.42	25.99 40.74	24.99 40.38	28.20 41.91
	86.51 82.05	84.15 88.03	84.81 87.68	84.69 87.75	82.29 89.44	81.11 90.04	84.98 87.82	61.78 61.95	58.60 69.32	59.48 68.95	59.33 69.06	56.44 71.48	55.22 72.45	58.74 70.50
	98.06 94.75	97.71 98.18	97.83 98.04	97.81 98.07	97.28 98.75	96.94 98.94	97.80 98.12	84.99 79.32	83.33 88.68	83.97 88.33	83.83 88.41	81.87 90.84	80.84 91.76	82.62 90.01
	100 99.97	100 100	100 100	100 100	100 100	100 100	99.99 100	99.86 98.82	99.85 99.91	99.87 99.90	99.86 99.90	99.81 99.97	99.77 99.98	99.32 99.96
	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	100 100	99.98 100

Type	Shapiro - Wilks Test p-value	Bartlett Test p-value
Non-transformed	0.033	0.066
$\sqrt{x}$	0.049	0.081
$\sqrt{x + 0.5}$	0.049	0.080
$\sqrt{x + (\frac{3}{8})}$	0.049	0.080
$\sqrt[3]{x}$	0.060	0.085
$\sqrt[4]{x}$	0.066	0.088
$L o g_{10}$	0.082	0.095

Universidade Federal de Minas Gerais, Escola de Veterinária Caixa Postal 567, 30123-970 Belo Horizonte MG - Brazil, Tel.: (55 31) 3409-2041, Tel.: (55 31) 3409-2042 - Belo Horizonte - MG - Brazil
E-mail: abmvz.artigo@gmail.com

Acompanhe os números deste periódico no seu leitor de RSS

[1] Corresponding author: malikergin@isparta.edu.tr