Validation of a scale to evaluate the abusive use of technologies (Computer, cell phone, tablet, among others)

Abstract

BACKGROUND INFORMATION:

Daily, prolonged interactivity of individuals with technologies (computer, cell phone, tablet, among others) impacts life and significantly changes habits, behaviors, personal and social relationships. Technologies lead to a multitude of advantages, but attention is required concerning possible damages.

OBJECTIVE:

Validation of a scale to evaluate the abuse use of technologies (TAUS).

METHOD:

TAUS validation was carried out in 5 phases: (1) initial scale construction with 20 questions, (2) expert evaluation, (3) application to 200 volunteers, (4) statistical analysis and results, (5) preparation of the final version of the validated TAUS. We used the R statistical program and the “dplyr” package version 3.4.2 to present descriptive statistics, to test hypotheses of means differences and for factorial analysis. Factor analysis was used for the orthogonal model. The method used was Principal Components based on Spearman’s correlation matrix.

RESULTS:

The results provided a final, validated version of a TAUS suitable for clinical and research contexts. The last step of the study was to calculate Cronbach’s alpha, in order to measure the internal consistency of the scale. The value found was 0.910, which is considered good.

CONCLUSIONS:

This Technology Abuse Scale may contribute to future studies, to the conscious use of technologies, to a reduction of physical and emotional damage and to an improvement of the subjects’ quality of life.

KEYWORDS:
Abusive use; digital dependence; technologies; mental disorder

INTRODUCTION

Technologies (computers, cell phones, tablets, among others. (CCPT&O)) are in continuous and rapid evolution and interact increasingly with our lives.¹ Improvement is so rapid that very few of us are prepared to absorb or follow this progression. Abusive use is further intensified by the use of the internet, social networks, applications and everything else offered to us.

New technologies CCPT&O have promoted changes² in habits, behaviors, personal and social relationships. Now and in the foreseeable future it seems impossible to avoid all the ensuing effects (benefits and losses) resulting from this interactivity.² It is thus of paramount importance to investigate and understand such changes in all possible contexts.

The “normal” use of technologies is one that allows us to take advantage of personal growth, work, social relationships, among others.³ Daily prolonged use does not constitute, per se, a pathological dependence. Pathological dependence is characterized by a sufficient level of inadequate use and must be accompanied by a history of symptoms in order to be diagnosed.³

The purpose of the construction of validated scales in the area of digital dependence⁴ is to provide researchers with appropriate instruments for carrying out specific studies. We intend to perfect and train health professionals to deal with this new demand for help, namely the diagnosis and treatment of such “technology-dependent” individuals who increasingly seek this kind of support.

The objective of this study is to validate a scale to evaluate the abusive use of technologies in general through a TAUS in the daily life of individuals.

MATERIALS AND METHODS

At the time of writing, there are no specific scales for the quantitative evaluation of certain forms of abuse of digital technologies. To produce and validate any such scale, it is necessary to develop its content rigorously aligned with the subject and the objectives of the end-product. This task must be undertaken by experts in the field: then it must be tested on volunteers and the results statistically analyzed for validity. There is no consensus to define the number of specialists who should participate in the validation of a scale; this is left to the discretion and accessibility of the researcher: the greater the number of specialists, the greater the disagreement; conversely, the smaller the number (less than 3) the greater the risk of the agreement being 100%.

Accordingly, the production, validation and testing of this scale to evaluate abuse of digital technologies was carried out in 5 phases.

1. Construction of an initial questionnaire scale; six specialists trained in the area of digital dependence⁴ were selected. Based on published studies,^5,6,7 they constructed a scale with 20 questions.

2. Evaluation of the questions by a second group of six similarly trained specialists, who analyzed the content regarding presentation, clarity, relevance and understanding. Thus, a preliminary validation was provided.

3. Application of the scale to 200 volunteers, divided into two groups: MAIN Group, including 100 participants with presumed abuse of technologies (CCPT&O); CONTROL Group including 100 participants with no presumed abuse of technologies (CCPT&O). For inclusion in the Main or Control groups, volunteers were previously submitted to the Internet Addiction Test (IAT) scale⁸ The Main Group included volunteers with IAT scores ≥ 50, whereas the Control Group included IAT scores < 50.

4. Statistical analysis and evaluation of the results.

5. Preparation of the validated final version.

The 200 volunteers participating in the research were asked to insert values opposite each question, as follows: Never/Rarely (0 points); Often (1 point), Always (2 points). The final sum of the results obtained ranked responders as follows. no dependence - 0 to 10 points; mild dependence - 11 to 20 points; moderate dependence - 21 to 30 points; severe dependence - 31 to 40 points. Orientations referring to each range of points was offered.

Sample. Volunteers included in the TAUS were (i) patients seen at our facility with complaints of abuse symptoms and prolonged daily use of technologies (CCPT&O). (ii) accompanying persons (iii) students, employees, any persons who agreed to participate. All were randomly recruited through posters at the institution, verbal communication from person to person and on social networks.

Factor analysis was used for the orthogonal model. The method used was Principal Components based on Spearman’s correlation matrix. For data analysis we used the R statistical program, version 3.4.2. ⁹and packages “dplyr” .¹⁰ “psy” .¹¹ “paran” ¹² into R.

Inclusion Criteria. Participants should be between the ages of 17 and 65 and have a cell phone, tablet, computer, etc. with or without internet access.

Exclusion Criteria. illiterate candidates and persons with some kind of mental impairment that would prevent them from using technologies.

We discarded 5 Main and 10 Control group participants. Discarded volunteers presented incomplete questionnaires, discontinued participation or lack of accompanying persons when minors. The included results were entered into a database for statistical analysis.

RESULTS

Table 1 shows the results of the demographic statistics (age group, gender, degree of education) of the sample. For each characteristic, the absolute number of elements with the characteristic and the proportion within its group are displayed. Demographic data were collected for statistical purposes and not considered in the statistical evaluation.

Table 1
Descriptive statistics of sample.

Gender
	Male			Female
Control	28 (31.1%)			62 (68.9%)
Main	34 (36.2%)			60 (63.8%)
Age range
	15-25	26-36	37-47	48-58	59-69
Control	29 (32.2%)	23 (25.6%)	11 (12.2%)	11 (12.2%)	16 (17.8%)
Main	44 (46.8%)	23 (24.5%)	20 (21.3%)	5 (5.3%)	2 (2.1%)
Education
	Middle	College	Graduate	Master	Doctoral	NI
Control	21 (23.3%)	26 (28.9%)	37 (41.1%)	2 (2.2%)	3 (3.3%)	1 (1.1%)
Main	53 (56.4%)	26 (27.7%)	9 (9.6%)	5 (5.3%)	0 (0%)	1 (1.1%)

• NI = Not informed

Scores for the 20 original question scale. The mean ± standard deviation score for the Control group was 12.71 ± 8.42, while the corresponding value for the Main group was 19.47 ± 7.27 The t-test of means between the two groups produced a p-value < 0.001 (t-statistic = 5.820); this indicates a significantly higher level of damage in the Main group vs. the Control group. This difference ratifies, prima facie, the characteristics of the groups, mainly dependence in the main group and little or no dependence in the control group.

Factor analysis. The first test performed was the Bartlett sphericity test to verify if the variables are correlated with each other. In this test, the null hypothesis is that the correlation matrix, based on Spearman’s correlation, is equal to the identity matrix. For the data set, a statistic equal to 1806.758 and a p-value <0.001 was found, implying that the covariance matrix is not equal to the identity.

The next criterion used to verify the adequacy of the factor analysis was the Kaiser-Meyer-Olkin (KMO) criterion. Its value found was equal to 0.877; values above 0.8 are considered good.¹³ Table 2 DISPLAYS the Measurement of Sampling Adequacy (MSA) indices for each of the variables.

Table 2
Measure of Sampling Adequacy (MSA).

TAUS.1	TAUS.2	TAUS.3	TAUS.4	TAUS.5
0.819	0.811	0.922	0.888	0.925
TAUS.6	TAUS.7	TAUS.8	TAUS.9	TAUS.10
0.913	0.892	0.864	0.910	0.827
TAUS.11	TAUS.12	TAUS.13	TAUS.14	TAUS.15
0.756	0.851	0.897	0.893	0.870
TAUS.16	TAUS.17	TAUS.18	TAUS.19	TAUS.20
0.875	0.914	0.874	0.919	0.893

Due to the results for the Bartlett test and the KMO criterion, we considered it appropriate to carry out the factorial analysis for the scale.

The next step was to check the factor loads to determine the number of relevant factors. We used 3 criteria: Factorial Load, Screeplot and Parallel Analysis. Table 3 shows the factorial loads, using the Principal Components as method:

Table 3
Factorial loads of Principal Components.

	PC1	PC2	PC3	PC4	PC5
Standard deviation	2.762	1.328	1.205	1.114	1.059
Variance proportion	0.381	0.088	0.073	0.062	0.056
Cumulative proportion	0.381	0.470	0.542	0.604	0.660
	PC6	PC7	PC8	PC9	PC10
Standard deviation	0.942	0.904	0.874	0.775	0.732
Variance proportion	0.044	0.041	0.038	0.030	0.027
Cumulative proportion	0.705	0.746	0.784	0.814	0.841
	PC11	PC12	PC13	PC14	PC15
Standard deviation	0.684	0.668	0.620	0.600	0.573
Variance proportion	0.023	0.022	0.019	0.018	0.016
Cumulative proportion	0.864	0.886	0.906	0.924	0.940
	PC16	PC17	PC18	PC19	PC20
Standard deviation	0.545	0.514	0.511	0.442	0.428
Variance proportion	0.015	0.013	0.013	0.010	0.009
Cumulative proportion	0.955	0.968	0.981	0.991	1.000

• PC = Principal Components

Factor loads with cumulative proportions above 0.9 are considered satisfactory.¹³ For this data set, we would have to use 13 factors, which in practice would not solve the problem of data reduction.

The Screeplot criterion of the correlation matrix was tested: in this test we eliminate the factors related to Eigenvalues > 1. Figure 1 presents this criterion:

Figure 1
Screeplot Chart. The components with variance values greater than 1 are highlighted, above the red line, because these are the relevant components.

By this criterion, we should use 5 factors, and in this case, the commonalities of the variables are presented in Table 4.

Table 4
Commonalities for 5 factors.

TAUS.1	TAUS.2	TAUS.3	TAUS.4	TAUS.5
0.750	0.701	0.617	0.503	0.651
TAUS.6	TAUS.7	TAUS.8	TAUS.9	TAUS.10
0.623	0.700	0.744	0.679	0.702
TAUS.11	TAUS.12	TAUS.13	TAUS.14	TAUS.15
0.846	0.801	0.613	0.569	0.683
TAUS.16	TAUS.17	TAUS.18	TAUS.19	TAUS.20
0.720	0.637	0.485	0.583	0.598

Analyzing the commonalities, it was observed that the question 18 (How often do you usually have the feeling of being accompanied when you are using the technologies (CCPT&O) should be excluded from the initial scale with 20 questions because of a commonality less than 0.5. Therefore, we decided to use the Screeplot as a basis for recompose the questionnaire scale

The third criterion used to find the number of factors was the Parallel Analysis. By this criterion, the number of factors found was equal to 3, and its commonalities are presented in Table 5.

Table 5
Commonalities for 3 Factors.

TAUS.1	TAUS.2	TAUS.3	TAUS.4	TAUS.5
0.610	0.481	0.585	0.472	0.642
TAUS.6	TAUS.7	TAUS.8	TAUS.9	TAUS.10
0.592	0.611	0.586	0.649	0.684
TAUS.11	TAUS.12	TAUS.13	TAUS.14	TAUS.15
0.652	0.680	0.462	0.458	0.473
TAUS.16	TAUS.17	TAUS.18	TAUS.19	TAUS.20
0.597	0.591	0.195	0.418	0.405

For the number of factors equal to 3, we must remove questions 2, 4, 13, 14, 15, 18, 19 and 20 because they have a commonality less than 0.5, what would eliminate important issues as the goal of the research.

The last step of the study was to calculate Cronbach’s alpha,¹³ in order to measure the internal consistency of the scale. The value found was 0.910, which is considered good.¹³

DISCUSSION

Samples from the so-called Main and Control Groups were randomly formed without pre-established concerns about the quantitative distribution of male and female subjects, as well as the distribution of age groups and education level. This resulted in a considerable level of variability, which reinforces the random characteristic of a data collection ensuring realistic results.

Thus, the significant differences in IAT scores for the “Main” and “Control” groups, formed by dependents and non-dependents of digital technologies respectively, were ratified by the Main group with an IAT score 60% higher than that of the Control group, securing the quality of the results obtained in the collection. A final validated scale was constructed, with the purpose of being used in clinical practice which fully met what was proposed, namely the evaluation of abusive use of technologies (Computer, cell phone, tablet, among others).

Factorial Analysis was then performed based on the results of the Bartlett sphericity tests that confirmed the correlation between the variables that constitute the questionnaire, in addition to the very satisfactory KMO index, equal to 0.877, a value above the statistically reference value of 0.8.¹³ The first of three criteria, Factorial Loads, signaled a high value of 13 factors for an initial scale with 20 questions, which made it a useless criterion.

The second criterion was the Screeplot which, despite pointing to 5 factors, suggested the withdrawal of only one of the 20 questions from the initial scale mentioned above. The indication of this withdrawal was due to the fact that the commonality of this question, extracted by the Screeplot Criterion was 0.485, thus below 0.5, which is a criterion for withdrawing a question from a scale. Withdrawing just one question leads to an excellent Cronbach’s Alpha of 0.910, corroborating with judicious scale construction.

As the third and last criterion, the Parallel Analysis pointed to only 3 factors, but with the suggestion to withdraw 8 questions out of a total of 20, which would compromise the consistency of the scale.

Thus, the Screeplot Criterion was used to recompose the TAUS Scale, now with 19 questions, pointing to a positive and consistent adequacy to obtain data on dependence on digital technologies.^14-16

As a limitation of the study, we came across an absence of specific validated instruments capable of investigating behavior using technologies on a day-to-day basis, which might have helped us in the preparation of the present scale. Future studies are recommended so that we can refine the research in all areas and especially on the subject of digital dependence.

CONCLUSION

The results obtained provided a validated version for the TAUS, with 19 questions appropriate to clinical and research contexts for clarity and accuracy.

Statistical results showed that the issues of the final version of the scale presented alignment among them, qualifying it as positive to measure the abusive use of technologies (CCPT&O). The final version of TAUS can be used, whenever it is necessary to carry out research projects related to the subject digital dependency.

With the results of new studies using the TAUS, we can better observe the clinical, cognitive-behavioral, social and professional effects resulting from the impact caused by the interference of the technologies (CCPT&O) in the daily life of individuals. We can also expand scientific knowledge, improve outpatient care and develop forms of conscious use of technologies such as prevention and reduction of physical and/or psychological damage in the population.

We recommend that the study be replicated in a larger sample and representative of the target population.

ACKNOWLEDGEMENTS

This work was supported by: Carlos Chagas Filho Foundation for Research Support of the State of Rio de Janeiro (FAPERJ); Institute of Psychiatry (IPUB) of the Federal University of Rio de Janeiro (UFRJ); Delete - Conscious Use of Technologies.

Annex 1 _ Validated final version

Scale to evaluate the Abusive Use of Technologies (TAUS)

Technologies = Computer, cell phone, tablet, among others (CCPT&O).

Date: _____/____/______ Age: ________________

Volunteer name: _______________________________________________________

Gender: F ( ) M ( )

Works: Yes ( ) No ( )

Unemployed: Yes ( ) No ( )

Level of Education: ( ) Middle ( ) Higher ( ) Graduate ( ) Master ( ) Doctoral

Signature of Volunteer: _________________________________________________

e-mail:_______________________________________________________________

Phone:_______________________________________________________________

Interviewer: __________________________________________________________

The test is a scale with 19 questions that measure the mild, moderate and severe levels of abusive use of technologies (computer, cell phone, tablet, etc. - CCPT&O) in everyday life.

Please enter the number corresponding to the answer next to the question:

Never/Rarely (0)

Frequently (1)

Always (2)

QUESTIONS

1. How often do you usually use CCPT&O technologies for more than three hours throughout your day?

2. How often do you usually use CCPT&O for more than four hours throughout your day?

3. How often when you stop using the CCPT&O do you usually go back to it?

4. How often do you usually feel happier when using CCPT&O in your day?

5. How often do you usually feel sad when you cannot access CCPT&O in your day?

6. How often do you usually feel anxious when you realize you have no access to CCPT&O technologies?

7. How often do you usually experience some kind of physical discomfort, such as chest tightness, a sore throat, palpitation, shortness of breath, or dizziness when you realize that you have no access to the CCPT&O?

8. How often do you usually feel afraid when you realize that you are without access to the CCPT&O?

9. How often do you usually feel nervous when you realize that you have no access to the CCPT&O?

10. How often do you usually feel rejected when you realize that someone has read and not immediately responded to your messages or emails on the CCPT&O?

11. How often do you usually use CCPT&O to post something to see the reaction of others and if it is positive do you feel more important/valued?

12. How often do you usually use CCPT&O to post something and see the reaction of others and if not, does it make you feel devalued/less important?

13. How often do you usually use CCPT&O to avoid the feeling of being alone?

14. How often do you usually consult or use CCPT&O even when you are with friends or with your partner?

15. How often do you usually consult or use CCPT&O even when you are at work, in classrooms or in other public places?

16. How often do you usually consult or use CCPT&O even when you are with the family?

17. How often during your day do you usually check the messages on your CCPT&O?

18. How often do you stop exercising or any other activity in your real life to stay connected in the virtual world of CCPT&O?

19. How often do you usually ignore the people who are by your side in the real world to stay corresponding with people in the virtual world of CCPT&O?

RESULTS

Once you have answered all the questions, add up the numbers you selected for each answer to get a final score. The higher the score, the higher the level of abuse of technologies CCPT&O in everyday life and related problems.

Check your score:

Up to 8 points: You are a user with no signs of abuse of CCPT&O in your daily life and with full control over its use.

9 to 18 points: Mild - You show signs of using CCPT&O technologies in everyday life at a light level. You may experience occasional problems due to the start of abusive use of the technologies in certain situations. You may have future impacts on your quality of life if you use CCPT&O more often than necessary. Be aware that the use of technologies in your daily life will not harm your personal, social, family, professional or academic life.

19 to 28 points: Moderate - You show signs of abusive use of technologies CCPT&O at a moderate level. You start having frequent problems due to the abusive use of the technologies in certain situations. You should consider the impacts that are present today in your personal, social, family, professional and academic life because you use CCPT&O more intensively in your daily life than what is recommended. You must learn to deal with technologies more consciously.

29 to 38 points: Severe - The use of excess CCPT&O in your daily life may already be causing significant problems in your personal, social, family, professional and academic life at a serious level. You should seek to assess the consequences of these impacts that may also be causing physical and/or emotional harm and compromising your quality of life. We recommend seeking guidance through professional help in specialized centers.

REFERENCES

• ¹
  King ALS, Nardi AE, Cardoso A (Organizadores). Nomofobia - Dependência do computador, internet, redes sociais? Dependência do telefone celular? O impacto das novas tecnologias interferindo no comportamento humano. Editora Atheneu, RJ, 2015.
• ²
  King ALS, Valença AM, Silva AC, Sancassiani F, Machado S, Nardi AE. Nomophobia: Impact of Cell Phone Use Interfering with Symptoms and Emotions of Individuals with Panic Disorder Compared with a Control Group. Clinical Practice & Epidemiology in Mental Health. 2014;10:28-35 doi: 10.2174/1745017901410010028.
  » https://doi.org/10.2174/1745017901410010028
• ³
  King ALS, Valença AM, Nardi AE. Nomophobia: the Mobile Phone in Panic Disorder With Agoraphobia Reducing Phobias or Worsening of Dependence? Cog Behav Neurol 2010;23(1):52-4. doi: 10.1097/WNN.0b013e3181b7eabc
  » https://doi.org/10.1097/WNN.0b013e3181b7eabc
• ⁴
  Gonçalves LL. Dependência Digital: tecnologias transformando pessoas, relacionamentos e organizações. Barra Livros, RJ, 2017.
• ⁵
  Guedes E, Nardi AE, Guimarães FMC, Machado S, King ALS. Social networking, a new online addiction: a review of Facebook and other addiction disorders. MedicalExpress. 2016;3(1):M160101. DOI:10.5935/MedicalExpress.2016.01.01..
  » https://doi.org/10.5935/MedicalExpress.2016.01.01.
• ⁶
  King AL, Nardi AE. Novas tecnologias: uso e abuso. In: Associação Brasileira de Psiquiatria; Nardi AE, Silva AG, Quevedo JL, organizadores. PROPSIQ Programa de Atualização em Psiquiatria: Ciclo 3. Porto Alegre: Artmed/ Panamericana; 2013. p. 9-27. (Sistema de Educação Médica Continuada a Distância, v. 2).
• ⁷
  King ALS; Valença AM; Silva ACO; Baczynski T; Carvalho MR; Nardi AE. Nomophobia: dependency on virtual environments or social phobia? Comp Human Behav. 2013;29(1):140-4. DOI:10.1016/j.chb.2012.07.025
  » https://doi.org/10.1016/j.chb.2012.07.025
• ⁸
  IAT -The Center for Internet Addiction Recovery, Copyright 2009-2010 by The Center for Internet Addiction; Web Site Designed by Next Sunrise Studios, Bradford PA.
• ⁹
  R Core Team (2017). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/
  » https://www.R-project.org/
• ¹⁰
  Wickham H, François R, Henry L, Müller K (2017). dplyr: A Grammar of Data Manipulation. R package version 0.7.4. https://CRAN.R-project.org/package=dplyr
  » https://CRAN.R-project.org/package=dplyr
• ¹¹
  Falissard B. (2012). psy: Various procedures used in psychometry. R package version 1.1. https://CRAN.R-project.org/package=psy
  » https://CRAN.R-project.org/package=psy
• ¹²
  Dinno A (2012). paran: Horn's Test of Principal Components/Factors. R package version 1.5.1. https://CRAN.R-project.org/package=paran
  » https://CRAN.R-project.org/package=paran
• ¹³
  Hair-Jr JF, Rabin B, Money AH, Samouel P . Fundamentos de métodos de pesquisa em administração. Tradução: Lene Belon Ribeiro. Porto Alegre: Bookman, 2005.
• ¹⁴
  King ALS, Guedes E, Nardi AE. Cartilha Digital. Porto Alegre: EducaBooks, 2017.
• ¹⁵
  King ALS, Guedes E, Nardi AE. Ergonomia Digital. Porto Alegre: EducaBooks,2017.
• ¹⁶
  King ALS, Guedes E, Nardi AE. Etiqueta Digital. Porto Alegre: EducaBooks, 2017.

Publication Dates

• Publication in this collection
  04 July 2019
• Date of issue
  2019

History

• Received
  21 Aug 2018
• Reviewed
  25 Oct 2018
• Accepted
  26 Feb 2019
• Published
  17 Apr 2019