Scale validation to evaluate WhatsApp dependence (WADS)

ABSTRACT

BACKGROUND INFORMATION:

One of the most popular social networks is WhatsApp. Abusive use of this tool has produced significant changes in behavioral and social behaviors and habits of individuals. Therefore, research is needed to allow us to specifically evaluate the interaction of WhatsApp with the daily lives of individuals.

OBJECTIVE:

Production and validation of a scale to evaluate WhatsApp dependence (WADS).

METHODS:

Validation performed in 5 phases: 1- initial scale construction with 20 questions, 2- expert evaluation, 3- application in volunteers, 4- statistical analysis and results, and 5- elaboration of the final version of the validated WADS.

RESULTS:

We obtained a descriptive statistical analysis, a clear-cut separation of dependents vs. non-dependents and a successful factorial analysis. These results provided a validated version of WADS.

CONCLUSIONS:

We were able to construct a final version of WADS adequate to clinical contexts and to be used in future research to evaluate dependence of this digital tool. WADS will contribute to the conscious use of WhatsApp, hopefully reducing harmful effects and improving quality of life.

KEYWORDS:
WhatsApp; digital dependence; digital technologies, social networks

INTRODUCTION

As we write, it has become virtually impossible to think about life without the daily and frequent use of technology.¹ Virtual connection immediately brings to mind the use of social networks to make such contacts, for personal or professional reasons.¹

WhatsApp² is one of the most used among the social networks, with an estimated daily access by 1 billion people sending more than 50 billion messages; it provides users with communication by text, image, voice (recordings and phone calls), documents, video camera, all obtained through a simple internet connection.² Such daily interactivity through WhatsApp brings copious benefits; however, we cannot fail to note physical and emotional losses linked thereto.^3,4 Abusive, daily use (for long hours) is a threat to health and to quality of life.

In today’s world the use of the conventional telephone to interact has become a “boring” procedure, especially when compared to the instantaneousness demanded by the speed of information.⁵ Instead of calling, one has become accustomed to WhatsApp as a means to write or record messages and send them instantly so that they find the “target” wherever “it” may be.²

The use of the WhatsApp instant messaging application has revolutionized the communication paradigm in social, academic and professional environments.⁶ It allows for a more complete functional communication, a newly found imperative model for the current social context.

In order to specifically evaluate the use of WhatsApp we need a specific instrument to investigate routines, durations, frequencies of access, among other issues, in order to plan and consciously use the tool.⁷ No such instrument is presently available.

The objective of this project is the production and validation of a scale to evaluate the dependence of WhatsApp and arrive at an instrument of investigation and data collection related this ubiquitous tool.

MATERIALS AND METHOD

For a newly created scale to be validated it is necessary to develop its content strictly aligned with the subject and the research objectives and then to promote and submit the same to expert evaluation. There is no consensus to define the number of experts who should participate in the validation of a scale; this is left to the discretion of the research team. However, the greater their number, the greater the disagreement; in contrast, a smaller number (for instance, less than 3) leads to a risk of 100% agreement. The production and validation of the scale was performed in 5 phases:

1. Initial scale construction with 20 questions, by six experts, trained in the area of digital dependence⁵ and based on published studies.^8-10

2. Critical assessment and primary validation, by six different experts, who analyzed the content regarding presentation, clarity, pertinence and understanding.

3. Scale application to 200 volunteers, divided into a Main Group, with 100 volunteers suspected of WhatsApp abusive and daily use and a Control Group with 100 participants without apparent abusive use of WhatsApp.

4. Statistical analysis of data and results.

5. Adjustments based on the collected data and construction of the final validated version.

In the initial version of the 20-question scale to assess WhatsApp, the level of dependence was stratified into absent, mild, moderate and severe. The volunteers participating in the research were asked to insert the corresponding response value next to each question, as follows:

• Never/Rarely (0 points);

• Often (1 point)

• Always (2 points).

Upon completion, the points obtained were to be added to provide rating and, if necessary, clinical orientation.

The following demographic data were collects: (a) age, (b) gender, (c) professional circumstance, (c) degree of education. These were only collected for informational purposes and not taken into account for statistical analysis and expert validation of the scale.

Sample, Inclusion and Exclusion Criteria. Volunteers included to participate in the scale validation came from the following categories: (i) people who came to our facility because of suspected abusive use of WhatsApp; (ii) Companions of the previous group; (iii) students and employees, (iv) any who agreed to participate. They were randomly recruited through posters at the institution, verbal communication from person to person and on social networks. Participants should be between the ages of 16 and 65 and have a cell phone or computer with internet access.

Exclusion criteria were: illiterate people and persons with some mental impairment that would prevent them from using WhatsApp.

The 200 selected volunteers individuals were divided into a Main or Control through the application of the Internet Addiction Test (IAT).¹¹ The Main group included volunteers with IAT scores ≥ 50, whereas the Control Group included participants with IAT scores < 50. We were able to use 95 of the questionnaires in the Main Group and 90 in the Control Group. Discarded questionnaires presented incomplete scales, discontinued participation or lack of accompanying persons when minors. Results were entered into a database for statistical analysis.

RESULTS

We present the results of the descriptive statistics, the differences of means and the factorial analysis. Throughout the work, the data were divided into Control and Main Groups. To analyze the data we used Student’s t-test and the R statistical program Version 3.4.2,¹² “dplyr”,¹³ “psy”,¹⁴ and “Paran”¹⁵ packages.

Descriptive Statistics. Table 1 shows the results of the descriptive statistics of the sample. For each characteristic, the absolute number of elements with the characteristic and the proportion within each group are displayed.

Table 1
Demographics of included participants

Gender
	Male			Female
Control	28 (31.1%)			62 (68.9%)
Main	34 (36.2%)			60 (63.8%)
Age ranges
	15-25	26-36	37-47	48-58	59-65
Control	29 (32.2%)	23 (25.6%)	11 (12.2%)	11 (12.2%)	16 (17.8%)
Main	44 (46.8%)	23 (24.5%)	20 (21.3%)	5 (5.3%)	2 (2.1%)
Instruction
	Middle	Superior	Graduate	Master	Doctoral	NI
Control	21 (23.3%)	26 (28.9%)	37 (41.1%)	2 (2.2%)	3 (3.3%)	1 (1.1%)
Main	53 (56.4%)	26 (27.7%)	9 (9.6%)	5 (5.3%)	0 (0%)	1 (1.1%)

• NI: not informed.

Scores for the 20-question original scale. The Main Group scored (mean ± standard deviation) 17.03±8.19 points in the IAT test, while the Control group scored 8.43±6.12 points The t statistic was 8.09 generating a p-value < 0.001. This difference ratifies the presence of dependence in the Main Group and its absence in the Control Group).

Factorial analysis. The Bartlett sphericity test was applied to verify if the variables are correlated with each other. In this test, the null hypothesis is that the correlation matrix is equal to the identity matrix. For the data set, a statistic p-value <0.001, implying that the covariance matrix is not equal to identity; this shows a clear correlation between the variables.

The Kaiser-Meyer-Olkin (KMO) criterion was used to verify the adequacy of the factor analysis. The value found was 0.883; values above 0.8, are considered good.¹⁶ In Table 2, we exhibit the MSA indices for each of the variables.

Table 2
MSA of the MSA Questions - Measure of Sampling Adequacy

WADS.1	WADS.2	WADS.3	WADS.4	WADS.5
0.862	0.913	0.901	0.894	0.857
WADS.6	WADS.7	WADS.8	WADS.9	WADS.10
0.936	0.904	0.918	0.903	0.885
WADS.11	WADS.12	WADS.13	WADS.14	WADS.15
0.913	0.876	0.873	0.863	0.868
WADS.16	WADS.17	WADS.18	WADS.19	WADS.20
0.738	0.880	0.821	0.903	0.869

Due to the results found for both Bartlett and KMO, we considered that it was appropriate to carry out the factorial analysis for the scale.

The next step was to check the factor loads to determine the number of relevant factors. We used 3 criteria: Factorial Load, Screeplot and Parallel Analysis. Table 3 shows the factorial loads.

Table 3
Principal Components (PC).

	PC1	PC2	PC3	PC4	PC5
Standard deviation	2.730	1.457	1.344	1.053	0.969
Proportion of Variance	0.373	0.106	0.090	0.055	0.047
Cumulative proportion	0.373	0.479	0.569	0.624	0.671
	PC6	PC7	PC8	PC9	PC10
Standard deviation	0.880	0.836	0.784	0.756	0.750
Proportion of Variance	0.039	0.035	0.031	0.029	0.028
Cumulative proportion	0.710	0.745	0.776	0.804	0.832
	PC11	PC12	PC13	PC14	PC15
Standard deviation	0.700	0.641	0.624	0.619	0.576
Proportion of Variance	0.025	0.021	0.019	0.019	0.017
Cumulative proportion	0.857	0.878	0.897	0.916	0.933
	PC16	PC17	PC18	PC19	PC20
Standard deviation	0.568	0.546	0.540	0.468	0.462
Proportion of Variance	0.016	0.015	0.015	0.011	0.011
Cumulative proportion	0.949	0.964	0.978	0.989	>0.999

It is advisable¹⁶ to use factor loads with cumulative proportions > 0.9. However, for the data set, we would have to use 11 factors, which in practice would not solve the problem of data reduction. We then moved to the Screeplot criterion of the correlation matrix, where we eliminate the factors related to Eigenvalues greater than 1. Figure 1 presents this criterion.

Figure 1
Screeplot for the variance of components. in which the four points are above the red line and whose variance are greater than 1 are the relevant components.

According to this criterion, we must use 4 factors, and in this case, the commonalities of the variables are presented in the table below (Table 4).

Table 4
Communalities for 4 Factors.

WADS.1	WADS.2	WADS.3	WADS.4	WADS.5
0.659	0.659	0.686	0.707	0.571
WADS.6	WADS.7	WADS.8	WADS.9	WADS.10
0.684	0.534	0.499	0.680	0.554
WADS.11	WADS.12	WADS.13	WADS.14	WADS.15
0.690	0.593	0.603	0.723	0.647
WADS.16	WADS.17	WADS.18	WADS.19	WADS.20
0.679	0.657	0.524	0.610	0.528

Analyzing the commonalities, question 8 (“How often do you usually post your photos in WhatsApp to see if someone comments, in order to feel more valued?”) should be excluded due to a community of less than 0.5. However, due to its value being very close to 0.5, it was considered valid.

The third criterion used to find the number of factors was the Parallel Analysis. By this criterion, the number of factors found was equal to 3, and its commonalities are presented in Table 5.

Table 5
Communalities for 3 Factors

WADS.1	WADS.2	WADS.3	WADS.4	WADS.5
0.653	0.629	0.620	0.470	0.532
WADS.6	WADS.7	WADS.8	WADS.9	WADS.10
0.608	0.533	0.465	0.678	0.427
WADS.11	WADS.12	WADS.13	WADS.14	WADS.15
0.511	0.578	0.549	0,654	0.630
WADS.16	WADS.17	WADS.18	WADS.19	WADS.20
0.679	0.635	0.523	0.552	0.455

For the number of factors equal to 3, we must withdraw four questions (from the initial scale with 20) that will be cited in the discussion of this study, because they present commonalities smaller than 0.5. The deleted questions were:

“How often do you feel anxious when you realize that you are without access to WhatsApp?”

“How often do you usually post photos of yourself on WhatsApp to see if anyone comments, in order to feel more valued? “

“ How often do you usually join WhatsApp groups? “

“ How often do you ignore people who are your side in the real world to communicate with people through WhatsApp? “

The last step of the study was to calculate the Cronbach’s alpha¹⁶ in order to measure the internal consistency of the scale. The value found was 0.909, which is considered good.¹⁶

DISCUSSION

For the elaboration of a final validated scale that meets the proposed goals, namely the evaluation of the physical damages related to the abusive use of WhatsApp, it would have been necessary that all evaluation stages should be fulfilled and that the suggested final adaptations after all statistical analyzes and by experts, ought to be successful. We believe that these goals have been attained, and that the final set of included questions is satisfactory.

It has been noted¹ that people who are dependent on digital technologies tend to commit themselves to its abusive use in their personal, social, academic and professional lives. The prognosis of this scenario is increased digital dependence⁵ by intensifying the use of these resources, in the particular case of WhatsApp, because of its evident advantages. It is necessary to evaluate how much this resource impacts the life of people and for this it is necessary to have a validated scale that can provide elements for this evaluation, as described next.

The descriptive statistics of our Main and Control samples was satisfactory, in the sense that gender, age groups and educational levels showed no abnormal concentrations for any parameter.

The scores of Main and control groups in the Internet Addiction Test sharply and significantly separated addicted Main Group (score 17.03) from Control Group (score 8.18) entries (p < 0.001).

The factorial analysis was performed in consequence of the satisfactory results of the Bartlett sphericity test, which presented a significantly low p-value, indicating a correlation between the variables. The KMO test ratified the adequacy of the factorial analysis with 8 of the 20 items greater than 0.9, 11 items greater than 0.8 and 12 greater than 0.7.

Regarding the factor loads, 3 criteria were chosen for factor selection, and the Factor Load Criterion found 11 factors, which is a very high number. The Screeplot Criterion indicated 4 factors and the withdrawal of only one question from the scale. The Parallel Analysis Criterion pointed out 3 factors and the withdrawal of 4 questions because they presented commonalities less than 0.5. Within this scenario we chose to adopt the Parallel Analysis Criterion with the least number of factors and with the withdrawal of the four questions displayed in the Results section. The withdrawal of these four questions from the initial scale with 20 questions made the final scale more consistent and concise, with 16 questions.

Finally, we consider it important to highlight the excellent Cronbach Alpha ¹⁶ value of 0.909; This confirms the consistency of the scale within the objectives for which it was developed and tested.

Because of the novelty of the scale, we decided to clarify to the participating volunteers what this research represented, its objectives and the importance of the understanding of our intentions; we also indicated the use to be made of the data; this leads us to firmly believe that responses were reliable; this scenario was built jointly between the research team and the participants as a way to obtain the best results.

The limitation of the study was related to the difficulty of producing a totally new scale without the benefit of similar results, upon which we could base ourselves.

CONCLUSION

The results obtained provided a validated final version for the WhatsApp Dependence Scale (WADS) with 16 clear, accurate and reliable questions, appropriate to clinical contexts hopefully useful for future research. The statistical results showed that the issues of the final version of WADS presented alignment among them, qualifying it as positive to measure WhatsApp dependence. The final version can be used as a pioneer scale to assess WhatsApp dependence whenever a specific search of this nature is required.

ACKNOWLEDGEMENTS

This work was supported by: Carlos Chagas Filho Foundation for Research Support of the State of Rio de Janeiro (FAPERJ); Institute of Psychiatry (IPUB) of the Federal University of Rio de Janeiro (UFRJ); Delete - Conscious Use of Technologies.

Annex 1. Validated final version:

Scale to evaluate WhatsApp dependence (WADS)

Date: _____ / ____ / ______ Age: ________________

Name of Volunteer: ______________________________________________

Gender: F () M ()

Works: Yes () No ()

Unemployed: Yes () No ()

Level of Education: () Middle () Upper () Postgraduate () Master

( ) Doctorate degree

Signature of Volunteer: _________________________________________

Email:_______________________________________________________

Tels .________________________________________________________

Interviewer: __________________________________________________

The test is a 16-question scale that measures mild, moderate, and severe dependence levels of WhatsApp.

Please enter the number corresponding to your answer next to the question:

1. Never/Rarely (0)

2. Frequently (1)

3. Always (2)

Questions:

1. How often do you use WhatsApp throughout your day?

2. How often do you feel the need to access WhatsApp?

3. How often, when you quit WhatsApp, do you usually go back in as soon as possible?

4. How often do you experience any physical discomfort, such as chest tightness, sore throat, palpitation, shortness of breath or dizziness when you realize that you are out of WhatsApp?

5. How often are you afraid of being without access to WhatsApp?

6. How often do you feel rejected when you realize that someone has read and did not immediately respond to your WhatsApp messages?

7. How often do you use WhatsApp to avoid feelings of being alone?

8. How often do you feel nervous when you realize you have no access to WhatsApp?

9. How often do you check WhatsApp on your device even when you are with friends or with your partner?

10. How often, when you are in a bar or restaurant, do you regularly look for or send messages through WhatsApp?

11. How often do you check out WhatsApp when you are with your family?

12. How often do you join WhatsApp groups with people you do not know in your real life?

13. How often do you send photos in WhatsApp showing a reality different from your real life?

14. How often do you feel depressed when you see in WhatsApp that your friends have a more interesting life than yours?

15. How often do you, in WhatsApp, have the feeling of having company?

16. How often do you check if someone is online through WhatsApp?

Results:

Once you have answered all the questions, add up the numbers you selected for each answer to get a final score. The higher the score, the higher the WhatsApp dependence level and related problems.

Below is a ranking of the values obtained in your score:

Up to 2 points: You are a user with no signs of abusive use of WhatsApp and full control over its use.

3- 12 points: Light - You show signs of a possible dependence on WhatsApp at a light level. You may start having occasional problems because of the light abusive use of WhatsApp in certain situations. You may have future impacts on your personal, social, family, professional, or academic life by using WhatsApp more often than recommended. Make sure that abusive use of WhatsApp will not impair your quality of life.

13 - 22 points: Moderate - You show signs of a possible dependence on WhatsApp at a moderate level. You start having frequent problems due to abusive use of WhatsApp in certain situations. You should consider the impacts on your personal, social, family, professional, or academic life by using WhatsApp with greater intensity than recommended. You must learn to deal with WhatsApp more consciously.

23 - 32 points: Severe - Using WhatsApp is causing significant problems in your personal, social, family, professional or academic life at a serious level. It must assess the consequences of these impacts that may be causing physical and emotional damages in the various areas, significantly impairing their quality of life. We recommend seeking guidance through professional help in specialized centers.

REFERENCES

• ¹
  King ALS, Nardi AE, Cardoso A (Organizers). Nomofobia - Computer addiction, internet, social networks? Dependence on the cell phone? The impact of new technologies interfering with human behavior. Publisher Atheneu, RJ, 2015.
• ²
  Guedes E, Nardi AE, Guimarães FMC, Machado S, King ALS. Social networking, a new online addiction: a review of Facebook and other addiction disorders. MedicalExpress. 2016: 3 (1): M 160101. DOI: 10.5935/MedicalExpress.2016.01.01.
  » https://doi.org/10.5935/MedicalExpress.2016.01.01.
• ³
  King ALS, Guedes E, Nardi AE. Digital Ergonomics. Porto Alegre: EducaBooks, 2017.
• ⁴
  King AL, Nardi AE. New technologies: use and abuse. In: Brazilian Association of Psychiatry; Nardi AE, Silva AG, Quevedo JL, organizers. PROPSIQ Update Program in Psychiatry: Cycle 3. Porto Alegre: Artmed / Panamericana; 2013. p. 9-27. (System of Continuing Medical Education at a Distance, v. 2).
• ⁵
  Gonçalves LL. Digital dependence: technologies transforming people, relationships and organizations. Bar Books, RJ, 2017.
• ⁶
  King ALS, Guedes E, Nardi AE. Digital Label. Porto Alegre: EducaBooks, 2017.
• ⁷
  Guimarães LAM, Veras AB (Organizers). Psychic health and work. Cambridge: Cambridge University Press.
• ⁸
  King ALS, Valença AM, Silva AC, Sancassiani F, Machado S, Nardi AE. Nomophobia: Impact of Cell Phone Use Interfering with Symptoms and Emotions of Individuals with Panic Disorder Compared with a Control Group. Clin Pract Epidemiol Ment Health. 2014; 10: 28-35. DOI: 10.2174/1745017901410010028
  » https://doi.org/10.2174/1745017901410010028
• ⁹
  King ALS, Valencia AM, Nardi AE. Nomophobia: The Mobile Phone in Panic Disorder With Agoraphobia Reducing Phobias or Worsening of Dependence? Cogn Behav Neurol. 2010; 23 (1): 52-4. DOI:10.1097/WNN.0b013e3181b7eabc
  » https://doi.org/10.1097/WNN.0b013e3181b7eabc
• ¹⁰
  King ALS; Valença AM; Silva ACO; Baczynski T; Carvalho MR; Nardi AE. Nomophobia: dependence on virtual environments or social phobia? Comp Behav. 2013; 29 (1): 140-4. DOI: 10.1016/j.chb.2012.07.025
  » https://doi.org/10.1016/j.chb.2012.07.025
• ¹¹
  IAT-The Center for Internet Addiction Recovery, Copyright 2009-2010 by The Center for Internet Addiction; Web Site Designed by Next Sunrise Studios, Bradford PA.
• ¹²
  R Core Team (2017). A: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/
  » https://www.R-project.org/
• ¹³
  Hadley Wickham, Romain Francois, Lionel Henry and Kirill Müller (2017). dplyr: A Grammar of Data Manipulation. R package version 0.7.4. https://CRAN.R-project.org/package=dplyr
  » https://CRAN.R-project.org/package=dplyr
• ¹⁴
  Falissard B. (2012). psy: Various procedures used in psychometry. R package version 1.1. https://CRAN.R-project.org/package=psy
  » https://CRAN.R-project.org/package=psy
• ¹⁵
  Dinno A (2012). paran: Horn's Test of Principal Components / Factors. R package version 1.5.1. https://CRAN.R-project.org/package=paran
  » https://CRAN.R-project.org/package=paran
• ¹⁶
  HAIR et al. Fundamentals of research methods in administration. Translation: Lene Belon Ribeiro. Porto Alegre: Bookman, 2005.

Publication Dates

• Publication in this collection
  04 July 2019
• Date of issue
  2019

History

• Received
  03 Aug 2018
• Reviewed
  15 Aug 2018
• Accepted
  04 Sept 2018
• Accepted
  17 Apr 2019