Research Reporting Guidelines in Dentistry: A Survey of Editors

Abstract

The use of reporting guidelines has an important role in the development of health research, improving the quality and precision of the publications. This study evaluated how dental journals use reporting guidelines. All editors of dental journals registered on the 2013 Journal Citation Reports list (n=81) were invited to participate. The data were collected by a self-reported web-based questionnaire. Information about the profile of journal/editor and on the use of reporting guidelines by journals was gathered. Information/recommendations about the use of reporting guidelines were collected from the websites of all journals. Data were descriptively analyzed and frequencies were summarized. Thirty-four (42%) editors completed the questionnaire. Most journals are members of Committee on Publication Ethics (64.7%) and/or follow the International Committee of Medical Journal Editors recommendations (20.6%), while 26.5% are not members of any editorial group. Most editors are unfamiliar with the EQUATOR Network (55.9%), do not work full time (85.3%) and 88.2% have some income/payment. Most of them received educational training for this position (55.9%). The CONSORT Statement was endorsed by 61.8% of journals. Information from websites showed that 44.4% journals do not recommend any reporting guideline, 51.9% mention CONSORT Statement in the website and 28.4% only recommend the use of CONSORT Statement. There is clearly room for improving the use of reporting guidelines in dental journals. Broadening the understanding and the endorsement/adherence/implementation of reporting guidelines by journals may promote quality and transparence of published dental research.

Key Words:
guideline as topic; evidence-based dentistry; publishing

Resumo

O uso de guias de reporte tem um papel importante no desenvolvimento das pesquisas na saúde, melhorando a qualidade e a precisão das publicações. Esse estudo avaliou como periódicos de odontologia usam os guias de reporte. Todos editores de periódicos de odontologia registrados na lista do 2013 Journal of Citation Reports (n=81) foram convidados a participar. Dados foram coletados através de um questionário online autoaplicável. Informações sobre o perfil do periódico/editor e do uso de guias de reporte pelos periódicos foram coletados. Informações/recomendações sobre o uso de guias de reporte foram também coletados dos sites de todos os periódicos. Dados foram analisados descritivamente e frequências foram sumarizadas. Trinta e quatro (42%) editores completaram o questionário. Maioria dos periódicos é membro do Committee on Publication Ethics (64,7%) e/ou seguem as recomendações do International Committee of Medical Journal Editors (20,6%), enquanto 26,5% dos periódicos não são membros de nenhum grupo editorial. A maioria dos editores não é familiar com o EQUATOR Network (55,9%), não trabalha em tempo integral como editor (85,3%) e 88,2% recebem algum pagamento. A maioria deles recebeu treinamento para a posição de editor (55,9%). O CONSORT Statement foi endossado por 61,8% dos periódicos. Informações oriundas dos sites demonstraram que 44,4% dos periódicos não recomendam nenhum guia de reporte, 51,9% mencionam o CONSORT Statement no site e 28,4 apenas recomenda o CONSORT Statement. Existe um espaço claro para melhora no uso de guias de reporte em periódicos de odontologia. Um maior entendimento e endosso/aderência/implementação de guias de reporte por periódicos de odontologia pode promover a qualidade e transparência das pesquisas odontológicas publicadas.

Introduction

The problems with reporting health research and possible consequences have been pointed out in the literature ^1,2.^.Chalmers and Glasziou ³ suggested that at least half of published researches present low quality or insufficient information, wasting ten billion pounds. The completeness and transparency of reporting is necessary to allow reviewers and readers to make a correct judgment about the quality and risk of bias of these studies ¹. A recent study revealed significant growth of dental literature, the total number of publications more than doubled in the last years including all types of articles ⁴.Several papers were published discussing problems of reporting in various areas of dental research ^5,6,7,8,9 and the use of reporting guidelines could improve the quality and transparency to reports on oral health research ¹⁰.

A series of papers published by The Lancet discusses 17 recommendations to reduce waste in health research, including waste from incomplete or unusable reports. One suggestion of the authors is that a better understanding of initiatives as the EQUATOR network and the active use of reporting guidelines supported by that initiative (i.e. CONSORT, PRISMA, STROBE) could improve this situation ^1,3,11,12,13.The literature has been also suggesting that authors and reviewers should improve the quality of research reporting by training in topics such as reporting guidelines, publication ethics and research integrity ^1,14.

The present study evaluated the profile of dental journals and how dental journals use reporting guidelines by the actions of journals related to the use of the following guidelines: Consolidated Standards of Reporting Trials (CONSORT) ¹⁵, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) ¹⁶ and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) ¹⁷ statements. In addition, it verified the information/recommendations about the use of reporting guidelines by journals from the journals' websites and compared with the answers of editors.

Material and Methods

In this cross sectional study all editors-in-chief of dental journals, i.e., journals on Dentistry, Oral Surgery & Medicine registered on the 2013 Journal Citation Reports (JCR) list (n=81) were invited to participate. E-mail addresses of the editors-in-chief were obtained from the journals' websites or from recent publications of editors. Data were collected by a self-reported web-based questionnaire developed using Google forms (Google Inc. Mountain View, CA, USA) between June and September 2015. The following information was obtained from the questionnaire: impact factor and content area of the journal, membership of Committee on Publication Ethics (COPE) or any editorial group, familiarity with the EQUATOR Network, occupation as a full-time editor of the journal, time length as editor, editor income/payment for the position, educational training for the position, information about the endorsement and implementation of CONSORT ¹⁵, PRISMA ¹⁶ and STROBE ¹⁷ and adherence of other reporting guidelines.

First, a pilot questionnaire was tried out with three dental journal editors not included in JCR 2013. The data collected did not include any information that could identify the editor or the journal. The questionnaire was sent individually by e-mail for each editor. A reminder was sent if the editor did not return the questionnaire after 3 weeks. After four attempts, if the editor did not return, his/her participation was eliminated. All respondents read the terms of study and agreed to participate (Local research and ethics committee approval #44345815.0.0000.5317).

Information/recommendations about the use of reporting guidelines by journals and impact factor of journal were also collected directly from the journals' websites by a member of the research team. Data were descriptively analyzed and frequencies were summarized using Excel program (Microsoft Office; Albuquerque, NM, USA).

Results

Among the 81 editors invited to participate in the study, 34 (42%) returned the complete questionnaire. Results about profile of dental journals are presented in Table 1. Results about the use of reporting guidelines by journals are presented in Table 2.

Table 1
Profile of dental journals (n=34)

• # IQR: Interquartile Range, $Journals following the ICMJE , Recommendations, *Authors could check all responses that apply

Table 2
Use and actions taken by journals related to the use reporting guidelines

• *Authors could check all responses that apply

The median of impact factor of respondent journals was 1.4 (IQR=1.076-2.025). The most prevalent content subarea the papers published in the journals was orthodontics (n=11; 32.4%). Most journals are members of Committee on Publication Ethics (n=22; 64.7%) but not familiar with EQUATOR Network (n=19; 55.9%). Considering the editorial teams, 26.5% are not members of any editorial team, whereas 20.6% followed the International Committee of Medical Journal Editors (ICMJE) recommendations. Most editors do not work full time (85.3%) and the majority of editors (n=30; 88.2%) receive some income/payment to carry out their functions as editor-in-chief. Furthermore, more than half underwent some sort of educational training for this position (n=19; 55.9%).

Most of the journals (n=25; 73.5%) advise the peer reviewers to use reporting guidelines as part of their review assessment. The CONSORT Statement was endorsed by 61.8% (n=21) journals. Among these 21 journals, 66.7% (n=14) answered that authors are required to follow the CONSORT recommendations and checklist in the instructions to authors, and 57.1% (n=12) require authors to submit the completed checklist, which is reviewed by the editorial team. More than half the journals (n=18; 52.9%) endorsed PRISMA Statement. Out of these 18 journals, 66.7% (n=12) answered that authors are required to follow the PRISMA recommendations and checklist in the instructions to authors, and 61.1% (n=11) require submission of the completed checklist with editorial team review. Most journals (70.6%) do not endorse the STROBE Statement.

Figure 1 summarizes the results of information/recommendations about the use of reporting guidelines by the journals, collected from their websites. Thirty-six journals (44.4%) did not advise any reporting guideline; these journals were classified with the lowest impact factors in Dentistry (n=21; 58.3%). Forty-two journals (51.9%) mention the CONSORT Statement in their website and 23 (28.4%) journals only advise the use of CONSORT Statement. Five journals mentioned the EQUATOR Network library of reporting guidelines including the Journal of Dental Research that currently has the highest impact factor in dentistry (IF=4.139).

Figure 1
Number of journals endorsing each initiative

The results indicate that most journals are not familiar with EQUATOR Network (55.9%). This information corroborates the information collected from journals' websites: only 5 journals mention this initiative. In both the survey and the websites, the CONSORT Statement was the most endorsed initiative.

Discussion

Reporting guidelines aim to improve the completeness, quality and transparency of research with specific guidance for a broad range of study designs and types of data. Use of reporting guidelines has been associated with improvements in the completeness of reporting. As such, reporting guidelines might be an effective tool to help reduce waste and increase the value of research ^1,18,19. This is the first survey and evaluation of instructions to authors developed in dental research to evaluate the profile of dental journals and actions taken by these journals related to reporting guidelines.

It is encouraging that from the present results, most of dental journals endorsed or mention in their instructions to authors the CONSORT Statement ¹⁵, since in a recent survey evaluating the high impact factor medical journals, 63% journals refer to CONSORT Statement in the instructions to authors ^20,21. Some dental journals (44.4%), however, do not refer/endorse any reporting guideline, confirming that reporting guidelines remain much less used/endorsed than they should be.

A better understanding of the profile of editors and journals is important since they have a key role to ensure that articles published are as transparent as possible and with complete details ²². Editorial teams, ethics committee and initiatives as EQUATOR Network are also critical to help editors and journals to guarantee the transparency of article journals. However, in dentistry, there is a huge number of journals not members of any editorial team or member of the Committee on Publication Ethics and are still not familiar with the EQUATOR Network. These initiatives can provide resources with important information about ethics, publications and journalology for editors, since many dental and medical editors are untrained and uncertified.

A recent article discusses proposals helping to improve reporting the medical research literature ¹⁴ and one highlighted topic is the importance to develop core competencies especially by training editors and peer reviewers. The present results suggest that in dentistry, many editors (44.1%) have not received any educational training for the position of editor and still 26.5% dental journals do not recommend peer reviewers to use reporting guidelines as part of their review.

Hirst and Altman ²³ evaluated the use of reporting guidelines for the peer-reviewers of submitted manuscripts of 116 health research journals. Their findings demonstrated low percentage of journals mentioning reporting guidelines in instructions to peer-reviewers whereas the present results demonstrated that most dental journals (73.5%) recommend peer reviewers to use reporting guidelines as part of their review assessment. A possible reason of this increase could be a series of papers published after 2012 recommending and encouraging journals to use reporting guidelines during peer review process ¹. Also, most respondents are members of some editorial team that could recommend this use of reporting guidelines. However, there are few studies evaluating this topic as the randomized trial by Cobo et al. ²⁴ in which the authors found that manuscripts reviewed using reporting guidelines presented better quality than articles reviewed without reporting guidelines. Examples like the BMC Oral Health Journal, where peer reviewers are asked to refer to checklists when evaluating. Such studies should be encouraged in oral health research community. Also, a more active training of peer reviewers on how to use guidelines and checklists would improve the oral health science directly and indirectly.

One important topic evaluated in the present study is the endorsement and implementation of reporting guidelines by journals. The results of this study show that by far the CONSORT Statement is the most endorsed and most extensively evaluated initiative, as also pointed out for medical journals ^18,19. Results of this survey showed that the PRISMA Statement is endorsed by 52.9% journals. However, with the increased number of systematic reviews published in the last few years, more journals should endorse PRISMA statement ¹⁶ for the same above mentioned reasons considering the CONSORT Statement. In contrast, two important findings are worrying: 1) other initiatives are far less endorsed than CONSORT and only 22.2% of dental journals referred 2 or more initiatives in their instructions to authors and 2) 44.4% of journals do not refer any initiative in the instructions to authors at all.

Findings of a recent systematic review ^18,19 are clearly showing the positive impact of the endorsement of reporting guidelines on the completeness of study reporting. Yet, it seems apparent that this process of endorsement is a passive process, the journals merely advising adherence to key guidelines by submitting a completed checklist. A recent article proposed actions and potential benefits for supporting adherence to PRISMA-P (Preferred Reporting Items For Systematic Review and Meta-Analysis Protocols) by stakeholders including journal editors and one of the actions is to encourage compliance to PRISMA-P for authors submitting protocols for publication and offer PRISMA-P as a template to assist in protocol writing for publication ²⁵.

Results of the conducted survey showed that in most journals that endorse the CONSORT and PRISMA Statements, authors are required to follow recommendations and checklist in the manuscript guidelines of the journal and submit the complete checklist to be reviewed by the editorial team. In contrast, in many journals authors are asked to use the checklist, but no action is taken if it is not used. The American Journal of Orthodontics and Dentofacial Orthopedics, in order to help authors understand and apply the standards, prepared a separate document about the CONSORT and PRISMA Statements. It also showed the benefits of a recent active implementation of reporting guidelines in this journal, where the submission process and the editorial team are actively using the CONSORT statement to guide the RCT manuscript authors from initial submission throughout the whole process (preview to peer review process). The results demonstrated a significant improvement in reporting of RCTs after the assessment ²⁶. This approach should be encouraged and extended to other journals.

One limitation of our survey is that despite the four attempts to obtain the editor's reaction, there was a relatively low response rate (42%). Still, this response rate concurs with a previous survey that evaluated the CONSORT endorsement by high impact factor medical journals, with a 39% response rate ²⁰. Another limitation is that only 7 journals with impact factor higher than 2.0 answered the survey.

Results of this study evidence that reporting guidelines are used less than ideal in dentistry and in consequence the quality of reporting oral health research remains suboptimal. Here are some suggestions to improve these problems:

1. 1. Editors should be trained for the position;

2. 2. Authors should be trained to make research articles fit for purpose and to use reporting guidelines;

3. 3. An active implementation of reporting guidelines by journals is encouraged;

4. 4. Journals should recommend peer reviewers to use reporting guidelines as part of their review and should be trained for their use;

5. 5. Journalology should be included into the training curriculum of universities.

In conclusion, the use of reporting guidelines has an important role in the development of oral health research but their use is suboptimal. Thus, without a broad understanding of the edorsement/adherence/implementation of reporting guidelines it is difficult to achieve the benefits for which those guidelines were developed.

Acknowledgements

We thank Coordenação de Aperfeiçoamento de Pessoa do Nível Superior (PDSE-CAPES, 99999.014351/2013-07) for the scholarship provided for the first author. We also thank Thais Piccolo Carvalho for collecting the emails of editors and the journal editors who kindly responded our request. Dr. Moher is supported by University of Ottawa Research Chair. The authors declare that there were no funders in study conception or design, data acquisition and analysis, decision to publish, or preparation of the article. The authors declare no potential conflicts of interest with respect to the authorship and/ or publication of this article.

References

• ¹
  Glasziou, P; Altman, DG; Bossuyt, P; Boutron, I; Clarke, M; Julious, S; et al.. Reducing waste from incomplete or unusable reports of biomedical research. Lancet2014;383:267-276.
• ²
  Simera, I; Altman, DG. Writing a research article that is "fit for purpose": EQUATOR Network and reporting guidelines. Evid Based Med 2009;14:132-134.
• ³
  Chalmers, I; Glasziou, P. Avoidable waste in the production and reporting of research evidence. Lancet2009;374:86-89.
• ⁴
  Jayaratne, YS; Zwahlen, RA. The evolution of dental journals from 2003 to 2012: a bibliometric analysis. PLoS One 2015;10:e0119503.
• ⁵
  Cairo, F; Sanz, I; Matesanz, P; Nieri, M; Pagliaro, U. Quality of reporting of randomized clinical trials in implant dentistry. A systematic review on critical aspects in design, outcome assessment and clinical relevance. J Clin Periodontol 2012;39:81-107.
• ⁶
  Jokstad, A; Esposito, M; Coulthard, P; Worthington, HV. The reporting of randomized controlled trials in prosthodontics. Int J Prosthodont 2002;15:230-242.
• ⁷
  Kloukos, D; Papageorgiou, SN; Doulis, I; Petridis, H; Pandis, N. Reporting quality of randomised controlled trials published in prosthodontic and implantology journals. J Oral Rehabil 2015;42:914-925.
• ⁸
  Montenegro, R; Needleman, I; Moles, D; Tonetti, M. Quality of RCTs in periodontology - a systematic review. J Dent Res 2002;81:866-870.
• ⁹
  Savithra, P; Nagesh, LS. Have CONSORT guidelines improved the quality of reporting of randomised controlled trials published in public health dentistry journals? Oral Health Prev Dent 2013;11:95-103.
• ¹⁰
  Sarkis-Onofre, R; Cenci, MS; Demarco, FF; Lynch, CD; Fleming, OS; Pereira-Cenci, T; et al.. Use of guidelines to improve the quality and transparency of reporting oral health research. J Dent 2015;43:397-404.
• ¹¹
  Al-Shahi, Salman R; Beller, E; Kagan, J; Hemminki, E; Phillips, RS; Savulescu, J; et al.. Increasing value and reducing waste in biomedical research regulation and management. Lancet2014;383:176-185.
• ¹²
  Chan, AW; Song, F; Vickers, A; Jefferson, T; Dickersin, K; Gotzsche, PC; et al.. Increasing value and reducing waste: addressing inaccessible research. Lancet2014;383:257-266.
• ¹³
  Ioannidis, JP; Greenland, S; Hlatky, MA; Khoury, MJ; Macleod, MR; Moher, D; et al.. Increasing value and reducing waste in research design, conduct, and analysis. Lancet2014;383:166-175.
• ¹⁴
  Moher, D; Altman, DG. Four proposals to help improve the medical research literature. PLoS Med 2015;12:e1001864.
• ¹⁵
  Schulz, KF; Altman, DG; Moher, D; CONSORT, Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ2010;340:c332.
• ¹⁶
  Moher, D; Liberati, A; Tetzlaff, J; Altman, DG; Group, P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ2009Jul;339:b2535.
• ¹⁷
  Von, Elm, E; Altman, DG; Egger, M; Pocock, SJ; Gotzsche, PC; Vandenbroucke, JP; et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Med 2007;4:e296.
• ¹⁸
  Turner, L; Shamseer, L; Altman, DG; Schulz, KF; Moher, D. Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Syst Rev2012;1:60.
• ¹⁹
  Turner, L; Shamseer, L; Altman, DG; Weeks, L; Peters, J; Kober, T; et al.. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev 2012;11:MR000030.
• ²⁰
  Hopewell, S; Altman ,DG; Moher, D; Schulz, KF. Endorsement of the CONSORT Statement by high impact factor medical journals: a survey of journal editors and journal 'Instructions to Authors'. Trials;9:20.
• ²¹
  Shamseer, L HS; Altman, DG; Moher, D; Schulz, KF. Update on the endorsement of CONSORT by high impact factor journals: a survey of journal "Instructions to Authors" in 2014. Trials2016;17:301.
• ²²
  World Medical A. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA2013;310:2191-2194.
• ²³
  Hirst, A; Altman, DG. Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals. PLoS One 2012;7:e35621.
• ²⁴
  Cobo, E; Cortes, J; Ribera, JM; Cardellach, F; Selva-O'Callaghan, A; Kostov, B; et al.. Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. BMJ2011;343:d6783.
• ²⁵
  Moher, D; Shamseer, L; Clarke, M; Ghersi, D; Liberati, A; Petticrew, M; et al.. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1
• ²⁶
  Pandis, N; Shamseer, L; Kokich, VG; Fleming, PS; Moher, D. Active implementation strategy of CONSORT adherence by a dental specialty journal improved randomized clinical trial reporting. J Clin Epidemiol 2014;67:1044-1048.

Publication Dates

• Publication in this collection
  12 Dec 2016
• Date of issue
  Jan-Feb 2017

History

• Received
  14 Sept 2016
• Accepted
  24 Nov 2016