Intra and interobserver concordance of the AO classification system for fractures of the long bones in the pediatric population^☆

ABSTRACT

OBJECTIVE:

The AO classification for fractures of the long bones in the pediatric population was developed and validated in 2006. However, the complexity of this system has limited its use in clinical practice and few studies in the literature have evaluated its reproducibility and applicability. The present study had the objective of determining the intra and interobserver agreement using the pediatric AO system, among physicians with different levels of experience.

METHODS:

After making the sample calculation, 108 consecutive radiographs on long-bone fractures in patients aged 0-16 years, coming from the digital files of the quaternary-level hospital, were selected. The radiographs were classified by five examiners with different levels of experience after prior explanations about the system. A chart containing images from the classification was made available for consultation. The evaluations were made at two different times by each observer. The Fleiss kappa index was used to ascertain the intra and interobserver agreement.

RESULTS:

Intraobserver agreement that was at least substantial was obtained for all the items of the classification and it reached excellent levels for all observers in relation to five of the seven items considered. The interobserver evaluation presented excellent levels of agreement in two items, substantial in two items, moderate to substantial in one item and poor to moderate in one item. No influence from the observer's experience was observed with regard to obtaining higher or lower levels of agreement, either in the intraobserver or in the interobserver evaluation.

CONCLUSIONS:

In this study, the intra and interobserver agreement was considered to be good or excellent for the pediatric AO classification system, for the parameters of bone, segment, paired bone, subsegment, standard and deviation. However, the intra and interobserver agreement was statistically unsatisfactory for the parameter of severity/side of avulsion. The levels of agreement obtained did not depend on the observer's level of experience within pediatric orthopedics.

Keywords:
Bone fractures/classification; Child; Orthopedics; Method

resumo

Objetivo:

Aclassificação AO para fraturas dos ossos longos na população pediátrica foi desen volvida e validada em 2006. Entretanto, a complexidade desse sistema tem limitado o seu uso na prática clínica. Poucos estudos na literatura avaliam sua reprodutibilidade e aplicabi lidade. Este trabalho teve como objetivo determinar a concordância intra e interobservadores com o uso do sistema de classificação AO pediátrica entre médicos de diferentes níveis de experiência.

Métodos:

Após a feitura do cálculo amostral, foram selecionadas 108 radiografias consecuti vas de fraturas de ossos longos de pacientes de 0-16 anos, provenientes do arquivo digital de um hospital de nível quaternário. As radiografias foram classificadas por cinco examina dores com diferentes níveis de experiência após uma explicação prévia sobre o sistema. Foi mostrada uma planilha que continha as imagens da classificação para consulta. As avaliações foram feitas em dois momentos distintos por cada observador. O índice Kappa de Fleiss foi usado para verificar a concordância intra e interobservadores.

Resultados:

Foram obtidas concordâncias intraobservadores no mínimo substanciais em todos os itens da classificação, alcançaram níveis excelentes por todos os observadores em cinco dos sete itens considerados. A avaliação interobservadores apresentou níveis de concordância excelentes em dois itens, substancial em dois itens, moderada a substancial em um item e pobre a moderada em um dos itens. Não se observou influência da experi ência do observador na obtenção de maiores ou menores níveis de concordância, intra ou interobservadores.

Conclusões:

Neste estudo, a concordância intra e interobservadores foi considerada boa ou excelente para o sistema de classificação AO pediátrico para os parâmetros: osso, seg mento, osso pareado, subsegmento, padrão e desvio. No entanto, a concordância intra e interobservadores foi estatisticamente insatisfatória no parâmetro gravidade/lado da avulsão. Os níveis de concordância obtidos independem da experiência do observador em ortopedia pediátrica.

Palavras-chave:
Fraturas ósseas/classificação; Crianca; Ortopedia Método.

Introduction

The main reason for hospitalization within pediatric orthopedics is fractures of the long bones.1Classification of the fractures is essential for determining the epidemiology, facilitating communication between orthopedists and defining treatment algorithms.2Several classification systems have been developed based on the location and morphology of the injuries, in order to categorize each type of injury of long bones in children.1

The AO classification for long-bone fractures in adults is not used for the pediatric population because it does not take into consideration the bone elasticity, presence of the growth plate and anatomical characteristics of the epiphysis.1The same trauma mechanism may produce different fracture patterns in children, such as plastic deformities, greenstick fractures and complex fractures.1Another important characteristic is the greater fragility of the growth plate, which is less resistant than the surrounding bone, thus meaning that this structure is more easily injured.

Any orthopedic classification system needs to be clinically relevant, reproducible and valid. To meet these objectives, the system needs to go through three investigative stages, as proposed by Audigé et al2In the case of pediatric fractures, the first stage should involve experienced pediatric orthopedists, in order to define a common language for describing the fracture patterns and classification process. The second stage relates to developing international multicenter agreement studies that involve surgeons with different levels of experience. The third stage relates to implementation of a prospective clinical study.3

The pediatric AO classification takes into consideration the AO system for long-bone fractures in adults and the most relevant pediatric fractures. The location of the fracture and its morphology are taken into consideration. The bone is subdivided into three segments: proximal (epiphysis + metaphysis), diaphyseal and distal (epiphysis + metaphysis). Regarding morphology, the disease code for the child and the fracture severity and displacement, which depend on the type of fracture, are considered.1

The authors of the pediatric AO classification have already reached the third stage of the validation process, i.e. application of the proposed system within the context of a prospective clinical study.2However, the degree of complexity of this method and the difficulty in incorporating it into clinical practice lead us to believe that studies evaluating its reproducibility and accuracy are still needed, especially if less experienced orthopedists are taken into consideration.

Thus, we conceived this study with the aim of estimating the intra and interobserver agreement of the AO classification system for long bones in children, among examiners with different levels of experience.

Materials and methods

This research project was submitted to the research ethics committee of the Brazil Platform for assessment and approval (approval number: 29073114.3.0000.5505).

Sample calculation

Firstly, we determined the number of radiographs that would be needed to obtain kappa values greater than 0.70, through tests with a significance level of 5% and power of 80%. The calculation showed that we would need to evaluate at least 95 radiographs. The formula used for this calculation was as follows4:

In which z(alpha) and z(beta) are obtained from the normal distribution; Q₀ and Q₁ are obtained from the table of the reference article for the sample size; and K₁ and K₀ are the kappa values obtained from the hypotheses of the test.For this analysis, we obtain:

Sample selection

These examinations were obtained consecutively between January 2013 and March 2014 in the imaging diagnostics department of a quaternary-level university hospital, with prior authorization. All the radiographs produced during this period that were identified in the digital files as images of segments of the appendicular skeleton were obtained for evaluation. These segments included the pelvis, thigh, knee, lower leg, ankle, shoulder, upper arm, elbow, forearm and wrist. Examinations performed on children aged 0-16 years who presented fracture of the long bones were included. The radiographs were selected so as to include examinations with two views and good radiographic quality. This was done by two orthopedists who did not participate in the classification process. Thus, 119 radiographs on fractures of the long bones were collected, in anteroposterior and lateral views. Among these, six were excluded due to poor quality and five because the growth plate had already closed. The study therefore included 108 radiographs.

Process of classifying the radiographs

The radiographs were classified by five examiners with different levels of experience. One was at expert level (>10 years of experience as a pediatric orthopedist - examiner 5), one was at advanced level (>5 years of experience as a pediatric orthopedist - examiner 4), one was at medium level (>1 year of experience as a pediatric orthopedist - examiner 3) and two were at basic level (general orthopedists - examiners 1 and 2).

With the aim of minimizing bias due to difficulties in interpretation and inexperience with the classification system, the observers were given prior explanations regarding the classification systems used. Furthermore, during the classification process, a brochure containing the entire AO classification for pediatric long-bone fractures was available for each participant.

The radiographs were organized in chronological order in a closed digital file. The classifications were made by five observers, at two different times, with an interval of 15 days between one evaluation and the other. Each of the five researchers evaluated and classified the radiographs independently. The observers were given all the time that they needed to evaluate the radiographs.

The participants were instructed not to discuss the classification systems until the end of the classification stage. Furthermore, they did not have access to the patients' histories or to any clinical data.

Statistical analysis

The statistical analysis on the results obtained was performed by a specialist professional in the field of medical statistics. The Fleiss kappa test was used to assess the intra and interobserver agreement for each scale.5^and6It is considered that using the Fleiss kappa coefficient is the most appropriate method for situations in which multiple examinations or evaluations are made and when the scale evaluated presents several categories.7

The tests were interpreted in accordance with Altman8as "proportional agreement with correction of random occurrences". The kappa agreement coefficient has values ranging from +1 (perfect agreement), through 0 (agreement equal to chance) to -1 (complete discordance). There are no definitions regarding the agreement levels that are accepted, but some studies have suggested that results in the range of 0-0.2 show very low agreement, 0.21-0.40 poor agreement, 0.41-0.60 moderate agreement and 0.61-0.80 substantial agreement. Values greater than 0.80 are considered to be practically perfect agreement.4 7 8^and9

Fracture classification system

The overall structure of the classification is based on the location of the fracture and its morphology. The fracture locations covered are the different long bones and their respective segments and subsegments. The morphology of the fracture is described by a specific code that represents the fracture pattern, with a code for the severity and an additional code that is used for certain types of fractures (displaced supracondylar fractures of the humerus, displaced fractures of the head and neck of the radius and fractures of the femoral neck).10

The numbering system for the long bones (1-4) and for the segments (proximal = 1, diaphyseal = 2 and distal = 3) is similar to that of the AO system described by Müller for fractures of the long bones in adults.11It differs in relation to the coding for malleolar fractures, such as fractures of the distal tibia or fibula. Moreover, the definitions of the three bone segments differ from those of adults. The letters R, U, T and F refer to the radius, ulna, tibia and fibula and are added to the code for the segment, in relation to paired bones, when only one bone is fractured or when both bones are fractures but with different patterns.10

With regard to the subsegments, segments 1 and 3 are subdivided into two subsegments: the epiphysis (E) and the metaphysis (M). Segment two is the same as the diaphyseal subsegment (D).10

The metaphysis is defined as a square in which the sides have the same length as the widest part of the growth place. In relation to paired bones such as the radius/ulna and tibia/fibula, both bones should be included in the square. The proximal femur is an exception: its metaphysis is not defined as a square but is located between the growth plate and the subtrochanteric line.10

If the center of the fracture line is located inside the abovementioned square, it is a metaphyseal fracture. If the epiphysis and the respective growth plate (physis) are included, it is an epiphyseal fracture. Intra and extra-articular ligament avulsions are epiphyseal and metaphyseal injuries, respectively.10

A certain number of fracture patterns that are important in children are described by the so-called "child code". These fracture patterns are specific for the subsegments in which they are located and thus are grouped as E, M or D. This code also takes into consideration some internationally accepted classification systems for pediatric fractures (such as the classification of Salter-Harris).3 10^and12

The severity code distinguishes between two grades: simple (.1) and multifragmented (.2). To describe the side of the avulsion, when necessary, the letter M would indicate medial ligament avulsion and the letter L, lateral.

Supracondylar fractures of the humerus, which are classified as 13-M/3, are described using an additional code that takes into consideration the degree of displacement (I-IV), which is very similar to the classification of Gartland.13

When the paired bones (radius/ulna or tibia/fibula) both present the same fracture pattern, they should be documented by only one classification code. In this case, the severity code will be that of the bone that is more severely fractured. On the other hand, when only one bone is fractured, a lower-case letter defines this bone (r, u, t or f) and should be added to the code for the segment. For example, 22u describes a diaphyseal fracture of the ulna in isolation. Furthermore, when the two bones are fractured with different fracture patterns, each fracture should be classified separately and a lower-case letter should be included in the classification. For example, a complete spiral fracture of the radius and plastic deformity of the ulna are classified as 22r-D/5.1 and 22u-D/1.1.10

Fractures of the head and neck of the radius are described by an additional code (I-III) that takes into account the angle and grade of displacement. Fractures of the femoral neck are proximal metaphyseal fractures (M), with an intertrochanteric line that limits the metaphysis. These metaphyseal fractures can be divided into three types, which are represented by an additional code (I-III) that takes into account the position of the fracture in the proximal metaphysis: transcervical, basicervical and transtrochanteric.10

Results

Intraobserver agreement

The data relating to the statistical evaluation on intraobserver agreement and the respective results according to the Fleiss kappa index are shown inTable 1,Table 2,Table 3,Table 4,Table 5,Table 6andTable 7. Each item that forms part of the classification was analyzed independently and is presented in a specific table.

Table 1:
Statistical analysis on intraobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the bone. CI, confidence interval.

Table 2:
Statistical analysis on intraobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the segment. CI, confidence interval.

Table 3:
Statistical analysis on intraobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the paired bone. CI, confidence interval.

Table 4:
Statistical analysis on intraobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the subsegment. CI = confidence interval.

Table 5:
Statistical analysis on intraobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the pattern. CI, confidence interval.

Table 6:
Statistical analysis on intraobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the severity and side of the avulsion. CI = confidence interval.

Table 7:
Statistical analysis on intraobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the displacement. CI, confidence interval.

In general terms, substantial correlation of agreement was found in relation to practically all the items addressed in the classification. Excellent agreement levels were obtained by all the observers in relation to the items bone, segment, paired bone, subsegment, pattern and displacement. On the other hand, the severity and side of avulsion presented substantial agreement for three observers and excellent for the other two. Lastly, it was seen that greater observer experience did not necessarily imply a higher level of agreement.

Interobserver agreement

Table 8,Table 9,Table 10,Table 11,Table 12,Table 13andTable 14show the results from the Fleiss kappa index relating to the interobserver analysis on the first and second assessments by the examiners involved in this study.

Table 8:
Statistical analysis on interobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the bone. CI, confidence interval.

Table 9:
Statistical analysis on interobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the segment. CI, confidence interval.

Table 10:
Statistical analysis on interobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the paired bone. CI, confidence interval.

Table 11:
Statistical analysis on interobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the subsegment. CI, confidence interval.

Table 12:
Statistical analysis on interobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the pattern. CI, confidence interval.

Table 13:
Statistical analysis on interobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the severity and side of the avulsion. CI, confidence interval.

Table 14:
Statistical analysis on interobserver agreement according to the Fleiss kappa index, described for each examiner and for each of the parameters analyzed in the pediatric AO classification: in this table, the displacement. CI, confidence interval.

The interobserver agreement index was considered to be excellent for the items of bone and segment and substantial for the items of paired bone and subsegment. The item pattern showed moderate agreement only for one of the observers in comparison with the others, excellent for two other examiners and substantial agreement in the correlation between the remaining observers. Lastly, the item of severity and side of the injury was the one that presented greatest disparity of results. It reached an excellent agreement index only in the comparative analysis between two of the observers, while the others ranged from poor to moderate, at most. Once again, the results do not allow any correlation between the agreement levels obtained and the observers' experience.

Discussion

The pediatric AO classification is a relatively new method for grouping and standardizing the descriptions of different types of long-bone fractures in children. In the orthopedic literature, only a very limited number of studies have addressed this topic. This stimulated our group to conduct the present study, with the aim of assessing the applicability and reproducibility of this system within our setting.

An ideal classification system should conform to very well defined criteria, such as being easy to apply, being highly reproducible, having high accuracy, being capable of adequately guiding the treatment and being capable of indicating the prognosis for the injuries.2 14 15 16^and17In addition, an ideal classification should enable comparisons between the results obtained from different series, and should allow better documentation of epidemiological data.

The AO group put forward a systematic method that covered all long-bone injuries in children and used Müller's classification for adults as its basis. This method is based on an alphanumeric system and aims to categorize the main descriptive elements of these fractures, such as their location and type.11This classification was validated in a study published by Slongo et al.,³ and started to be used in studies conducted by the authors who conceptualized it.

Until then, each body segment of the immature skeleton had been studied in isolation. The classification of each body segment of the immature skeleton was studied separately and the classifications of the different types of fracture were determined by authors with particular interest in each of the regions studied. We observed that for this reason, there was a large number of classifications for childhood and adolescence, guided by different criteria. For example, we can cite the systems ofPoland (1898),18 Bergenfeldt (1933),19 Aitken (1936)20 Salter e Harris (1963)12ePeterson (1994)21for growth plate injuries. We are aware that this multiplicity of classification methods is found for fractures of a variety of segments of the immature skeleton. However, Slongo et al.2emphasized that almost none of these systems have been subjected to proper validation for subsequent clinical application.

Independent of the classification method, it is ideally expected that there should be a high level of agreement among the professionals who use these methods. We observed in our study that for the variables of severity and pattern in the AO classification system for children, the level of agreement achieved was lower among some of the examiners. For the variable of pattern, there are nine subtypes for the length of the epiphysis and seven for the length of the diaphysis. Therefore, we take the view that the large number of options for each of these variables allows each examiner to have more choices that can be made, and that this is independent of the expertise and/or experience of those involved. The inference that we can make is that, despite the logic of the classification systems available, as advocated by their respective authors, they can be considered to be very complex, regardless of the detailing of each category. Therefore, this did not allow there to be an adequate level of confidence between the observers, when applied.

A smaller number of options may also generate a more reliable classification system, but this may not resolve the problem of the classifications, in a general manner. For example, in the study by Sidor et al.,22reduction of the number of types of fracture in order to apply the modified Neer classification for the proximal humerus was not found to provide any increase in agreement.

We believe that, in a general manner, our study presents several important points. Firstly, we brought together a large number of cases (108) that presented great variability of injuries. We observed that other studies presented series ranging in size from 10 to 275 cases.10^and14In studies in the literature that involved the type of analysis used in our study, there was an average participation of five evaluators for every 50 cases.14Secondly, our observers had a variety of levels of experience, which also made it possible to ascertain whether the degree of learning might interfere with the application of the different classification systems. In our study, greater experience among the examiners did not increase the agreement among the items evaluated, which denotes that it may be possible to make general use of the classification system for the entire community of orthopedic surgeons, independent of their experience of managing pediatric fractures.2

We support the idea that simplified classification systems would be expected to present higher levels of intra and interobserver agreement than would the systems evaluated in this study. They would also be expected to more efficiently predict what the best treatment method would be and what type would give rise to the lowest late complication rates. Thus, a system that encompasses the predicates of an ideal classification needs to be planned for long-bone fractures of the immature skeleton.

In this manner, in our opinion, an ideal classification system has not yet been achieved. The complexity of the analysis on fractures that involve the locomotor apparatus during childhood and adolescence is directly related to several factors: age; differences in growth between different bone segments; growth patterns; bone remodeling rates; mechanical action on the bone; state of the adjacent structures; difference in growth rates between the proximal and distal growth plates; growth of the epiphysis; status of the circulation; energy of the trauma involved, etc. The need for comprehension of the influence of all these variables that change with growth of the locomotor apparatus makes creation of a single acceptable classification system a very complex task.

Conclusions

In this study, the intra and interobserver agreement for the pediatric AO classification system was considered to be good or excellent for the parameters of bone, segment, paired bone, subsegment, pattern and displacement. However, the intra and interobserver agreement relating to the parameters of severity and side of the avulsion was statistically unsatisfactory.

Referências

• Meling Terje, Harboe K, Enoksen HC. Reliable classification of children's fractures according to the comprehensive classification of long bone fractures by Müller. Acta Orthop. 2013;84(2):207-12.
• Audigé L, Bhandari M, Hanson B, Kellam J. A concept for the validation of fracture classifications. J Orthop Trauma. 2005;19(6):401-6.
• Slongo T, Audigé L, Schlickewei W, Clavert JM, Hunter J. Development and validation of the AO pediatric comprehensive classification of long bone fractures by the Pediatric Expert Group of the AO Foundation in col laboration with AO Clinical Investigation and Documentation and the International Association for Pediatric Traumatology. J Pediatr Orthop. 2006;26(1):43-9.
• Cantor AB. Sample-size calculations for Cohen's Kappa. Psychol Methods. 1996;1(2):150-3.
• Rosner BA. Fundamentals of biostatistics. 4 ed. Belmont: Duxbury Press; 1995. p. 426.
• Fleiss JL. Statistical methods for rates and proportion. 2 ed. New York: Wiley; 1981.
• Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37(5):360-3.
• Altman DG. Practical statistic for medical research. 3 ed. London: Chapman and Hall; 1995. p. 403-9.
• Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-74.
• Slongo T, Audigé L, Clavert JM, Lutz N, Frick S, Hunter J. The AO comprehensive classification of pediatric long-bone fractures: a web-based multicenter agreement study.. J Pediatr Orthop 2007;27(2):171-80.
• Müller M, Narzarian S, Koch P, Schatzker J. The comprehensive classification for fractures of long bones. Berlin, Heidelberg, New York: Springer-Verlag; 1990.
• Salter RB, Harris WR. Injuries involving the epiphyseal plate. J Bone Joint Surg Am. 1963;45(3):587-622.
• Gartland JJ. Management of supracondylar fractures of the humerus in children. Surg Gynecol Obstet. 1959;109(2):145-54.
• Audigé L, Bhandari M, Kellam J. How reliable are reliability studies of fracture classifications? A systematic review of their methodologies. Acta Orthop Scand. 2004;75(2):184-94.
• Garbuz DS, Masri BA, Esdaile J, Duncan CP. Classification systems in orthopaedics. J Am Acad Orthop Surg. 2002;10(4):290-7.
• Burstein AH. Fracture classification systems: do they work and are they useful?. J Bone Joint Surg Am 1993;75(12):1743-4.
• Martin JS, Marsh JL. Current classification of fractures. Rationale and utility. Radiol Clin North Am. 1997;35(3):491-506.
• Poland J. Traumatic separation of the epiphysis. London: Smith, Elder & Co; 1898. p. 144-62.
• Bergenfeldt E. Beitrage zur Kenntnis der traumatischen Epiphysenlosungen an den langen Rohrenknochen der Extremitaten: eine klinisch-rontgenologische Studie. Acta Chir Scand. 1933;73 Suppl.28.
• Aitken AP. The end result of the fractured distal tibial epiphysis. J Bone Joint Surg. 1936;18(3):685-91.
• Peterson HA. Physeal fractures. Part 3: classification.. J Pediatr Orthop 1994;14(4):439-48
• Sidor ML, Zuckerman JD, Lyon T, Koval K, Cuomo F, Schoenberg N. The Neer classification system for proximal humeral fractures. An assessment of interobserver reliability and intraobserver reproducibility.. J Bone Joint Surg Am 1993;75(12):1745-50.

• 2
  ☆Work developed in the Discipline of Pediatric Orthopedics, Department of Orthopedics and Traumatology, Universidade Federal de São Paulo (Unifesp), São Paulo, SP, Brazil.

Publication Dates

• Publication in this collection
  Oct 2015

History

• Received
  08 Aug 2014
• Accepted
  21 Aug 2014