SCOLIOSIS ASSESSMENT VIA TELEHEALTH USING PHOTOGRAMMETRY: DEVELOPMENT AND ACCURACY

AVALIAÇÃO DA ESCOLIOSE POR TELEMEDICINA USANDO FOTOGRAMETRIA: DESENVOLVIMENTO E PRECISÃO

EVALUACIÓN DE LA ESCOLIOSIS MEDIANTE TELEMEDICINA CON FOTOGRAMETRÍA: DESARROLLO Y PRECISIÓN

ABSTRACT

The aim of this study was to present the development and assessment of the measurement accuracy of variables obtained with the Digital Imaged-based Postural Assessment (DIPA©) Capture app and Analysis software as part of a telehealth assessment protocol for idiopathic scoliosis. To enable the correct capture of photographic images for further analysis, the development of an application, called DIPA© Capture, was performed. A photo is automatically obtained only when the smartphone is aligned. The sample of this prospective study was composed of consecutive images of a real plumb line (RPL) taken by using the DIPA© Capture app. Once the photo was captured the app automatically drew a virtual plumb line (VPL). The inclination of the RPL and VPL was measured using the DIPA-S© eHealth Analysis software. A total of 50 images using the App comprised this sample. The median (min-max) inclination angle of the real and virtual plumb lines was 89.8° (88.6°-90°) with a 1.4° range and 90° (89.4°-90°) with a 0.6° range, respectively. The mean difference between the inclination of the two plumb lines (RPL - VPL) was very small -0.1°±0.04° (p= 0.017). The RMS error was 0.3°. The DIPA© Capture app and Analysis software for image acquisition and measurement was developed and is ready for testing with patients. The app accurately captures the alignment of the smartphone during the image acquisition and the software shows adequate measurement accuracy for future assessment of idiopathic scoliosis by photogrammetry. Level of Evidence III; Diagnostic Studies - Investigation of a Diagnostic Test.

Keywords:
Telemedicine; Remote Consultation; Posture; Scoliosis

RESUMO

O objetivo deste estudo foi apresentar o desenvolvimento e a avaliação da precisão das medidas de variáveis obtidas com o aplicativo Digital Imaged-based Postural Assessment (DIPA©) Capture e o software de Análise, como parte de um protocolo de avaliação de telemedicina para escoliose idiopática. Para possibilitar a correta captura de imagens fotográficas para análises posteriores, foi realizado o desenvolvimento de um aplicativo chamado DIPA© Capture. Uma foto é automaticamente obtida apenas quando o smartphone está alinhado. A amostra deste estudo prospectivo foi composta por imagens consecutivas de uma linha prumo real (LPR) tiradas usando o aplicativo DIPA© Capture. Uma vez capturada a foto, o aplicativo desenhou automaticamente uma linha prumo virtual (LPV). A inclinação da LPR e da LPV foi medida usando o software de Análise DIPA-S© eHealth. Um total de 50 imagens usando o aplicativo compôs esta amostra. A mediana (min-máx) do ângulo de inclinação das linhas prumo real e virtual foi de 89,8° (88,6°-90°) com uma variação de 1,4° e 90° (89,4°-90°) com uma variação de 0,6°, respectivamente. A diferença média entre a inclinação das duas linhas prumo (LPR - LPV) foi muito pequena, -0,1°±0,04° (p= 0,017). O erro RMS foi de 0,3°. O aplicativo DIPA© Capture e o software de Análise para aquisição e medição de imagens foram desenvolvidos e estão prontos para testes com pacientes. O aplicativo captura com precisão o alinhamento do smartphone durante a aquisição de imagens, e o software apresenta uma precisão de medição adequada para avaliações futuras da escoliose idiopática por fotogrametria. Nível de Evidência III; Estudos diagnósticos - Investigação de um exame para diagnóstico.

Descritores:
Telemedicina; Consulta Remota; Postura; Escoliose

RESUMEN

El objetivo de este estudio fue presentar el desarrollo y la evaluación de la precisión de las medidas de variables obtenidas con la aplicación Digital Imaged-based Postural Assessment (DIPA©) Capture y el software de Análisis, como parte de un protocolo de evaluación de telemedicina para la escoliosis idiopática. Para permitir la correcta captura de imágenes fotográficas para análisis posteriores, se llevó a cabo el desarrollo de una aplicación llamada DIPA© Capture. Una foto se obtiene automáticamente solo cuando el smartphone está alineado. La muestra de este estudio prospectivo estuvo compuesta por imágenes consecutivas de una línea de plomada real (LPR) tomadas utilizando la aplicación DIPA© Capture. Una vez capturada la foto, la aplicación dibujó automáticamente una línea de plomada virtual (LPV). La inclinación de la LPR y de la LPV se midió utilizando el software de Análisis DIPA-S© eHealth. Un total de 50 imágenes utilizando la aplicación compusieron esta muestra. La mediana (mín.-máx.) del ángulo de inclinación de las líneas de plomada real y virtual fue de 89,8° (88,6°-90°) con un rango de 1,4° y 90° (89,4°-90°) con un rango de 0,6°, respectivamente. La diferencia media entre la inclinación de las dos líneas de plomada (LPR - LPV) fue muy pequeña, -0,1°±0,04° (p= 0,017). El error RMS fue de 0,3°. La aplicación DIPA© Capture y el software de Análisis para la adquisición y medición de imágenes fueron desarrollados y están listos para ser probados con pacientes. La aplicación captura con precisión el alineamiento del smartphone durante la adquisición de imágenes y el software muestra una precisión de medición adecuada para la evaluación futura de la escoliosis idiopática mediante fotogrametría. Nivel de Evidencia III; Estudios Diagnósticos - Investigación de una Prueba Diagnóstica.

Descriptores:
Telemedicina; Consulta Remota; Postura; Escoliosis

INTRODUCTION

Idiopathic scoliosis is an orthopedic condition of the spinal column and trunk that tends to progress rapidly during periods of growth spurts in children and adolescents, requiring constant monitoring.¹ The increasing technological innovation has opened up new possibilities in assessment and treatment tools for various orthopedic issues, particularly concerning spinal conditions.²

The teleconsultation mode of care substantially increases the reach of healthcare professionals and patients access to specialized care in different areas. The screening and evaluation of patients with scoliosis have been widely conducted through the scoliometer and direct observation of postural asymmetries, characteristic of the condition. However, these methods either require in-person presence or are limited to an observational assessment.³

The development of applications and software and the use of artificial intelligence are strengths of the current period, optimizing certain procedures in the assessment of patients with scoliosis.⁴ Furthermore, new reliable, affordable, and low-cost alternatives are desirable for advancing screening programs and the periodic assessment of patients with scoliosis.

The DIPA-S eHealth© capture and analysis system is presented as a new way of evaluating patients with scoliosis. It allows for the extraction of quantitative information from clinical parameters, and measurements can be objectively conducted by professionals, all done entirely remotely.

This technical report aims to present the development and assessment of the measurement accuracy of variables obtained with the (Digital Imaged-based Postural Assessment - Scoliosis) DIPA-S eHealth© Capture app and Analysis software as part of a telehealth assessment protocol for patients with idiopathic scoliosis.

TECHNICAL REPORT

Development

This study was approved by the research ethics committee (52894421.7.0000.5347). To enable the correct capture of photographic images for subsequent analysis, the development of an application named DIPA-S eHealth© Capture was proposed to minimize artifacts stemming from the traditional method of acquiring photographic images and facilitating the collection process, as image acquisition will be performed by users in a home environment, without the presence of a healthcare professional, in other words, without a professional or trained evaluator.

The smartphone application DIPA-S eHealth© Capture was programmed in Java with the assistance of the Android Studio 4.1.3 Integrated Development Environment (IDE). Its internal accelerometer was used to determine the orientation of the smartphone’s axes (Figure 1). The accelerometer is a sensor that can operate on various principles, such as piezoresistive, piezoelectric, capacitive, etc. However, they all convert the displacement of a known mass into an electrical signal that can then be amplified and used in a wide range of applications. It’s worth noting that accelerometers are designed to read in a specific direction of force. Therefore, to obtain 3D acceleration, three orthogonal sensors are required.

Figure 1
Orientation of local axes on a smartphone.

Among the sensors available on the smartphone, only the accelerometer was used in this project, as the axis orientations in this research project can be determined statically since the user will be holding the phone to capture a photograph.

In programming languages like Java, the use of interfaces is widely spread. An interface is a tool developers use to require the programmer to implement methods according to their application so that the functions of that interface execute correctly and yield the expected results in each solution. For this application, the SensorEventListener interface was used, making it possible to use the onSensorChanged method (SensorEvent event), executed whenever a change in sensor values occurs at a maximum time interval. This method takes, as an argument, an object of type SensorEvent, which contains various information such as the sensor type (in case multiple types are used with the same interface), values read by the sensor, the date on which the values were read, precision, and more.

It’s worth noting that the way the interface obtains these values is not within the scope of this technical report and can vary drastically depending on the sensor’s operating principle and the Android version in which the application is running. From the application’s perspective, this is a significant advantage of using interfaces since the hardware and software developer of the device is responsible for correctly obtaining and processing the values read by the sensor. In contrast, the application programmer is responsible for correctly using the information obtained from the sensor.

Therefore, with each change in the sensor at a specified time interval, the device’s rotation around the Z-axis can be determined by equations 1 and 2:

equation 1 $$\overrightarrow{G}=g_x\overrightarrow{ı}+g_y\overrightarrow{ȷ}+g_z\overrightarrow{k}\left[\frac{m}{S^2}\right]$$

equation 2 $$r_z=\arctan \left(\frac{g_x}{g_y}\right)[rads]$$

Where:

$\overrightarrow{G}$=> Vector acceleration obtained by the sensor

r_z=> Rotation around Z

The rotation around the X-axis of the device is given by equation 3:

equation 3 $$r_x=\arccos \left(\frac{g_z}{|\overrightarrow{G}|}\right)[\text{rads}]$$

Where:

$\overrightarrow{G}$=> Vector acceleration obtained by the sensor

r_x=> Rotation around X

Once the values of rotation around X and Z are obtained, image capture is enabled when the rotation around X is between 89° and 91° and the rotation around Z is between -1° and 1°.

To capture images with the device’s camera, the android.hardware.camera 2 package was used. By using the createCaptureRequest method of a CaptureRequestBuilder object, the camera’s preview was created, and with the events of a CameraCaptureSession object, CaptureCallback in conjunction with CaptureRequestBuilder, a photo is automatically obtained and saved on the device.

Measurement accuracy

The sample of this prospective study was composed of consecutive images of a real plumb line (RPL) taken for a physiotherapy student using the DIPA-S eHealth© Capture app (Figure 2). To assess the measurement accuracy, determining whether the image acquisition occurs without artifacts and with the smartphone perfectly vertical, once the photo was captured, the app the DIPA-S eHealth© Capture automatically drew a virtual plumb line (VPL) to ensure the absence of inclination (greater than 1°) during the image acquisition. (Figure 2)

Figure 2
Photos taken using DIPA-S eHealth© capture.

The DIPA-S eHealth© Analysis program was developed in the C# (C Sharp) programming language using the Visual Studio development platform with a community license. The programming language was chosen because of its widespread use in developing applications in the Windows environment. At the same time, the development platform was selected for its free use in educational projects and academic research. In Figure 3, the application screen is presented, where it is possible to open an image to select an analysis plane. (Figure 3)

Figure 3
Example of analysis selection.

Depending on the selected analysis, variables in each plane can be measured. The mathematical definitions and equations of each variable are described below.

In the frontal plane, the user inputs five points, with the first two exclusively for system calibration by entering a known distance in the image. Next, the midpoint (Pm) between the following two points, P1 and P2, is calculated, and the horizontal variation (∆x) from this midpoint to the third user-input point (P3) is determined, as illustrated in Figure 4 and Equation 4.

Figure 4
Ilustration of the calculation and Equation performed in the frontal plane analysis.

Where:

∆x => distance in millimeters between Pm and P3.

Pm => calculated point

P1, P2 and P3 => clicked points

In the sagittal plane, four points are specified by the user, with the first two dedicated to calibration. Then, two points (P1 and P2) are provided, and the application returns the horizontal distance between them (∆x) in millimeters based on the current mm/pixel conversion. Figure 5 and Equation 5 illustrate the calculation performed in this analysis.

Figure 5
Illustration of the calculation and equation performed in the sagittal plane analysis.

Where:

∆x => distance in millimiters between P1 and P1.

P1 and P2 => cliked points

The user provides only two points (P1 and P2) in the axial plane, and then the angle between the horizontal line and the line formed by these points is calculated (α), as shown in Figure 6 and Equation 6.

Figure 6
Illustration of the calculation and equation performed in the axial plane analysis.

Where:

α => angle formed by the line connecting the two clicked points (P1 and P2) and the horizontal.

P1 and P2 => cliked points

To measure the inclination of the plumb line (RPL and VPL), the DIPA-S eHealth© Analysis software for the measurement in the axial plane was used. (Figure 7) In the software, the image can be input directly from the server, where the images were automatically saved by the DIPA-S eHealth© Capture app, or by opening it from a local folder. This system feature will allow the professionals to take measurements using the software from images collected in the home environment by parents or caregivers, eliminating the need for the professional’s in-person presence. In the imaging acquisition process, the only clinical variable analyzed thus far has been the plumb line (both real and virtual).

Figure 7
Measurement of the inclination angle of the real and virtual plumb line obtained using the DIPA-S eHealth© analysis software.

In the future, this vertical line will be used to measure actual clinical postural variables such as angle of trunk rotation and frontal and sagittal trunk imbalance. The rater clicked on the two extremities of the RPL (greatest distance). The software automatically calculated the angle formed between the straight line that passes through the two points clicked on the RPL and the horizontal reference line. The rater did the same procedure for the VPL on the same image.

The Wilcoxon test, median, minimum, maximum, range, mean difference and RMS error were calculated (p<.05). A total of 50 images using the App comprised this sample. There was no exclusion or sample loss. The median (min-max) inclination angle of the real and virtual plumb lines was 89.8° (88.6°-90°) with a 1.4° range and 90° (89.4°-90°) with a 0.6° range, respectively. The mean difference between the inclination of the two plumb lines (RPL - VPL) was very small -0.1°±0.04° (p= 0.017). The RMS error was 0.3°.

DISCUSSION

The DIPA-S eHealth© capture and analysis system was developed to enable a reliable and accurate assessment of patients with scoliosis through teleconsultation, applied entirely remotely. In addition to its strengths, such as dispensing with in-person presence and expanding the reach of specialized professionals in spinal deformities, its use will represent a significant advancement in acquiring quantitative postural variables through teleconsultation. A relevant feature of the system is that it will allow the image capture by the parents or caregivers in the home environment. At the same time, the measurement of clinical variables in patients with scoliosis can be performed directly by the professionals themselves using the software, reducing sources of error, and increasing its applicability in the follow-up of these patients.

Photogrammetry has been widely explored and is a valuable assessment tool in clinical practice for patients with scoliosis.⁵,⁶ Photogrammetry provides a reliable method for evaluating various relevant variables in monitoring patients with scoliosis. It is easy to use, cost-effective, and does not involve ionizing radiation.⁷ Although highly useful, its application is based on and dependent on a specific protocol. This protocol requires the preparation of the environment for camera positioning with fixed distance and height, including a visible real plumb line during image acquisition, palpation, and marking anatomical reference points, among other aspects related to the detailed protocol associated with the technique.

The intrinsic characteristics of photogrammetry protocols currently require in-person execution, and, as far as we know, there are no protocols for using photogrammetry in a virtual environment, specifically in the context of telemedicine. The DIPA-S eHealth© capture and analysis system introduces a new way of employing photogrammetry to evaluate patients with scoliosis through telemedicine, eliminating the need for in-person presence.

Recognizing the existing gap in the scientific literature and clinical practice regarding the quantitative clinical assessment of scoliosis patients through telemedicine,⁸ the DIPA-S eHealth© Capture App and Analysis Software could contribute to screening, diagnosing, and monitoring patients in the virtual environment. Future studies using the DIPA-S eHealth© system by different evaluators and at different times, and even in comparison with in-person clinical assessment and X-ray examination, should be conducted to verify its reproducibility, validity, and diagnostic accuracy.

CONCLUSION

The DIPA-S eHealth© Capture app and Analysis software for image acquisition and measurement were developed and are ready for testing with patients. The app accurately captures the smartphone’s alignment during image acquisition, and the software shows adequate measurement accuracy for future assessment of idiopathic scoliosis by photogrammetry.

• Study conducted by the School of Physical Education, Physiotherapy, and Dance of the Universidade Federal do Rio Grande do Sul.

REFERENCES

• ¹ Alfraihat A, Samdani AF, Balasubramanian S. Predicting curve progression for adolescent idiopathic scoliosis using random forest model. PLoS One. 2022;17(8):e0273002.
• ² Watanabe K, Aoki Y, Matsumoto M. An application of artificial intelligence to diagnostic imaging of spine disease: estimating spinal alignment from Moiré images. Neurospine. 2019;16(4):697-702.
• ³ Scaturro D, de Sire A, Terrana P, Constatino C, Lauricella L, Sannasardo CE, et al. Adolescent idiopathic scoliosis screening: Could a school-based assessment protocol be useful for an early diagnosis?. J Back Musculoskelet Rehabil. 2021;34(2):301-6.
• ⁴ Negrini F, Cina A, Ferrario I, Zaina F, Donzelli S, Galbusera F, et al. Developing a new tool for scoliosis screening in a tertiary specialistic setting using artificial intelligence: a retrospective study on 10,813 patients: 2023 SOSORT award winner. Eur Spine J. 2023;32(11):3836-45. doi: 10.1007/s00586-023-07892-1.
  » https://doi.org/10.1007/s00586-023-07892-1.
• ⁵ Navarro IJ, Candotti CT, do Amaral MA, Dutra VH, Gelain GM, Loss JF. Validation of the Measurement of the Angle of Trunk Rotation in Photogrammetry. J Manipulative Physiol Ther. 2020;43(1):50-6.
• ⁶ Navarro IJRL, Candotti CT, Furlanetto TS, Dutra VH, do Amaral MA, Loss JF, et al. Validation of a mathematical procedure for the cobb angle assessment based on photogrammetry. J Chiropr Med. 2019;18(4):270-7.
• ⁷ Furlanetto TS, Sedrez JA, Candotti CT, Loss JF. Photogrammetry as a tool for the postural evaluation of the spine: A systematic review. World J Orthop. 2016;7(2):136-48.
• ⁸ Satin AM, Lieberman IH. The Virtual Spine Examination: Telemedicine in the Era of COVID-19 and Beyond. Global Spine J. 2021;11(6):966-74.

Edited by

• Reviewed by:
  Marcelo Wajchenberg

Publication Dates

• Publication in this collection
  10 Feb 2025
• Date of issue
  2025

History

• Received
  11 June 2024
• Accepted
  19 Nov 2024