An Adaptation of Computer Vision of Artificial Intelligence for the Assessment of Postural Ergonomics in Laparoscopic Surgery

INTRODUCTION

The Federation of European Ergonomics Societies defines ergonomics (or human factors) as the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principles, data, and methods to design in order to optimize human well-being and overall system performance. Human anatomical, anthropometric, physiological, and biomechanical characteristics all together comprise physical ergonomics. This includes working postures, material handling, repetitive movements, work-related musculoskeletal disorders, workplace layout, safety, and health.¹

The concept of postural ergonomics evolved as a topic of interest in laparoscopic surgery with the increase in the prevalence of work-related musculoskeletal disorders among laparoscopic surgeons, which has been reported to be as high as 74%.² The physical complaints reported by the surgeons as a result of improper ergonomics have varied from generalized pain, pain in the upper back, lower back, neck, shoulder, and lower extremities, and fatigue to eyestrain, carpal tunnel syndrome, and cervical spondylosis.^3–5 The assessment of postural ergonomics thus assumes importance in the recognition and subsequent correction of the awkward/prolonged postures held by the surgeons.

The existing methods to assess postural ergonomics are many and vary from real-time assessment by placing inertial sensors on the body of the surgeon and subsequent kinematic analysis^6,7 to analysis using scores such as rapid upper limb analysis⁸ or rapid entire body analysis.⁹ These have the disadvantages of being cumbersome, time consuming, and often require the presence of an expert for the analysis.¹⁰

The advent of artificial intelligence (AI) has opened a new horizon for the assessment of ergonomics. Machine learning is a branch of AI where training data and/or past experiences are used to create algorithms to successfully execute a performance.¹¹ Computer vision is a subset of machine learning which aids computers in seeing and making inferences from observed picture data.¹²

The use of AI and computer vision for the assessment of ergonomics is a relatively recent development. This would be a much easier, less time consuming, and more accurate approach to assess ergonomics as opposed to traditional methods.^13,14 However, studies on the use of this technology in the field of laparoscopic ergonomics are limited. This study is an attempt to use the technology of AI and computer vision to assess the postural ergonomics of laparoscopic surgeons.

OBJECTIVES

Using AI to assess postural ergonomics among laparoscopic surgeons.

METHODOLOGY

This was a prospective observational study conducted at Victoria Hospital in Bengaluru between November 2022 and January 2023. After obtaining prior consent, laparoscopic surgeons were observed while performing various laparoscopic surgeries. A total of 30 laparoscopic surgeries were included in the study. This study analyzed postures held for more than 30s and repetitive movements. These were photographed from the side and front views using a tablet. These photos were then imported onto an AI posture evaluation software app and analysis of the head and neck postures was done. From the front view, the app detected landmarks like outer angles of both eyes, glabella, tip of nose, subnasale, lip junction, and angles of the mouth and mentum. From the side view, it detected landmarks like the outer canthus of the eye, tragus, C7 vertebra, and the acromion process.

Using these landmarks, the app calculates various parameters such as craniohorizontal angle (CHA), craniovertebral angle (CVA), straight sagittal posture (SSP), upper head posture (UHP), lower head posture (LHP), and vertical posture (VP). The app then generates a report containing the analysis based on the angles measured. The reports obtained from the app from various postures across multiple surgeries were tabulated. Data analysis was done using SPSS 23 software and reported using descriptive statistics.

RESULTS

The postures of surgeons while performing 30 surgeries were analyzed, and results were tabulated. Of the surgeries performed, four (13.33%) were laparoscopic appendectomy, nine (30%) were laparoscopic cholecystectomy, eight (26.66%) were intra peritoneal only mesh repair (IPOM), five (16.66%) were diagnostic laparoscopy, one (3.33%) was totally extraperitoneal repair (TEP), and three (10%) were other surgeries such as laparoscopic varicocelectomy, laparoscopic Nissen’s fundoplication, and laparoscopic diaphragmatic hernia repair.

Using the above-mentioned landmarks, CHA, CVA, SSP, UHP, LHP, and VP were calculated by the app as shown in Figures 1 to 3.

The mean CHA was 22.19 ± 7.02, with a median of 21.75 (25.20–16.75). The mean CVA was 44.70 ± 18.90, with a median of 44.00 (49.10–35.70). The mean SSP was 58.90 ± 15.24, with a median of 56.65 (68.55–44.92).

The mean UHP was 8.36 ± 5.71, with a median of 7 (11.52–3.60). The mean LHP was 9.13 ± 8.24, with a median of 6.25 (14.12–3.07). The mean VP was 14.80 ± 12.64, with a median of 11.5 (17.25–7.12) (Table 1). The surgeons in all the scenarios showed face tilt towards the right.

Among the 30 scenarios, 16 (53.33%) showed rounded/protracted shoulder posture.

Forward head posture (FHP) was present in 28 (93.3%) scenarios, healthy head posture was present in 1 (3.33%) scenario. Forward head and rounded shoulder posture (RSP) was present in 5 (16.66%) scenarios (Table 2).

DISCUSSION

Artificial intelligence is concerned with the design and implementation of computer systems capable of solving problems that usually require the ability of human beings. Such problems are of a high complexity and/or involve natural tasks (e.g., vision or natural language understanding), which classical algorithmic methods cannot usually solve. For solving them, AI programs mainly manipulate symbolic information and not just numerical data, as usual in computer science.¹⁵

Computer vision is a branch of AI that combines concepts, techniques, and ideas from digital image processing, pattern recognition, AI, and computer graphics.¹⁶ It works on the principle of recognition. Recognition is defined by the trial to determine whether or not an input data contains or resembles some specific object, feature, or activity. In computer vision, action recognition refers to being able to detect a particular component from a video or image scenes.¹⁷

Of particular interest to this study is the branch of face recognition. Facial landmarks, like the corners of the eye, corners of the mouth, tip of the nose, chin, and cheek are located topographically.¹⁸ Separate rectangular search regions are established for the mouth and the eyes. Then appropriate algorithms are used for the extraction of the borders.¹⁹

Applications of computer vision are varied and range from automatic classification of blood cells in medical images to control of an unnamed lunar rover, from surveillance of parks, streets, and venues to sports video analysis.^20,21 The use of this technology for the assessment of postural ergonomics is a relatively recent development. Although face recognition can detect the landmarks relevant for ergonomics (using which appropriate angles can be measured and deviations from normal can be accurately detected), its use for assessing the same in laparoscopic surgeons has been minimal.

The AI posture evaluation software app we used for the study imports pictures taken during the surgery from front and side views and detects landmarks as described before.

With these landmarks, the software calculates the following angles:

• Craniohorizontal angle: The angle between the intersection of a horizontal line that passes through the tragus of the ear and a line joining the tragus of the ear and the external canthus of the eye.²² It gives an estimate of the head on neck angle or the position of the upper cervical spine.²³ A higher value indicates FHP.²⁴

• Craniovertebral angle: The angle between the intersection of a horizontal line through the spinous process of C7 and a line to the tragus of the ear. It provides an estimation of the position of the neck on the upper trunk. A smaller angle indicates a more FHP.²⁵

• Sagittal straight posture: The angle formed by the intersection of a horizontal line that passes through C7 and a line that connects the midpoint of the greater tuberosity of the humerus and the posterior aspect of the acromion. It assesses the shoulder position. A smaller angle means that the shoulder lies anteriorly in relation to C7, in other words, rounded shoulder.²²

• Vertical posture: Angle between vertical and the line joining glabella to subnasal point.

• Upper head posture: Angle between horizontal and line joining outer canthi of both eyes.

• Lower head posture: Angle between horizontal and line joining angles of the mouth.

The VP, UHP, and LHP assess the tilt of the face.

Forward head posture is seen when the head and upper cervical vertebra extend and the lower cervical vertebra flex.²⁶ It is known to cause shoulder and neck pain.^27,28 Rounded shoulder posture or protracted shoulder posture or forward shoulder posture occurs when the acromion processes are placed more anteriorly as compared with the mastoid processes. It means that the shoulders are bent forward, caused by elevation of the scapulae and protraction of the acromion. This causes pain in the head, shoulders, and arms.^29–31 It is associated with a risk of increased muscle load, degenerative disc disease, back pain, and chronic shoulder pathologies.^32,33

In our study, we found the mean CHA to be 22.19 ± 7.02. The mean CVA was 44.7 ± 18.9. The mean SSP was 58.9 ± 15.24. This is suggestive of a FHP with a RSP. The mean UHP was 8.36 ± 5.71. The mean LHP was 9.13 ± 8.24. The mean VP was 14.8 ± 12.64. This is suggestive of a face tilt toward the right.

Among the 30 scenarios, 16 (53.33%) showed rounded/protracted shoulder posture.

Forward head posture was present in 28 (93.3%) scenarios, healthy head posture was present in 1 (3.33%) scenario. Forward head and RSP was present in 5 (16.66%) scenarios.

The severe consequences of musculoskeletal pain occurring as a result of poor ergonomics make it imperative to study and assess postural ergonomics in the field of laparoscopic surgery. Over 50% surgeons have reported that musculoskeletal pain negatively impacts their performance during surgery.^34,35 In contrast to the other fields, laparoscopic surgeons assume an upright posture with a straight back and have fewer trunk movements and weight shifting.^36,37 They perform repetitive movements like looking back and forth from the monitor to the surgical site, and repeated insertion and removal of long laparoscopic instruments. These increase the risk of overuse injuries.³⁸ Foot pedals are also a source of discomfort for the surgeons.³⁹ This is because they are often not placed in the direct visual field of the surgeon, and hence the surgeons maintain dorsiflexion of the foot over the pedal to avoid losing contact. This causes an imbalance in the weight distribution across both the legs.^10,40

Thus, an ideal assessment of the postural ergonomics during laparoscopic surgery would be to assess in real time the whole body (from head to toe) of the surgeon from front, side, and back views, giving importance to the awkward postures held, the duration of time for which each posture is held and the repetitive movements performed.

The real-time ergonomic risk assessment methods in use at present involve the placement of sensors over various landmarks over the surgeons and measuring the movement data.^6,10 However, these methods are difficult to implement due to issues related to sterility, cooperation, and acceptance from surgeons.¹⁰ These issues are alleviated by the use of the technology of AI and computer vision, which has multiple advantages like not requiring the subjects to wear special sensors or special clothing with markers and needing very less and basic equipments.¹⁴ The software used in this study does the ergonomic analysis by using photos of surgeons captured during surgery. There is scope to further develop this into a real-time analysis done during the performance of surgery.

The limitations of the study include inability to capture the entire length of the surgeon from the front while performing the surgery, hence making an assessment of the posture of the whole body difficult. The images captured represent only either the repetitive movements or the postures held for more than 30s, and a comprehensive assessment of the surgeon during the whole procedure, as would occur with a real-time analysis was not possible. Also, the presence of a surgical mask during the procedure makes detection of the facial landmarks difficult.

CONCLUSION

The technology of AI and computer vision can revolutionize the assessment of postural ergonomics in laparoscopic surgery, making the assessment much easier, quicker, and more accurate. Studies need to be conducted on a larger scale for better validation of results. Further developments in the software need to be made for real-time assessment of postural ergonomics and to provide immediate red alerts for posture correction. Development of a customized software catering to the specific needs of laparoscopic ergonomics, taking into consideration the above-mentioned limitations would be ideal.

ORCID

Prem Kumar A https://orcid.org/0000-0001-9126-1115

Sindhu S https://orcid.org/0009-0003-4416-5659

Mallikarjuna Manangi https://orcid.org/0000-0002-1509-8501

Santhosh Shivashankar Chikkanayakanahalli https://orcid.org/0000-0003-4656-9465

Sunil Kumar Venkatappa https://orcid.org/0000-0002-8991-2756

REFERENCES

1. Definition and domains of ergonomics. In: What is Ergonomics? International Ergonomics Association; 2017. http://www.iea.cc/whats/. Accessed 4/11/2017.

2. Alleblas CCJ, de Man AM, van den Haak L, et al. Prevalence of musculoskeletal disorders among surgeons performing minimally invasive surgery: A systematic review. Ann Surg 2017;266(6):905–920. DOI: 10.1097/SLA.0000000000002223.

3. Stucky CH, Cromwell KD, Voss RK, et al. Surgeon symptoms, strain, and selections: Systematic review and meta-analysis of surgical ergonomics. Ann Med Surg (Lond) 2018;27:1–8. DOI: 10.1016/j.amsu.2017.12.013 PMID: 29511535; PMCID: PMC5832650.

4. Hemal AK, Srinivas M, Charles AR. Ergonomic problems associated with laparoscopy. J Endourol 2001;15:499–503. DOI: 10.1089/089277901750299294.

5. Gutierrez-Diez MC, Benito-Gonzalez MA, Sancibrian R, et al. A study of the prevalence of musculoskeletal disorders in surgeons performing minimally invasive surgery. Int J Occup Saf Ergon 2018;24(1):111–117. DOI: 10.1080/10803548.2017.1337682.

6. Sánchez-Margallo FM, Sánchez-Margallo JA. Ergonomics in Laparoscopic Surgery [Internet]. Laparoscopic Surgery. InTech; 2017. DOI: 10.5772/66170.

7. Gómez-Galán M, Pérez-Alonso J, Callejon-Ferre AJ, et al. Musculoskeletal disorders: OWAS review. Ind Health 2017;55(4):314–337. DOI: 10.2486/indhealth.2016-0191.

8. McAtamney L, Nigel Corlett E. RULA: A survey method for the investigation of work-related upper limb disorders. Appl Ergon 1993;24(2):91–99. DOI: 10.1016/0003-6870(93)90080-s. PMID: 15676903.

9. Hignett S, McAtamney L. Rapid entire body assessment (REBA). Appl Ergon 2000;31(2):201–205. DOI: 10.1016/s0003-6870(99)00039-3. PMID: 10711982.

10. Dabholkar T, Dabholkar YG, Yardi S, et al. An objective ergonomic risk assessment of surgeons in real time while performing endoscopic sinus surgery. Indian J Otolaryngol Head Neck Surg 2020;72(3):342–349. DOI: 10.1007/s12070-020-01840-x.

11. Alpaydin E. Introduction to Machine Learning, 4th edition. London, England: The MIT Press; 2020.

12. Karn A. Artificial intelligence in computer vision. Int J Eng Appl Sci Technol 2021;6(1):249–254. DOI: 10.33564/IJEAST.2021.v06i01.037.

13. Fernández MM, Álvaro Fernández J, Bajo JM, et al. Ergonomic risk assessment based on computer vision and machine learning. Comput Ind Eng 2020;149:106816; ISSN: 0360-8352; DOI: 10.1016/j.cie.2020.106816

14. Fan C, Mei Q, Yang Q, et al. Computer vision based rapid entire body analysis (REBA) estimation. J Ind Constr. In Proceedings MOC Summit: Edmonton, Alberta, Canada; 2022. pp. 90–97.

15. Haton JP. A brief introduction to artificial intelligence. IFAC Proc Vol 2006;39(4):8–16. ISSN: 1474-6670; ISBN: 9783902661050. DOI: 10.3182/20060522-3-FR-2904.00003.

16. Oscar C, Iglesias A, Galvez A, et al. Hybridization of convergent photogrammetry, computer vision, and artificial intelligence for digital documentation of cultural heritage – A case study: The Magdalena Palace. In 2014 International Conference on Cyberworlds (CW). IEEE; 2014. pp. 369–376. DOI: 10.1109/CW.2014.58.

17. Lui YM, Beveridge JR. Tangent bundle for human action recognition, 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG), Santa Barbara, CA, USA, 2011, pp. 97–102. DOI: 10.1109/FG.2011.5771378.

18. Li SZ, Jain AK. Handbook of Face Recognition, 2nd edition. New York: Springer; 2011.

19. Hess M, Martinez G. Facial feature extraction based on the smallest univalue segment assimilating nucleus (Susan) algorithm. In: Proceedings of Picture Coding Symposium, Vol. 1. Los Alamitos: IEEE Computer Society Press; 2004. pp. 261–266.

20. Nagabhushana S. Computer Vision and Image Processing. New Delhi: New Age International (P) Ltd; 2005.

21. Ahad MAR. Computer Vision and Action Recognition: A Guide for Image Processing and Computer Vision Community for Action Understanding. France: Atlantis Press; 2011.

22. Raine S, Twomey L. Posture of the head, shoulder and thoracic spine in comfortable erect standing. Aus Physiother 1994;40:25–32.

23. Mosaad D, Abdel-Aziem A. Postural balance and neck angle changes in school children while carrying a traditional backpack versus a double-sided bag. Biomed Hum Kinet 2018;10(1):59–66. DOI: 10.1515/bhk-2018-0010.

24. Ramprasad M, Alias J, Raghuveer AK. Effect of backpack weight on postural angles in preadolescent children. Indian Pediatr 2010;47(7):575–580.

25. Rugel JD, Sevek F. The influence of load placement on postural sway parameters. 13th International Conference on Biomedical Engineering. Vol. 23 of the Series IFMBE Proceedings, December 3–6, 2008, Singapore; pp. 1821–1824.

26. Yoo WG, Kim MH. Effect of different seat support characteristics on the neck and trunk muscles and forward head posture of visual display terminal workers. Work 2010;36(1):3–8.

27. Chiu TT, Lam TH, Hedley AJ. Correlation among physical impairments, pain, disability, and patient satisfaction in patients with chronic neck pain. Arch Phys Med Rehabil 2005;86(3):534–540. DOI: 10.1016/j.apmr.2004.02.030.

28. Cote P, Hogg-Johnson S, Cassidy JD, et al. The association between neck pain intensity, physical functioning, depressive symptomatology and time-to-claim-closure after whiplash. J Clin Epidemiol 2001;54(3):275–286. DOI: 10.1016/S0895-4356(00)00319-X.

29. Lukasiewicz AC, McClure P, Michener L, et al. Comparison of 3-dimensional scapular position and orientation between subjects with and without shoulder impingement. J Orthop Sports Phys Ther 1999;29(10):574–586. DOI: 10.2519/jospt.1999.29.10.574.

30. Raine S, Twomey LT. Head and shoulder posture variations in 160 asymptomatic women and men. Arch Phys Med Rehabil 1997;78(11):1215–1223.

31. Michener LA, McClure PW, Karduna AR. Anatomical and biomechanical mechanisms of subacromial impingement syndrome. Clin Biomech (Bristol, Avon) 2003;18(5):369–379.

32. Wong CK, Coleman D, di Persia V, et al. The effects of manual treatment on rounded-shoulder posture, and associated muscle strength. J Bodyw Mov Ther 2010;14(4):326–33. DOI: 10.1016/j.jbmt.2009.05.001.

33. Yoo WG. Comparison of shoulder muscles activation for shoulder abduction between forward shoulder posture and asymptomatic persons. J Phys Ther Sci 2013;25(7):815–816. DOI: 10.1589/jpts.25.815.

34. Esposito C, Najmaldin A, Schier F, et al. Work-related upper limb musculoskeletal disorders in pediatric minimally invasive surgery: A multicentric survey comparing laparoscopic and sils ergonomy. Pediatr Surg Int 2014;30(4):395–399. DOI: 10.1007/s00383-013-3437-y.

35. Filisetti C, Cho A, Riccipetitoni G, et al. Analysis of hand size and ergonomics of instruments in pediatric minimally invasive surgery. Surg Laparosc Endosc Percutan Tech 2015;25(5):e159–e162. DOI: 10.1097/SLE.0000000000000125.

36. Berguer R, Rab GT, Abu-Ghaida H, et al. A comparison of surgeons’ posture during laparoscopic and open surgical procedures. Surg Endosc 1997;11(2):139–142.

37. Szeto GP, Cheng SW, Poon JT, et al. Surgeons’ static posture and movement repetitions in open and laparoscopic surgery. J Surg Res 2012;172(1):e19–e31. DOI: 10.1016/j.jss.2011.08.004.

38. Reyes DA, Tang B, Cuschieri A. Minimal access surgery (MAS)-related surgeon morbidity syndromes. Surg Endosc 2006;20(1):1–13. DOI: 10.1007/s00464-005-0315-2.

39. Wauben LS, van Veelen MA, Gossot D, et al. Application of ergonomic guidelines during minimally invasive surgery: A questionnaire survey of 284 surgeons. Surg Endosc 2006;20(8):1268–1274. DOI: 10.1007/s00464-005-0647-y.

40. Kranenburg G. Ergonomic problems encountered during video-assisted thoracic surgery. Minim Invasive Ther Allied Technol 2004;13(3):147–155. DOI: 10.1080/13645700410033661.