Force Sensing in Robot-assisted Keyhole Endoscopy: A Systematic Survey

Instrument-tissue interaction forces in Minimally Invasive Surgery (MIS) provide valuable information that can be used to provide haptic perception, monitor tissue trauma, develop training guidelines, and evaluate the skill level of novice and expert surgeons.Force and tactile sensing is lost in many Robot-Assisted Surgery (RAS) systems. Therefore, many researchers have focused on recovering this information through sensing systems and estimation algorithms. This article provides a comprehensive systematic review of the current force sensing research aimed at RAS and, more generally, keyhole endoscopy, in which instruments enter the body through small incisions. Articles published between January 2011 and May 2020 are considered, following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. The literature search resulted in 110 papers on different force estimation algorithms and sensing technologies, sensor design specifications, and fabrication techniques.


Introduction
In Minimally Invasive Surgery (MIS), surgical access is provided through small incisions or natural orifices in the body.A surgical instrument is operated by the surgeon for tissue manipulation.Compared to open surgery, MIS provides less tissue trauma, postoperative pain, patient discomfort, wound complications and immunological response stress Wottawa et al. (2016), lower risk of infection Soltani-Zarrin et al. (2018) and blood loss Dai et al. (2017), shorter hospital stay Bandari et al. (2020), faster recovery Lee et al. (2015), and improved cosmetics Aviles et al. (2016) all of which lead to improved therapeutic outcome and efficiency Otte et al. (2016) and lower morbidity and morality Aviles et al. (2017) making MIS cost-effective Faragasso et al. (2014).Nonetheless, the ergonomically cumbersome posture increases surgeon fatigue.The limited instrument dexterity and visual perception of the scene Haghighipanah et al. (2017); Haouchine et al. (2018), and the non-intuitive hand-eye coordination due to fulcrum motion reversal decrease accuracy and contribute to surgeon fatigue Hadi Hosseinabadi et al. (2019).The high level of psychomotor skills needed increases the operation time and require a longer learning curve Shahzada et al. (2016).The sense of touch is reduced by friction in the access port and instrument mechanism.
In Robotic MIS (RMIS), the surgical instrument is controlled by a robotic manipulator and operated by a remote surgeon.The robotic operation restores hand-eye coordination Aviles et al. (2015a) and innovations in tool design improve dexterity leading to improved ergonomics that reduce surgeon fatigue Stephens et al. (2019); Bandari et al. (2020).The enhanced 3D surgical vision, automatic movement transformations, fine motions, filtering of physiological hand tremor and motion scaling lead to improved surgery precision Sang et al. (2017).However, the surgeon is isolated from the surgical site by robotic manipulators that do not provide the haptic perception Juo et al. (2020).This deprives the surgeon of a rich source of information.Thus, many studies are targeted towards the reconstruction and evaluation of haptic feedback.
Haptics can be either tactile or kinesthetic Okamura (2009).The tactile perception is through the cutaneous receptors in the skin which can sense, for example, texture or temperature Mack et al. (2012).The kinesthetic force feedback is perceived by mechanoreceptors in the muscle tendons to detect force, position and velocity information about objects Juo et al. (2020).Traditionally surgeons use palpation to characterize tissue properties, detect nerves and arteries Bandari et al. (2017), and identify abnormalities such as lumps and tumors Lv et al. (2020); Puangmali et al. (2012).Moreover, the surgeons rely on the sense of touch to regulate the applied forces.Excessive forces can lead to tissue trauma, internal bleeding, and broken sutures.Insufficient forces however can lead to loose knots and poor sutures.Sang et al. (2017); Li et al. (2016).
Direct Force Feedback (FF) and Sensory Substitution (SS) are the most common approaches of presenting surgeons with force information.While the direct method provides the most intuitive interaction Li and Hannaford (2017), it is the most challenging one to implement, as it requires a method of force sensing and a safe and robust teleoperation interface for force reflection.A compromise between transparency and stability of different teleoperation frameworks is reported by Hashtrudi-Zaad and Salcudean (2001).In sensory substitution, visual, auditory, or vibrotactile signals provide haptic perception to the surgeon.While safety can be easily guaranteed, this method can cause discomfort, distraction, and cognitive overload.In general, visual methods are shown to be the most effective feedback modality.Abdi et al. (2020) In summary, the introduction of haptic perception is proven to decrease operation time Abiri et al. (2017), facilitate training, improve accuracy, and enhance patient safety for novice surgeons in complex tasks Juo et al. (2020).More experienced surgeons learn to infer force information from visual cues such as the tissue and instrument deformations and the stretch in sutures Aviles et al. (2017).Additionally, force information can be used to automate surgical robot tasks in dynamic and unstructured environments Kuang et al. (2020), to identify tissues in real time, to create tissue-realistic models and simulators for training Stephens et al. (2019), and to perform surgical skills assessment Soltani-Zarrin et al. (2018).

Comparison to the Existing Reviews
An extensive review of haptic perception and its efficacy in RMIS is presented by Amirabdollahian et al. (2018).This review concluded that while there is a consensus on the need for haptic and tactile feedback, no commercial system is yet available that addresses this need.More recently, El Rassi and El Rassi (2020) presented a brief overview of haptic feedback in teleoperated robotic surgery.Overtoom et al. (2019) and Rangarajan et al. (2020) surveyed virtual haptics in surgical simulation and training.The latter followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines to identify the relevant literature.The authors similarly affirm the efficacy of haptic feedback in surgical education.None of the publications above review the developments in the field of force sensing and estimation.Abdi et al. (2020) reviewed research since 2000 on the efficacy of haptic feedback in teleoperated medical interventions.The authors present a concise overview of the force-sensing literature with 44 references cited over a wide range of medical applications.Although the review provides a general understanding of the challenges and complexities in instrument-tissue force measurement, it is not a comprehensive presentation of the prominent developments and the articles were subjectively selected with no evaluation criteria.Additionally, the records were only classified based on the sensing technology and the sensor location; However, the instrument's dexterity level, the sensing Degrees of Freedom (DoFs), and the performance measures were not compared.A comparison of its references with the records cited in our review shows an overlap of only 20 out of 110 papers.Bandari et al. (2020) reviewed tactile sensing literature over the past twenty years.It also includes some literature on force-sensing in neurosurgery and microsurgery procedures.Although the authors presented a comprehensive review with 121 references, a comparison of the included articles with the records in this paper shows an overlap of only 8 out of 110 papers which are mostly on developments related to the gripping force sensing.
This article is a systematic review based on the PRISMA guidelines that expands on the sensor design requirements and presents the most recent developments in force sensing and estimation in keyhole endoscopy.We discuss how research has evolved over the past decade and provide suggestions for future research directions.The closest publications to our review are the surveys by Puangmali et al. Puangmali et al. (2008) and Trejos et al. Trejos et al. (2010) which were published about a decade ago, and therefore there are no overlapping papers with those reviews.

Methodology
A systematic survey was conducted by following the PRISMA guidelines (see Figure 1) and it was based on Google Scholar, Web-of-Science, PubMed, and IEEE Xplore Digital Library repositories.The period for the review is over the past decade, from January 2011 until May 2020.The following keywords were used for identification: Force sensing, Kinesthetic, Tactile, Haptics, Minimally Invasive Surgery, MIS, MIRS, Robot-Assisted, RMIS, RAS, RAMIS, Laparoscopy, and Endoscopy.For every year, the first 20 pages of search results in Google Scholar were surveyed (total of 2000 records).The same approach was used for the identification of records through the other repositories Prepared using sagej.cls(PubMed: 213,and IEEEXplore: 40).For screening, the duplicates were removed and the identified records were skimmed through to mark the ones that are relevant to keyhole endoscopy.The articles that refer to force sensing in microsurgery, neurosurgery, and needle insertion were excluded because they involve a different set of requirements and challenges.Specifically, microsurgical instruments such as those used in neurosurgery and retinal surgery Gonenc et al. (2017) have a much smaller diameter (less than 2 mm) and do not require an articulated wrist, which complicates the actuation system and sensors' power and signal routing.Moreover, Bandari et al. (2020) briefly discussed the force-sensing literature in microsurgery, neurosurgery, and needle insertion.114 articles were found eligible for a complete review.Throughout the review, the references of the selected papers were surveyed and the relevant articles that were not initially identified were added, thus increasing the total number of eligible records to 129.The work progressions and duplicate publications were removed to lead to the 110 articles included in this survey.The included articles are tabulated for an easier comparison of the method, the sensor location, the sensing DoFs, the dexterity of the instrument under study, and the results.The Dexterity Index (DI) for different instruments is defined according to the Table 1 and Figure (2017) which can be considered as a requirement on the sensor accuracy.The human's temporal resolution is 320 Hz for force discrimination and up to 700 Hz for vibration detection Puangmali et al. (2012).However, the desired bandwidth of the force sensor is usually dictated by the application (FF, SS, vibration detection, etc.) and desired noise and resolution performance.A sample rate of 500 Hz is considered appropriate for direct force feedback applications Jones et al. (2017).Sample rates as low as 30 Hz can be effective in visual SS modality.

Size, Mass, and Packaging
Surgical instruments are inserted into the body through a cylindrical port of 12-15 mm in diameter Spiers et al. (2015); Li et al. (2015b).The outside diameter of the instrument is desired to be less than 10 mm Shi et al. (2019).The sensor should be lightweight to not significantly increase the instrument inertia.The operation rooms are filled with equipment that can cause electromagnetic interference, and the electrocautery tools operate at high voltages Lim et al. (2014); Seok et al. (2019).Thus, the sensors require insulation for electrostatic protection and shielding against electromagnetic interference Peña et al. (2018).The sensors that enter the body also require sealing against humidity and debris ingression Trejos et al. (2014).

Sterilizability
Surgical instruments are cleaned and sterilized for reuse; the former refers to removing debris from the device and the latter is the elimination of microorganisms that can cause disease Trejos et al. (2014).The common sterilization methods are plasma and gamma radiation, the use of chemicals (alcohol, ethylene oxide or formaldehyde), and steam sterilizations Bandari et al. (2020).Steam sterilization is the fastest and the most preferred method Soltani-Zarrin et al. ( 2018) which is performed in an autoclave at 120-135°C, 207 kPa and 100% humidity for 15-30 minutes Spiers et al. (2015); Zhao and Nelson (2015).This harsh environment can be destructive to many transducers, signal conditioning electronics, wire insulations, bondings, and coatings.

Biocompatibility
The sensors for use in MIS must abide by ISO 10993 which entails a series of standards for evaluating the biocompatibility of medical devices Trejos et al. (2014).
For biocompatibility electrical components often require coatings that interfere with sterilizability Spiers et al. (2015).

Adaptability and Cost
Instruments are disposed after 10 to 15 uses due to accelerated cable fatigue Anooshahpour et al. (2014); Kim et al. (2018a); Xue et al. (2018).The EndoWrist instruments retail at $2k-$5k Spiers et al. (2015).If the sensor is integrated into the instrument and is to be disposed, it should not increase the instrument price significantly.An adaptable solution that can be easily used on different instruments is desirable.

Location
The sensors can be placed in the instrument mounting interface, the instrument base, proximal (outside the body) and distal (inside the body) shafts, the actuation mechanism (cables/rod), the trocar mount and its distal end, the articulated wrist, and the gripper jaws (Figure 4).While the size, sterilizability, biocompatibility, and insulation requirements are more relaxed for the sensors placed outside Sensors placed at the gripper jaw provide the most accurate readings and have the most stringent design constraints.They are difficult to fabricate, package, mount Kim et al. (2018b); He et al. (2014), and shield Seok et al. (2019); Wang et al. (2014) and have limited adaptability which makes them cost-prohibitive for disposable instruments Xue et al. (2019).Additionally, the electronics are usually placed away from the transducer which deteriorates the Signal to Noise Ratio (SNR) Suzuki et al. (2018).Placing the force sensor at the grasper may also conflict with functional requirements for monopolar or bipolar cautery instruments He et al. (2014).
Figure 5 summarizes the severity level of different sources that contribute to the sensing inaccuracy as a function of the sensor location (scale of 1 to 3; 1 is minimum, 3 is maximum and is no effect).It also compares how stringent the listed design requirements are for each sensor location (scale of 1 to 3; 1 is the least, 3 is the most and refers to not a requirement).The distribution of the records included in this survey as a function of the sensor locations and the sensing technologies are shown in the same figure.It is evident that the sensorless techniques have the majority of publications over the past ten years.Additionally, the Micro-Electro-Mechanical (MEM) and Fiber Brag Grating (FBG) technologies have been widely adopted in the fabrication of miniature transducers that can be integrated into the gripper jaws.An overview of different transduction technologies is presented in the next section.

Sensing Technologies
Sensing technologies and the corresponding number of articles in this survey is shown in Figure 6.

Sensorless
Sensorless refers to the case where the sensors used for force estimation are already inherent in the surgical robot Stephens et al. (2019).In model-based approaches, the sensors are the encoders and the motor current measurements.In the visionbased techniques, the sensor is the visual feedback of the surgical site through mono or stereo cameras.
Model-Based Model-based techniques can be categorized into 1) analytical models developed based on first principles, 2) disturbance observers and Kalman filters that utilize a dynamic model and the control loop commands and feedback signals, and 3) data-driven models which consider the instrument as a black-box and fit a quantitative model to a customized set of input and outputs.The model-based literature is summarized in Tables 2 and 3.They include 15 analytical models, eight of which are physics-based and 7 studies use observers or Kalman filters.There are 5 articles on the use of data-driven models.
The accurate dynamic model of the surgical instruments is challenging to obtain due to the many sources of nonlinearities e.g.friction, backlash Sang et al. (2017)  Proposed the compensation of the gripping torque which was experimentally identified as a function of the instrument posture.The experiments were on: EndoWrist 1) ProGrasp 2) Large Needle Driver 3) Dissecting forceps.

7-Haraguchi et al. (2015)
The articulated wrist was replaced by a machined spring.The instrument was pneumatically driven.A 3-DoF continuum model of the spring distal joint and the pneumatic pressure were used for force estimation.
Forceps tri-axial Proposed the use of Sliding Perturbation Observer (SPO) to estimate the reaction force.The presented method compensated for the Coulomb friction.Evaluated two approaches for force estimation on the 3 rd link of the Raven-II system: 1) Further expanded on the approach in Li et al. (2016) by adding cable tension estimation.2) The force was estimated by measuring the cable stretch using a linear encoder.

Instrument axial force
Raven-II mm gripper / 6 ERR 1) < 4N, 2) < 3N RNG: 0-10N #2 Provided better estimation at lower forces 14- Li and Hannaford (2017) Used the Gaussian Progress Regression (GPR) supervised learning approach because of its ability to deal with uncertainties and nonlinearity.The model inputs were motors encoder, velocity, and current.

Vision-Based
The existing literature affirms that experienced surgeons use visual cues (tissue and instrument deformations and the stretch in the suture) as sensory feedback surrogates Martell et al. (2011); Noohi et al. (2014).
With the 3D stereoscopic view in robotic surgery providing depth information, and the developments in the available computational power (high-performance Graphic Processing Units-GPUs, cluster computers, and cloud platforms), a noticeable shift towards adoption of vision-based techniques was observed.While mechanical models of the tissue are presented, they are mostly complex and computationally expensive Aviles et al. (2016).Most of the literature (Tables 4 and 5) implement supervised learning architectures (Recurrent Neural Network (RNN) and Long-Short-Term-Memory (LSTM) Aviles et al. (2016); Marban et al. (2019)) with the video stream as inputs to estimate the instrument-tissue interaction forces.The vision-based techniques are robust to many sources of inaccuracy listed in Figure 5.However, they can be affected by the instrument occlusion, smoke and changes in the tissue properties, lighting conditions and camera orientation.The estimation update rate cannot be faster than the video frame rate which is usually 30 Hz.This limitation makes the vision-based approached not suitable for FF applications in which the control loop is desired to execute faster than 500 Hz Jones et al. (2017).The current literature highlights that force estimation through video processing is easier in pushing tasks (characterized by smooth deformations) than those produced by pulling tasks that are characterized by irregular tissue deformations due to grasping Marban et al. (2019).
Force estimation based on using Optical Coherence Tomography (OCT) as the reference sensor is proposed by Otte et al. (2016) and Gessert et al. (2018).OCT images provide volumetric data with a resolution of a few micrometers in which the tissue compression and subsurface deformations can be reflected.Thus, they contain a richer Prepared using sagej.clsImage processing algorithms were utilized for suture strain estimation by identifying the suture line and tracking the displacement of markers.The achieved resolution in strain estimation was two orders of magnitudes smaller than the known strain to failure of most suture materials (20+%)./ Suture pull Force magnitude Mono Strain resolution of 0.2% and 0.5% was achieved in one-marker tracking on stationary suture and twomarker tracking on moving suture, respectively.

2-Kim et al. (2012)
The soft-tissue deformation was obtained by processing the stereoscopic depth image as a surface mesh.It was compared against the original organ shape from preoperative images.A spring damper model was used for force estimation./ Tissue push and pull Gripper tri-axial forces

Stereo
No results were presented

3-Noohi et al. (2014)
A virtual template, based on assuming soft tissue local deformation to be a smooth function, was used to estimate the tissue deformation without a-priori knowledge of its original shape.The force magnitude was estimated by using a biomechanical model./ Tissue push Gripper tri-axial forces

Mono
In force magnitude:  signal space compared to the mono and stereo visions that provide only the surface information.

Strain Gauge
Strain gauges are the most commonly used transducers for force sensing Yu et al. (2018a).They are accurate and small and can be designed in different configurations for multi-axis force sensing.Although the transducers are low cost with a price of $10-$25 per unit Srivastava et al. (2016), they require special surface preparation, adhesives, and coatings for optimal performance that increases the assembly and integration cost Li et al. (2017b).When used in Wheatstone bridge arrangements, they require multiple wires for connection that makes packaging difficult for quick and seamless integration with surgical instruments Shi et al.Trejos et al. (2014Trejos et al. ( , 2017) ) conducted an extensive study on biocompatible adhesives and coatings that can withstand the harsh environment during steam sterilization.However, none of the combinations showed reliable measurements after seven cycles.Tables 6 to 8 summarizes the articles which   2019), and low cost Bandari et al. (2020).However, optical fibers cannot be routed into small bending radii Trejos et al. (2014).Additionally, The presence of small and intricate parts can make fabrication and assembly of fiberbased sensors costly Li et al. (2017b).
The Light Intensity Modulation (LIM) based sensors are vulnerable to light intensity variations due to the temperature or fiber bending Lv et al. (2020).This can be improved by normalizing the optical signal against the emitted power Hadi Hosseinabadi et al. (2019).Alternatively, a redundant strain-free fiber can be used to compensate for the effect of temperature or other sources of uncertainty Puangmali et al. (2012);Song et al. (2014).Table 9 summarizes the articles that address MIS force estimation based on LIM.
The FBG sensors are wave-length coded and insensitive to the changes in the light intensity.FBGs are very sensitive, have calibration consistency, and exhibit high SNR which provide repeatable and high-resolution strain measurements Shahzada et al. (2016).Multiple gratings can be accommodated into one fiber Soltani-Zarrin et al. (2018) simplifying the design and signals routing.Thus, they are also used in shape sensing Lv et al. (2020).Nonetheless, FBGs require interrogators for signal processing which the commercial systems cost between $10k to $100k Yurkewich et al. (2014).The articles which used FBGs for MIS force estimation are summarized in Table 10.

Capacitive
Capacitive methods are attractive solutions for high resolution and compact force sensor designs.Compared to strain gauges, they provide limited hysteresis in microscale and increased sensitivity Sang et al. (2017).However, they have a limited range Li et al. (2017b) and are prone to thermal and humidity drift Bandari et al. (2020).The change in capacitance can be due to the change of the overlapping area or the distance between the two electrodes; the latter provides higher sensitivity and a more linear response Kim et al. (2017).The commercially available Capacitance to Digital Converter (CDC) chips such as the AD7147 from Analog Devices significantly simplify the signal processing, which was believed to be challenging for capacitive transducers Trejos et al. (2014).However, they provide a low sampling rate.
Two sensors consisting of a triangular prism shape and two capacitive-type transducers with an elastomeric polymer dielectric were integrated into the grasper.
Molding was used to fabricate a prototype./ Gripper (tip) Gripper normal (F N ), shear (F S ), pull (F P ), grip (F G ) forces CD RMIS instrument for RAVEN-II / 6 RES: F N :6.4%,F S :3.4%, F P :8.6%, T P : 5.7% RNG: F P , F N , F S :±5N, T P :±3Nmm 7- Seok et al. (2019) Extension of Kim et al. (2018b) with humidity compensation.An AC shield was added to minimize the temperature effects on parasitic capacitance.An anodizing process was applied for electric insulation.The sensors range was extended to 20 N. / Gripper (base) Gripper normal (F N ), shear (F S ), pull (F P ), grip (F G ) forces, axial torque (T P ) CD RMIS instrument for S-surge robot / 6 RNG: ±20 N sterilizability and biocompatibility of the existing literature are not evaluated.

MEMS
MEM sensors (Table 12) operate based on the same physical principles discussed so far.However, MEM fabrication techniques such as deposition, etching, and lithography allow for the cost-effective production of small, fully-integrated, monolithic sensors Radó et al. (2018) with reduced lead time in prototypes and high throughput batch volumes Gafford et al. (2014).Typically, MEMS do not require manual assembly, bonding, and alignment, and provide functional devices after the fabrication process Gafford et al. (2013).
By utilizing MEMS technology, it is possible to develop smart parts (e.g.grippers) with integrated sensing capability for micromanipulation Pandya et al. (2014).Biocompatible coatings can be added to MEM sensors for biomedical applications.

Other Technologies
Piezoelectric transducers do not require an external power supply and have high stiffness Li et al. (2017b).However, they are subject to charge leakage and are not suitable for low frequency and static loads Sang et al. (2017).They are also sensitive to temperature.Piezoresistive transducers used in force-sensitive resistors are scalable with low hysteresis Prepared using sagej.clsand noise Bandari et al. (2017).Nonetheless, their linear response is limited to a small range and they drift under constant load Juo et al. (2020).They do not have the challenges associated with the integration of strain gauges, are relatively insensitive to humidity, and can be used in high temperatures above 170°C Li et al. (2017b).Shape Memory Alloys (SMAs) like Nitinol have a higher gauge factor compared to common metallic strain gauges and provide a larger range due to their stretchability.SMAs require an insulating coating for use on conductive surfaces.They can be clamped at two ends and do not require a backing material with special surface preparation.They are low cost and available at diameters as small as few microns Srivastava et al. (2016).Quantum tunneling composite (QTC) pills are flexible polymers that act as insulators in resting-state but increase conductivity when compressed.They are very sensitive, provide a wide dynamic range, and are low cost (less than 1$/pill).However, they are temperature sensitive and inaccurate in dynamic loading applications Novoseltseva (2018).Recently, vibration frequency and phase shifting due to an applied force have been measured for force estimation by the use of accelerometers.This approach is slow as it needs a few vibration cycles to generate stable and repeatable signals Kuang et al. (2020).

Discussion and Conclusion
In keyhole endoscopy, the surgeon's interaction with the surgical site is via slender instruments that are inserted into the body through small incisions.Despite the many benefits to the patient, the operation is more challenging for the surgeon due to the instruments' limited dexterity, fulcrum motion reversal, uncomfortable posture, and limited visual presentation.Additionally, the surgeon's force perception is affected by the forces between the instrument and the skin and the instrument's dynamics.The adoption of robotic and computer vision technologies resolves the limitations above and significantly improves the accuracy and efficiency in RMIS.However, most telesurgical systems completely isolate the surgeon from the tissue through the local/remote architecture of robotic telemanipulation.This deprives the  surgeon of the rich information in palpation and direct interaction with the tissue.Without force feedback, the interaction of the surgeons with the environment is not as intuitive as direct manipulation and therefore extensive training is required.Moreover, the lack of haptic feedback leads to a higher risk of errors and longer task completion Prepared using sagej.clstime, up to 2 orders of magnitude in complex tasks Hannaford et al. (1991), which may lead to higher surgery costs.
One active research stream in the field of robotic surgery is improving the sense of telepresence for the surgeons, also known as "transparency".Direct force feedback is the most intuitive approach to improve transparency.For a fully transparent haptic experience, reliable interaction force sensing at the surgeon console and the instrument-tissue interface is required.This is in addition to a safe bilateral teleoperation architecture, and a local manipulator that is capable of reflecting the force commands, known as a haptic display.The extensive literature on haptic control indicates a trade-off between transparency and stability Hashtrudi-Zaad and Salcudean (2001).Alternatively, sensory substitution was proposed instead of haptic feedback, in the form of visual, auditory, or vibrotactile cues of force information.Although the safety can be easily guaranteed in systems with SS, it is not intuitive and can cause cognitive overload for the surgeon.The SS methods can also be used in MIS systems because no robotic manipulator is required for force reflection.The efficacy of different haptic feedback modalities in improving the surgical training and surgeon performance metrics has been studied extensively Amirabdollahian et al. (2018); Rangarajan et al. (2020); Overtoom et al. (2019); Abdi et al. (2020).It is shown that a transparent haptic experience and visual feedback of force information improve the performance metrics and shorten the training time for novice surgeons in complex tasks.Apart from haptic feedback, the instrument-tissue interaction forces can be used for tissue damage monitoring, surgical skills assessment, development of surgical training guidelines, and to automate tasks.
Extensive research has been conducted to estimate or sense the instrument-tissue interaction forces.The functional requirements depend on the application.While it is not necessary to estimate the tissue forces precisely to provide an appropriate haptic experience Jones et al. (2011), the bandwidth and sampling rate are important requirements to ensure low latency and smooth interaction with the remote environment.The sampling rate and bandwidth are less critical in SS.
Sensorless approaches utilize the information that is already available in the robotic manipulator; the axes positions and velocities, motors currents, and visual display of the surgical theatre.With the exponential growth, over the past decade, in the available computational power to researchers, data-driven approaches based on supervised learning Li and Hannaford (2017); Aviles et al. (2017) have been widely adopted.Among them, neural networks have shown promising results when trained and used on one particular instrument.However, they require a long and computationally-expensive training phase that is yet clinically-prohibitive.The training is based on a set of measurements at the beginning of the surgery that is used afterward for force estimation throughout the entire surgery.Proposed approaches that have an instrument's operational parameters as inputs, do not consider the variations between instruments and the change of instruments behavior throughout its use Kong et al. (2018).Considering how the research direction has evolved over the past decade, experimentation with different model architectures, development of efficient training, and identification methods that can be automatically performed at the system start-up Spiers et al. (2015), improving the computation time and incorporation of online adaptation techniques are attractive research areas to be further investigated.Moreover, all the existing literature uses the information at the patient manipulator for force estimation, but the inclusion of the operating parameters at the surgeon console may also improve the quality of force estimation.
The sensor design is another avenue towards collecting force data at the instrument tip.The sensor can be located inside or outside the body.The closer the sensor is to the instrument tip the more accurate the measurements are.However, the size, biocompatibility, sterilizability, insulation, and sealing requirements are more stringent when such an approach is followed.Design proposals for sensor integration into the instrument tip have limited adaptability because the instruments for different types of surgery have different shapes at the tip (e.g.EndoWrist cautery forceps, graspers, dissectors, needle drivers, etc.).Therefore, a custom sensor needs to be designed for every instrument which increases the development, fabrication, and maintenance costs.
A variety of transduction principles, including resistive, capacitive, optical, piezoelectric, and magnetic have been used in the development of sensing solutions for minimally invasive procedures.While strain gauges are still the most commonly used transducers, the study by Trejos et al. Trejos et al. (2017) showed that biocompatible adhesives and coatings can only survive a maximum of 6 steam sterilization cycles.Considering that the instruments are typically used 10 times before disposed, this would lead to a 40% increase in the cost of the instruments with integrated strain gauges.Additionally, the installation of strain gauges is labor-intensive that contributes to increased cost.
A comparison of the publications summarized in this article with the surveys by Puangmali et al. (2008) and Trejos et al. (2010) indicates a noticeable shift towards utilizing FBG and MEMS technologies for the development of gripper integrated miniature sensors (Figure 5).FBGs are compact, sterilizable, biocompatible, electrically passive, and immune to electromagnetic noise.They provide high sensitivity with sub-micron resolution and can have multiple gratings embedded in one fiber which simplifies fiver and therefore optical signal management.While the commercial interrogators are expensive, there are signal conditioning solutions proposed to decrease the electronics cost Yurkewich et al. (2014).The developments in MEMS technology have overcome the barrier of scale and cost in the fabrication of delicate miniature sensors.Additionally, MEMS sensors typically do not need manual assembly and can be integrated into the desired application after production.
Another observable trend is the utilization of datadriven regression approaches for sensor calibration.Models based on neural networks and other supervised learning methods such as Gaussian Process Regression have shown unprecedented performance in handling nonlinearities and uncertainties in sensor calibration.Compared to the surgical instruments, the transducers show a more consistent response and do not need regular calibration unless removed and reintegrated.Efficient calibration approaches that can be quickly and automatically performed without operator intervention (e.g based on payload estimation) would benefit the RMIS systems.

Figure 2 .
Figure 2. Instrument's degrees of freedom 2. Depending on the sensor location, the sensing DoFs are defined as instrument or wrist tri-axial forces (F X , F Y , F Z ) and moments (M X , M Y , F Z ), and the gripper normal (F N ), shear (F S ), and pull (F P ) forces as depicted in Figure 3.In summarizing the results, the following acronyms were used: ACC: Accuracy, ERR: Maximum absolute error, MAE: Mean-Absolute-Error, NRMSE: Normalized-Root-Mean-Square-Error, RES: Resolution, RMSE: Root-Mean-Square-Error, RNG: Range, and SENS: Sensitivity.Design Requirements DoF, Range, Resolution, Accuracy, Bandwidth and Sampling Rate The grasping force, the instrument lateral and axial forces, and the axial torque are the most relevant DoFs to improve accuracy and provide an effective haptic experience in MIS applications Wee et al. (2016); Soltani-Zarrin et al. (2018);

Figure 3 .
Figure 3. Sensing degrees of freedom depending on the sensor location

Figure 4 .
Figure 4.The options for sensor location

Figure
Figure 5. Overview

Figure 6 .
Figure 6.The sensing technologies , tendons compliance Anooshahpour et al. (2014) and creep Haghighipanah et al. (2017), elastic deformations, actuators performance variations (the motors' brush conductivity and change in the armature winding resistance) Li et al. (2016), hysteresis Anooshahpour et al. (2016), inertia, and gravity Wang et al. (2014).Additionally, any model relies on a set of measurements (calibration or training set) that are usually taken at the beginning and used throughout the estimation.It is experimentally shown that the tool behavior changes with time which deteriorates the estimation accuracy Kong et al.
(2017)    Developed and identified a dynamic model for the Patient Side Manipulator (PSM) of the daVinci Standard system and the surgical instrument.The identified model was used for external force estimation.
-Faragasso et al. (2014) A force sensing device composed of a linear retractable mechanism and a spherical visual feature was installed on the endoscope.The force was estimated as a function of the size of the spherical feature in the image./ Palpation Instrument axial force Mono RES: 0.08N RMSE = 0.13N RNG: 0-1.96N 5-Aviles et al. (2014)The method used a 3D lattice to model the deformation of soft tissue.An RNN estimated the force by processing the information provided by the 3D lattice and the surgical tool motion./ s full feedback architecture inAviles et al. (2014) was replaced by local and global feedback.The RMSE and computation time were improved.Aviles et al. (2015a) was upgraded to a recursive neural network LSTM based architecture which improves the force estimation accuracy.from Optical Coherence Tomography (OCT) and the instrument trajectories were used as inputs to a Generalized Regression Neural Network (GRNN) to estimate the instrument-tissue forces. of dimensionality reduction on the performance of the RNN+LSTM architecture proposed inAviles et al. (2015b).It showed that implementation of a Probabilistic Principal Component Analysis (PPCA) significantly reduced dimension (75% reduction) and improved accuracy.were estimated by finding stereocorrespondences based on tissues salient features and the use of probabilistic soft tissue tracking and thin-plate splines (TPS).The displacements were used to estimate forces based on a biomechanical model.extension toAviles et al. (2016) where the proposed RNN+LSTM architecture was extended to three-axis force components.Z-axis was normal to the tissue, X and Y axes were planar with the tissue surface.et al. (2015b) with a deeper network, fully connected layers, and a sequence of mono 2D images as inputs.The results were on a sponge, a PET bottle, and a human arm with changes of light and pose.17N, Arm:0.1NRNG: Sponge:0-3N, PET bottle:0-7N, Arm:0-2N 13-Haouchine et al. (2018) A biomechanical map of the organ shape was built on-thefly from stereoscopic images.It used 3D reconstruction and meshing techniques./ Tissue push and pull Force magnitude Stereo Plot comparison, not quantified.

(
2019); Lv et al. (2020).Strain gauges are highly influenced by electromagnetic noise and are not suitable for use close to other tools with strong magnetic fields (e.g.electrocautery) Choi et al. (2017); Xue et al. (2018).They have low sensitivity and often require custom flexures or modifications in the load-carrying structure to amplify local strains Hadi Hosseinabadi et al. (2019).Strain gauges are fragile and require mechanical overload protection Kuang et al. (2020).They typically do not survive multiple sterilization cycles Shi et al. (2019) and lose repeatability.Trejos et al.

Four
L = 0.213N, Dir = 4.37°, F G = 0.747N RNG: F L : ±10N, Fg: 0-20N 5-Shahzada et al. (2016) Four FBG sensors were attached to the instrument distal shaft in a two crosssection layout which is insensitive to the error caused by combined force and torque loadsFBGs were integrated into the articulated wrist.The differential wavelength shift was used to achieve robustness to temperature and gripping force.with sliding stretchable T-shaped parts for enhanced axial strain.Axial FBGs were at the grasper's bending neutral axes and its surface.N < 0.57N, F P < 0.78N 2: F N < 0.81N, F P < 0.9N RNG: F N : 0-10N, F P : 0-6N9-Xue et al. (2018)The cable tensions were measured by FBGs pasted in the grooves on inclined cantilevers integrated into the Instrument base./ Instrument base (Cable tension) had a miniature flexure based on a Sarrus mechanism to achieve high axial sensitivity and a large measurement range.An FBG was tightly suspended along the central axis of the flexure.

)
Two sensors with 3 electrodes and common grounds were integrated into the Gripper.The dielectric was air and the signal processing electronics were local./ Gripper (base)Lee et al. (2014) with the 3-axis force sensor moved into the articulated wrist and two capacitive torque sensors in the tendon drive pulleys of the gripper jaws./ Articulated wrist and instrument base Wrist tri-axial forces, grip Force CD RMIS instrument for RAVEN-II / 6 NRMSE: F X =0.039 , F Y =0.056 , F Z =0.026 RNG: F X : ±1N, F Y : ±1N, F Z : ±1.6N 5-Kim et al. (2018a) Proposed a capacitance sensing PCB with 8 electrodes and a CDC chip and a conductive deformable structure as the common ground.A spherical cap was added to the sensor for testing it in a palpation taskX , F Y , F Z : 5.5%FSO, M X , M Y , M Z : 2.7%FSO RES: F X : 0.22mN, F Y : 0.31mN, F Z : 0.11mN, M X : 0.47mNmm, M Y : 0.41mNmm, M Z : 0.17mNmm RNG: F X , F Y , F Z : 1N, M X , M Y : ±20Nmm, M Z : ±10Nmm 6-Kim et al. (2018b) Extension of Kim et al. (2016) with a slight modification in the proximal gripper jaw and calibration scheme so that the combination of the capacitive transducers also resolved the axial torque about the gripper./ Gripper (base)

F
axis force-torque sensor chip composed of 16 piezo-resistive beams was fabricated by using ion beam etching and surface doping.The sensor is 2x2 mm and installed onto the grasper. 0-40N, F S : ±12.5N,F P :±12.5NM N : ±15Nmm, M S : ±100Nmm,M P : ±100Nmm 6-Dai et al. (2017)Proposed a 3-axis capacitive force sensor with differential electrodes.The compressive load reduced the dielectric thickness, and shear forces changed the overlap area.The sensor was fabricated using MEMS lithographyion etching (DRIE) to fabricate a monolithic silicon-based 3-axis force piezoresistive sensor.The sensor was covered with a semi-sphere PDMS polymer.

F
N < 1.58N, F S , F P < 0.31N RNG: F N : 0-35N, F S , F P : sensor that used a piezoresistive transducer to measure normal force and LIM in optical fibers to estimate the tissue deformation.The sensor was out of silicon for biocompatibility. of a pneumatic balloon in a cavity on the surgical instrument or endoscopic camera.When needed, the membrane is inflated to contact the tissue.The pneumatic pressure and volume are monitored to estimate the force.was excited by using a vibration motor.The structure's triaxial acceleration signals in time-domain showed discernible ellipse-shaped profiles when a force was applied.The acceleration profiles were characterized via regression to estimate the direction and magnitude of the applied force./ Vibration monitor / Proximal & distal shaft Instrument tri

Table 1 .
Dexterity index definition for MIS instruments

Table 2 .
Sensorless force estimation: model-based Pneumatic Actuation Muscles (PAM) were used in design of a custom forceps.Disturbance observers were used on the pneumatic actuation system and the robot joints.A wrist actuation design using planetary gears was proposed to decouple the motions in different DoFs.The motor currents were used to estimate the forces in static and dynamic scenarios.

Table 3 .
Zhao and Nelson (2015)ation: model-based -continue.Developed the dynamic model of one jaw by using the Benson model to describe the dry friction.The parameters were experimentally identified for an instrument designed based on the concept inZhao and Nelson (2015).Proposed the use of NNs optimized by a Genetic Algorithm (GA) for force estimation.The model inputs were the motors' positions, velocities, and currents.The performance of NNs, decision tree, random forest, and support vector machine models were compared in the angle and gripping torque estimation of each jaw.It concluded that the NN estimations were reliable when trained and tested on each jaw, on the same tool, and withing the frequency of the training data.

Table 4 .
Vision-based force estimation

Table 5 .
Vision-based force estimation -continue Took an undeformed reference volume and a deformed sample volume from OCT as inputs into a Siamese 3D-CNN architecture and output a 3D force vector.The results were compared with CNNs that take the difference or the sum of the undeformed and deformed volumes, and a CNN that takes 2D projected surface images as inputs.
(CS at an external sensor) Prepared using sagej.cls

Table 10 .
Table 11 lists the articles that are based on the capacitive transduction principle.The Optical force sensing: FBG

Table 13 .
Other force sensing technologies