ASSET Rater Training Information

ASSET Global Rating Scale

Similar to global rating scales validated in other surgical fields, a 5-point Likert scale with descriptors at 1, 3, and 5 is utilized for ASSET. Rather than assigning arbitrary numbers to the descriptors, the ASSET was designed to correspond to the levels of the Dreyfus model of skill acquisition with “1” representing the novice level, “3” representing the competent level, and “5” representing the expert level of arthroscopic skill performance. It was thought that this would help improve reliability and would lend it toward use within the context of an already accepted model of professional skill acquisition.

The total score of the ASSET is the sum of the 8 domains with a maximum of 38. However, for an individual to “pass” and be considered competent for the technical portion of the procedure they must achieve a minimum score of “3” in each of the 8 domains being assessed. It was felt that some of the previously proposed arthroscopic assessment tools had multiple domains that assessed similar skills which may place too much weight on a particular skill and could lead to confusion by the raters, therefore effecting reliability. For the ASSET, each domain was designed to assess a unique facet of arthroscopic skill acquisition and every attempt was made to limit overlap.

ASSET Domains

Safety

This assesses the ability of the surgeon to use the arthroscope and instruments in a manner that protects the articular cartilage and soft tissue from unnecessary damage. A surgeon is considered competent when they are able to avoid significant damage to articular cartilage and surrounding soft tissue. Lower scores are given when there is clear video evidence of damage to articular cartilage or surrounding soft tissue that was caused by the operating surgeon. If not clear evidence of such damage exists the surgeon is given the benefit of the doubt and is given a higher score. This domain may be more useful for simulated arthroscopic environments where examinees will be allowed greater autonomy and there is no risk of patient harm. In the live assessments it is expected that attending intervention will occur prior to any significant damage.

Field of View

This domain evaluates ability to position the arthroscope tip and orient the light source on an angled arthroscope to achieve an optimal field of view. A competent score is given when a surgeon demonstrates an ability to use the arthroscope and light source to fully visualize structures and treat pathology within the joint. Novice scores are given for inadequate positioning resulting in narrow or inadequate views. Examples of novice performance include placing the arthroscope tip too close to the structure being evaluated or leaving the light source angled inappropriately away from the intended target (for example having the light source oriented to 9’o clock when 3 o’clock would optimize visualization).

Camera Dexterity

This domain measures the surgeon’s ability to move the camera within the joint. A competent score is given for the surgeon that is able to keep the camera steady, oriented to the appropriate plane (horizontal or vertical), and centered throughout most of the procedure. Novice scores are given for those who are unable to keep the camera centered and demonstrate rough or uncoordinated movements that increase the potential to cause articular cartilage damage. Since competent surgeons must be able operate the camera with both hands depending on the side of the patient that is being operated on, this domain can be combined with “instrument dexterity” to assess the effect of surgeon hand dominance.

Instrument Dexterity

Similar to the previous domain, this assesses the ability of the surgeon to use instruments within the joint. Competence is given to the surgeon who demonstrates appropriate use of instruments and an ability to accurately direct them to targets. Lower scores are given for individuals who are overly awkward or aggressive and frequently miss or overshoot targets. For example, a novice surgeon would demonstrate slow, wide swinging movements and difficulty directing the probe to its target and may struggle to use appropriate force possibly resulting in the probe being pulled out of view or out of the portal.

Bimanual Dexterity

This measures the ability of the surgeon to triangulate within the joint and coordinate the use of both hands. To be considered competent, a surgeon must demonstrate an ability to use both hands, such as being able to follow one’s probe from one compartment to the next in the knee. Lower scores are given for those are unable to coordinate movements between hands such as constantly losing the probe or struggling to place the probe in the field of view.

Flow of Procedure

Flow is the ability of the surgeon to complete tasks and progress from one portion of the case to the next in an efficient manner. A surgeon is considered competent when he or she is able to follow a planned, logical sequence of tasks and progresses from one task to the next with only minor difficulty. Lower scores are given for surgeons that follow a disorganized sequence of tasks, frequently pause to determine the next step, or require multiple attempts to successfully complete each step of the procedure.

Quality of Procedure

This domain assesses the overall quality of the procedure performed. A surgeon gains a competent rating in this domain by performing an adequate procedure that does or would not require the supervising surgeon to correct any flaws in the final product. Lower scores are given for performing an inadequate or incomplete procedure.

Autonomy

The final domain measures the ability of the surgeon to perform the procedure independently. This domain scored on a scale from 1 to 3 because these are the only levels of autonomy that can be assessed using intraoperative video collected through the arthroscope. For ASSET assessments, physical interventions by the attending surgeon are documented by stopping the recording. Once the surgeon being evaluated regains control, recording is resumed. A surgeon must be able to perform the procedure without intervention to earn a score of “3” in this domain and be considered competent.