July/August 2016 Issue

Social Services Software: Evaluating Therapist Empathy With Psychotherapy Software
By Susan A. Knight
Social Work Today
Vol. 16 No. 4 P. 8

When choosing a psychotherapist, it's not uncommon for someone to feel a sense of vulnerability, anxiety, and perhaps even confusion around the process. How does one choose the right therapist? Besides official credentials and accreditation, what assurance is there that a therapist is truly competent? How can practitioner competence be objectively evaluated in the context of therapeutic counseling—a treatment method that is relationship and conversation based?

A multidisciplinary team of researchers has taken a major step forward in addressing this by developing software that can be used for training and evaluating psychotherapists. With backgrounds that span several different disciplines including psychology, engineering, computer science, and linguistics, the team has developed a software tool that can analyze the vocal content of a psychotherapy session and measure the level of empathy conveyed by the practitioner.

"Right now, there's no way to evaluate the interaction that takes place between a therapist and a client in an objective way," explains Zac Imel, PhD, an assistant professor of counseling psychology at the University of Utah. "This software offers a powerful tool for service providers to give them feedback on the work they're doing."

What Makes a Therapist Effective?
Many factors contribute to a therapist's effectiveness. While this continues to be an active area of research, the research to date has established that the best therapists are those who are able to form the best relationships with their clients. "The most effective therapists are the ones that are best at making their clients feel understood, the ones that are best at inspiring hope," Imel explains. "Interpersonal qualities distinguish good therapists from those who struggle more." Currently, however, there is no mechanism for measuring such skills among practicing therapists.

David Atkins, PhD, a University of Washington School of Medicine research professor of psychiatry and behavioral sciences, is part of the software development team. In the university's news release for the software, he highlighted the difficulty arising from the current lack of objective evaluation tools: "In terms of individual quality of the therapist, it's like not knowing whether the pill in your hand is as effective as the others in the bottle."

The lack of effective evaluation mechanisms has a significant impact on the training and feedback that therapists receive. The traditional evaluation approach has been for third-party evaluators, known as coders or raters, to provide feedback based on audio or video recordings of counseling sessions. While this approach has its strengths, it is highly inefficient. There's the initial time spent providing training in the process and then the actual time required to review and analyze the recorded sessions. Using human coders also makes the evaluation process a highly subjective one. Atkins points out that two coders, in spite of their training, won't necessarily arrive at the same conclusions from reviewing the same therapy session.

Imel notes that it's been several decades since Carl Rogers, PhD, credited with developing the person-centered approach to psychotherapy, started recording therapy sessions for the purpose of analysis. "We've had no fundamental shift in how psychotherapy is evaluated since then," Imel explains, "whereas other areas have been able to utilize advances in technology."

Another member of the software development team is Shrikanth S. (Shri) Narayanan, PhD, a professor of electrical engineering, computer science, linguistics, psychology, neuroscience, and pediatrics with the University of Southern California's Viterbi School of Engineering. He is also the director of the Signal Analysis and Interpretation Lab at the university. The computational model of empathy that the software uses is largely based on the lab's research and development work in behavioral signal processing.

Computational Algorithms That Inform Clinical Interventions
Using data collected from more than 1,000 psychotherapy sessions, the software has essentially been taught to recognize speech patterns that are deemed to be empathetic in nature. The result is a computational model of empathy which can be used for evaluation purposes to assess whether a particular exchange exhibits language qualities that are reflective of empathetic speech patterns. In addition, through the use of computational algorithms that process all of the data, the software is able to shed further light on the composition of effective therapeutic communications.

"What works? When does it work? Why do some strategies work more effectively than others?" Narayanan asks. He explains that by developing computational algorithms that provide an increased understanding of human behavior, these tools can be used to support and inform clinical interventions that care practitioners rely on.

A major advantage of these computational algorithms is their ability to process large volumes of data in a consistent, reliable, and structured way, allowing for additional insights and learning. In this regard, the computer's processing capability can reasonably be considered superior to human observation of an isolated therapeutic exchange. "We have the ability to identify patterns in the data that would be extremely hard to detect from just a single interaction," Narayanan says. "We're able to glean information at a level of detail more granular than just an overall impression."

This type of analysis and the ability to quickly detect patterns across a large dataset creates an enormous amount of potential. "The speed of computers, combined with algorithms and the amount of data we have, is joining together in a way where you can do so much more with it," Imel says. "If you feed them [computers] enough data, be it text data or audio data, they can learn patterns. Based on those patterns, they can end up making predictions that are accurate."

Imel sees a variety of applications for the software, such as training in academic settings and ongoing professional training, as well as research applications. In the case of training, for example, there is usually limited faculty dealing with large numbers of students. These conditions limit the amount of direct observation that students receive. While there will always be a role for humans, he says, this software supplements the human work.

Broad Applications in the Community
Beyond traditional psychotherapy settings, Narayanan sees broad applications in the community for the software and the related technologies that it draws on. The ability to model and evaluate the expression of empathy has relevance in any environment where there are human-centered interactions taking place. Given the ability for speech and language patterns to provide insight into underlying conditions and mental states, these technologies could be applied in the treatment of autism, in pediatrics, in behavioral care, and elsewhere within the social work profession.

He cites treatment programs for mental health as a specific example where use of the technology could have far reaching benefits. "Mental health and well-being is important to society on a whole. The possibilities offered by these new technologies have enormous potential for helping both the service providers and the people they provide services to."

Elaborating on the role of mental health interventions within the continuum of care, Narayanan points out that mental health issues can be especially challenging to treat. "Mental health differs from other medical conditions in that it is often pervasive and dynamic," he says. "This research and these new technologies can help to connect the dots, fill in the gaps, provide new insights, and offer ways to support the work of clinicians and care providers within the [health care] ecosystem."

Collaborative and Multidisciplinary Problem Solving
Narayanan says that it's exciting to see clinicians, researchers, and technologists coming together and working collaboratively in order to find new ways to advance science and help people in meaningful ways. He appreciates how projects such as the development of this evaluation software allow researchers to apply scientific and technical knowledge in a way that has direct relevance for improving human health and well-being.

Imel echoes these sentiments and emphasizes that "this sort of work is multidisciplinary at its core." He shares that an exciting and rewarding aspect of this type of research is how it brings individuals together across disparate disciplines, such as clinical psychology and social work with electrical engineering and computer science. The work is unique in that it has people coming together from completely different backgrounds to work collaboratively on solving problems.

The Next Stage: Real World Settings
The next stage of research, Imel says, is to start using the software "in real world clinical settings, under real conditions, with real providers." In the long term, the team would like to have a software tool that can record and then unobtrusively generate real-time feedback during a therapy session. The ability to provide this type of immediate feedback has enormous potential for transforming and improving training and evaluation processes, and facilitating therapists' ongoing skill maintenance and development. And unlike human-based assessment and feedback, a software tool providing real-time feedback can readily be scaled up for widespread use across training and clinical settings.

As Atkins has expressed, the ability to assess the quality of psychotherapy services is critical in order to ensure that patients receive quality treatment. However, relying solely on human judgment for this is insufficient. The team's software has the potential to serve as an efficient and reliable quality assessment tool, contributing to the type of objective quality assessment that is very much needed in the field but currently lacking.

"This is an opportunity to move the ball forward on measuring the quality of interaction we have with our clients," Imel says. "For therapists, this could really impact the work we do and how we do it."

— Susan A. Knight works with organizations in the social services sector to help them get the most out of their client management software.