Full Length Research Paper
ABSTRACT
We assessed patients’ perceptions of the utility and impact of a same-day self-administered tablet-based patient-reported outcomes (PRO) assessment integrated into routine HIV care in two North American clinics. People with HIV (PWH) self-administered a PRO assessment inclusive of psychosocial and behavioral topics prior to their routine care visit. Providers were given succinct summary results before seeing the patient. We conducted semi-structured 1:1 interviews after appointments assessing utility and perceived impact of PROs. We coded themes using qualitative software. PWH (n=30; mean age 48) reported PROs facilitated honest responses, improved recall, enriched communication, expanded comprehensiveness of care, and promoted self-evaluation. Several PWH reported feeling more ‘cared about’ when asked about mental health and quality of life-related needs. PWH found PROs administered before the routine care appointments were useful for prioritizing discussion topics with their providers, initiating discussion of sensitive issues, and improving comprehensiveness of/satisfaction with care.
Key words: Patient-reported outcomes, HIV care.
INTRODUCTION
HIV, in its emergence as a manageable chronic condition (Hogg et al., 1998; Palella et al., 1998; Murphy et al., 2001) has ushered in an era in HIV care of emphasizing the management of comorbidities over the life course and an opportunity to focus on improving quality of life. As such, modern HIV care requires a multidimensional assessment of patient health that queries and helps contextualize mental health, symptoms, and health behaviors. These areas are often most easily and accurately elicited by patient report. Both people with HIV (PWH) and their HIV care providers have prioritized the importance of assessing domains such as depression, medication adherence, substance use, sexual risk behavior, HIV-related stigma, intimate partner violence, and social support as part of routine care (Fredericksen et al., 2015, 2019). However, in HIV care many of these domains are commonly under-addressed and inadequately or not assessed (Messiah et al., 2001; Gross et al., 2002; Conigliaro et al., 2003; Morin et al., 2004; Crane et al., 2017). Time constraints, social desirability bias, and language/culture barriers are common reasons (Kissinger et al., 1999; Williams et al., 2002; Narayan, 2010).
Routine, systematic elicitation of patient report, through the use of patient reported measures, also known as “patient reported outcomes” (PROs) (U.S. Department of Health and Human Services and Federal Drug Administration 2006), improves provider detection and ability to address multiple difficult-to-measure domains of care "Crane et al 2017". PROs have been useful to providers in HIV and other care settings (Wolfe et al., 2003; Irwin et al., 2015; Fredericksen et al., 2016), improving patient outcomes (Ruland et al., 2010; Cleeland et al., 2011; Basch et al., 2016, 2017), detection of symptoms (Velikova et al., 2004; Mark et al., 2008a; Sharma et al., 2016), adverse health behaviors (Berry et al., 2011; Crane et al., 2017), and mental health issues (Espallargues et al., 2000; Crane et al., 2017), and by helping providers to prioritize the most relevant concerns (Mark et al., 2008a; Fredericksen et al., 2016). PROs improve patient-provider communication, by helping patients take inventory of their needs, empowering them to raise concerns, and facilitating discussion of sensitive topics (Wagner et al., 1997; Taenzer et al., 2000; Brown et al., 2001; Detmar et al., 2002; Velikova et al., 2004).
From the patient perspective, studies have shown that PROs increase satisfaction with care (Wasson et al., 1999; Taenzer et al., 2000; Chen et al., 2013; Nelson et al., 2015). However, to ensure sustained success of use of PROs in clinical care, patients must find PROs acceptable to use, easy to self-administer, and useful to their goals. A multisite quantitative study among PWH assessing patients’ perceived ease and acceptability of using PROs in their care found that a 10-12 minute assessment of health domains including antiretroviral (ART) adherence, sexual risk behavior, intimate partner violence, depression, substance use, and others, self-administered via touch screen computer tablets on-site prior to the appointment with results delivered to the provider prior to meeting, to be highly acceptable (Fredericksen et al., 2020). However, few studies have interviewed patients at length about the acceptability of PROs. While a few qualitative studies with patients in other types of care settings have found such assessments acceptable and usable (Wasson et al., 1999; Wolpin et al., 2008; Wu et al. 2011, Jones et al., 2014; Sarabia et al., 2015, Stover et al., 2015; Sharma et al., 2016), to date, there has not been a published qualitative assessment asking PWH themselves how they perceive the value and impact of PRO assessments in their care. To address this, we assessed patient perceptions of the utility and impact of a same-day, self-administered, tablet-based patient-reported outcomes (PRO) assessment that had been integrated into their routine HIV care in two North American clinics.
METHODS
Background
These interviews were part of a larger evaluation project, the PROgress study, which sought to understand the effectiveness and impact of implementing a self-administered touch screen electronic PRO into clinical HIV care in two North American HIV care outpatient clinics: the Midway Specialty Care Center (MSCC) in Ft.
Pierce, FL, USA, and St. Michael’s Hospital (SMH), Toronto, Canada, between August 2018 and July 2020. A full description of the PROgress study is available at https://progresshivcare.org/.
Sites
We selected the study sites based on interest in exploring PRO implementation in their practice, the number of providers to experience the intervention, caseload, and patient demographic clinical, and geographic diversity. MSCC in rural Ft. Pierce, FL, serves ~1500 patients, with a high rate of uninsured and impoverished patients, 60% of whom are women and over half of patients are ethnic minorities. SMH in Toronto, Ontario, is an urban outpatient hospital-based clinic serving ~1800-2000 PWH; 70% are men, with a high proportion of men who have sex with men (MSM).
Participating PWH
The study included English-speaking adults (aged ≥ 18 years at study entry) with a diagnosis of HIV infection who were attending one of the participating clinics for a routine visit during the study period and had completed the PRO on that day. Those with known acute or significant prohibitive psychiatric, cognitive, or motor impairment, or those who appeared acutely intoxicated, were excluded.
Recruitment
The local research coordinator invited a convenience sample of patients at each site to participate in an individual qualitative interview occurring immediately after their routine clinic visits (same day) during which they had been administered the PRO. Patients were either telephoned in advance of their appointment to invite them to stay later for the interview, or offered the interview immediately after the appointment. Patients were assured that choosing to participate or not would not impact their care and that all information would be kept confidential prior to de-identification. We intentionally oversampled members of ethnic minority groups and women. All project activity was approved by respective Human Subjects Review entities (SMH through its institution-based Research Ethics Board, and MSCC through the University of Washington’s Institutional Review Board).
PRO assessment
We selected instruments for the PRO assessment based on their brevity, validity, and potential to inform decision-making during the visit. We built on the lessons learned from PRO integration into the Centers for AIDS Research Network of Integrated Clinical Systems cohort (Kitahata et al., 2008). Input from clinical site team facilitated further tailoring for site-specific needs. The PRO assessment at both sites included: mental health (depression/suicidal ideation [PHQ-9] (Spitzer et al., 1999, Kroenke et al 2001), anxiety [single item from HIV symptom index] (Justice et al., 2001); health behaviors (antiretroviral adherence (Simoni et al., 2006; Lu et al., 2008), substance use [ASSIST, AUDIT-C] (Bush et al., 1998; Bradley et al., 2003, Newcombe et al., 2005), nicotine use (Kiechl et al., 2002; Nance et al., 2017), sexual risk behavior (Fredericksen et al., 2018); circumstantial factors (housing status (Whitney et al., 2020), intimate partner violence [IPV] (Fitzsimmons et al., 2019), and other forms of screening (nutrition [Canadian Nutrition Screening Tool] (Laporte et al., 2015), attitudes toward medications [2 items from HATQOL] (Holmes and Shea, 1998), sexual orientation, gender identity). Measures unique to individual sites included a review of symptoms measure at MSCC; SMH used a measure querying Canadian citizenship status, as well as a measure querying sex practices under the influence of illicit drugs. We applied skip logic wherever possible to minimize patientresponse burden. The number of questions included in the assessments ranged from 65 to 101 at MSCC and from 51 to 1 at MH. Patients self-administered the PRO assessment in electronic format on a touch-screen iPad tablet. A small number of questions appear to the patient on the screen at a time with large radio-buttons next to each response option. Once the patient has made their selection on the final question on the page, a new screen with subsequent questions automatically appears. Patients have the ability to move forward and backward through the PRO using navigational arrow icons at the base of the screen, and may skip a section without answering by using the forward button. They may also change their response from an earlier screen by navigating there using the “back” button. A progress bar at the bottom of the screen indicates the proportion of the PRO assessment that has been completed and how much content is left. A “help” button is visible throughout the assessment, in case patients have questions about or difficulty with the assessment; use of this button prompts real-time response from staff to check in.
Automated algorithms generated PRO scores and results. Patient completion of the PRO automatically prompted the printing and/or electronic transmission of a 1-page summary of results (Figure 1) which was shared with the provider immediately prior to seeing the patient.
Interview guide development
Two seasoned qualitative researchers with expertise in evaluating patient perceptions of PRO data collection in HIV care developed the interview guide in conjunction with team members with expertise in health evaluation research and clinical HIV care. Areas of inquiry were informed by previous study findings with PWH on this topic (Fredericksen et al., 2019), as well as content from a validated scale of acceptability adapted for evaluating PROs ((Tariman et al., 2011), and included ease of use, perceived impact on patient-provider communication, and perceived impact on quality of care. Sample questions included: “Do you think having your provider see your answers to these questions affected their ability to take care of you/give you good quality care? Why or why not?”, “How do you imagine that it would be different, if at all, if your provider were instead verbally asking you these questions?”, and “In your opinion, do you think taking the PROs affected how the appointment went? If so, how?
Interviews
We conducted semi-structured individual interviews lasting up to 60 minutes, querying ease of completing the PROs, interest in responding to the questions, any perceived changes in the patient-provider communication due to the provider receiving a summary of the PROs, perceived impact on their care, and interest in having the clinic implement PROs as part of their standard practice during routine office visits. Patients were remunerated at the equivalent of $50 USD. Qualitative data were collected by digital recorder and transcribed.
Analysis
Qualitative data were coded using Dedoose qualitative software (v. 8.3.41) by two trained, seasoned qualitative researchers (RF, EF).
We initially coded within general pre-established thematic areas based on the interview questions: usability of PRO platform, perception of general value of the PROs to their care, perceived impact on communication with their provider, and criticisms/ recommendations for improving its use in care. Two coders independently identified key concepts within each category using an open-coding method, and met to discuss these, reconciling differences in interpretation to create a unified coding scheme for categorizing the data. New coding categories included usability of PRO platform; improvement of recall of health concerns; promotion of self-evaluation; facilitates honest responses; improves patient-provider communication; expands comprehensiveness of care; enhances sense of being cared for; criticisms/concerns regarding 43% Black, 36% white (Table 1) reported primarily favorable views of self-administered, electronic PROs. PWH unanimously reported finding the the PRO interface easy to navigate and use:
Easy, and I’m not a big computer person (Male, 57, FL)
It was self-explanatory. You hit [the icon that says] “Next”. You read and get through with it and go to the next page. It was easy. (Male, 54, FL)
The interface was fantastic. Like I said, I'm glad the [progress] bar was there, so you can kind of see where you're going. (Male, 29, ON)
I don't really know much about electronics. I have my cell phone and every time I'm using it, if I have a little difficulty, I have to call my kids. But this was kind of fun experience. (Female, 55, ON)
Patients reported PROs added value to their care in several ways. These included (1) helping feel prepared for their appointment through improved recall of their own health concerns and needs; (2) promoting self-evaluation in psychosocial and behavioral areas; (3) facilitating honest responses, by increasing comfort level in answering topically sensitive questions; (4) improving patient-provider communication; (5) expanding comprehensiveness of care, and (6) enhancing a sense of feeling ‘cared about’.
Improves recall of health needs and sense of preparedness for visit
Most PWH found PROs were a practical tool for helping them remember what issues to bring up with their provider, thereby allowing them to feel better prepared for the visit. It also gave them a better idea of the spectrum of issues that might possibly be discussed.
I think the questionnaire is a good thing to quickly filter out what needs to be addressed…not everybody comes mentally prepared in terms of having questions and a goal, ‘I should have asked at the doctor that. Why didn't I ask him that when I was there?’ Sometimes you don't realize that something needs to be discussed until you have to fill out a questionnaire. So that's a good thing. (Male, 47, ON)
It's good to have a ‘light bulb’ of certain things to remind yourself to ask. (Male, 39, ON)
…maybe if I didn’t saw it [the PRO questions], and [had not had] an idea of what the question gonna be, the person asked me, I wouldn't even answer. But just the fact that I already read it and aware of it, so when the question asked, I was more prepared. (Female, 55, ON)
[The PROs] just reminded me of some questions that I wanted to ask my doctors. I didn't know what my appointment was for or what to expect when I came in today, so it was good just for me to be able to know some questions that I should ask on my own. (Male, 29, ON)
Promotes self-evaluation
Several PWH indicated that the PROs prompt them to take inventory of their health status and act as a tool for self-reflection:
[The PROs] make you think about life and what you’ve been up to the past, since the last time. (Male, 57, FL)
Well, I think it maybe helps you look at yourself more clearly and get a better understanding of who you are. (Female, 64, FL)
It make[s] you like kind of just open your eyes to…your life at a certain point. (Male, 21, FL)
The [PROs] make you think…I'm thinking about last time, if I'm doing better at certain things. If there's anything showing up that I didn't have before... there's a lot of depressions and things… just kind of see where I'm at with all that. Because a lot of times, when you're in it, you don't know you're in it, or you don't think about [it]. (Male, 61, FL)
I actually really liked the question about how the medication – whether it impacts your day. It's strange...it was kind of nice that it was acknowledged…reading it made it become real, like, "Yes, it does get in the way sometimes." (Male, 29, ON)
Facilitates honest responses
PWH noted that the automated format of PROs helped allow for more honest responses, particularly to psychosocial and behavioral questions, relative to being asked by their provider face-to-face.
[The PROs] ask you questions that nobody – that other people don’t ask you. That’s the way of getting to know how a person really feels inside…sometime, you don’t normally talk to people like you want to. Friends and family and sometimes you keep stuff all up inside, so you really don’t open up to nobody. (Female, 53, FL)
It’s just easier to answer [PRO questions] that way [on the iPad tablet]...if somebody was asking those questions [in person], it would be like you’re being, I don’t know, on trial. (Male, 57, FL)
Social desirability bias was the key factor in why answering questions on a tablet felt easier than face-to-face conversation:
It’s harder if [your provider is] somebody you know all the time. You don’t want to let them down…you build up a façade. I guess we all do. You know, trying to be somebody that we’re not. Or we think ‘this is how they want us to be’. We do things. But we don’t want people to know about it..[but] the computer is impartial. It’s completely impartial. (Male, 38(a), FL)
Sensitive topics, such as substance use and sexual behavior, were easier to answer on the tablet:
It's a lot different to [talk about drug use] to somebody because you're scared of the facial expressions, you're scared of the reactions. The iPad is not going to look at you because you say you do crack every day. So you can be a lot more open when you're [answering questions on] the iPad by yourself. So if you're not feeling… comfortable talking to a doctor about it at first, it's a lot easier at least when they come in and they already know. (Male, 21, FL)
I find that it's easier to answer to an iPad than it is to a person. I found certain questions, like the alcohol question, I was a little more honest on the iPad versus [face-to-face] – because I feel like you're expecting a judgmental look back, even though my doctor is actually extremely great no matter what I've done. Or not done. But the iPad makes it sort of less [judgmental]. It takes away from that, so you don't have to worry about – because it's on a sheet of paper, someone else is going to make their look while they're reading it, so you don't have to see it. You don't have to acknowledge it. (Male, 29, ON)
I mean, [PROs] save the doctor and everyone a lot of time asking a bunch of questions that you might not be honest about. You know what I mean? But if it’s just between you and an iPad…[it’s a] tool a little bit more confidential…rather than trying to explain yourself to someone, so…I quit smoking for a year and then my dumb ass started smoking again. And [my provider] found out, because on the questionnaire…because she asked about the smoking… for sure she must have read it, because she ain’t going to know I started smoking again. I aired myself out real good before I come in here. I was going to hide it until [the PRO] asked me the question. I’m like, damn! (laughs)...so it helps you. You got to be a little bit truthful too…it helps. (Male, 45, FL)
I think [PROs] probably to get people to open up…I wouldn't just randomly go to my doctor and say, "Okay, I have – [this sexual issue]", no. But just the fact of seeing the question there and get to actually look at the question, look at the [response options], I was able to answer that, yeah. (Female, 55, ON)
Improves patient-provider communication
PWH reported feeling that PROs helped improve communication with their provider, particularly by identifying mental health and quality of life needs that might otherwise have gone unaddressed. Some PWH emphasized that by doing so, PROs made the appointment more efficient:
It definitely helps the doctor, because they come in and he already knew certain things…they print it out I guess, and [the provider] kind of looked at a few things and started asking me himself…he already knew where to start. (Male, 57, FL)
I know the doctor would be sitting and asking me questions that probably [are] not relevant…[PROs] kept it focused on what's going on. (Male, 30, ON)
[Providers] know more…about you. They can go through [the PRO results] and figure out instead of asking. Because a lot of times they see a lot of people every day, so it’s hard to ask them what’s wrong. It’s like I said, it helps them remember…about you. She knew I quit smoking and now I smoke again...without that she wouldn’t know. I think it benefits the doctors as much as it benefits everyone else, because they don’t have to ask a bunch of embarrassing questions. (Male, 45, FL)
A common theme was the sentiment that PROs help improve the tone of communication, by depersonalizing and ‘defusing’ sensitive questions:
If you [as a provider] ask some sensitive question, "So have you done crystal meth or have you did crack cocaine?" Someone could be, "What are you guys asking the question for?" [Laughs]. But…the iPad is a good method, if it's coming from the iPad. (Male, 27, FL)
…some [questions from providers] can hurt your feelings. They might not intend to hurt your feelings, but it could be the way it comes out. Which they probably wouldn’t mean no harm. But I would prefer [answering questions] on the pad. To me that was much easier and more sensitive, and I got a better understanding, and I just told the truth. But sometimes [providers take] the tone of a person’s boss. To me, that was better on the pad. That was better for me. (Female, 64, FL)
Expanded comprehensiveness of care
Several patients appreciated that the PRO assessment increased the general comprehensiveness of their care:
The questionnaire was all things that you think about at different times, but it's all good that it was all compiled into one thing….there's times that I am anxious…there's times that I am worried about my housing or my rent. There's different times that you feel one of those things that was in that questionnaire, so it's good that it's all compiled into one thing. (Male, 39, ON)
It’s sort of a nice little check with the doctor to make sure they're not missing anything. Because it's sort of a
structure and it's easy for them to just go right through the list to see if there's anything – there's indicators on the form that say what they should be looking for, which is great, because you might not think it's a problem, but then it is [a] problem, so it's good we would identify it…very valuable. (Female, 65, FL)
Some patients noted the comprehensiveness of PROs by the fact that they covered issues that might not have otherwise been discussed:
…these questions…on the iPad… really did a lot for me because you had a lot of questions that I probably wouldn't have asked the doctor, or she probably wouldn't have asked me. My symptoms. If I'm taking my medication on time. How does the medication make me feel? My anxiety level or my depression level and my insomnia or whatever the case may be. (Male, 54, FL)
[My provider] brought up [my anxiety] briefly…we had a brief discussion about that. He did mention that there's a social worker, maybe to chat, and things like that. Which was great…it's not something I would have brought up if it weren't for the [PROs on the] iPad. (Male, 29, ON)
Enhanced feeling of ‘being cared about’
Patients reported that the discussions transpiring from having been asked questions about well-being, mental health, and health behaviors in the PROs helped personalize their appointment and enhanced their sense of being "cared about".
It felt personal, and I felt like, based on what I responded and the doctor briefly touching on or following up on those [things], I felt really impressed to know that, okay, I didn't come in for that! I just thought that, okay, I'm gonna come, do my blood work, blah-blah-blah, and move on. But then it felt a little bit more personal to me, and I felt good about that, to know that there's a little bit of care or relationship building in it.. [the PROs were] a conversation starter. I felt like there was a little bit of relationship…deepened based on that. (Male, 30, ON)
It's not like you're just coming in and getting your injections and getting out...it just shows that they are attentive, that they are doing their stuff. (Male, 39, ON)
Criticisms of the PROs
While patients broadly found PROs to be highly acceptable and felt they were likely useful for PWH in general, some patients felt that PROs were less necessary for their own purposes. These patients felt themselves already able to communicate well with their provider:
I already came knowing the questions I wanted to ask my doctor…so being asked the other questions, I was like, "okay, I wasn't planning on this." (Male, 47, ON)
I brought up the fact that I was depressed…that’s what I always do with them. I’m very straightforward. (Female, 37, FL)
I put it on myself to speak about [my health issues], if there’s a change in something. (Male, 38[b], FL)
Some patients felt the issues or behaviors addressed by the PROs were not relevant to them:
There wasn’t really anything in [the PROs] that was new or made me think about something else.…if I was a different patient, I think it could’ve spurred me to ask different questions…or maybe to be more thoughtful about, “Hey, is this a problem,” [e.g., if] talking about drug use. But it didn’t change much for me. (Male, 54, FL)
Another felt historical questions to be less relevant, given that their provider already knows them well:
[My drug use] was over 20 years ago…I’ve been good, and I don’t do hard drugs…so the things that are going on in my life right now, have nothing to do with what I did that many years ago. And they know that. They know everything. (Female, 37, FL)
Some felt the questions about sex/sexuality were too personal:
I was really taken back. The sex questions were very direct. I was like, "Really? Seriously?" (Male, 51, ON)
Sexuality is not important. If you're asking about those things, it's not important…that's not anything medical, that's just your private life, who you're screwing and what gender you want to be. (Female, 54, FL)
As far as your [sex] partner – as far as [whether they have] HIV…if he is or if he ain’t. That’s not for us to say. I thought that was a little personal. (Female, 57, FL)
Some patients felt that the PROs lacked context when answering, and preferred to answer questions in person to humanize the experience and provide appropriate context:
The iPad’s good… [but] I'd rather talk to someone one-on-one. Talking one-on-one, you get a feeling of someone instead of on the iPad…it's a machine. You talking to someone one-on-one, they can get to how you feeling and do a little one-on-one at the person you talking to. (Male, 54, FL)
I think [the PRO questions] are more appropriate in person…I think it should be more of a human touch. (Male, 47, ON)
On the iPad, two or three questions where I marked, it was like, okay. I’ll just mark this because I can’t explain why I feel this way, because it’s not a person. (Female, 53, FL)
Two patients expressed concerns surrounding the use of PRO data and its confidentiality, fearing potential legal consequences of their response. As one patient summarized:
I find that it's the scariest part about being…HIV positive is the law…you cannot even engage in oral sex…unless it's protected, you have to disclose. That's why it's awkward when those [sexual risk behavior] questions came up. I was just like, I don't know how to answer this question, because I don't want to put myself in a position where, yes, I'm disclosing we might be having unprotected sex, but can this sort of bite me in the end. (Male, 29, ON)
DISCUSSION
PWH experienced electronic, self-administered, tablet-based PROs as easy to navigate and use, and reported PROs added value to their care in several ways. PROs improved their preparedness for the visit by helping to recall health needs; and the PROs promoted self-evaluation, as a comprehensive means of ‘taking stock’ of their overall health, particularly mental health. PWH valued the use of PROs as a mechanism for disclosing highly personal or potentially stigmatizing symptoms and behaviors, particularly depressive symptoms, sexual behavior, and substance use; most, but not all, found it easier to convey these to an “impartial” platform prior to the appointment rather than being asked verbally by the provider. PWH described improved communication with their providers, in part due to feeling able to answer questions more honestly, but also due to the comprehensiveness of the assessment across several health topics. PWH believed that the provider receipt of PRO results prior to the appointment resulted in discussion of issues that otherwise might have been overlooked. As a result, several patients reported experiencing an enhanced quality of care that felt highly personalized, comprehensive, attuned to hidden needs, and focused on well-being.
Our findings echo evidence from primarily HIV and cancer care (Fredericksen et al., 2020), in which PROs were found acceptable and useful to patients across a broad range of populations and care modalities (Crane et al., 2007; Wolpin et al., 2008; Howell et al., 2015; Sarabia et al., 2015). Patients have found electronic/ computer-based PRO assessments easy to use (Basch et al., 2005; Basch et al., 2007; Mark et al., 2008a; Mark et al., 2008b; Wolpin et al., 2009; Dunn et al., 2016; Sinha et al. 2020), useful for recall and promoting discussion of symptoms (Basch et al., 2005; Mark et al., 2008a; Mark et al., 2008b; Sarabia et al., 2015), and helpful in improving providers’ awareness of their needs (Detmar et al., 2002; Basch et al., 2005, 2007; Thewes et al., 2009). The use of PROs has helped patients feel more in control of their own care (Fredericksen et al., 2019). Patients also have found that PROs help increase honesty with their provider, as well as themselves; indeed, prior work comparing computer-based assessments with interviewer-based assessments notes higher accuracy when using the former when reporting sensitive information (Sinha et al., 2020). Our interviews suggest PWH perceive similar value of PROs found in other clinic populations, with an emphasis of the value of an enhanced communication dynamic afforded by reducing social desirability bias.
While overall PWH found PROs to be valuable, some expressed reservations and concerns which are instructive for considering implementation of PROs. Some felt that the issues assessed, such as drug use, were not relevant personally; for example, if without history, or only distant history, of drug use. This highlights the need to tailor PROs as much as possible to individual patient needs. This is addressable in several ways. One is the use of carefully considered skip patterns, easily programmed in electronic PROs; for example, a patient would be asked about specific types of drug use only if they had indicated lifetime use of that drug. In addition, algorithms can help administer certain PRO measures with varying levels of frequency based on patients’ historic responses. For example, a patient over 50 that has reported no prior drug use history might be shown drug use items only annually; a patient indicating IPV on a previous assessment might be shown the same items at each visit, whereas a patient indicating no prior history might be shown these items less frequently. Others found certain PRO items, particularly the items querying sexual risk behaviors, to be too personal, highlighting the usefulness of allowing patients to “skip” questions if they wish. To help offset these concerns, introductory language could be included at the beginning of the assessment indicating that the PROs are meant as a broad screening tool, and that not all questions are necessarily applicable to individual patients.
Some PWH reported concerns about PROs lacking sufficient contextual detail to accurately represent or explain their behaviors, which was felt to be easier in a human interaction. A small number of PWH expressed concerns regarding confidentiality of the data and uncertainty regarding the purpose and uses of the assessment. In this population, tasked with navigating HIV-related stigma and concern for ‘feeling judged’ regarding health behaviors, the use of data and confidentiality protocols should be made clear, ideally by the clinic personnel that are introducing the assessment. Framing the purpose of the PROs to PWH at first administration, in fact, is critical to their ongoing relationship with PROs; PWH should be informed that their providers will see the answers, that they have a choice in whether or not they answer particular questions, and that the PROs are intended as a tool for ensuring their needs are heard in their care.
With the goal of promoting the healthiest patient outcomes possible among PWH, the argument for implementing PROs into clinical HIV care is strong. First, as an increasingly aging population, PWH are often managing multiple chronic conditions and reporting on their associated symptoms; PROs, with the ability to quickly and comprehensively assess many symptoms, are highly effective for this purpose. Second, relative to other conditions or disease groups, the health of PWH is interwoven tightly with a social dimension, which includes navigating social support and managing perceived and real HIV-related stigma, with mental health implications. Related symptoms may evade report or detection when assessing solely in-person; here, PROs offer a more honest assessment of patient needs. Third, for PWH, healthy outcomes often rely on accurate reporting to providers of health behaviors that may be embarrassing or stigmatizing to admit to in-person, such as substance use, HIV/STI transmission risk behavior, and ART adherence; again, PROs offset social desirability bias that often obscures these behaviors. These attributes, supported by evidence of high patient acceptability, usability, and perceived value, underscore the importance and potential of electronic, self-administered PROs in optimizing clinical HIV care.
Strengths
We interviewed PWH from two geographically diverse clinics, with highly diverse patient populations.
LIMITATIONS
PWH willing to be interviewed may not be representative of the population of PWH that self-administered the PROs in these clinics. We did not track interview refusal rate, and we lacked data on socioeconomic status which may preclude our ability to characterize those agreeing to participate vs. those who declined. There exists the possibility that reviewers external to the study may reach different conclusions. We also acknowledge that this study utilized a full-time research coordinator at each site who ensured smooth implementation of the PRO workflow for each patient; a lack of this resource in other settings may yield a different experience from the perspective of the patient.
CONCLUSION
PWH receiving care in two North American HIV clinics found electronic, self-administered tablet-based PROs administered on-site immediately before the appointment to be useful for prioritizing discussion topics with their providers, helping initiate discussion on sensitive issues, and improving comprehensiveness of and satisfaction with care.
CONFLICT OF INTERESTS
The authors have no conflict of interest.
ACKNOWLEDGMENTS
This study was funded by ViiV Healthcare PO# 3001390801. We are thankful to the patients, providers, and research staff from the two PROgress study clinics, St. Michael’s Hospital in Toronto, Ontario, Canada, and the Midway Specialty Care Center in Fort Pierce, Florida, USA. Additionally, we thank the PROgress study steering committee, comprising patients living with HIV, HIV care providers, and HIV care researchers, for their oversight and guidance.
REFERENCES
|
Basch E, Artz D, Dulko D, Scher K, Sabbatini P, Hensley M, Mitra N, Speakman J, McCabe M, Schrag D (2005). Patient online self-reporting of toxicity symptoms during chemotherapy. Journal of Clinical Oncology 23(15):3552-3561. |
|
|
Basch E, Artz D, Iasonos A, Speakman J, Shannon K, Lin K, Pun C, Yong H, Fearn P, Barz A, Scher HI, McCabe M, Schrag D (2007). Evaluation of an online platform for cancer patient self-reporting of chemotherapy toxicities. Journal of American Medicine Informatics Association 14(3):264-268. |
|
|
Basch E, Deal AM, Dueck AC, Scher HI, Kris MG, Hudis C, Schrag D (2017). Overall Survival Results of a Trial Assessing Patient-Reported Outcomes for Symptom Monitoring During Routine Cancer Treatment. Journal of American Medicine Association 318(2):197-198. |
|
|
Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, Rogak L, Bennett AV, Dueck AC, Atkinson TM, Chou JF, Dulko D, Sit L, Barz A, Novotny P, Fruscione M, Sloan JA, Schrag D (2016). Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. Journal of Clinincal Oncology 34(6):557-565. |
|
|
Berry DL, Blumenstein BA, Halpenny B, Wolpin S, Fann JR, Austin-Seymour M, Bush N, Karras BT, Lober WB, McCorkle R (2011). Enhancing patient-provider communication with the electronic self-report assessment for cancer: a randomized trial. Journal of Clinical Oncology 29(8):1029-1035. |
|
|
Bradley KA, Bush KR, Epler AJ, Dobie DJ, Davis TM, Sporleder JL, Maynard C, Burman ML, Kivlahan DR (2003). Two brief alcohol-screening tests from the Alcohol Use Disorders Identification Test (AUDIT): validation in a female Veterans Affairs patient population. Archives of Internal Medicine 163(7):821-829. |
|
|
Brown RF, Butow PN, Dunn SM, Tattersall MH (2001). Promoting patient participation and shortening cancer consultations: a randomized trial. British Journal of Cancer 85(9):1273-1279. |
|
|
Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA (1998). The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Archives of Internal Medicine 158(16):1789-1795. |
|
|
Chen J, Ou L, Hollis SJ (2013). A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organizations in an oncologic setting. BMC Health Services Research 13(211). |
|
|
Cleeland CS, Wang XS, Shi Q, Mendoza TR, Wright SL, Berry MD, Malveaux D, Shah PK, Gning I, Hofstetter WL, Putnam JB JR, Vaporciyan AA (2011). Automated symptom alerts reduce postoperative symptom severity after cancer surgery: a randomized controlled clinical trial. Journal of Clinical Oncology 29(8):994-1000. |
|
|
Conigliaro J, Gordon AJ, McGinnis KA, Rabeneck L, Justice AC (2003). How harmful is hazardous alcohol use and abuse in HIV infection: do healthcare providers know who is at risk? Journal of Acquired Immune Deficiency Syndromes 33(4):521-525. |
|
|
Crane HM, Crane PK, Tufano JT, Ralston JD, Wilson IB, Brown TD, Davis TE, Smith LF, Lober WB, McReynolds J, Dhanireddy S, Harrington RD, Rodriguez CV, Nance RM, Delaney JA, Safren SA, MM Kitahata, Fredericksen RJ (2017). HIV provider documentation and actions following patient reports of at-risk behaviors and conditions when identified by a web-based point-of-care assessment. AIDS and Behavior 21(11):3111-3121. |
|
|
Crane HM, Lober W, Webster E, Harrington RD, Crane PK, Davis TE, Kitahata MM (2007). Routine collection of patient-reported outcomes in an HIV clinic setting: the first 100 patients. Current HIV Research 5(1):109-118. |
|
|
Detmar SB, Muller MJ, Schornagel JH, Wever LD, Aaronson NK (2002). Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial. Journal of the American Medical Association 288(23):3027-3034. |
|
|
Espallargues M, Valderas JM, Alonso J (2000). Provision of feedback on perceived health status to healthcare professionals: a systematic review of its impact. Medical Care 38(2):175-186. |
|
|
Fitzsimmons E, Loo S, Dougherty S, Mathews W, Mugavero M, Mayer K, Potter J, O'Cleirigh C, Crane P, Crane H, Fredericksen RJ (2019). Development and Content Validation of the IPV-4, a Brief Patient- |
|
|
Reported Measure of Intimate Partner Violence for Use in HIV Care. International Society of Quality of Life Research Conference. San Diego, California. |
|
|
Fredericksen RJ, Crane HM, Lober W, Fitzsimmons E, Short D (2020). Progress evidence review and summary: impact of administering patient-reported measures and outcomes (PROs) within routine HIV care. |
|
|
Fredericksen RJ, Edwards TC, Merlin JS, Gibbons LE, Rao D, Batey DS, Dant L, Paez E, Church A, Crane PK, Crane HM, Patrick DL (2015). Patient and provider priorities for self-reported domains of HIV clinical care. AIDS Care 27(10):1255-1264. |
|
|
Fredericksen RJ, Fitzsimmons E, Gibbons LE, Loo S, Dougherty S, Avendano-Soto S, Anderson WA, Gutierrez C, Shurbaji S, Burleson S, Christopoulos K, Poceta J, Mayer KH, Mugavero MJ, Mathews WC, Crane PK, Crane HM (2020). How Do Treatment Priorities Differ Between Patients in HIV Care and Their Providers? A Mixed-Methods Study 24(4):1170-1180. |
|
|
Fredericksen RJ, Fitzsimmons E, McReynolds J, Brown S, Smith L, Lober W, Crane HM (2019). Patients' perceptions of value of a self-administered electronic patient-reported outcomes assessment on quality of their HIV care. International Society of Quality of Life 26th Annual Conference. San Diego, CA. |
|
|
Fredericksen RJ, Harding BN, Ruderman SA, McReynolds J, Barnes G, Lober WB, Fitzsimmons E, Nance RM, Whitney BM, Delaney JAC, |
|
|
Mathews WC, Willig J, Crane PK, Crane HM (2020). Patient acceptability and usability of a self-administered electronic patient-reported outcome assessment in HIV care: relationship with health behaviors and outcomes. AIDS Care pp. 1-11. |
|
|
Fredericksen RJ, Mayer KH, Gibbons LE, Edwards TC, Yang FM, Walcott M, Brown S, Dant L, Loo S, Gutierrez C, Paez E, Fitzsimmons E, Wu AW, Mugavero MJ, Mathews WC, Lober WB, Kitahata MM, Patrick DL, Crane PK, Crane HM (2018). Development and Content Validation of a Patient-Reported Sexual Risk Measure for Use in Primary Care. Journal of General International Medicine 33(10):1661-1668 |
|
|
Fredericksen RJ, Tufano J, Ralston J, McReynolds J, Stewart M, Lober WB, Mayer KH, Mathews WC, Mugavero MJ, Crane PK, Crane HM (2016). Provider perceptions of the value of same-day, electronic patient-reported measures for use in clinical HIV care. AIDS Care, pp. 1-6. |
|
|
Gross R, Bilker WB, Friedman HM, Coyne JC, Strom BL (2002). Provider inaccuracy in assessing adherence and outcomes with newly initiated antiretroviral therapy. AIDS Care 16(13):1835-1837. |
|
|
Hogg RS, Heath KV, Yip B, Craib KJ, O'Shaughnessy MV, Schechter MT, Montaner JS (1998). Improved survival among HIV-infected individuals following initiation of antiretroviral therapy. Journal of the American Medical Association 279(6):450-454. |
|
|
Holmes WC, Shea JA (1998). A new HIV/AIDS-targeted quality of life (HAT-QoL) instrument: development, reliability, and validity. Medical Care 36(2):138-154. |
|
|
Howell D, Molloy S, Wilkinson K, Green E, Orchard K, Wang K, Liberty J (2015). Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors. Annals of Oncology 26(9):1846-1858. |
|
|
Jones J, Stephenson R, Smith DK, Toledo L, La Pointe A, Taussig J, Sullivan PS (2014). Acceptability and willingness among men who have sex with men (MSM) to use a tablet-based HIV risk assessment in a clinical setting. Springerplus 3:708. |
|
|
Justice AC, Holmes W, Gifford AL, Rabeneck L, Zackin R, Sinclair G, Weissman S, Neidig J, Marcus C, Chesney M, Cohn SE, Wu AW (2001). Development and validation of a self-completed HIV symptom index. Journal of Clinical Epidemiology 54(Suppl. 1):S77-90. |
|
|
Kiechl S, Werner P, Egger G, Oberhollenzer F, Mayr M, Xu Q, Poewe W, Willeit J (2002). Active and passive smoking, chronic infections, and the risk of carotid atherosclerosis: prospective results from the Bruneck Study. Stroke 33(9):2170-2176. |
|
|
Kissinger P, Rice J, Farley T, Trim S, Jewitt K, Margavio V, Martin DH (1999). Application of computer-assisted interviews to sexual behavior research. American Journal of Epidemiology 149(10):950-954. |
|
|
Kitahata M, Rodriguez B, Haubrich R, Boswell S, Mathews WC, Lederman MM, Lober WB, Van Rompaey SE, Crane HM, Moore RD, Bertram M, Kahn JO, Saag MS (2008). Cohort profile: the Centers for AIDS Research Network of Integrated Clinical Systems. International Journal of Epidemiology 37(5):948-955. |
|
|
Kroenke K, Spitzer RL, Williams JB (2001). The PHQ-9: validity of a brief depression severity measure. Journal of General Internal Medicine 16(9):606-613. |
|
|
Laporte M, Keller HH, Payette H, Allard JP, Duerksen DR, Bernier P, Jeejeebhoy K, Gramlich L, Davidson B, Vesnaver E, Teterina A (2015). Validity and reliability of the new Canadian Nutrition Screening Tool in the 'real-world' hospital setting. European Journal of Clinical Nutrition 69(5):558-564. |
|
|
Lu M, Safren SA, Skolnik PR, Rogers WH, Coady W, Hardy H, Wilson IB (2008). Optimal recall period and response task for self-reported HIV medication adherence. AIDS and Behavior 12(1):86-94. |
|
|
Mark TL, Fortner B, Johnson G (2008a). Evaluation of a tablet PC technology to screen and educate oncology patients. Supportive Care in Cancer 16(4):371-378. |
|
|
Mark TL, Johnson G, Fortner B, Ryan K (2008b). The benefits and challenges of using computer-assisted symptom assessments in oncology clinics: results of a qualitative assessment. Technology in Cancer Research and Treatment 7(5):401-406. |
|
|
Messiah A, Loundou AD, Maslin V, Lacarelle B, Moatti JP (2001). Physician recognition of active drug use in HIV-infected patients is lower than validity of patient's self-reported drug use. Journal of Pain and Symptom Management 21(2):103-112. |
|
|
Morin SF, Koester KA, Steward WT, Maiorana A, McLaughlin M, Myers JJ, Vernon K, Chesney MA (2004). Missed opportunities: prevention with HIV-infected patients in clinical care settings. Journal of Acquired Immune Deficiency Syndromes 36(4):960-966. |
|
|
Murphy EL, Collier AC, Kalish LA, Assmann SF, Para MF, Flanigan TP, Kumar PN, Mintz L, F. R. Wallach, Nemo GJ, I. Viral Activation Transfusion Study (2001). Highly active antiretroviral therapy decreases mortality and morbidity in patients with advanced HIV disease. ANnals of Internal Medicine 135(1):17-26. |
|
|
Nance DR, Fredericksen RJ, Cropsey K, Chander G, Mugavero M, Christopoulos K, Geng E, Mathews W, Hahn A, Mayer K, O'Cleirigh C, Eron J, Saag M, Kitahata M, Crene H (2017). E-cigarette use among persons living with HIV. International AIDS Society Meeting. Paris, France. |
|
|
Narayan MC (2010). Culture's effects on pain assessment and management. American Journal of Nursing 110(4):38-47. |
|
|
Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S (2015). Patient reported outcome measures in practice. The BMJ 350:g7818. |
|
|
Newcombe DA, Humeniuk RE, Ali R (2005). Validation of the World Health Organization Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): report of results from the Australian site. Drug and Alcohol Review 24(3):217-226. |
|
|
Palella FJ Jr, Delaney KM, Moorman AC, Loveless MO, Fuhrer J, Satten GA, Aschman DJ, Holmberg SD (1998). Declining morbidity and mortality among patients with advanced human immunodeficiency virus infection. HIV Outpatient Study Investigators. The New England Journal of Medicine 338(13):853-860. |
|
|
Ruland CM, Holte HH, Roislien J, Heaven C, Hamilton GA, Kristiansen J, Sandbaek H, Kvaloy SO, Hasund L, Ellison MC (2010). Effects of a computer-supported interactive tailored patient assessment tool on patient care, symptom distress, and patients' need for symptom management support: a randomized clinical trial. Journal of the American Medical Informatics Association 17(4):403-410. |
|
|
Sarabia S, Perez-Cosio A, Brown C, Leung Y, Gill G, Liang M, Yeung S, Liu A, Wong L, Liu G, Howell D (2015). Are pallative patients less accepting to self-report symptom measures for clinical management than curative patients? Journal of Clinical Oncology 33(29):79. |
|
|
Sharma P, Dunn RL, Wei JT, Montie JE, Gilbert SM (2016). Evaluation of point-of-care PRO assessment in clinic settings: integration, parallel-forms reliability, and patient acceptability of electronic QOL measures during clinic visits. Quality of Life Research 25(3):575-583. |
|
|
Simoni JM, Kurth AE, Pearson CR, Pantalone DW, Merrill JO, Frick PA (2006). Self-report measures of antiretroviral therapy adherence: A review with recommendations for HIV research and clinical management. AIDS and Behavior 10(3):227-245. |
|
|
Sinha N, Yang A, Pradeep A, Bhuvaneswari R, Kumarasamy N, O'Cleirigh C, Mayer KH, Chan BT (2020). Feasibility and acceptability of a psychosocial and adherence electronic patient reported outcomes (PROs) system at an HIV care center in southern India. AIDS Care 32(5):630-636. |
|
|
Spitzer RL, Kroenke K, Williams JB (1999). Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. Journal of the American Medical Association, 282(18):1737-1744. |
|
|
Stover A, Irwin DE, Chen RC, Chera BS, Mayer DK, Muss HB, Rosenstein DL, Shea TC, Wood WA, Lyons JC, Reeve BB (2015). Integrating patient-reported outcome measures into routine cancer care: cancer patients' and clinicians' perceptions of acceptability and value. The Journal of Electronic Health Data and Methods 3(1):1169. Washington, DC. |
|
|
Taenzer P, Bultz BD, Carlson LE, Speca M, DeGagne T, Olson K, Doll R, Rosberger Z (2000). Impact of computerized quality of life screening on physician behaviour and patient satisfaction in lung cancer outpatients. Psycho-Oncology 9(3):203-213. |
|
|
Tariman JD, Berry DL, Halpenny B, Wolpin S, Schepp K (2011). Validation and testing of the Acceptability E-scale for web-based patient-reported outcomes in cancer care. Applied Nursing Research 24(1):53-58. |
|
|
Thewes B, Butow P, Stuart-Harris R, G. Greater Southern Area Health Service Screening Collaborative (2009). Does routine psychological screening of newly diagnosed rural cancer patients lead to better patient outcomes? Results of a pilot study. Australian Journal of Rural Health 17(6):298-304. |
|
|
U.S. Department of Health and Human Services and Federal Drug Administration (2006). Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health and Quality of Life Outcomes 4:79. |
|
|
Velikova G, Booth L, Smith AB, Brown PM, Lynch P, Brown JM, Selby PJ (2004). Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. Journal of Clinical Oncology 22(4):714-724. |
|
|
Wagner AK, Ehrenberg BL, Tran TA, Bungay KM, Cynn DJ, Rogers WH (1997). Patient-based health status measurement in clinical practice: a study of its impact on epilepsy patients' care. Quality of Life Research 6(4):329-341. |
|
|
Wasson JH, Stukel TA, Weiss JE, Hays RD, Jette AM, Nelson EC (1999). A randomized trial of the use of patient self-assessment data to improve community practices. Effective Clinical Practice 2(1):1-10. |
|
|
Whitney BM, Fitzsimmons EF, Jackson MK, Brown SM, Nguyen J, Nance RN, Ruderman S, Delaney JAC, Crane HM, Fredericksen RJ (2020). Patient perceptions and understanding of a housing status measure for use in HIV care. 27th Annual Conference of International Society of Quality of Life. (Virtual) |
|
|
WHO ASSIST Working Group (2002). The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): development, reliability and feasibility. Addiction 97(9):1183-1194. |
|
|
Williams MV, Davis T, Parker RM, Weiss BD (2002). The role of health literacy in patient-physician communication. Family Medicine 34(5):383-389. |
|
|
Wolfe F, Pincus T, Thompson AK, Doyle J (2003). The assessment of rheumatoid arthritis and the acceptability of self-report questionnaires in clinical practice. Arthritis and Rheumatology 49(1):59-63. |
|
|
Wolpin S, Berry D, Austin-Seymour M, Bush N, Fann JR, Halpenny B, Lober WB, McCorkle R (2008). Acceptability of an Electronic Self-Report Assessment Program for patients with cancer. Computer Informatics Nursing 26(6):332-338. |
|
|
Wu WW, Johnson R, Schepp KG, Berry DL (2011). Electronic self-report symptom and quality of life for adolescent patients with cancer: a feasibility study. Cancer Nursing 34(6):479-486. |
|
Copyright © 2024 Author(s) retain the copyright of this article.
This article is published under the terms of the Creative Commons Attribution License 4.0
