The United States ranks 37^th in the world on the most commonly measured health outcomes, including infant mortality, adult female mortality, adult male mortality, and life expectancy [1]; yet, our per capita health care costs are the highest in the world [2,3]. The combination of chronic health conditions and mental health issues leads to poorer outcomes [4,5]. For instance, Garfield et al, (2014) noted that Anxiety disorders, Major Depressive Disorder (MDD), and co-occurring anxiety and MDD are associated with increased risk of heart failure. Paradise et al. (2014) concluded that patients with specific mental health conditions (bipolar disorder, depression, and other psychotic disorders) experienced worse anticoagulation control. Patients with chronic health conditions who also suffer from a mental illness incur double the health costs compared to those solely with chronic health conditions [6]. Given that 70% of mental health services are provided solely in primary care [7], this setting has become the basis for health care reform in most nations and is becoming so in the United States.

      Decades of research suggest that integrated care improves patient access to mental health services [8-10], and may overcome disparities in mental health access for minority groups [11]. Integrated care has also shown to reduce wait times for mental health treatment [12], enhance treatment engagement and adherence [13], improve patient satisfaction [12], and create better clinical and functional outcomes [8].

      Psychologists are being called on to function in new ways and in new roles during this transformation. The integration of psychology into public health and primary care has been increasingly embraced in the health care system both in the United States and internationally [14,15]. Large-scale organizations including the United States Military, Veterans Health Administration, and Private Health Care Organizations have increasingly focused in delivering innovative models of integrated care [16]. Practice redesign and payment models to enhance the success of integration have been developed through the Center for Medicaid and Medicare for the public sector. Thus, it is not traditional clinical psychology, or even traditional health psychology that psychologists in integrated settings are being asked to provide. The changes in the field are creating new opportunities for psychologists interested in largescale system change, innovative practice, and team-based care [17].

Primary Care

      With these changes in mind, it is important to provide context to the setting to better facilitate understanding the function of behavior health professionals and the role of psychologists within primary care. Primary care is the setting where the general population receives health care. Primary care refers to the provision of integrated accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community [18]. Behavioral health refers to the broad area of mental health and substance abuse conditions, health behaviors (including their contribution to chronic medical illnesses), life stressors and crises, stress-related physical symptoms, and ineffective patterns of health care utilization [19]. It includes social factors that influence health, creating preventive services that increase the health of a population, managing the effects of having a chronic illness as well as anxiety, depression, trauma, emotional distress and substance use. There is a critical need for a workforce trained to deliver these services.

      TPsychologists have a unique role to play in this newly re-conceptualized workforce. They have knowledge about program development and evaluation, and have leadership and supervisory experiences that will enable them to serve as “health care providers” prepared for the 21st Century [14,20]. As the need for psychologists to work as members of primary care health teams grows, it has become essential that graduate education and training programs prepare future generations of psychologists to meet workforce demands. The great majority of psychology faculty does not have experience working in primary care. As a result, the APA’s Society for Health Psychology’s Committee on Integrated Primary Care (IPC) developed an innovative curriculum to be used as a resource by graduate programs and training sites. It was designed as a “plug and play” course that any faculty could use to provide knowledge and skills in this area.

Curriculum

      The curriculum was developed by a core group of nine psychologists with extensive clinical and training experience in primary care [21]. The curriculum is based on the most recent scientific literature in order to complement the expertise and experience of the course developers. The course was designed to provide students with the skills necessary to prepare them for work in the newly developing models of collaborative, population-based health care. The curriculum strives to enhance teaching and student learning by providing instructors with detailed lecture notes, videos, interactive exercises, and sophisticated graphic illustrations to accompany each module of the course. Moreover, the course contains the most current research data and references to accompany and support all of the information presented across the modules. There is a process in the Education Directorate to update the curriculum as new research is conducted and published.

      The curriculum includes four 120-minute foundation modules (each with 90-minute versions), and eleven 90-minute topic modules. The four foundational modules provide students with groundwork in IPC psychology. Module 1 (introduction to IPC) defines the field, identifies key factors leading to integration, and discerns the role of the traditional mental health provider to the role of the psychologist in primary care. Module 2 (across the continuum: Psychology’s role in IPC) discusses the range of patient needs presented in primary care. Modules 3 (Primary care patients: Who are they and how can psychologists be helpful) and 4 (IPC interventions) describe common patient behavioral health concerns in primary care, and identify the clinical skills associated with providing interventions suited for primary care. The eleven topic modules were written by experts in particular subject areas, in collaboration with the core team to ensure that they retained a foundation in primary care. These topic modules with many more in development, focusing on working with common diagnoses in primary care, such as depression, anxiety, ADHD, substance misuse, and chronic pain; as well as topics relevant to working in primary care, including health promotion and disease prevention, motivational interviewing, chronic disease self-management, psychologists as scientists, and working with older adults.

      The goal of the curriculum is to increase the knowledge, skills, and attitudes related to IPC competencies; as such, this study sought to assess the educational outcomes of this course. The Department of Clinical Medical Psychology (CMP) at Mercer University was the first doctoral program in the nation to assess the efficacy of this new educational program. Specifically, the study intended to examine whether exposure to the curriculum would result in significant changes (within-group), and significant differences (between-group) in (1) students’ interest levels and attitudes pertaining to IPC, and (2) students’ actual and perceived competency levels in IPC. Overall, we aimed to (1) discern whether those who completed the course moved towards mastering the core competencies for working in IPC, (2)determine if the course resulted in heightened interest in the field of IPC, (3) provide preliminary feedback to the Society for Health Psychology’s IPC Committee regarding potential modifications or enhancements to the course and data collection instruments, and to (4) justify whether the course should be a required component of the CMP program’s overall curriculum.It was hypothesized that exposure to the curriculum would result in significant improvements in students’ interest levels and subjective and objective competency attainment. Moreover, it was hypothesized that significant differences in interest and competency in IPC Psychology would exist between the students exposed to the curriculum and a control group of similar students not exposed to the course.

Participants

      Participants included Mercer University Clinical Medical Psychology doctoral students. The experimental group consisted of students enrolled in the Introduction to Integrated Primary Care Psychology (Health Psychology II) course (n=14). These individuals were fourth-year, third-year, and second-year cohort students. The control group (n=18) was comprised of the students of similar training within the same program, who were not enrolled in the course. All procedures involving human participants were in accordance with the ethical standards of the Institutional Review Board (IRB H1610298) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. For this type of study, formal consent was required.

Measures

      A subjective assessment pertaining to student interest and attitudes related to working as a psychologist in IPC (See Appendix A), and a subjective assessment pertaining to student competency in IPC (See Appendix B) was administered at two time periods: (1) prior to the initiation of the course, (2) after completion of the course. Both of these measures were developed by the Society for Health Psychology’s Committee on IPC [21].

      Student interest and attitudes was assessed across the following domains: perceived level of knowledge, importance, preparation, and comfort of working in an IPC setting, as well as, interest in pursuing training, and pursuing a career in the field. Self-reported competency was assessed across each of the six core competencies for practice in IPC identified by McDaniel and colleagues (2014): science, systems, professionalism, relationships, application, and education. To further assess competency attainment, objective measures in the form of 4 short quizzes developed by the Society for Health Psychology’s Committee on Integrated Primary Care (See Appendix C) were distributed to students enrolled in the IPC course immediately after the completion of each of the first four foundational modules.

      As a control condition, a group of Mercer Clinical Medical Psychology doctoral students (n=18), at a similar level in the program, who were not enrolled in the class, were recruited and completed the same assessments pertaining to student interest and attitudes and competency in IPC psychology to serve as a between-group point of analysis.

      Pre-course interest was relatively high, and did not significantly differ between groups (t(30)=-0.584, p=0.563); (M = 3.54, SD = 0.54) (M = 3.66, SD = 0.59) intervention and control; respectively (Figure 1). This was not unexpected as all the students were enrolled in the Clinical Medical Psychology Program, a program geared toward preparing psychologists to function as members of interdisciplinary healthcare teams. Thus, there is a pre-existing interest in the course material. Results were similar regarding perceived pre-course competency for both groups; (t(28) = -0.111, p = 0.913); (M = 2.74, SD = 0.69) (M = 2.76, SD = 0.74), intervention and control; respectively (Figure 2).

Figure 1: Pre-interest BETWEEN group (IPC and non-IPC). Pre-interest was relatively high and did not significantly differ between groups (t(30)=-0.584, p=0.563); (M = 3.54, SD = 0.54) (M = 3.66, SD = 0.59) intervention and control; respectively.

Figure 2: Pre-competency BETWEEN groups (IPC and non-IPC). Results were similar regarding pre-competency for both groups; (t(28) = -0.111, p = 0.913); (M = 2.74, SD = 0.69) (M = 2.76, SD = 0.74), intervention and control; respectively.

      Students enrolled in the course yielded significantly higher overall interest rates after completion of training when compared to pre-assessment measures (t(13) = -5.21, p<.001), including each cluster (Figure 3). Level of knowledge: (t(13) = -5.21, p<.001); Importance: (t(13) = -9.89, p<.001), Level of Preparation: (t(13) = -2.86, p =.013), Comfort: (t(13) = -3.82, p =.002), Level of Interest – Training: (t(13) = 1.47, p = .165), Level of Interest – Career: (t(13) = 1.295, p =.218); respectively (Figure 3).

Figure 3: Interest in IPC (WIHIN-GROUP) from Pre to Post Course Completion. Students enrolled in the course yielded significantly higher overall interest rates after completion of training when compared to pre-assessment measures (t(13) = -5.21, p<.001), including each cluster. Level of knowledge: (t(13) = -5.21, p<.001); Importance: (t(13) = -9.89, p<.001), Level of Preparation: (t(13) = -2.86, p =.013), Comfort: (t(13) = -3.82, p =.002), Level of Interest – Training: (t(13) = 1.47, p = .165), Level of Interest – Career: (t(13) = 1.295, p =.218).

      Overall, subjective self-competency was significantly higher at post-assessment than pre-intervention (t(13) = -7.50, p < .001). Thus, exposure to the material increased perceived skills, knowledge, and understanding of the field. Science cluster: (t(13) = -6.56 p < .001); Systems cluster: (t(13) = -4.19, p = .001); Professionalism cluster: (t(13) = -4.05, p = .001); Relationships cluster: (t(13) = -3.50, p = .004); Application Cluster: (t(13) = -7.59, p < .001), and Education cluster: (t(13) = -9.54, p < .001); respectively (Figure 4).

Figure 4: Competency in IPC (WITHIN-GROUP) from Pre to Post Course Completion. Overall Subjective self-competency was significantly higher at post-assessment than pre-intervention (t(13) = -7.50, p < .001). Thus, exposure to the material increased perceived skills knowledge and understanding of the field. Science cluster: (t(13) = -6.56 p < .001); Systems cluster: (t(13) = -4.19, p = .001); Professionalism cluster: (t(13) = -4.05, p = .001); Relationships cluster: (t(13) = -3.50, p = .004); Application Cluster: (t(13) = -7.59, p < .001), and Education cluster: (t(13) = -9.54, p < .001).

      Students enrolled in the course demonstrated significantly higher overall interest rates after completion of training when compared to non-IPC controls (t(30) = 3.89, p =.001), including each cluster. Level of knowledge: (t(30) = 6.15, p<.001); Importance: (t(30) = 1.38, p = .179), Level of Preparation: (t(30) = 4.17, p<.001), Comfort: (t(30) = 2.825, p =.008), Level of Interest – Training: (t(30) = 0.27, p = .979), Level of Interest – Career: (t(30) = -.756, p =.456); respectively (Figure 5).

Figure 5: Interest in IPC (BETWEEN-GROUP) at Post- Course Completion. Students enrolled in the course yielded significantly higher overall interest rates after completion of training when compared to non-IPC control measures (t(30) = 3.89, p =.001), including each cluster. Level of knowledge: (t(30) = 6.15, p<.001); Importance: (t(30) = 1.38, p = .179), Level of Preparation: (t(30) = 4.17, p<.001), Comfort: (t(30) = 2.825, p =.008), Level of Interest – Training: (t(30) = 0.27, p = .979), Level of Interest – Career: (t(30) = -.756, p =.456).

      Subjective self-competency was significantly higher for the IPC group at the completion of the course when compared to controls (t(24.78) = 4.24, p < .001). Science cluster: (t(30) = 5.89, p < .001); Systems cluster: (t(30) = 3.46, p = .002); Professionalism cluster: (t(30) = -3.75, p = .001); Relationships cluster: (t(30) = 2.38, p = .024); Application Cluster: (t(30) = 6.15, p < .001), and Education cluster: (t(30) = 5.85, p < .001); respectively (Figure 6).

Figure 6: Competency in IPC (BETWEEN-GROUP) at Post- Course Completion. Subjective self-competency was significantly higher for the IPC group at the completion of the course when compared to controls (t(24.78) = 4.24, p < .001). Science cluster: (t(30) = 5.89, p < .001); Systems cluster: (t(30) = 3.46, p = .002); Professionalism cluster: (t(30) = -3.75, p = .001); Relationships cluster: (t(30) = 2.38, p = .024); Application Cluster: (t(30) = 6.15, p < .001), and Education cluster: (t(30) = 5.85, p < .001).

      Comparative analysis between the intervention and control group showed that exposure to materials yielded higher objective knowledge of each of the targeted four modules [t(18.66)=3.10, p=.006, t(21.91)=3.42, p=.002, t(20)=3.26, p=.004), t(27)=6.07, p<.001]; respectively. This is relevant, as no difference was found at baseline when comparing perceived (subjective) competency between groups (t(28)=-0.111, p=0.913); respectively (Figure 7).

Figure 7: Module quiz performance with both IPC and non-IPC groups. Comparative analysis between the intervention and control group demonstrated exposure to materials yielded higher objective knowledge of each of the targeted four modules [t(18.66)=3.10, p=.006, t(21.91)=3.42, p=.002, t(20)=3.26, p=.004), t(27)=6.07, p<.001]. This is relevant, as no difference was found when comparing perceived (subjective) competency between groups (t(28) = -0.111, p = 0.913).

      This study provided a closer examination of factors that may influence future generations of psychologists to work as members of integrated primary care health teams. Results suggest that the APA’s Society for Health Psychology’s committee on Integrated Primary Care’s curriculum can be utilized as an instrumental resource for graduate programs and training sites. Specifically, data gathered from this preliminary analysis show that this curriculum can be an effective teaching tool for future psychologists considering working in integrated primary care settings, as it invokes significant student interest and competency attainment.

      After completing the entire curriculum, students reported significantly higher knowledge, skills, and attitudes related to primary care psychology (i.e., the mission of primary care, types of patients, types of providers, and the frequency of mental health issues seen in this setting). In particular, analyses revealed significantly elevated levels of knowledge about the role and responsibilities of an integrated care psychologist in primary care, and student’s perceived importance of psychologists and medical providers working collaboratively in primary care settings. In addition, analyses demonstrated a significant increase in students‘ reported level of preparation for working collaboratively with primary care medical providers, as well as, increased comfort levels in working with physicians, nurses, and other medical providers on collaborative treatment plans in the primary care setting. This is a very important finding, in that one of the driving factors in developing the curriculum was to increase awareness among clinical psychology trainees within a rapidly growing field, and to foster interest in the role of psychology in this burgeoning career path. While medicine is commonly recognized for having primary care and specialty care settings, the field of IPC is fast developing with the inclusion of a behavioral health professional such as a psychologist as a key member of the team.

      With respect to competency attainment, the preliminary analysis boasted impressive results. Exposure to the curriculum significantly improved students’ perceived levels of competencies across a variety of areas. Specifically, results were significant across all six competency domains (science, systems, professionalism, relationships, application, and education) [22] both within the IPC group, and between the IPC group and control group at post-course completion. Importantly, students expressed increased levels of knowledge related to the biopsychosocial approach in integrated primary care (e.g., knowledge of biological components of health and illness, knowledge and understanding of evidence-based practice and its application to the practice of IPC psychology), as well as, research and evaluation in IPC. Furthermore, students reported increased competencies with respect to leadership and administration in IPC (e.g., promoting effective communication and collaborative decision-making in healthcare teams), interdisciplinary systems in integrated primary care, and professional values and attitudes in integrated primary care (e.g., willingness to adapt to the primary care environment, including frequent interruptions, fast pace of clinic, and unpredictable access to space). Moreover, following the course, students reported increased levels of competency attainment among the following additional areas: individual, cultural, and disciplinary diversity in integrated primary care, ethics in integrated primary care (e.g., ability to identify and address the distinctive ethical issues encountered in primary care), reflective practice, self-assessment, and selfcare in IPC, inter-professionalism in integrated primary care (e.g., appreciation of the unique contributions that different health care professionals bring to the primary care team), building and sustaining relationships in integrated primary care, practice management in IPC (e.g., appreciation of the need to operate at a variety of paces, consistent with the needs and realities of primary care), assessment in IPC (e.g., selection and implementation of screening methods using evidence-based assessment measures to identify patients at risk or in need of specialized services), intervention in IPC (e.g., knowledge of current evidence-based interventions appropriate for primary care to treat health and mental health-related issues), clinical consultation in IPC, education, and supervision in IPC.

      To help control for potential biases in self-reporting, objective measures of competency were administered in the form of short quizzes assessing knowledge acquired from the four foundational modules of the curriculum (Appendix C). Comparative analysis between the IPC group and the control group revealed that exposure to course materials yielded higher objective knowledge among the students who took the course for each of the targeted four modules. This is relevant, as no difference was found when comparing perceived (subjective) competency levels between groups at baseline. Moreover, it is consistent with results obtained on the self-reported competency measure, increasing confidence in the utility of the tool developed by the IPC committee in assessing these constructs. 

      Overall, results from this preliminary analysis show that this curriculum can serve as a vehicle to enhance psychology students’ knowledge base within this new and transforming area of study. As such, it appears that students who successfully complete the course will have gained a proficient understanding of the skills necessary to work within an integrated healthcare team (e.g., the ability to communicate effectively with other health professionals; provide differential diagnoses and participate in the development of unified treatment plans; promote communication between treatment team members; and incorporate knowledge of the roles and responsibilities, values and ethical standards of other health professions). Moreover, results suggest that the curriculum inspires interest among psychology graduate students and trainees to pursue the knowledge, skills, and attitudes related to primary care psychology. Given the overall results of this analysis, it is recommended that this course be a required component of any graduate program focusing in clinical health psychology, or any program geared toward training psychologists to serve as members of integrated interdisciplinary healthcare teams.

      To our knowledge, this is the first study focusing exclusively on students’ interest and competency in IPC. Yet since this curriculum is new, our participant pool imposed limitations resulting in a small sample size. It is noteworthy that for our study we utilized a sample size comprised of doctoral students in a clinical program with an emphasis in health psychology. As a consequence, the level of interest for pursing clinical training in integrated primary care and pursuing a career within IPC were not significantly increased, as it can be assumed students entered the program with the intention to work in the health psychology realm. Future studies should utilize larger, more heterogeneous samples will likely correct for this and reveal differences with respect to levels of interest in pursuing clinical training, and levels of interest in pursuing a career within IPC. Despite these limitations, the study provides insight into factors associated with the development and implementation of an IPC psychology curriculum.

Directions for future research

      Future studies should consider testing samples from various psychology programs nationwide, and to develop new measures. Specifically these should focus on the skills of program development and evaluation as well as quality improvement skills, which make psychologists stand out from other more purely clinical behavioral health providers. Future research can focus on refining properties and standardization of current measures. Also, studies should assess technological mediums (e.g., online surveys). Another area of interest may be faculty perspective data collection in addition to their graduate students. More research is necessary to understand the specific mechanisms that impact IPC curriculums.