Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and private study only. The thesis may not be reproduced elsewhere without the permission of the Author. THE RELATIONSHIP BETWEEN CASE CONCEPTUALIZATION AND HOMEWORK IN COGNITIVE BEHAVIOURAL THERAPY (CBT) FOR DEPRESSION A thesis in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Psychology At Massey University, Albany, New Zealand. Michael Haig Easden 2010 ii iii ABSTRACT Comprehensive case conceptualization is central to Cognitive Behavioural Therapy ( CBT; A. T. Beck, Rush, Shaw, & Emery, 1979). Despite the importance attributed to case conceptualization there is li mited empirical support for the utility of case conceptualization in CBT. In particular , there is limited research on the relationship between therapist competence in case conceptualization, in - session and between session treatment planning (i.e., homework ), and outcomes. Furthermore, little is kn own about the evolution of case conceptualization over a course of CBT . In order to facilitate the empirical investigation of case conceptualization in CBT the primary aim of the current thesis was to develop a new method for evaluating case conceptualization, t he Conceptualization Rating Scale (CRS ; Easden & Kazantzis, 2008 ; 2009 ). Study 1 investigated how patients’ ( n =10) written case conceptualizations change over a course of CBT for depression using 53 J. Beck Case Conceptualiza tion Diagrams (J. Beck, 1995). Therapist’s resultant case conceptualizations became more complete over the course of therapy with an average of 33% more information being recorded in case conceptualizations from intake to session 10 . Consistent with cognitive- behavioural theory, therapists tended to conceptualize core beliefs about the ‘self’ with relatively minimal reporting of beliefs about the ‘world / others’ and the ‘future’. Study 2 provided the training, development and preliminary psychometrics of the CRS. Independent observers ( N = 4) rated 2 25 DVD recorded therapy se ssions. The CRS was demonstrated to possess adequate internal consistency (α = .61 ) an d excellent total scale interrater reliability for each of the four domains integration ( k = .8 3 ) , importance ( k = . 65 ), competence (ICC = .93 ), and f it / match (ICC = .86 ). Results revealed that independent iv observers were able to agree on different aspects of CBT therapist’ s utilization of case conceptualizations during therapy sessions. In Study 3, u sing the total sample of 28 patients , therapist’s ( N = 7) levels of competence were assessed by independent observers ( N = 4 ) using the CRS based on 225 DV D recorded therapy sessions . A multilevel modelling (MLM) analysis revealed that after controlling for time, taken together, thera pist competence in homework use and therapist competence in case conceptualization explained 40 % of within patient variance and 19 % of between patient variance associated with positive change on the BDI - I I after controlling for patient beliefs about homework, symptom severity and personality beliefs . The results from each study contribute towards an understanding of the relationship between different domains of therapist competence and outcomes in psychotherapy. More specifically, empirical support is provided for the utility and systematic integration of case conceptualization in CBT for depression . Implica tions are discussed for supervision, training and clinical practice in CBT to ensure positive patient outcomes and evidenced - based interventions in CBT. v ACKNOWLEDGEMENTS This thesis would not have been possible without the dedication of a number of people playing different roles in the creation and running of the CBT Depression Study. I would like to express my sincere thanks to Nikolaos Kazantzis (PI), Paul Merrick, Janet Leathem, Jan Scott, Frank Dat tilio, Kei th Dobson, Rona Moss - Morris, Freder ick Newman, and Kevin Ronan who comprised the original research team involved in the CBT Depression Study and who were responsible for the initial design of the methods which enabled the gathering of my own data. I would like to acknowledge the Massey Univ ersity Albany Strategic Fund, the Massey University School of Psychology, and the Lottery Health Shared Equipment Grant for helping to fund the CBT Depression Study. To the patients and clinicians that participated in this research project thank you for your dedication and strength to succeed in making this project worthwhile . I would like to offer a special thanks to the all the raters that facilitated the collec tion of data, in particular, Ly n Birrell and Jenni Becket t for their persistence and flexibili ty. Also, thanks to my fellow coordinators, Margo Munro and Nicole Foster for keeping the CBT Depression Study running. Thanks to my supervisors Richard Fletcher and Nikolaos Kazantzis for their expertise and support at different stages in my thesis when it was most needed. I would also like to thank Beverly Haarhoff, Judith Beck, Willem Kuyken, Keith Dobson, Paul Salkovskis and Gregory Mumma for their expert feedback on the measure created and developed in this thesis. In particular, I would like to ackno wledge Judith Beck not only for her valuable feedback in developing the measure which facilitated this research but for providing permission to use her conceptualization diagram for the purpose of this research. vi Finally, I would like to thank my family f or their ongoing support. Without your love and encouragement I would not have started let alone completed this research project. vii TABLE OF CONTENTS ABSTRACT........................................................................................................................ iii ACKNOWLEDGEMENTS ................................................................................................ v LIST OF TABLES ........................................................................................................... xiii LIST OF FIGURES ........................................................................................................ xvii CHAPTER I ........................................................................................................................ 1 Introduction......................................................................................................................... 1 1.1 Overview ............................................................................................................. 1 1.2 Research Rationale and General Aims ................................................................. 1 1.3 Methodology and Findings .................................................................................. 4 1.4 Overview of Chapters .......................................................................................... 5 CHAPTER II ....................................................................................................................... 9 Theoretical Basis of Case Conceptualization in CBT ........................................................ 9 2.1 Defining Case Conceptualization ......................................................................... 9 2.2 The Process of Case Conceptualization .............................................................. 11 2.3 Cognitive Case Conceptualization ..................................................................... 13 2.4 Levels of Cognitive Case Conceptualization ...................................................... 14 2.5 Case Conceptualization Terminology ................................................................ 16 2.6 Formats of Cognitive Case Conceptualization .................................................... 17 2.7 Functions of Cognitive Case Conceptualizations ................................................ 20 2.8 Case Conceptualization Personality, and Personality Disorders .......................... 21 2.9 Summary ........................................................................................................... 22 viii CHAPTER III ................................................................................................................... 23 Empirical Basis of Case Conceptualization in CBT......................................................... 23 3.1 Evaluating Case Conceptualization in CBT ....................................................... 23 3.2 Measures of Cognitive Case Conceptualizations ................................................ 24 3.3 Reliability of Case Conceptualization in CBT .................................................... 26 3.4 Validity of Case Conceptualization in CBT ....................................................... 34 3.5 Summary ........................................................................................................... 41 CHAPTER IV ................................................................................................................... 45 Cognitive Behavioural Therapy (CBT) for Depression and the Use of Homework ....... 45 4.1 Overview of CBT for Depression ...................................................................... 45 4.2 Theory of CBT for Depression .......................................................................... 45 4.3 Research into CBT for Depression ..................................................................... 48 4.4 The Use of Homework in CBT for Depression .................................................. 49 4.5 Standard Manualized Treatment Protocols in CBT for Depression ..................... 55 4.6 Summary ........................................................................................................... 58 CHAPTER V ..................................................................................................................... 61 Therapist Competence in CBT for Depression ................................................................ 61 5.1 Determinants of Psychotherapy Outcome .......................................................... 61 5.2 Therapist Adherence and Competence ............................................................... 62 5.3 Therapist Competence for the Practice of Psychology in New Zealand .............. 63 5.4 Measuring Therapist Competence in CBT ......................................................... 65 5.5 Levels of Therapist Competence in CBT ........................................................... 68 5. 6 The Shape of Therapist Competence .................................................................. 69 ix 5.7 Outcome and Therapist Competence in CBT ..................................................... 69 5.8 Therapist Competence in CC and Homewor k Use ............................................. 71 5.9 Summary ........................................................................................................... 72 5.10 Research Objectives for the Current Project ....................................................... 73 5.11 Specific Hypotheses Investigated ....................................................................... 74 CHAPTER VI ................................................................................................................... 79 The CBT Homework Project ............................................................................................ 79 6.1 Overview ........................................................................................................... 79 6.2 Method .............................................................................................................. 80 6.3 Relevant Measures ............................................................................................ 84 6.4 Procedure .......................................................................................................... 90 CHAPTER VII .................................................................................................................. 97 Study One - The Development and Change of Case Conceptualizations throughout a Course of Cognitive Behavioural Therapy (CBT) for Depression .................................. 97 7.1 Outline and Aims ............................................................................................... 97 7.2 Method ............................................................................................................ 100 7.3 Procedure ........................................................................................................ 101 7.4 Results ............................................................................................................. 104 7.5 Discussion ....................................................................................................... 115 CHAPTER VIII .............................................................................................................. 119 Study Two - The Development of the Conceptualization Rating Scale (CRS).............. 119 8.1 Outline and Aims ............................................................................................. 119 8.2 Item Selection and Measure Construction ........................................................ 121 x 8.3 E xpert Feedback .............................................................................................. 127 8.4 Method ............................................................................................................ 130 8.5 Procedure ........................................................................................................ 131 8.6 Measure ........................................................................................................... 139 8.7 Statistical Analyses .......................................................................................... 139 8.8 Results – Training Sample ............................................................................... 142 8.9 Results – Total Sample .................................................................................... 151 8.10 Discussion ....................................................................................................... 157 CHAPTER IX ................................................................................................................. 161 Study Three - The Relationship between Therapist Competence in Case Conceptualization and Therapist Competence in Homework in CBT for Depression . 161 9.1 Outline and Aims ............................................................................................. 161 9.2 Statistical Analyses .......................................................................................... 163 9.3 Specifying the Model ....................................................................................... 167 9.4 Treatment of Data ............................................................................................ 171 9.5 Evaluating the Tenability of Model Assumptions ............................................. 177 9.6 Results – Descriptive Statistics of Observed Data ............................................ 180 9.7 Results – Development of a Multilevel Model for Change ............................... 194 9.8 Discussion ....................................................................................................... 206 CHAPTER X ................................................................................................................... 217 General Discussion .......................................................................................................... 217 10.1 Summary ......................................................................................................... 217 10.2 The Evolution of Case Conceptualization ........................................................ 219 xi 10.3 The Reliability of Case Conceptualization ....................................................... 222 10.4 The Validity of Case Conceptualization ........................................................... 142 10.5 Limitations ...................................................................................................... 226 10.6 Recommendations for Future Research ............................................................ 227 10.7 Implications ..................................................................................................... 228 10.8 Final Conclusions ............................................................................................ 232 REFERENCES ................................................................................................................ 235 APPENDIX A .................................................................................................................. 265 Participant Information Sheet ............................................................................................ 265 APPENDIX B .................................................................................................................. 267 Participant Consent Fo rm .................................................................................................. 267 APPENDIX C .................................................................................................................. 268 Participant Demographic Questionnaire ............................................................................ 268 APPENDIX D .................................................................................................................. 269 Therapist / Rater Demographic Questionnaire ................................................................... 269 APPENDIX E .................................................................................................................. 270 CRS – Final Draft ............................................................................................................. 270 APPENDIX F .................................................................................................................. 280 CRS – Therapist Competence Scale .................................................................................. 280 APPENDIX G .................................................................................................................. 281 CRS - Component Identification Guide ............................................................................. 281 APPENDIX H .................................................................................................................. 282 Training Material .............................................................................................................. 282 xii APPENDIX I ................................................................................................................... 295 Depression Study Feedback Questionnaire ........................................................................ 295 APPENDIX J ................................................................................................................... 301 Depression Study Advertisement Example ........................................................................ 301 APPENDIX K .................................................................................................................. 302 Depression Study Advertisement Pamphlet ....................................................................... 302 APPENDIX L .................................................................................................................. 303 Graphs of li near regressions .............................................................................................. 304 xiii LIST OF TABLES Table 1 Sample sizes, aims, model integrity, and outcome measures in cognitive case conceptualization reliability and validity studies.... ................................39 Table 2 Sources of data and levels of conceptualization in selected cognitive case conceptualiza tion reliability and validity studies… … ….... .....................40 Table 3 Total sample: Participant demographics, diagnosis, severity of d epression and number of therapy sessions................. ………........ ..................................81 Table 4 Study 1 : Participant demographics, diagnosis, severity of d epression and number of therapy Sessions. …… ……………. .........................................9 9 Table 5 Study 1 : Descriptive statistics of data units c oded from J. Beck Case Conceptualization Diagrams ..................................................................105 Table 6 Integration of conceptualization component for training session one ............143 Table 7 Integration of conceptualization co mponent for training session two...........143 Table 8 Integration of conceptualization co mponent for training session three.........144 Table 9 Importance of the integration of conceptualization co mponent for training session one........................................................................................145 Table 10 Importance of the integration of conceptualization co mponent for training session two........................................................................................145 Table 11 Importance of the integration of conceptualization co mponent for training session three......................................................................................146 Table 12 Int raclass Correlation Coefficients of competence for rated therapy sessions at each training session.....................................................................147 Table 13 Comparison of “surface” level and “deeper” level average competence Scores .............................................................................................................147 xiv Table 14 Intraclass Correlation Coefficients of fit or “match” for rated therapy sessions at each training session.....................................................................149 Table 15 Comparison of “surface” level and “deeper” level average fit or “match” scores................................................................................................150 Table 16 Individual ite m and total scale reliability estimates for the CRS ...................155 Table 17 Session specific reliability estimates for the CRS........ ..................................156 Table 18 Mean scores and descriptive statistics for therapist competence scor es in case conceptualization for each patient on the CRS by session..... ............182 Table 19 Mean scores and descriptive statistics for therapist competence scores in homework use for each patient on the HAAC S by session... ..........184 Table 20 Int ercorrelations among predictor, control and dep endent variables.............187 Table 21 Descriptive statistics for the individual growth parameters obtained by fitting separate within- person OLS regression models for BDI _Change as a function of curvilinear time.....................................................................189 Table 22 Correlations between intercepts and slopes for substantive predictors, BDI - I I changes scores and time........................... .........................................192 Table 23 Estimates of fixed effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms................................................195 Table 24 Estimates of random effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms..... ..................196 xv Table 25 Estimates of fixed effect s from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after the introduction of salient variables..............................................................................................199 Table 26 Estimates of fixed effects for rate of change towards a final model from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts chang e in depressive symptoms after the introduction of salient variables....................200 Table 27 Estimates of random effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after the introduction of salient variables..............................................................................................201 Table 28 Common features of in- session therapist competence in CBT case conceptualization ............................................................................................23 1 Table 29 Estimates of random effects from a series of individual growth models of time for predicting change in depressive symptoms......................................30 3 xvi xvii LIST OF FIGURES Figure 1 A model for understanding the relationship between case conceptualization and homework in CBT for depression..................................4 Figure 2 Diagram outlini ng the general process of case conceptualization ...................12 Figure 3 E xample of Judith Beck’s (1995) Case Conceptualization Diagram completed after session 5……… .....................................................................18 Figure 4 Participant recruitment, screening and final treatment sample ........................90 Figure 5 Changes in mean amount of information in different components of J. Beck Case Conceptualization Diagrams across sessions......... ...................105 Figure 6 Changes in mean number of automatic thoughts in J. Beck Case Conceptualization Diagrams (average frequency of data units for all ten participants)................ .........................................................................106 Figure 7 Changes in mean number of meanings of automatic thoughts in J. Beck Case Conceptualization Diagrams (average frequency of data units for all ten participants)..................................... ....................................................107 Figure 8 Changes in number of emotions recorded in J. Beck Case Conceptualization Diagrams (total frequency of data unit s for all ten participants)............................................................... ...............................108 Figure 9 Changes in number of behaviours recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participants) ..............................................................................................108 Figure 10 Changes in number of conditional assumptions recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participants)............... ..........................................................................108 xviii Figure 11 Changes in number of compensatory strategies recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participants) .........................................................................................109 Figure 12 Changes in number of patient strengths recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participants)............................................... ..........................................109 Figure 13 Changes in mean amount of information for situations in J. Beck Case Conceptualization Diagrams (average frequency of data units for all ten participants)......................... ................................................................110 Figure 14 Changes in number of relevant childhood information / background data recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participants )....................... .......................111 Figure 15 Changes in number of core beliefs recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participants) ..............................................................................................111 Figure 16 E xample of b eliefs about ‘future’ as recorded in J. Beck Case Conceptualization Diagrams..................................................... .....................113 Figure 17 E xample of b eliefs about ‘world / others’ as recorded in J. Beck Case Conceptualization Diagrams... .......................................................................113 Figure 18 E xample of beliefs about ‘self’ as recorded in J. Beck Case Conceptualization Diagrams......... .................................................................114 Figure 19 Number of core beliefs related to ‘self’, ‘world / others’, and ‘future’ in J. Beck Case Conceptualization Diagrams at corresponding therapy sessions...........................................................................................................114 xix Figure 20 Diagram of Conceptualization Components and Core Therapist Competencies in Case Conceptualization........... ...........................................126 Figure 21 Diagram summarising the CRS rater training and measure development process......................................................................................132 Figure 22 D iagram of conceptualization component categorisation at conceptualization levels.................................................................................134 Figure 23 “Surface ” level conceptualization components ..............................................1 35 Figure 24 “Deeper” level conceptualization components..... ..........................................1 35 Figure 25 “Alternative” le vel conceptualization components.... ....................................136 Figure 26 Residual plot of depressive symptom change scores ( BDI change)............ ..175 Figure 27 Residual plot of scores for therapist competence in case conceptualization ( CRS comp) in relation to scores for change in depressive symptoms ( BDI change) ...............................................................180 Figure 28 Linear regression of therapist competence scores in case conceptualization as measured on the CRS f or individual patient by therapy session...............................................................................................182 Figure 29 Linear regression of therapist competence scores in homework use as measured on the HAACS f or individual patient by therapy session..............184 Figure 30 Linear regressions of BDI - I I change scores for individual patient by therapy session...............................................................................................186 Figure 31 Mean centred data trajectories for therapist competence in case conceptualization, homework use and d epressive symptom change.............187 Figure 32 Linear regressions for change in depression severity as measured by BDI - I I change scores for individual patients by therapy session...................30 4 xx Figure 33 Linear regressions of HAACS competence scores for individual patient by therapy session..........................................................................................30 5 Figure 34 Linear regressions of CRS competence scores for individual patient by therapy session...............................................................................................30 5 Figure 35 Linear regressions of CRS fit / match s cores for individual patient by therapy session..............................................................................................30 6 Figure 36 Linear regressions for change in the fit of written and in - session content of CCs as measured by CRS fit / match scores for individual patients by therapy session..........................................................................................30 6 xxi xxii 1 CHAPTER I Introduction 1.1 Overview This introductory chapter provides a brief rationale and backgr ound for the construction of a new measure of therapist competence in case conceptualization (CC) . A practical and theoretical context for understanding the relationship between case conceptualization and homework as two major interrelating processes withi n CBT for depression is outlined. An overview of chapters in the present thesis affords an understanding of the present thesis as a single cohesive piece of research derived of a series of chapters and interrelated studies. 1.2 Research Rationale and General Aims CC in psychotherapy is an ever- evolving product of an ongoing process with the goal of integrating a patient’s presenting problems into a justifiable and consistent description (Eells, 1997a, 2007; Kuyken, Fothergill, Musa, & Chadwick, 2005). In cognitive CC, cognitive- behavioural theory exists as a framew ork for understanding the patient’s presenting problems and informs inte rvention (Beck, 1995; N eedleman, 1999; Persons, 1989). I ntervention, in turn, re - informs the evolving CC process over the course of therapy (Dozois, Covin, & Brinker, 2003) . Therapist CC has frequentl y been described as a central component of Cognitive Behavioural Therapy as espoused by Aaron T. Beck (C BT ; A. T. Beck, Rush, Shaw, & Emery, 1979) ; researchers and clinicians referring to CC as the “first principle” , an “essential component” the “cornerstone” or the “heart” of CBT (A. T. Bec k et al., 1979; B ieling & Kuyken, 2003; Chadwick, Williams, & Mackenzie, 2003; 2 Persons, 2001). Thus, the organisa tion and integration of the information from a patient’s presentation and the quality of the r esulting written CCs have traditionally been considered to impact how therapists approach a patient’s case and implement a treatment plan (Crits - Christoph, Cooper & Luborsky, 1988). Despite wid ely accepted theory commonly implemented in the contemporary practice of psychotherapy, there has been a paucity of research into the utility of a general processes that might account for mechanisms of change in CBT ( Greenburg, 1991; Ilardi & Craighead, 19 94; Kazdin, 2006 ; Whisman, 1993 ) . This present thesis identifies CC as one such process which might account for hypothesised mechanisms of change within CBT ( Eells, Kendjelic, & Lucas, 1998; Persons, Mooney, & Padesky, 1995). In particular, specific formats of CC have not been thoroughly evaluated in the context of CBT. The present research seeks to address this research gap. Another central component of CBT, ‘homework’, exists as an exception to the dearth of research into central CBT processes (Kazantzis, 2005). Homework is broadly defined as collaborative and planned between- session activities tailored to meet a patient’s therapeutic goals; these tasks are based on an individualized CC consistent with the empirically supported CBT model ( Kazantzis, Deane, Ronan, & L’Abate, 2005 ). In theory, processes involved in homework integration facilitate underlying mechanisms that enable generalization of content discussed in - session and promotes cognitive and behavioural skill acquisition. In research, homework has increasingly been related to treatment benefits (for a review see Kazantzis, Deane & Ronan, 2000) and therapist competence (Davidson et al., 2004; Shaw, et al., 1999; Trepka, Rees, Shapiro, Hardy & Barkham, 2004 ). During therapy sessions the therapist guides the collaborative selection of in- session and between- session homework tasks based on patient s individualized CC s. Homework is both informed 3 by, a nd informs the CC process, in a dynamic interplay between areas of assessment and therapy (Dozois et al. , 2 003). Figure 1 provides a model for understanding the broad processes of CBT in relation to CC, treatment planning and homework. However, no study has specifically studied the re lationship between how CC changes over the course of therapy and homework in C BT and the subsequent relationship with therapeutic outcome. Subsequently, the current investigation into CC and homework provides a means of investigating the relationship between fundamental in- session and between- session processes in CBT . Furthermore the overall process by which therapists arrive at formal CCs warrants empirical investigation in the context of CBT. In order to addres s this gap in the literature the primary research aims for the current project are to: 1) design a new measure of therapist competence in CC suitable for use in CBT for depression, 2) investigate how therapist CCs change over a standard course of CBT for depression, and 3) to investigate the relationship between therapist CC, homework and outcomes in CBT for depression. It is intended that through these aims, and facilitated by a critical review of the current literature that some guidelines towards a model of therapist competence in CC in CBT can be presented in the final discussion. 4 Figure 1. A model for understanding the relationship between case conceptualization and homework i n CBT for depression 1.3 Methodology and Findings In order t o enable an empirical investigation of CC, the process by which CCs are constructed by therapists must be operationalized. To achieve this, the focus of the present thesis was to develop a new measure of therapist competence in CC. A combination of descriptive analysis of written CCs and reliability analysis of training of independent observers in the use of the new measure of therapist competence in CC provided information for the subsequent revision of the new measure of therapist competence in CC. Further larger scale reliability analysis and quantitative analysis using non- parametric methods of testing were used to demonstrate psychometric properties of the new measure. Finally multilevel modeling methods were employed Three broad goals a therapist has over the course of psychotherapy are to: Understand the patient and their presenting problems and strengths Apply a treatment plan promoting skill acquisition in session Promote generalization of these skills into areas of deficit in the patient’s life building upon patient strengths Case conceptualization Treatment Plan / Standard Treatment Protocol Homework 5 culminating in a statistical model for understanding the relationship between CC and homework in the context of CBT for depression. The research provides data on how therapists use CC in clinical practice both between sessions and during psychotherapy sessions. Also, the psychometric properties of a new measure of therapist competence in CC are assessed and discussed. A link between other psychotherapeutic processes (e.g., homework) and outcomes or treatment benefits (e.g., reduction in depr ession severity, functioning, and cognitive change) is also reported and discussed . Broadly speaking , this thesis provides empirical support for components or therapist skills contained within the process of CC that determine the final CC product central and specific to the practice of CBT in the context of depression. In summary, the aim of this research was to establish the importance of therapist competence, spec ifically in cognitive processes related to CBT in depression. This thesis presents a measure for assessing and monitoring therapist competence in CC that has been constructed. This thesis describes the development, construction and robust psychometric properties the measure has demonstrated. 1.4 Overview of Chapters Subsequent chapters explore necessary areas of research and present interrelated studies in order to achieve the research aims of the present thesis. Chapters two and three provide a critical review of CC in the context of CBT for depression. To provide this discussion, in chapter two a definition of CC, the main construct of interest in this thesis, and a discussion of key features that constitute a cognitive CC are discussed. Different level s of information contained within cognitive CCs and the proposed function of cognitive CCs are outlined. Commonly used 6 formats of CC used in CBT are outlined and discussed. Finally, the proposed functions of CC that might account for change in CBT are pres ented. In chapter three measures and associated methods for arriving at the final written CC products used in routine CBT practice are discussed. The empirical demonstration of reliability and validity in prior research as it applies to CC is reviewed and discussed. Chapter three identifies a lack of means to measure the process by which therapists construct CCs in CBT. Problems and strengths with current methodology used in research to investigate case conceptualization are presented and used as a guide and rationale for designing and interpreting studies one, two and three in the latter portion of the present thesis. Chapter four outlines the theory and research of CBT for depression and the use of homework. Secondly, s tandard manual- based treatment protocols are discussed providing a rationale for the importance of a new measure of therapist competence and giving an overview of Beckian cognitive - behavioural theory of psychopathology that exists as the framewo rk of the CC process. Chapter five discusses therapist factors as they relate to the use of homework and CC in CBT for depression. Consideration is given to the relevance of therapist competence for practice as a registered psychologist in Aotearoa / New Zealand. At the conclusion of chapter five, spe cific research objectives and hypotheses that have emerged are then presented to outline the primary goals of this thesis. In addition , the ethical considerations, limi tations of this projects and potential contribution of the proposed area of research will be discussed. Chapter six provides an overview of the CBT Homework Project or ‘ Depression Study’ which exists as the framework or context for the present thesis . A description of participants, therapists, therapist training, patient inclusion and 7 exclus ion criteria, overall project procedure as well as ethical considerations is provided. The remainder of this thesis presents three studies designed to test hypotheses emergent from the introductory chapters. In chapter seven , Study O ne presents data on the development or change of therapists’ written CCs during the early phases of a course of CBT for depression. This chapter adopts an exploratory and largely qualitative approach providing a survey of sorts by graphically representing how therapists use written CCs. In chapter eight , the development of a new measure of therapist competence in CC, the Conceptualization Ra ting Scale (CRS , Easden & Kazantzis, 2008 ; 2009 ) , is described. The training process for independent observers and feedback process for arriving at the final version of the CRS is included in the lead up to Study Two. In Study T wo, d ata on the reliability of therapist CCs in CBT for depression is provided for both the training sample and the total sample used in the present thesis. In chapter nine, S tudy Three explores the relationship between CC and homework. Firstly, d ata from the CRS on therapist competence in CC in CBT for depression is presented. Second ly, the relationship between CC and homework is explored towards a multilevel model which accounts for the proposed relationship in the context of CBT for depression and offers preliminary empirical support for the importance of therapist competence in CC in CBT for depression . The final chapter of this thesis, chapter ten , presents a discuss ion of the overall findings from all three studies in relationship to the propositions stated here and primary research aims and hypotheses outlined in the present thesis. Implications of the current research on clinical practice, training, supervision an d future research are 8 discussed and reconciled with existing research and in the context of potentially meaningful avenues for future research. 9 CHAPTER II Theoretical Basis of Case Conceptualization in CBT Look into the depths of your own soul and learn first to know yourself, then you will understand why this illness was bound to come upon you and perhaps you will thenceforth avoid falling ill. Sigmund Freud (1856 – 1939) 2.1 Defining Case Conceptualization It is generally considered that case conceptualization (CC) in CBT and across a range of psychotherapies is a working, evolving hypothesis based on guiding theory about a patient that seeks to identify and link underlying mechanisms accounting for a patient’s psychological, interpersonal and behaviour al deficits (Eells , Lombart, Kendjelic, Turner, & Lucas, 2005; Curtis & Silberschatz , 1997; Bieling & Kuyken , 2003; Persons, 1989). Numerous definitions of what exactly should be included in a CC and what form it should take have been documented. Methods for generating a range of theory- specific CCs are grounded in a wide range of theoretical propositions (Beck, 1995; Luborsky & Crits - Christoph, 1998; Curtis, Silberschatz, Sampson, & Weiss, 1994 ; Horowitz & Rosenberg, 1994; Needleman, 1999; Perry, 1994; Persons, 1989; Sperry & Maniacci, 1992) . Examples of specific methods or formats of CCs across different psychotherapies include the Core Conflictual Relationship Theme (CCRT; Luborsky, 1977; 1986; Luborsky & Crits - Christoph, 1998 ; Luborsky, Crits - Christoph, & Mellon, 1986) , the Plan Formulation Method (PFM; Curtis, Silberschatz, Weiss, Sampson, & Rosenberg, 1988; Curtis & Silberschatz, 1991), the Cyclic Maladaptive Pattern (CMP; Henry, Schacht, & Strupp, 1986; Johnson, Popp, Schacht, Mellon, & 10 Strupp, 1989), the Connected Central Relationship Patterns (CCRP; Albani, Pokorny, & K ä chele, 1998 ) , the Consensual Response Psychodynamic Formulation (Horowitz & Rosenberg, 1994) to name a few. The vast majority of methods for constructing a CC are grounded in psychodynamic or and interpersonal theoretical orientations and draw from or build upon the CCRT. In contrast, CCs from different theoretical perspectives also share a number of commonalities. In particular, m ost comprehensive CCs are suggested to comprise of a description of manifest problems, relevant developmental history, causal factors (distil and proximal), maintaining factors, as well as coping strengths or protective factors and weaknesses or perceived obstacles (Persons, 1989; Bieling & Kuyken, 2003). Ho wever some approaches, primarily those stemming from a psychodynamic theoretical orientation, focus on specific aspects. For example, the CCRT method focuses on a patient’s narrative around central relationship patterns considered the ‘heart’ of a CC (Lubo rsky, 1997). In this way, psychotherapists and clinical researchers from different psychotherapeutic orientations place emphasis on the relative importance of different aspects of the CC (Eells, 1997b; Eells et al., 2005; Flitcroft, James, Freeston, & Wood - Mitchell, 2007 ) . This lack of agreement suggests a need for empirically supported guidelines for the training and practice of CC construction and use in psychotherapy. This need is particularly important for the advancement of CBT as an empirically supported treatment where research in support of CC driven by CBT theory has not advanced to the extent as other CC methods specific to other psychotherapeutic orientations (see Chapter III ) . 11 2.2 The Process of Case Conceptualization A formal written CC is the product of ongoing process throughout the course of therapy; more specifically in an ever evolving process a t herapist is continuously making mental conceptualizations about their patient’s presenting problems and symptoms both in session and across sessions (Persons & Tompkins, 1997). Figure 2 provides a representation of the process by which formal written CCs are constructed. The figure is useful to acknowledge that CC is not a discrete output. Instead, CC is often an ongoing mental and verbal process. A therapist then takes “snap shots” that reflect this process, though formal ( or informal) written records that are then revised incrementally throughout the course of therapy. However, recently emphasis has been placed upon the patient’s role in developing the CC collaboratively with the therapist (Kuyken, Dobson, & Padesky, 2008). Building upon this notion, therapists commonly experience a “holding in mind” of their patients between sessions further to collaborative discussions with patients and active construction of a formal written CC ( Sch röder, Wiseman, & Orlinsky , 2009). Consistent with this definition CC has been described as a mode of experiential learning: Conceptualizing involves patients in developing objectivity about their experiences and hav ing a more accurate, deeper or perceptive understanding of their problem, which might lead to new understanding and to synthesis. The therapist provides accessible theoretical ideas to help inform the patient’s thinking, gives feedback and facilitates re - evaluation in order to help the patient to make fresh sense of the situation . (Milne, Claydon, Blackburn, James, & Sheikh, 2001, p.25) 12 Figure 2. Diagram outlining the general process of case conceptualization Accordingly, therapists construct CCs through in- session discussion with patients as well as between therapy sessions (i.e. , during supervision and personal reflection). Furthermo re, at various stages in therapy written or formal CCs are essential for the therapist to organize and monitor patients’ mental conceptualizations as well as to integrate rather than summarize seemingly contradictory pieces of information about a patient (Eells, 1997a, 2007). In this way, CC is both a product and a process within psychotherapy. W hile the formal written CC product evolves at discrete points in therapy, the process of CC is continuous. Eells provides a further definition CC consistent with this notion: 13 A psychotherapy case formulation is a hypothesis about the causes, precipitants, and maintaining in fluences of a person’s psychological, interpersonal and behavioural problems. A case formulation helps organize often complex and contradictory information about a person. It should serve as a blueprint guiding treatment, as a marker for change, and as a s tructure enabling the therapist to understand the patient better. A case formulation should also help the therapist anticipate therapy- interfering events and experience greater empathy for the patient. (Eells, 1997 a, p.430) 2.3 Cognitive Case Conceptualization The current project focuses on comprehensive cognitive CCs as intended for use in a Beckian Cognitive Behavioural Therapy (CBT; A. T. Beck, Rush, Shaw, & Emery, 1979). A cognitive CC differs from those associated with other ‘schools’ of theory or those suggested to be “pantheoretical” or “transtheoretical” ( Eells, 1997a ; Jose & Goldfried, 2008; Meier, 1999; Prochaska & DiClemente , 2005 ) in that they are grounded in cognitive- behavioural theory as espoused in A. T Beck’s theory of psychopathology (19 76 ) . Similarly, i n cognitive CC, the terminology and content is drawn heavily from the cognitive- behavioural model. Conversely, most CC research has been produced around a psychodynamic framework. Aaron T. Beck’s cognitive model (1976) advocates the importance of conceptualizing different situations by exploring and monitoring corresponding surface level automatic thoughts and attached meaning, thoughts, emotion, and behaviour. Furthermore , the importance of underlying assumptions or deeper level cognitions, such as conditional assumptions and core beliefs are emphasised as a means to explain the etiology and maintenance of a patient’s presentation across multiple situations. It follows that, i n order to put into 14 practice A. T. Beck’s cognitive behavioural t heory it is necessary to integrate all areas of CC into practice as well as to provide empirical support for comprehensive CC to better inform the use of CC in CBT. Kuyk en, Padesky, & Dudley (2009 ) provide a brief definition of CC in CBT which synthesizes the definition discussed: Case conceptualization [in CBT] is a process whereby therapist and client work collaboratively first to describe and then to explain the issue a client presents in therapy. Its primary function is to guide therapy in order to relieve client distress and build client resistance. ( p.3) In arriving at a definition of CC in CBT which is assumes such a central role in the overarching goals of psychotherapy, a rationale emerges for the empirical investigation and operationalization of CC as an hypothesised process that helps to identify and facilitate mechanisms of change in psychotherapy. 2.4 Levels of Cognitive Case Conceptualization In order to provide a fair and robust test of cognitive CC it is necessary to define the different levels of cognitive CCs so they can be investigated in a structured and uniformed manner. Three unique levels of CC in C B T have been identified (Persons, 1989). The first is described as the ‘situation’ level CC involving details of cognitions and behaviours specific to an immediate triggering situation or event. Examples include the five - part model (Padesky & Mooney, 1990) , Jacqueline Persons’ ‘Problem List’ (Persons, 1989) , and the Daily Record of Dysfunctional Thoughts or ‘Thought Record’ ( A. T. Beck et al., 1979) . A further example, while not 15 exclusively used in the practice of CBT, includes the Behavioural Functional Analysis as commonly implemented in the treatment of borderline personality disorder (Linehan, 1980; 1993 ; Waltz & Linehan, 1999) . The second level of CC occurs at the ‘diag nosis’, ‘syndrome’ or ‘problem’ level which focuses on disorder- specific theory or research and associated symptoms. A. T. Beck, Emery, and Greenberg’s (1985) book Anxiety Disorders and Phobias: A cognitive perspective offers examples of this second level of conceptualization incorporating conceptual frameworks for understanding patient’s distress characterized by generalized anxiety disorder (GAD), obsessive compulsive disorder (OCD), posttraumatic stress disorder (PTSD) , social anxiety disorder, panic disorder among other categories for understanding anxiety. Further examples of this level of CC include formulations of disorder- specific theory on OCD (Clark, 2004) , substance abuse (Basco & Rush, 2004) , bipolar disorder ( Newman et al., 2002) , depression (Beck et al., 1987), eating disorders (Garner & Garfinkel, 1997) , schizophrenia ( Chadwick, Birchwood, & Trower, 1996) , and personality disorders ( J. Beck , 2005 ) . These disorder- specific models provide a context for cognitio ns at surface, intermediate and core levels that promote understanding of clusters of cognitions, behaviours, emotions and physiology that are associated with the etiology and maintenance of particular clinical disorders and syndromes (Clark & A. T. Beck, 1999; Riso et al., 2007). While this second level of CC describes generalized patterns of common psychological phenomena, it does not account for idiosyncratic differences. Specific causes or maintaining factors that may account for psychological distress such as abuse or neglect, and the individualized attributions developed through such experiences (e.g., I am a failure as a mother – I am a failure in eve ry way) are generally not included in disorder- specific conceptualizations. 16 The third level of CC occurs at the ‘case’ level of conceptualization and aims to integrate all levels of information about a case to arrive at an individualized and comprehensive CC. This third level is defined as the “compreh ensive CC” throughout the present thesis to recogniz e this level as the most complete and meaningful conceptualization as it integrates information from all levels of inference in CBT . Examples of these comprehensive CCs include Jacqueline Person’s Case Formulation Approach (1989) which incorporates and extends the situational - level problems list and the Judith Beck Case Conceptualization Diagram (1995). Comprehensive cognitive CCs such as these are a parsimonious account of a patients’ presentation including relevant historical information, hypothesised s chemas and conditional rules, core beliefs, compensatory strategies or maintaining factors, and problems or situations including the behaviours, thoughts and associated emotions (J. S. Beck, 1995; Bieling & Kuyken, 2003; Eells, 1997; 2007; Needleman, 1999; Persons, 1989; Persons & Tomkins, 1997). Such comprehensive CCs are not intended to capture a patient’s entire life, but to target psychopathology through identifying cognitive and behavioural weaknesses and strengths (Bieling & Kuyken, 2003) , also referr ed to as overdeveloped and underdeveloped cognitions and behavioural strategies (A. T. Beck et al., 1979). 2.5 Case Conceptualization Terminology For the purposes o f the current project the term ‘case formulation’ will be used synonymously with ‘case conceptualization ’ as defined above. This is necessary in order to reflect upon literature which has adopted one term over another due to personal preference, theoretical orientation or a researcher’s need to refer to another researcher’s approach incorporat ing the use of a specified term. Furthermore, the J. 17 Beck Conceptualization Diagram (CCD; J. Beck, 1995) is the primary CC format of interest in this thesis. This is evident in that the J. Beck CCD is the format used in the CBT Homework Project or ‘Depress ion Study’ which provides the data set on which the thesis is based. Subsequently, the use of the terminology ‘case conceptualization’ over ‘case formulation’ has been selected to remain consistent with the terminology adopted by Judith Beck. 2.6 Formats of Cognitive Case Conceptualization E xamples such as the above that describe comprehensive case level conceptualizations have been developed to facilitate the creation of a concise and parsimonious CC where written or graphically displayed formats are often advocated (Beck, 1995; Needleman, 1999; Persons, 1989). Jacqueline Persons (1989) has proposed a single- page written format, the Case Formulation Sheet (see Persons, 1989 ) . The key components of this cognitive CC format are the problem list, schema or core beliefs, precipitants and activating situations, strengths and assets, the working hypothesis, predicted obstacles to treatment, as well as the origins of a patient’s disorder and problems (for a discussion see Persons, 2001). Judith Beck’s (1995) CC diagram (see Figure 3 for an example of a completed diagram ) includes text boxes pertaining to relevant childhood data, core beliefs, conditional assumptions and rules, coping strategies, as well as situational information as each situation relates to automatic thoughts and subsequent meaning to the patient, emotion and associated behaviour. Additionally, Needleman (1999) has combined Judith Beck’s (1995) CC diagram format and Persons’ (1989) Problem Case Formulation Sheet into one hybrid format the Case Conceptualization Summary Plan encapsulating the important categories contained in both as well as a broad treatment plan. 18 Figure 3. Example of Judith Beck’s (1995) Case Conceptualization Diagram completed after session 5 19 Weerasekera (1996) presents the ‘Four Ps’ appr oach to constructing a CC. Although the approach is described as a “multiperspective” or “integrative” approach the author specifically addresses the cognitive perspective. The CC is constructed by generating cognitive ‘ predisposing’ , ‘ precipitating’ , ‘ perpetuating’ , and ‘ protective’ factors. More recently the ‘Five Ps’ has become more commonly used including ‘presenting’ factors or problems consistent with Persons (1989) problem list. For example, in the case of a depresse d woman with a child, the women’s positive thoughts towards her child and beliefs surrounding her responsibility to be a good mother may be conceptualized as a protective factor that could potentially buffer psychological distress should these beliefs be activated. While this approach is not based on cognitive- behavioural theory, it can facilitate the construction of a cognitive CC when used in conjunction with a cognitive - behavioural theoretical perspective. Moreover, it might be speculated that cognitive assessment should be considered an essential component of any psychological assessment. As with all CCs, a treatment plan or guide to intervention, though strictly speaking not a part of the CC, should be informed or driven by the CC, consistent with hypothesis and other information within the CC (Bieling & Kuyken, 2003; Turkat, 1987). Similarly, ongoing assessment of a patient during therapeutic interventions should serve as a further source of information gathering and hypothesis testing (Persons & Tompk ins, 1997; Eells, 1997b). In this sense, although CC formats are static, the CC is a flexible, ever evolving entity. The resulting CC at any stage should be supported by the evidence gained from a patient’s presentation in therapy as well as from homework tasks undertaken outside of therapy (Kaza ntzis et al., 2005; Rector, 2007 ). 20 I n relation to commonly used formats of CC the question is posed “how do cognitive therapists formulate in the ‘real world’? Although some systematic formats of CC in CBT exist a dearth of empirical evidence to support their utility in CBT has been identified ( Kuyken, 2006). 2.7 Functions of Cognitive Case Conceptualizations The precise functions of CCs in CBT have been the subject of speculation as opposed to empirical investigation by researchers. For example, CCs in CBT have been suggested to promote the individualization and tailoring of treatment protocols, improve the therapeutic relationship between the therapist and patient, facilitate and inform treatment planning, be of particular benefit with complex patients or patients with more severe presentations. However, due to the lack of reliable and valid measures of CC in CBT these presuppositions are yet to be empirically tested. K uyken, Padesky, and Dudley (2008 ) present ten proposed functions of CC in CBT: 1. Synthesizes client experience, CBT theory and research 2. Normalizes presenting issues and is validating 3. Promotes client engagement 4. Makes numerous, complex problems more manageable 5. Guides selection, focus and sequence of int erventions 6. Identifies client strengths and suggests ways to build client resilience 7. Suggests the simplest and most cost - efficient interventions 8. Anticipates and addresses problems in therapy 9. Helps understand non - response in therapy and suggests alternative routes for change 10. Enables high q uality supervision 21 The proposed functions exist as a guideline for the range of variables that should be investigated to provide empirical support for the utility of CC in CBT. To date, little research exists investigatin g these proposed functions the stand as relatively untested hypotheses regarding the importance and utility of CC in CBT. 2.8 Case Conceptualization Personality, and Personality Disorders CC has been suggested to have particular utility in understanding the etiology and maintenance of individual personality traits and personality disorders ( J. Beck, 2005; Stenhouse & van Kessel , 2002 ; Westen, 1998 ) . The idea of a functional diagnosis has been proposed as a link between CC and diagnosis of personality disorders in contrast to the conceptualization of CC and DSM - I V - TR (American Psychiatric Association, 2000) diagnosis as two separate processes ( Weste n, 1998 ; Scott & Sembi, 2006 ). The functional diagnosis includes exploration of the patient’ s experience of th e self and others which is particularly relevant to the CBT for depression and the use of the negative cognitive triad as a theoretical framework (Beck , 1976 ). It has been postulated that reducing patient levels of dysfunctional interpersonal beliefs associated with the self and others is a potential mechanism of change (Clark in, 2006 ). Furthermore, understanding particular beliefs and behaviours that are associated with different personality disorders is important to developing a useful conceptualization a nd forming a strong therapeutic relationship (J. Beck, 2005). For example, patients diagnosed with Borderline Personality Disorder (BPD) have a high rate of comorbidity with substance abuse as well as pervasive interpersonal relationship problems. In formi ng a CC of a patient that incorporates concerns associated with BPD, a therapist is cued to the potential for additional problems and points of intervention often associated with the particular personality 22 disorder (Nezu, Nezu, & Lombardo, 2004). However, it is unclear as to the extent or the direction of the relationship between therapist interventions (e.g., different domains of therapist competence in CC) and variables associated with personality and personality disorders in CBT (Con nolly Gibbons, Crits- Christoph, Levinson, & Barber, 2003) . 2.9 Summary CC is central to the theory and practice of CBT both as a verbal process and in written format. Although the definition, content and focus of CCs are different across various psychotherapeutic orientations, CC in CBT includes characteristic differences both in definition and in practice relative to other psychotherapeutic orientations that have been outlined in this chapter. These distinctive cognitive features are observable in the CBT - specific written CC formats that are used in the common practice of CBT. However, the utility and function of these CC formats remains largely untested in the context of CBT. The following chapter reviews existing literature bearing on the empirical basis of CC in CBT. 23 CHAPTER III Empirical Basis of Case Conceptualization in CBT 3.1 Evaluating Case Conceptualization in CBT Although CC is considered to be central to CBT within CBT training programmes extending into routine practice (Sudak, Beck, & Wright, 2003) , it is important to note that to date, due to a paucity of research into specific CBT process, only few studies have found empirical support for the reliability and validity of CC in CBT (Chadwick, Williams, & Mackenzie, 2003; Kuyken, 2006). A cognitive CC, like any other theory- related conceptualization, is considered to be a useful conceptual and clinical tool (Sim, Gwee, & Bateman, 2005). As such, thorough objective investigation of the psychometric properties of the clinical tool should be carried out, inclusive of reliability and validity estimates of measures or tests used to make this assessment (Anastasi & Urbina, 1997). Moreover, CC as a psychotherapeutic process affords opportunities for understanding the mechanisms by which cognitive interventions influence change in CBT and other psychotherapeutic modalities (Garrat, Ingram, Rand, & Sawalani, 2007 ; Haubert & Dobson, 2007; Kazdin, 2006) . The current chapter begins by discussing existing measures used to investigate CC in CBT. Subsequently a critical review of research on the reliability and validity of CC is presented incorporating the use CC measures and methods developed to investigate the empirical basis for CC in CBT. 24 3.2 Measures of Cognitive Case Conceptualizations Few measures exist designed for the investigation of CC suitable for use within CBT. The Case Formulation Content Coding Method (CFCCM; Eells, Kendjelic, Lucas, & Lombart, 1 998, Eells et al., 1998) is a comprehensive questionnaire designed to measure the content and quality of written CCs. The CFCCM requires a rater to code information into “idea units” based on several categories including a) descriptive information, b) diagnostic information, c) formulation / inferred information and includes d ) treatment p lanning. In addition a separate section assesses the ‘elaboration and quality’ of CCs rated on ‘1’ to ‘5’ Likert scales including categories that rate a) complexity , b) degree of inference, c) precision of language, d) overall coherence, and e) systematic process. Based on written intake reports from a sample of 56 patient s the CFCCM yielded a kappa of .86 (ranging from .67 – 1.00) across both content and quality scales Consistent with the Four P’s (Weerasekera, 1996) , the CFCCM is designed as a structured approach to generating and assessing CCs in terms of patient’s presenting, precipitating, perpetuating and protective factors suitable for use in a range of psychotherapeutic orientations but intended to be “theoretically neutral” . Although the CFCCM incorporates a number of items relevant to a broad range of psychotherapeutic perspectives the CFCCM also includes CBT - specific items, for example, dysfunctional thoughts, schemas, automatic thoughts, core beliefs and the assessment of beliefs towards the self, world and others consistent with A. T. Beck et al.’s (1979) treatment of depression . W hile the CFCCM h as been tested based on CCs developed using a CBT approach, it does not advocate a cognitive- behavioural focus. Rather the therapist’s own psychotherapeutic orientation drives the use of the measure. This is problematic as a therapist’s self - report of therapeutic orientation does not ensure adherence to a CBT 25 model. In turn, there are limitations in generalising findings towards the empirical support of CBT - specific CC. The Quality of Cogn itive- Behavioural Case Formulation Rating Scale (Fothergill & Kuyken, 2002) was developed based on the quality scale included in the CFCCM and using “best practice” guidelines as stated in the measure. An initial pilot study yielding ratings of agreement based on 10 independently coded J. Beck CC D s ( k = .85) with scores ranging from very poor to good (‘‘very poor’’ n = 22.1% ; ‘‘poor’’ n = 33.6% ; ‘‘good enough’’ n = 34.5% ; ‘‘good’’ n = 9.7%). The measure is rated on a 4 - point Likert scale with descriptive a nchors including “very poor”, “poor”, “good enough” and “good” . For example , a “good enough” CC requires that “The elements are more clearly integrated (links and connections are apparent betw een sections) and more relevant , than irrelevant information is included. The case formulation is generally coherent and accurate. However, there may be some element of doubt as to whether the participant has fully understood the task. One or two of the elements of the formulation are either too brief of too verbose”. However, the Quality of Cognitive Case Formulation Rating Scale does not bear on the process involved in CC beyond an indirect indication of therapist competence by examining the CC as the final written product. A second measure developed specifically for the measurement of CC in CBT the Cognitive Formulation Rating Scale (CFRS; Academy of Cognitive Therapy, 1998) is a 12 - item measure used to assess a therapist’s written CC in three main areas: 1) case history, 2) case formulation, and 3) treatment plan and course of therapy. Each item is rated on a 3 - point Likert scale (i.e., 0 = not present, 1 = present but inadequate, 2 = present and adequate). The CFRS has a possible maximum score of 24 where a score of 20 is used as a criterion to indicate a “passing” o r adequate CC. Again, t he 26 CFRS is limited in that is only provides an indication of the adequacy or quality of written CCs but does not account for the therapist’s competence in eliciting information relevant to the CC in session. The CFRS is suggested, ho wever, to be clinically useful in the training of cognitive therapists in writing CCs when presented with clinical case information whilst it does not provide the specificity to determine processes or skill - sets utilized by the therapist to arrive at their written CC and fully assess the quality of written infor mation (Sudak et al. , 2003). This section identifies an astonishing lack of measures used to formally assess the use of CC in CBT. The need for the creation of measures that address aspects of the therapeutic process as they relate to CC are highlighted. Having introduced the limited measures of CC and written formats commonly used in CBT, t he following section explores reliability and validity in CC within CBT as the research relates to proposed functions of CC that have been investigated to date using existent formats and measures designed to investigate CC in the context of CBT. Where appropriate the next section discusses further structured methods used to arrive at CCs in CBT as they relate to research examining the reliability and validity of CC in CBT. 3.3 Reliability of Case Conceptualization in CBT The limited research conducted specifically on the reliability of cognitive CCs has produced less than optimal results. In particular, few contemporary studies have addressed the research question ‘can cognitive therapists produce reliable cognitive CCs?’ and existing research has produced inconclusive results. In an early study 54 patients ranging from mild to severe depression were rated on the Cognitive Triad Inventory (CTI; Beckham, Leber, Watkins, Boyer, & Cook, 1986). The CTI items were based on cognitive theory and the cognitive triad of depression stipulating that 27 an interplay among negative views of the self, world and future underlie dep ression, and subsequent cognitive treatment interventions should be employed to target this mechanism (A. T. Beck et al., 1979). Internal reliability for three subscales ‘view of self’, ‘view of world’ and ‘view of future’ ranged from good to excellent res pectively (α = .91, α = .81, α = .93). Overall reliability for the CTI was excellent (α = .95). Since the study was limited to addressing a single disorder - specific model in CBT for depression, conclusions are restricted. Most importantly, this study does not investigate comprehensive CC. However, this study supports the notion that cognitive therapists are able to reliably formulate CCs at the level of the disorder or the symptom, although generalization of the findings is limited in that the research was carried out with a subset of depressed patients. Persons, Mooney, and Padesky (1995) used a sample of 46 clinicians in an attempt to test Persons’ (1989) case conceptualization model. Clinicians listened to “part or all” of an initial interview with two an xious, depressed patients. Percentage agreement for the eight problems identified on the problem lists for patient one ( n = 3) and patient two ( n = 5) averaged 77.7% (range from 13.0% - 100% ). Intraclass Correlation Coefficients (ICC; Shrout & Fleiss, 1979 ) for the dysfunctional b eliefs lists ranged from .07 to .92 when averaged over 5 judges. The design of Persons et al.’s (1995) study was problematic in that the limited portions of therapy examined, the use of pre- contrived multiple choice statements, as well as criterion lists to capture patient dysfunctional beliefs did not reflect a naturalistic therapy session. Furthermore, the wide range of agreement rates reflects disagreement between clinicians both at the more superficial level of patient problems and at the deeper level of patient beliefs. 28 Persons & Bertagnolli (1999) attempted to increase the reliability of their results by implementing structured CC training and revising previously used criterion on measures related to underlying beliefs. Forty - seven clinicians who participated in a day long workshop in CC and listened to audio tapes of initial sessions with anxious, depressed patients ( N = 2). On average two thirds of presenting problems were identified, however, no penalty was given for incorr ectly identified problems. Intraclass Correlation Coefficients (ICC; Shrout & Fleiss, 1979) for the Dysfunctional Beliefs lists ranged from .44 to .91 (average .71) when averaged over 5 judges and ranged from .13 to .66 (average .37) when judged in compari son with criterion the criterion list. In attempting to replicate Persons et al.’s (1995) study, Persons and Bertagnolli’s (1999) study also produced poor results. Overall, Persons’ (1989) model has yielded moderate reliability over multiple judges’ averag e rating but poor reliability for single judges’ ratings (Persons & Bertagnolli, 1999; Persons, Mooney, & Padesky, 1995; Persons & Tompkins, 1997). However, reliability tests of CCs have been identified as problematic in part due to a disjunctio n between research and practice. This is clear in the above studies that focused on specific levels and components of CCs and used data from isolated portions of therapy sessions. This review argues for a fundamental misapplication of reliability as it applies to ‘r eal world’ CC. One explanation for low reliability of CCs can be had by a theoretical understanding of the CC process where many equally efficacious cognitive CCs could include variations in content, thus rendering different CCs potentially equally as valid. This is referred to as the quality of a CC where the coherence and justifiability is considered to at times supersede the reliability of a CC (Bieling & Kuyken, 2003; Kuyken et al., 2005). Furthermore, reliability should therefore ideally be considered in terms of the consistency of a therapist’s written CC 29 both to a patient’s presentation in session and within the CBT model. In this way , it can be ensured that a cognitive CC is being assessed rather than an alternative CC lacking focus on cognitions or deviating from widely accepted theoretically orientated language consistent with cognitive behavioural approaches ( Goldfried, 1999, 2000). This notion is consistent with research investigating Person’s (1989) model where limited information about the patient’s presentation was available to raters and therapist adherence and competence to the model was not adequately controlled for. Furthermore raters were by self - definition relatively unskilled (Henry & Williams, 1997). Additionally , the idea of an evolving, ever changing CC process was not considered in the research (Needleman, 1999). Limited res earch has explored the evolution or change of CCs over time when investigating the reliability of cognitive content. In one such study, 10 clinicians independently constructed two to three CCs each based video tapes of interviews from a sample of four women with mood and anx iety disorders (Mumma & Smith, 2001). Each patient had two cognitive CCs completed on their case relating to 15 dimensions pertaining to cognition, affect, symptoms and interpersonal functioning. Reliability was excellent when averaged over 10 raters ( ICC >.83 for all dimensions). However, the reliability of paired ratings was not reported. Although encouraging, the findings were based on situation- level rather than a case- level or comprehensive CC. This study adds support to the notion that CBT therapists can reliably formulate CCs at the level of the situation and warrants further research investigating the reliability of comprehensive CC at the level of the entire case. Though little research has been conducted on comprehensive cognitive CC, some studies have focused on cognitive aspects of components of CCs producing findings that may bear on the way CCs are intended for use consistent with CBT 30 theory and practice (e.g . research that addressed individual patient situational CCs, or considers automatic thoughts). One study assessed self - scenarios representing patient self- schemas considered to be an essential aspect of cognitive CC (Muran et al., 1994). Self - scenarios were essentially situational chain analyses judged to be most clinically relevant by two observers based on audio tapes of assessment interviews for the suitability of CBT between patients ( N = 8) and therapists ( N = 5 ). Therapists, independent observers and patients each rated self- scenarios for relevance of self scenarios selected based on a series of 9 - point Likert scales which provided structure to the assessments in areas such a frequency, preoccupation and chronicity of self scenarios. Patients rated their own self- scenarios and those of other patients to determine aspects of convergent and divergent validity. Rates of agreement were excellent when averaged across all cases and components of rated self - scenarios (ICC (3, k) = .92) . These results reflect both concurrent reliability and validity dependent upon whether the client, observer or therapist ratings are viewed as the criterion. Moreover, the results suggest that observers, patients and therapists are able to come to consensus over cognitive aspects of CC. In an extension of this study self - scenarios were constructed including an interpersonal component alongside the self- scenarios previously explored (Muran, Samstag, Se gal, & Winston, 1998). While the research did not speci fically target CBT, the procedure involved patients ( N = 6) constructing descriptions of individualized interpersonal situations where patients considered themselves to be at their best and worst (i.e. targeting maladaptive or adaptive cognitive and behavi oural style). The average interrater reliability for ratings of ‘clinical relevance’ (ICC (2, k) = .90) and the entire scenario were excellent (ICC (2, k) = .94). Good test - retest reliability was found by assessing the consistency of interpersonal self- scenarios as 31 rated by the six patients after 29 therapy sessions with ICCs ranging from 0.79 to 0.97 although were averaged over three cases. Averaging ratings over more ratings can distort the interpretation of the reliability coefficients where the ICC incr eases as a function of the sample size. Furthermore, including more than one rater for comparison is not reflective of what might be expected in clinical practice where a single clinician is likely to write t he formal CC in consultation with supervisors and perhaps other clinicians. Muran, Samstag, Ventur, Segal, and Winston (2001) applied their method of constructing self- scenarios to a single case study yielding excellent interrater reliability when reliability scores pertaining to beliefs about the ‘sel f’ for various scenarios were averaged (ranging from ICC (2, k) = .83 - 1.00). Previous CC research reflects a lack of consensus among clinicians for deeper level inferences, for example, at the level of schema (Eells et a l., 1998; Kuyken et al., 2005). I n contrast, Muran et al.’s (1994; 1998; 200 1) research reflects a triangulation of agreement among clinicians, independent observers and patients. The results suggest that to reach agreement on high- inference conceptualization content, greater structure of the ‘top down’ process of accurately identifying schemas (e.g., through increased training in identifying appropriate information linked with theory) must be addressed in combination with increased structure of ‘bottom up’ methods that start with prescribed schemas that may tend to account for patient who present with particular diagnoses. Muran et al.’s (1994; 1998 ; 2001) research is limited in that while it investigates a clinician’s ability to identify broad underlying beliefs, schema or cognitions consistent with those targeted in CBT, the beliefs are better described as intermediate level beliefs restricted to a patient’s understanding of the ‘self’ in interpersonal situations rather than ‘deeper level’ core beliefs that may become activated across a range of situations. Moreover, beliefs and their levels are poorly 32 defined in CBT literature. This is not surprising given that terms such as “surface”, “intermediate” and “deeper” level beliefs are somewhat arbitrary considering that beliefs at different levels are likely to overlap and be conceptually intertwined (Alford & A. T. Beck, 1997; J. Beck, 1995). Furthermore, protocol checks pertaining to treatment delivery (e.g., adherence or competence to a CBT protocol) were not carried out beyond clinician self- report. Contrary to ‘bottom up’ methods that prescribe lists of schema, Muran et al.’s (1998; 2001) studies reflect a relatively naturalistic setting where therapists identifying clinically meaningful CC content do not begin with a prescribed list of possible content and apply this to a patient but use theory to inform the identification of clinically meaningful situations. The results suggests that when guided by theory, training and increased structure pertaining to the implementation of the theory leads to increased agreement on what makes an accurate CC. Kuyken et al. (2005) tested the reliability and quality of cognitive CCs by investigating the J. Beck cognitive CC Diagram format (1995) . One hundred and fifteen mental health practitioners participated in a Continuing Education workshop in CC conducted by the first author. Clinicians listened to audio tapes of intake assessment with a single patient “Anna” (an actress) diagnosed with MD D and “personality difficulties” . Using data from additional measures (e.g., intake summary) clinicians then provided CCs for a single case rated against a ‘benchmark’ CC created by J. Beck. Participants also completed J. Beck CC Diagrams based on information subsequently rated using the Quality of Cognitive Case Formul ation Rating Scale. J. Beck provided a “benchmark” CC assumed to be the correct, gold standard CC. The procedure involved transferring information onto cards and using two independent judges to form agreement on categories of information units (range k = .63 - .91) . However, these ratings were only related to correct placement of CC information in 33 the diagram rather than that content of this information. Subsequent percent agreement between individual participants and benchmark CC ranged from 7% to 73% for individual information units, on average less than half the benchmark CC info being identified, or less than 50% agreement between two raters. What can be concluded from this study is that clinicians are able to reach a reasonable rate of agreement on the structure of CC, however, the wide range of reliability scores are consistent with the same issues identified in previous studies in that the content of case conceptualization identified is not consistent between raters despite having predefined criterion. Furthermore, it is suggested that a higher rate of agreement on both the content and structure of cognitive CCs are likely to be found among well - trained clinicians. As well as the experience of the single rat er, the research reviewed reveals that there is a tendency for greater expert agreement when there are greater numbers of clinicians involved in determining what should be included in an accurate CC. The studies examined often indicate high reliability for multiple raters but poor reliability for pairs of raters (i.e. , an initial single rater and a “target” represented by a second rater where raters are interchangeable; Shrout & Fleiss, 1979) . Although agreement by multiple experts has been suggested to bear on validity (Persons et al., 1995; Persons & Bertagnolli, 1999) some research in other therapeutic orientations has suggested that averaging judge’s ratings increases the reliability but does not dictate an increase in validity (Horowitz, Inouye, & Siegelman, 1979). In this way increased reliability on the bases of averaged ratings can result in misleading interpretations around the validity of scores for a measure. Drawing inferences based on averaged multiple rater scores and neglecting single or individual rater performance in using measures of CC has practical implications. Namely, the ability to generalise the 34 reliability scores and have confidence that a single rater is sufficient in the use of the measure in question, is limited clinical practice, research and training. The studies discussed outline methods for studying individual aspects of CC. Based on the literature reviewed i mproved methods should be employed to investigate the reliability and validity of comprehensive cognitive CCs in large patient samples using sources of data that is representative of naturalistic therapy sessions. The studies discussed so far also highlight the specificity of cognitive- behavioural theory (e.g., the concept of schemas or core beliefs essential to CBT) which shows that increased structure by training the theory and practice of CBT as well as a method of measurement which should reflect what is taught in training in CC. It is important to note that while limited research on moderate to large samples of clinicians has been carried out, no study has i nvestigated the reliability of comprehensive CCs in a medium to large sample of patients. In summary, the literature reviewed in the present thesis concords with a call for improved methods to reach a consensus on why cognitive therapists have difficulty reaching agreement on the content of cognitive CCs (s ee Table 1) . 3.4 Validity of Case Conceptualization in CBT CC is considered to be an integral part of a standard course of CBT, and while we can be confident that CBT, presumed to include specific use of CC, is related to patient benefit in broad range of areas, it is unclear what impact CC has on patient outcomes. To date there has been little empirical support for the link between CC and patient outcomes over the course of CBT . Such a link is necessary to ensu re the empirically supported practice of CBT including hypothesised mechanisms theorised to produce reliable change (Duncan, Sparks, & Miller, 2005 ; Kazdin, 2006 ). 35 A single existing study investigated what therapists ( N =23) determine to be important in the content of a CC (Flitcroft , Freeston, & Wood - Mitchell, 2007). Therapists rank ordered 86 items pertaining to the purpose of CCs generated based on a literature search of case conceptualization. The results suggested that therapists primarily valued the “ explanatory power” and “function/utility” of CCs used in the context of CBT for depression but also reflected disagreement among clinicians. No survey data or information specifically addressing how CBT therapists use systematic CC formats, particularly ac ross sessions has currently been published. There have been few studies that have investigated how CCs are reconciled with patients’ experiences. One of these studies involved 11 patients where no evidence was found to support the direct impact of cognitive CC on the main targets of CB T for psychoses (Chadwick et al., 2003). The therapists in the study gathered information pertaining to the CC as a baseline, and in a second condition the therapist spent two sessions dedicated to the sharing and development of the CC. However, the results cannot provide compelling evidence that cognitive CC has no impact on the therapeutic outcome of CB T because information on therapist fidelity was not assessed. Moreover, a more rigorous empirical test would evaluate cognit ive CC on outcome once a reliable cognitive CC had been obtained. In an extension of this study the Chadwick et al. (2003) study , 13 patients each undergoing a course of CB T from two therapists undergoing weekly supervision, were interviewed regarding their experience of the sharing of a CC two to three weeks after the sharing had taken place (Pain, Chadwick, & Abba, 2008). Both a CC diagram encapsulating standard CC components and a letter outlining an individualized CC for each patient were given to each patient to take home to read and make changes. Audio recordings of sessions and interviews were analysed using 36 content analysis. A number of themes were produced revealing mixed feelings from participants. For example, the majority of patients reported ex periencing both positive and negative emotions in response to CCs shared in session. O verall, very few patients endorsed the idea that they would experience no change or ultimately feel worse as a result of the shared CC. F or therapists their experience of the sharing of the CC reflected increased understanding of the patient, perceived strengthening of the therapeutic relationship, and increased sense of direction with the pa tient. As with prior research, reliability and fidelity checks were not a strong f eature of the research while informal checks of CC accuracy were carried out and therapists reported the use of CCs in the study increased their adherence to the CB T model. In sum, patient reports of their experiences and outcomes related to CC were mixed while therapist experiences tend to be positive. These preliminary findings support the assertion that patients and therapists report different benefits and experiences of CCs . Apart from preliminary results, there is little evidence to suggest that the qu ality of decision- making or working hypotheses affect the process and outcome of CBT (Kuyken et al., 2005). While recent case studies provide anecdotal support for the impact of CC in particular disorders or by targeting particular beliefs ( González - Prendes, 2007; Gorenstein, Tager, Shapiro, Mo nk, & Sloan, 2007; Mansell, 2008 ; Morasco, Weinstock, Ledgerwood, & Petry, 2007) research investigating medium to large samples under controlled conditions is also needed to reach generalizable conclusions about the importance of CC in CBT. Of particular relevance to process research in CC, large scale research has been suggested necessary in order to detect small effect sizes proposed to account for a “weaker than expected” relationship between therapist competence and outcomes (Barber, Sharpless, Klostermann, & McCarthy, 2007) . 37 Here a gap in the research has been identified in CBT literature. However, this gap is not present throughout the literature in CC in other psychotherapeutic approaches or theoretical orientations ( Caspar, 1995; Crits - Christoph, Cooper, & Luborsky, 1988; Rosenberg, Curtis, Sampson, Silberschatz, & Weiss, 1986; Field, Nash, Handwerk, & Friman, 2004; Levenson, 1995; Levenson & Strupp, 2007; Silberschatz, Fretter, & Curtis, 1986; Strupp & Binder , 1984) . For example, Crits - Christoph, Cooper, and Luborsky’s (1988) results pertaining to focal dynamic psychotherapy suggested that therapist’s “ formulation- adherent” interpretations, or treatment interventions consistent with the CC in brief psychodynamic therapy yield greater therapeutic benefits than non- adherent interpretations. This research suggests is that although CCs have general support from other therapeutic approaches or paradigms, that re search specific to cognitive CC is needed to provide empirical support for this core process within CBT. Only two studies have investigated the treatment utility of the use of CC in CBT in the range of moderate to large sample sizes to support the assertion that CBT guided by cognitive CC will lead to improved therapeutic outcome and symptom reduction (Persons, 2006 ; Persons, Bostrom, & Bertagnolli, 1999; Persons, Roberts, Zalecki, & Brechwald, 2006). The first study ( Persons et al., 1999) , found comparative results between a sample of 45 depressed outpatients treated with CC- driven CBT alone ( n = 27) or CBT plus pharmacotherapy and other results from published randomized controlled trials (RCTs) as indicated by a reduction in depressive symptoms. Similarly, Persons et al. (2006) assessed 58 anxious depressed outpatients and found statistically and clinically significant results as well as a large effect size of CC - driven CBT similar to results from other successful CBT treatment outcome studies. Each of these studies used a ‘naturalistic’ design to investigate the 38 effectiveness of an intervention without the inclusion of a control within the trial. R esults were comparable to other efficacious RCTs of empirically supported treatments, although the study was an uncontrolled trial and a large number of patients ( n = 40) were also receiving adjunct therapies including pharmacotherapy. It has been suggested that large naturalistic studies are comparable supplements for RCTs which may be useful for generating new hypotheses (Barber et al., 2007 ; Blacker & Mortimore, 19 96). However , in Persons et al.’s ( 1999 ; 2006) studies the impact of any additional CC training over and above what is expected in a standard course of CBT was not directly measured. For example, the assessment of therapist competence in using techniques r elated to CC or measurement of the quality of CC would be required to attribute additional treatment benefits or an effect size statistic to a course of CBT with an improved CC component. Moreover, CC is an inherent part of standard CBT (A. T. Beck et al., 1979) where CBT should be “driven”, guided and informed by CC. In this way, t hese studies neglect the process of CC within CBT and focus on the outcome. However, e vidence is generated supporting the idea that therapists asked to pay special attention to C C over the course of therapy are at least equally effective as those therapists that intend to provide a course of standard CBT without particular emphasis on CC beyond what is usually expected. Overall, critique of the limited research bearing on treatme nt- outcome of CC within CBT highlights the need for future research on the validity of CC in CBT. Table 1 provides a summary of the studies critiqued in this literature review emphasising the dearth of research relating outcomes related to symptom change beyond single case study and case series research. The table also summarises the 39 Table 1 Sample sizes, aims, model integrity, and outcome measures in cognitive case conceptualization reliability and validity studies Note. Table 1 is based on information reported in selected papers specifically investigating aspects of reliability and validity cognitive content of case conceptualizations appropriate for use in CBT Sample Size ( n) Study Aims CBT model / CC application CC Outcome measures Raters Therapists Patients I nter- formulator agreement Comparison to expert benchmark / criterion I ntegrity checks Specific CC training L ink to symptom change Patient feedback on CC utility Beckham et al. (1986) 16 n/a 54 Persons et al. (1995) 46 n/a 2   Persons & Bertagnolli (1999) 47 n/a 2    Mumma & Smith (2001) 10 n/a 4  Mumma (2004) 1 1 1  Mumma & Mooney (2007) 2 1 1  Muran & Segal (1992) 2 1 1  Muran et al. (1994) 2 5 8    Muran et al. (1998) 4 6 6    Muran et al. (2001) 2 1 1     Kuyken et al. (2005) 115 n/a 1    Chadwick et al. (2003) 1 2 13   Pain et al. (2008) 1 2 13   Persons et al. (1999) n/a 1 45   Persons et al. (2006) n/a 1 58   40 Table 2 Sources of data and levels of conceptualization in selected cognitive case conceptualization reliability and validity studies Source of conceptualization data (e.g., therapist / patient) Level of conceptualization (e.g., surface, surface / intermediate, all) Intak e assessment/s vignettes Data from multiple therapy sessions Independent observation of therapy sessions or recordings Situational (“surface” thoughts, emotions, etc) Diagnosis (i.e., level of symptom or disorder) Comprehensive CC (including “deeper” und erlying mechanisms, etc) Beckham et al. (1986)   Persons et al. (1995)     Persons & Bertagnolli (1999)     Mumma & Smith (2001)    Mumma (2004)    Mumma & Mooney (2007)     Muran & Segal (1992)      Muran et al. (1994)    Muran et al. (1998)    Muran et al. (2001)   Kuyken et al. (2005)      Chadwick et al. (2003)   Pain et al. (2008)   Persons et al. (1999)   Persons et al. (2006)   Note. Table 2 is based on information reported in selected papers specifically investigating aspects of reliability and validity cognitive content of case conceptualizations appropriate for use in CBT 41 relative absence of integrity checks and structured CC training wit hin CC research to ensure adherence and competence to both the CBT model and interventions and techniques involved in CC. Table 2 provides an overview of different sources of CC data reflecting a trend towards research focusing on generating CCs from intak e assessments rather taking a ‘naturalistic’ sample of therapy sessions across a full course of therapy. In turn, research based on isolated ‘snap - shots’ of a patient’s total therapy exposure has limitations when making generalisation about the utility of CC across a full course of CBT. Compounding this issue, Table 2 also reflects the narrow focus of CC research relying on investigations based on vignettes, snippets or surface level CCs. While each piece of research makes a further contribution to the body of literature, this review has sought to emphasise that it is important to capture or measure comprehensive CCs and CC- related processes when conducting research to provide a robust empirical bases for CC in CBT. 3.5 Summary Despite the importance att ributed to CC currently no measure adequately evaluates therapist’s use of CC in CBT. Research bearing on the reliability of therapists CCs suggests that cognitive therapists a less able to agree on more ‘deeper’ level or more inferential aspects of CC. Ho wever, these conclusions are made based largely on formal written CC formats where results do not necessarily translate or generalise to the naturalistic ‘real life’ practice of CBT. Furthermore, while in theory CC in attributed to impact the outcomes in CBT, there is a paucity of research addressing the validity of CCs in CBT. This gap within the body of CBT research suggests a need for improved methods of research ex tending into the exploration of the process by CCs are 42 constructed and utilized in CBT . This conclusion is consistent with an ongoing and growing focus on psychotherapy process issues, particularly around therapist factors that account for therapeutic benefit within psychotherapy. In sum, the literature reviewed in chapter two identifies a number of key areas emergent from the existing body of literature that demand attention in future research in the context of CBT: 1. Construction of comprehensive measures of CC processes 2. Systematic testing of existing systematic CC formats 3. Inclusion of str uctured training in cognitive CC 4. Protocol checks pertaining to treatment delivery in: a. The model of treatment (i.e., a specific CBT protocol) b. The therapist integration and use of components and skills specific to CC (see Chapter IV for a discussion of level s of measurement as they relate to therapist competence) 5. Longitudinal i nvestigation into how CCs change across therapy sessions 6. Robust tests of interrater reliability 7. Direct comparisons of CCs with outcome measures 8. Investigation of CC measures across a wide range of possible diagnoses and patient presentations 9. Investigation of CC measures across different sites 10. The use of medium to larger patient samples as well as continued case study analysis 11. Optimisation of m ethods reflecting a naturalistic or “real wor ld” therapy (i.e. analysis of complete therapy sessions) 43 The present thesis aims to synthesize and integrate these findings as much as possible. This review recognises CC as a unique area of process research that may help to answer key q uestions that remain unanswered. N amely, ‘ can cognitive therapists be observed to agree on the structure and content of cognitive CCs?’ and questions surrounding the importance of CC in CBT relative to patient outcomes over time. Also, a focus on the process of CC construction is advocated rather than discrete written outputs alone. Where possible, studies in the present thesis have sought to follow to the guidelines suggested above in the context of CBT for depression. The next chapter provides a discussion of CBT for depr ession and the relevance of in- session and between session treatment planning (i.e., homework) to the investigation of CC. 44 45 CHAPTER IV Cognitive Behavioural Therapy (CBT) for Depression and the Use of Homework The stronger person is not the one making the most noise but the one who can quietly direct the conversation toward defining and solving problems. Aaron T. Beck (1921 – present) 4.1 Overview of CBT for Depression Cognitive Behavioural Therapy (CBT; A. T. Beck et a l., 1979) is a term used to describe a range of therapeutic approaches that share aspects of theory and practice. CBT is a commonly used psychotherapy both in New Zealand and internationally (Kazantzis & Deane, 1998). Three common fundamental propositions have been outlined in cognitive- behavioural theory found support in research (Dobson, 2001) , namely: 1) that cognitive activity affects behaviour, 2) that cognitive activity can be monitored and altered, and 3) that desired behaviour change may be affected through cognitive change. ‘Beckian’ Cognitive Therapy (CT; A. T. Beck et al., 1979) is considered the standard CBT, originally developed for the treatment of depression, recognized as one of the most widely researched, used and empiri cally supported treatments for depression ( A. T. Beck et al., 1967; 1976; A. T. Beck et al., 1979). 4.2 Theory of CBT for Depression In brief, Aaron T. Beck’s cognitive - behavioural theory explaining the etiology and maintenance of depression stipulates that ‘schemata’, or co gnitive processes that organize and process information, maintain dysfunctional or bias ed beliefs about the self, world and others, forming what is known as the negative cognitive triad. This process leads to the distortion of an individual’s experiences r esulting in a number of 46 cognitive errors (i.e., overgeneralization, catastrophizing, personalization, selective abstraction; J. Beck, 1995). For individuals experiencing depression, cognitive errors such as these are likely to be present across a number of different situations in their life. The most prevailing beliefs associated with this negative cognitive process, considered to be the deepest level of beliefs held, are known as core beliefs. Although core beliefs may be positive, negative core beliefs ar e those that maintain systematic distortions and emotional disturbance. A core belief is theorized to exist as an underlying cognitive mechanism. Negative core beliefs about one’s self might become activated in a particular situation, triggering particular schema associated with negative outcomes, or negative past experiences. For a depressed individual this can result in a self fulfilling prophesy where, for instance, harmful compensatory strategies can lead a downward spiral of depressed thinking, behavio ur, and emotion and dysfunctional beliefs (A. T. Beck, 1967; 1976; A. T. Beck et al., 1979). For example , an individual experiencing depression may receive a poor review from their employer (situation). On the surface level the individual describes feelin g angry towards her employer and internally sad (emotion) because she disputes the reasons behind the poor review (thought). The individual does not let the employer know that she thinks or feels this way but instead physically leaves the office in the immediate situation having said nothing (behaviour). Having not made her thoughts and feelings clear, the individual ruminates about the event with her cognitions revolving around the notion ‘ I am a failure’ (core belief). In a second situation the same ind ividual may have had a disagreement with a friend. Although the situation is different, the emotion, thoughts and behaviour that occur are similar to the previous example. The individual thinks that her friends will not speak to her again, feels sad, and l eaves her friend’s house during the argument. 47 Again the individual ruminates about the event with her cognitions revolving around the notion ‘ I am a failure’ (core belief). Across these two situations the core belief that ‘I am a failure’ is present. The avoidance demonstrated by the individual in each particular situation is likely to be present in other areas of the individual’s life (compensatory strategy). Furthermore, it may be the case that this avoidant strategy was adopted from childhood experience s, where, for example, in a third recurrent situation, the individual was verbally abused by her parent, told she was useless or a failure, and was able to diminish the distress of this experience by running away. In this way this individual has developed beliefs that have become generalized and interlinked within a range of situations that trigger similar cognitive affective behavioural processes. The above example provides a brief demonstration of how a therapist begins to build up a CC together with a patient within the context of CBT for depression. By recognizing and monitoring the patient’s self- perpetuating negative cycle, often experienced initially as automatic by the individual experiencing depression, cognitive restructuring can take place pr omoting balanced thinking. Specifically, and of particular relevance to the present thesis, the cognitive content- specificity hypothesis ( A. T. Beck, 1976 ; Clark & A. T. Beck, 1989 ) states tha t each emotional or psychological disorder or affective state can be characterized by cognitive content specific to that disorder or state. In the above example, while a poor review from an employer, or a disagreement with a friend may evoke a number of different thoughts, emotions or behaviours, it is our appraisal of the meaning of the event that determines these outcomes rather than the actual event or situation. Similarly, in the example of a depressed patient experiencing sadness, it is the interpretation that the event will lead to thoughts of significant loss or failure rather than the event itself. 48 4.3 Research into CBT for Depression Over the last 40 years a vast body of research has been conducted in support of many aspects of CBT ( A. T. Beck, 2005). Recently , Butler, Chapman, Forman, and A. T. Beck (2006) cond ucted a review of meta- analyses, identifying 16 methodologically rigorous meta- analyses including a collective sample of 9995 participants in 332 studies. They concluded from 562 comparisons that CBT is as effective or superior to antidepressants in the treatment of adults and at least as effective when compared to other treatments such as behaviour therapy (DeMaat, Dekker, Schoevers, & DeJonghe, 2006; DeRubeis et al 2005; Jarrett et al., 1999) and yields large effect sizes for a range of disorders. D isorders that have found empirical support for the efficacy of CBT in both theory and research include the treatment of anxiety and phobia (Beck, Emery, & Greenberg, 1985; Butler, Fennell, Robson, & Gelder, 1991; Chambless & Gillis, 1993 ; Clark, 2004; Clark et a l., 2003), personality (Beck, Emery, & Greenberg, 1985; Svartberg, Stiles, & Seltzer, 2004) , bipolar (Lam et al., 2003), addiction (Brown, Evans, Miller, Burgess, & Mueller, 1997) , eating (Pike, Walsh, Vitousek, Wilson, & Bauer, 2003) , and psychotic disord ers (Gould, Mueser, Bolton, Mays, & Goff, 2001 ; Rector, Seeman, & Segal, 2003; Sensky et al., 2000 ) . Research has also supported the efficacy of CBT in relapse prevention in the treatment of depression in adults (DeRubeis et al 2005; Hollon, 2003; Hollon e t al 2005; Paykel et al., 2005) as well as children and adolescents ( James, Soler, & Weatherall, 2005; Macdonald, Higgins, & Ramchandani, 2006; March et al., 2004). 49 4.4 The Use of Homework in CBT for Depression Introduction Although an increasing amount of literature has supported the treatment benefits of CBT as a treatment as a whole, there is a paucity of research into the efficacy of specific therapist techniques, components and processes within CBT that account for how, when and why change occurs ( Barber & DeRu beis, 1989, 2001; Freeley, DeRubeis, & Gelfand, 1999; Jacobson et al., 2000; Persons, 2005; Tang & DeRubeis, 1999; Whisman, 1993). However, one central component of CBT, the use of homework has been recently emphasized in lit erature (Kazantzis , Deane, Ronan, & L’Abate, 2005 ) . Homework as a specific process within CBT has received more attention than any other therapeutic process (Kazantzis, 2005) . Consistent with cognitive behavioural theory (A. T. Beck et al., 1979) , a collaborative and system atic approach to homework use has been promoted including the discussion and specification of frequency, duration and location of the homework task to be undertaken and a written note of what is intended to be completed, all in consideration of a patient’s capacity to undertake the homework task in question (Shelton & Ackerman, 1974; Shelton & Levy, 1981). Furthermore, special attention to review of homework as well as the design and assignment of homework tasks are laid out as three crucial steps in a guiding model for practice theorized to be essential to successful completion of homework tasks and therapeutic benefit ( Kazantzis, Macewan, & Dattilio, 2005). However, in practice a certain degree of therapist competence is required to integrate homework use expertly and efficiently into a therapy session (Kazantzis et al., 2005 ) . 50 Definition of Homework in CBT Homework is a term that is used within a range of psychotherapeutic orientations ( She lton, & Ackerman, 1974 ; Tomkins, 2004) . Broadly s peaking, homework encompasses any number of in - session and between- session activities undertaken by a patient, usually in collaboration with the therapist towards fulfilling therapeutic goals. To illustrate, therapy can be though t of as a ‘pin - hole’ of a patient’s life. In order to generalise what is discussed in therapy to a patient’s everyday life, patients and therapists are able to practice during therapy sessions and plan to for between- session integration of homework . This effectively extend s the patient exposure t o therapy- related engagement from 50 minutes during a session to, for example, two hours spent on homework outside of therapy . A common example of homework tasks might include behavioural experiments, the monitoring of cognitions and behaviours, pleasurable activities or self- care strategies. The precise form of the homework t ask can manifest in various ways ranging from, for example, a long term comprehensive thought record or diary to a long belated walk in the park to get fit, listening to music, or a simple phone call to a friend. Homework tasks extend to the disengagement, ‘non - behaviour’, for example thought stopping for sexual offenders or rumination in the case of depression. In CBT homewo rk is defined in relation to cognitive- behavioural theory originally described in Aaron. T. Be ck’s standard CBT . This is described in his book Cognitive Therapy and the Emotional Disorders ( 1976) which places an emphasis throughout the book on the importance of the patient documenting and monitoring in- session and between- session thoughts, emotions and behaviours (i.e. homework) in the process of patient skill acquisition . Kazantzis et al. (2005) present an operational definition for homework assignments: 51 Homework assignments are planned therapeutic activities under taken by clients between therapy sessions. Their content is derived primarily from the empirically supported cognitive behavioural therapy model for the particular presenting problem but is tailored for the client based on a n individualized conceptualizat ion. Designed collaboratively, homework assignments are focused on the client’s goals for therapy. Homework assignments represent the main process by which clients experience behavioral and cognitive therapeutic change, practice and maintain new skills and techniques, and experiment with new behaviours. Homework assignments also provide an opportunity for clients to collect information regarding their thoughts, moods, physiology, and behaviours in different situations and to read information related to therapy and their presenting problems. (p. 2) Homework and Therapy Outcome in CBT for Depression The use of homework in CBT has been supported by a growing body of research suggesting that the level of compliance in CBT homework tasks is associated with treatment benefits ( Addis & Jacobson, 2000; Bryant, Simons, & Thase, 1999; Burns & Auerbach, 1992 ; Burns & Nolen - Hoeksema, 1991 ; Burns & Spangler, 2000; Carroll, Nich, & Ball, 2005; McEvoy, & Nathan, 2005; Rees, Kazantzis, Deane & Ronan, 2000 ). Kazantzis et a l. (2000) carried out a meta - analysis of 27 studies ( N = 1327) exploring the effects of homework on treatment outcome with anxious and depressed patients. A medium effect size ( r = .36) of homework assignments in relation to CBT outcomes was found supporti ng a correlational relationship between patient compliance with homework assignments and therapy outcome. Based on these displays of effect sizes, it is suggest ed that 68% of depressed patients receiving CBT 52 would be likely to improve in therapy involving homework compared to 32% of patients not given homework. However, Kazantzis et al.’s (2000) analysis did not include estimates of effect sizes for “no homework” condition in consideration of the possibility that treatment without homework could potentially be equally or more beneficial than structured homework - augmented treatment protocols. A further limitation included the sub- grouping of studies by: a) those considered to be either correlational or causal, b) into diagnostic sub - populations (i.e., anxious , depressed, other out- patient), and c) by type of homework task undertaken (i.e., no single type, relaxation, other skills). More recently, Kazantzis, Whittington, and Dattilio (in press), replicated and extended the findings of Kazantzis et al.’s (2000) meta- analysis in 46 studies ( N = 107 2) yielding a pre and post- treatment effect size of .63 for control conditions and 1.08 for therapy conditions including structured homework use. When comparing control and treatment studies for the same therapy with or without homework (i.e., CBT compared with CBT with structured homework use, instead of CBT versus relaxation therapy incorporating homework use), a medium effect size of .48 was obtained. This more general and conservative approach compared to the Kazantzi s et al. (2000) study was able to increase the overall sample size for comparison of supporting the utility of homework in use in CBT versus CBT without structured homework use . Mausbach, Moore, Roesh, Cardenas and Patterson (2010) conducted a third meta- analysis based on 23 studies ( N = 2183) only a small to medium effect size of .26. Mausbach et al.’s (2010) meta - analysis produced very similar results to Kazantzis et al.’s (2000) original meta - analysis and adopted similar methods. Of note, Mausbach et al. ’s (2010) meta - analysis had almost twice the total sample of patients included in their meta- analysis with half the total number of studies. This is in part likely to be due to Mausbach et al.’s (2010) inclusion of some 53 very large studies that might produce effect sizes that are likely to be considered outliers when isolating the effects of homework use. For example, Mausbach grouped Burns and Spangler’s (2000) study into their analysis which accounted for 521 patients in their meta- analysis. This uncontrolled study included analysis of two separate groups completing the same treatment at different times. The researchers compared homework compliance assessed once at week 12 retrospectively with outcomes in various areas of symptom reduction (i.e., depressive symptoms) for patients randomly allocated to medication or no- medication groups. In fact, Burns and Spangler’s (2000) concluded that the size of the causal effect tended to be large despite producing relatively small effect sizes. After synthesising the r esults of three meta- analysis conducted, homework use is demonstrated to positively impact CBT in relation to positive therapy outcomes. Homework and Therapy Process in CBT for Depression In relation to the present thesis sample, homework has been sugge sted to be particularly important in the treatment of depression and prevention of patient relapse into depressive episodes (Thase & Callan, 2006). Consistent with this review, a clear positive relationship between the use of homework and positive therapeu tic outcomes has been established (Tomkins, 2004). It follows that increasing homework compliance in CBT for depression is likely to promote treatment benefit. Subsequently, the process by which homework is integrated into therapy or administered to a patient has been a focus in psychotherapy process research. Research into the utility of homework implies that to ensure best practice and to provide a thorough assessment of a patient, a therapist can formally test hypotheses in a systematized manner (Bruch, 1998 ) , ‘manipulating’ relevant variable s and gathering 54 data both from in vivo , or in session, experimentation t hat can be observed first- hand, and through the use of homework. For example, a patient may experience a cognitive shift in response to social reinforcement by undertaking a homework task involving behaviour paradoxical to how they would usually act in a situation selected for completion of a behavioural experiment. The therapist works to facilitate the undertaking and skill acquisition that occurs in the patient as a result of anticipating and monitoring situations (Kelly, 1955). As introduced previously, homework assignments are espoused as a means to generalise and put into practice what is discussed in therapy by experimentation outside of the t herapy room. This provides an opportunity to find evidence to support or to disprove hypotheses about the triggers and maintenance of automatic thoughts, emotions, behaviours, and related underlying assumptions in a range of situations in a patient’s every day life (Kazantzis et al., 2000). However, in a review of 16 empirical studies investigating the process by which therapists integrate homework into therapy it was identified that although there are a number of proposed purposes for homework in psychothe rapy, that no research exists that surveys therapist’s viewpoints in order to answer this question empirically (Sheel, Hanson, & Razzhavaikina, 2004) . While there is no empirical indication that clinicians and researchers can agree on the purpose of homework tasks within the therapeutic process research has identified elements within the process of homework integration that are related to homework compliance and achieving patient’s goals related to homework in CBT for depression. The degree of specificity in setting homework tasks has been propositioned to be an important mechanism of change and key indicator of homework utility. Homework utility refers to the extent to which homework adherence and competence is associated with symptom improvement over the course 55 of psychotherapy (Yovel & Safren, 2006). Detweiler - Bedell and Whisman (2005) used data from S. D . Hollon et al.’s (1992) Cognitive Pharmacotherapy Treatment Project to rate aspects of discussions between therapists and patients ( N = 24) diagnosed with depression during a course of CBT. Independent observers ( N = 4) rated taped therapy sessions on the extent to which patients were involved in homework discussion, the specificity of the homework task or goal, discussion of barriers or obstacles to homework completion, and patient compliance. Although reliability of ratings was modest with a reported intraclass correlation coefficient of .68 (range from .44 to .78) across categories rated, a key finding was that the specificity of the homework task appar ent in discussion between a therapist and patient was associated with decreased depressive symptomology. A secondary finding suggested that patients displaying disengagement in discussion around homework tasks were likely to experience treatment benefits f rom discussion of barriers or obstacles to homework completion. This finding is consistent with recently published case studies emphasising the importance of conceptualizing patient barriers or obstacles surrounding non- adherence to homework to gain insight into a patient’s difficulties and facilitate completion of subsequent homework tasks ( Garland & Scott, 2007; Kazantzis & Shinkfield, 2007 ; Rector, 2007; Thase & Callan, 2006). In this way, recent research on processes included in homework integration as a mechanism of change in CBT has began facilitate the potential for the systematic operationalization of homework use in CBT to afford optimal treatment outcomes. 4.5 Standard Manualized Treatment Protocols in CBT for Depression Processes and relationships involved in homework use are just some of many possible mechanisms of change within psychotherapy. With a propensity towards 56 enormous variation in how a particular form of therapy in delivered, increasingly manual- based treatment protocols have been proposed as a means to standardize treatment delivery and methodology. Such manuals might instruct on how treatment variables, techniques or strategies should be integrated into therapy (e.g., such as CC processes and the use of homework) or alternatively be used to fix particular variables that might fluctuate in therapy to manipulate variables otherwise fixed in the treatment manual or to compare with alternatives to the treatment manual. The use of manual- based treatment protocols has facilitated the assessment of treatment integrity and enabled the establishment of empirically supported treatments (EST; Chambless & Hollon, 1998; Wi lson, 1996). Although manual - based treatment protocols are standardized and provide structure to therapy sessions, a certain deg ree of therapist flexibility in implementing treatment is emphasized in contemporary literature (Beutler et al., 2006; Connolly Gibbons et al., 2003; Mumma, 1998; Persons, 2006; Sanderson, 2006). Such emphasis may serve as a reminder to therapists about th e importance of flexibility where, for example, in the practice of CBT, flexibility is crucial in order for the therapist to remain consistent with fundamental theory (A . T. Beck, 1979). Similarly, t he National Institute for Health and Clinical E xcellence (NICE ; 2009 ) present guidelines for the treatment of depression and are widely considered to be among the leading independent organizations regarding best practice in the treatment of depression. NICE guidelines recommend “person centered” care and advocate a flexible approach to the treatment of depression: Treatment and care should take into account patients’ needs and preferences. People with depression should have the opportunity to make informed decisions about their care and treatment, in partnershi p with their 57 practitioners...Good communication between practitioners and patients is essential. It should be supported by evidence - based written information tailored to the patient’s needs. (p.7 ) . While the integrity of protocols as a whole has received empirical support ( A. T. Beck et al., 1979) , there has been a shift towards investigation of individual components of protocols as exemplified by contemporary literature examining the use of homework in psychotherapy (Addis & Jacobson, 2000). It has been argued that standard manual- based approaches need to be adapted to the individual and in practice are somewhat tailored to the individual. For example, in the case of idiosyncratic CCs formulated to help inform individualized treatment plans utilized to mee t the needs and specific goals of the individual patient (Bond, 1998 ; Persons, 2006; Rogers, Reinecke & Curry, 2005). This is seen, for example, in the therapist’s choice of techniques or homework tasks and treatment interventions for individual patients ( A. T. Beck et al., 1979; Gibbons, Crits - Christoph, Levinson, & Barber, 2003 ; Nezu, Nezu, & Lombardo, 2004 ). Optimizing the process by which a CC is constructed has been suggested as a means to further systematize or structure the process of making ‘clinica l judgments’ more accurate (Persons, 1989). In contrast, a less structured approach even within a manual- based treatment protocol may result in ‘gaps’ or shortcomings in the testing of hypotheses about a patient’s presentations . For, example, a therapist w ho tests hypotheses by intuition, and does not keep a record of the data that has and has not been gathered might have a tendency to explore emotion but neglect behaviours that maintain the specified emotion. Alternatively, a therapist might have concluded that a hypothesis about a patient has been confirmed without 58 systematic reference to situations that challenge the hypothesis having not recorded previous important situations for future reference. It follows that while standard manualized protocol for p sychotherapy provide guidelines for practice in keeping with the particular manual, the practical performance of the therapist is not measured. Thus, standard manualized treatment protocols provide a means of reducing the variance or controlling therapist variables that might account for key therapy outcomes (e.g., reduction in depressive symptoms) but are not independently sufficient to ensure treatment fidelity. Moreover , therapist competence in different areas related to the treatment protocol (i.e., CC and homework use) is also likely to influence key therapy outcomes. In sum, the use of a manual- based standardized CBT protocol adapted to incorporate a standardized approach to a therapist’s homework adherence and competence, combined with a structured format for therapist’s use of CC with CBT Homework Project facilitates the investigation of the relationship of CC and homework processes in CBT for depression (See Chapter VII). 4.6 Summary This chapter has provided a brief introduction to the empirically supported theory and practice of CBT for depression. Homework has been identified as a central feature in facilitating mechanisms of change which has been demonstrated in recent burgeoning research to influence the process and outcome of CBT. Lastly, a discussion of the utility of standard treatment protocols such as A. T Beck’s treatment manual for the CBT of depression (A.T. Beck et al., 1979) serves to further reinforce the need to research in areas of therapist competence that serve to guide the use of standard manualized treatment protocols. 59 The present thesis aims largely to investigate the process by which therapists gather, record, and utilize conceptual data pertinent to patient presentations to provide clarity and support for existing theory. It must also be recognized that the focus of the present thesis was not to investigate specific homework tasks as they might relate to CC or therapeutic benefit. Instead, the competence of the therapist in using any number of individualized homework tasks was investigated, with the recognition also that some tasks in particular were likely to be used routinely with all patients consistent with a standard approach to CBT for depression (e.g., thought records and monitoring of thoughts and moods, or specific beh avioural experiments). More specifically, the present thesis aims to design a new measure of therapist competence of CC in CBT for depression. It is intended that this new measure will facilitate reaching additional aims to investigate how therapist CC change over the course of therapy and investigate the relationship between therapist CC, use of homework, and outcome. The next chapter provides a discussion of therapist competence in the context of CBT for depression and in relation to CC and broader therap y outcomes. The rationale for a new measure of therapist competence specifically assessing CC is expanded further. 60 61 CHAPTER V Therapist Competence in CBT for Depression 5.1 Determinants of Psychotherapy Outcome When c onsidering what factors contribute to psychotherapy outcome researchers have commonly examined the variables related to the therapy in question (e.g., the specific therapy is being employed, what techniques are being used) to the patient (e.g., the age, ge nder or diagnoses), or to the therapeutic relationship between patient and therapist. More recently research has specifically examined therapist factors ( Brosan, Reynolds, & Moore, 2006; Eells, 1999). Therapist factors are defined as therapist characteristics, training or abilities that promote positive change in psychotherapy. The study of therapist factors is based on the proposition that the therapist does in fact account for effects in psychotherapy (Nezu & Nezu, 2005). It must be recognised, however, t hat within any psychotherapeutic investigation therapist factors cannot be explored in isolation from patient and factors related to a specific psychotherapeutic modality. In a sample of N = 5000 patients and N = 71 therapists Okiishi et al. (2006) built on previous large studies (Brown et al., 2004; Okiishi et al., 2003) to determine the role of the therapist factors on treatment effect. The Outcome Questionnaire - 45 (OQ - 45) is a 45 - item self report measure measured on a 5 - point Likert scale ( ‘ 1 ’ = never, ‘ 2 ’ = rarely, ‘ 3 ’ = sometimes, ‘ 4 ’ = frequently, ‘ 5 ’ = almost always). The OQ - 45 was used as the primary outcome measure originally developed to assess patient outcome on three broad dimensions symptom distress, interpersonal relationships, and social role or occupational functioning. Therapists were rank - ordered based on their level and speed of therapeutic outcome as measured on OQ - 45. Results suggested 62 dramatic differences in the relative effectiveness of individual therapists where the “top ranked” the rapists were demonstrated to be effective in delivering a consistently high level of symptom relief in a short number of sessions compared with “bottom ranked therapists” were observed to produce less symptom relief in a longer number of sessions while having twice as many patients deteriorate than “top ranked” therapists. However, researchers did not find a relationship between therapist level of training, type of training, gender or theoretical orientation. In sum, while Okiishi et al.’s (2006) study prov ides convincing evidence that therapist factors are important in producing differential treatment effects, it does not identify any particular therapist factor that explains this relationshi p. Although therapist characteristics relevant to how a therapist might perform in therapy sessions were investigated (i.e., level, type and modality of training) the manifestation of potential therapeutic ability, skills or competence was not investigated and has received little attention in literature until recently. The present thesis hopes to address hopes to extend this research using similar methods with a focus on therapist competence as a key predictor variable of different outcomes. 5.2 Therapist Adherence and Competence The variation in the levels of reliability and validity of therapist’s cognitive CCs discussed in Chapter II have placed emphasis on therapist factors, in particular the role of therapist competence in delivering treatment ( Beutler, Castonguay, & Follette, 2006; Shaw & Dobson, 1988; Shaw, et al. , 1999; Startup, Jackson, & Pearce, 2002; Trepka et al., 2004). While the final products of therapy, for example a therapist’s written CC or a well designed homework task are likely to impact the 63 outcome of therapy, so too is the competence of the therapis t in delivering these therapy components and eliciting information in- session and between sessions. While literature has provided some inconsistencies surround the definition and distinguishing features of what constitutes adherence and competence, more r ecent literature has rectified these inconsistencies. Consistent with cognitive- behavioural theory it is suggested that a competent therapist is able to: 1) display adherence to a theoretical or conceptual framework in order to guide therapeutic interventi ons, 2) engage the client in a constructive therapeutic relationship, 3) skilfully implement interventions intended by the respective treatment manual, as well as 4) applying knowledge of when and when not to apply treatment interventions ( Kazantzis, 2003) . While a competent therapist should be adherent, adherence is described specifically as the occurrence or non- occurrence, frequency or consistency of therapist behaviour. Competence refers to therapist level of skills, comprehensiveness, appropriateness and timing of behaviour (Nezu & Nezu, 2007). Given that a competent therapist should be adherent, but with recognition of subtle differences in the precise definitions the term, ‘therapist competence’ will henceforth be used to refer to adherence and competence if not otherwise specified. 5.3 Therapist Competence for the Practice of Psychology in New Zealand The New Zealand Psychologists Board is responsible for setting standards of clinical and cultural competence as well as ethical conduct in New Zealand under Section 118 (i) of the Health Practitioners Competence Assurance Act (2003). The New Zealand Psychologists Board adopted new guidelines for Core Competencies for the practice of psychology in New Zealand on 13 April 2006. These Core Competencies are intended for use alongside the Code of Ethics for Psychologists 64 Working in Ao tearoa / New Zealand ( New Zealand Psychological Society, 2002). Continued development and maintenance of clinical and cultural competence is suggested to include observation, individualised developmental appropriateness, ongoing monitoring, include the use of valid methodology, be flexible with respect to practice individual therapist’s psychotherapeutic orientation, include repeated measures over time and to ensure the assessment modality should be appropriate to the training modality and content. However, for the purposes of the present thesis it is necessary to distinguish between therapeutic competence and professional competence (Geller, Norcross, & Orlinsky, 2005; Milne et al ., 2001) . The Core Competencies for the practice of psychology in New Zealand address both professional and therapeutic competence. Professional competence is the ability of the therapist to demonstrate a breath of knowledge applicable to a range of settin gs or patient presentations in the practice of psychotherapy. Professional competence, for example, incorporates ongoing professional development, personal style, values, philosophy, individual reflection or engagement in personal psychotherapy, and the learning of new psychotherapeutic modalities. However, in this review of literature the term ‘competence’ referrers to therapeutic competence as the main focus of this thesis. Broadly speaking, t herapeutic competence is the ability of the therapist to demonstrate skilful application their psychotherapeutic model of choice within therapy consistent with the definition of therapist competence provided thus far in this review of literature. In reference to therapeutic competence, t he guidelines for Core Competencies for the practice of psychology in New Zealand make specific reference to competence as it relates to CC for practice as a clinical psychologist stating “assessment and formulation [CC] are fundamental for understanding a client’s presentation, current 65 needs and devising appropriate interventions. Assessment is also an ongoing process which may lead to revised formulation [CC] and / or changes to the intervention” (2006, p.23) . However, the guidelines do not specify how therapist competence should be measured in New Zealand. Moreover, few measures exist to assess therapist competence in the context of CBT and few still incorporate the specific measurement on therapist competence in constructing and reconstructing CCs as specified in the Core Competency guidelines. 5.4 Measuring Therapist Competence in CBT In order to assess therapist competence in CBT researchers have focused on creating observational, supervisor - rated measures of therapist competence. Observational measures have been noted to be superi or to therapist self- report in consideration of findings suggesting that therapists over estimate their own levels of competence compared to expert raters (Brosan et al., 2006 ). It follows that external measures must be used by independent raters in order to accurately measure and monitor therapist competence to provide feedback to the therapist. A number of key measures have been developed to measure therapist adherence and competence in CBT. Most notably the CTS , (Young & Beck, 1980) an 11 - item scale designed to assess a therapist’s overall performance in administering CBT. Although Young and Bec k (1980) describe CC as a crucial determinant of a cognitive therapist’s effectiveness, only one component of a single item specifically focuses on the measurement of therapist CC of a patient’s presentation to determine how skill in CC may relate to the administration and use of homework. Other items on the CTS provide vague descriptions of aspects of therapy may provide approximations of the degree of therapist co mpetency relevant to formulating a 66 comprehensive CC (e.g. item - 3: Understanding). Such criticism of the CTS is reflected in its psychometric properties and the limits of use in supervision and research (Vallis, Shaw, & Dobson, 1986; Whisman, 1993). In this way the CTS provides a general indication of overall therapist competence to CBT as a modality, however, it lacks the specificity to provide detailed feedback on therapist competence in particular domains and components of CBT. For example, ‘h omework ’ features as item- 11 but gives no reference to the reciprocal utility of homework, or measurement there- of, for informing, and being informed by the CC. A number of revisions have been suggested to the CTS. Most notably the CTS - R (Blackburn et al., 2001) expa nds upon the CTS mainly by adding items intended to capture more aspects of the therapeutic relationship such as therapist charisma and flair and the facilitation of emotional expression. However, the CTS - R does not distinguish between adherence and competence. Here a common theoretical confound is reflected in research that fails to distinguish between or adequately define therapist competence ( Kazantzis et al., 2003). Another revision of the CTS, The Cognitive Therapy Adherence and Competence Scale ( CTACS; Barber, Liese, & Abrams, 2003 ) is a 21 - item scale that is suggested to provide a wider scope of therapist activity than the CTS. The CTACS distinguishes between adherence and competence, where the CTS does not, and produced relatively strong psychometric properties for both adherence and competence scales. Existing research emphasizes the need for conceptual clarity around an operational definition of therapist competence. Also, research further highlights gaps in the literature and practitioners understanding and competence in homework, CC, 67 and other fundamental process variables, specific domains or components within CBT are concerned. More recently, the importance of the use of homework as a core process in CBT and many other empirically supported th erapies (Kazantzis & Ronan, 2006) has lead to the development of a number of measures specifically designed to gauge therapist homework adhere nce and competence. In particular, the Homework Adherence and Competence Scale (HAACS; Kazantzis, Wedge, & Dobson, 2005) , is a 19 - item scale designed assess the therapist’s integration of patient between session tasks into therapy looking specifically at the therapist’s competence in ‘review’, ‘design’ and ‘assignment’ of homework. The HAACS distinguishes between adherence and competence and provides detailed descriptive anchors used to gauge varying levels of therapist competence in the use of homework generated from contemporary research. The HAACS format also provides a measurement framework, which has built on that adopted by process measures such as the CTS, suitable for adaptation and use in the construction of new psychometric process measures. Preliminary investigation of the HAACS suggests strong psychometric properties (see Chapter VI) . As discussed, there is a dearth of support for the reliability and validity of cognitive CC. However, with increased interest in the import ance of ensuring that therapists can reliably and validly formulate patients’ cases some researchers have developed scales for this purpose. Most relevant to this project is the Quality of Cognitive Case Conceptualization Rating Scale (Fothergill & Kuyken, 2002). The researchers drew from an existing measure, Tracey Eells’ Content Coding Method (Eells, Kendjelic, & Lucas, 1998) with the speci fic aim to rate the quality of cognitive CCs. Although Kuyken, Fothergill, Musa and Chadwick (2005) in their subsequent 68 study found some success in providing evidence for the importance of quality therapist CCs, no further evidence was provided to answer t he question ‘can therapists construct reliable cognitive CCs?’ (see Chapter II I ) . It is possible that the Quality of Cognitive Case Conceptualization Rating Scale (Fothergill & Kuyken, 2002) in its current form lacks the specificity to demonstrate the impo rtance of cognitive CCs in relation to outcome and to yield high rates of agreement between independent observers using the scale. 5.5 Levels of Therapist Competence in CBT Different levels or units of analysis have been suggested in literature on therapist competence in CBT. The first and broadest level of therapist competence in CBT refers general therapist competence in the CBT treatment manual being delivered. The second level of therapist competence in CBT refers to specific areas of therapist competence (e.g ., CC, homework). These two levels of therapist competence have been described as “macro” and “micro” (Manring, Beitman, & Dewan, 2008) or “global domain ” and “limited domain” (Barber et al., 2007) therapist competence. Research examining process and mechanisms of change in CBT has focused on general therapist competence as opposed to specific areas of therapist competence that may impact psychotherapy outcomes. It is conceivable that competence can be broken down further and that different aspects of therapist competence can be investigated in more or less detail. For ex ample, Mansell (2008) identifies individual 51 competencies measured on the CTS derived of between 8 and 13 elements per category. However it is emphasised that competence on the CTS should be interpreted as a global domain or macro level total rather than differentiating between competencies. This is suggested to be due to a 69 lack of research, underpinned by a lack of specificity in the items on the CTS to detect variations in competence at the limited domain or micro level. These limitations in measures designed to measure general competence provide the basis for the rationale for the creation of new measures designed to measure specific areas of therapist competence in CBT. 5.6 The Shape of Therapist Competence No consensus has been reached regarding the stability of therapist competence (Kazantzis, 2003). Disagreement has arisen surrounding whether therapist competence is best described as a “state” or a “trait” (Shaw & Dobson, 1988). For example, it is possible that t herapist competence can be demonstrated to varying degrees across different patient presentations, a range of settings or across session within a single patient (Milne et al., 2001). Conversely, i t has been suggested that therapists generally are able to demonstrate a consistency in their competence across therapy sessions ( Svartberg, 1999). This has implications for the frequency and timing of measurement of therapist competence in research, training and continued professional development. For instance, if a therapist is able to demonstrate competence consistently across therapy sessions, then fewer measurements of competence are necessary to gage a therapist true level of competence. This thesis gives special attention to what is dubbed the ‘ shape’ of therapist competence in examining the evolution of therapist competence across a course of therapy . 5.7 Outcome and Therapist Competence in CBT Although it is commonly assumed that a therapist’s competence in deliver ing treatment will result in therapeutic benefit for a patient, it has only been recentl y that 70 this notion has been the focus of empirical investigation (Davidson et al., 2004; Shaw, et al., 1999; Trepka et al, 2004). Trepka et al. (2004) demonstrated in a sample of 30 depressed patients that competence, as measured on the Cognitive Therapy Scale (CTS: Young & Beck, 1980) , was significantly related to therapeutic outcome, albeit weakened when controlling for the presence of a positive therapeutic alliance. Despite the “modest power” reported in Trepka et al.’s (2004) study, the results suggest that therapist competence is a good predictor of therapeutic change particularly in the context of a strong therapeutic relationship. Furthermore, for those that compl eted a full course of therapy ( N = 21) the results supported the idea that the impact of outcome is attributable mainly to therapist factors as opposed to client factors. Barber et al. (2007) provided a review of the primary findings from therapist competence measures focusing on in- session psychotherapy competence. The researchers reported that internal consistency for different measures tended to be high. This suggests that if therapists are competent in one area of competence they are likely to be compe tent in another. Further to this, it would be expected there would be a positive relationship between therapist competence in homework and CC. The results of Barber et al.’s (2007) review found, however, that interrater reliability was low across different measures of therapist competence. The authors also reported that the results of different studies reflected mixed results for the relationship between competence and outcomes. Results ranged from positive and weak to negative relationships between therapist competence and outcomes. The authors suggest that due to a possible small effect size of therapist competence, that large samples may be necessary to detect these small effects. Furthermore, the authors suggest that investigation of cumulative “limited domain” therapist competencies may strengthen the relationship of “global domain” or overall therapist competence and outcomes. In 71 light of these results the current project investigates the multiple limited domain therapist competencies (i.e. as measured by the CTS, CRS, and HAACS) in relation to outcomes (i.e. depressive symptoms) in CBT for depression. In a recent naturalistic study of 69 patients treated by 18 therapists in an outpatient clinic, a relationship was found between patient and expert ratin gs of therapist competence measured in a number of different domains. Therapist competence was observed to account for 15% of the variation in patient depression severity reduction (Kuyken & Tsivrikos, 2008). However, it has been suggested that the improved operationalization and assessment of therapist competence both in global and domain- specific levels of therapist competence are necessary to provide a full understanding of the impact of therapist competence on therapeutic outcomes ( Barber et al. , 2007 ) . In sum, in the current thesis it is expected that there will be a relationship between therapist competence in CC and outcomes expressed primarily by a decrease in depressive symptomology in CBT for depression. 5.8 Therapist Competence in CC and Homework Use No studies have specifically investigated the relationship between adherence and competence in homework and CC as it pertains to outcome. Kuyken et al., (2001) in a study of 162 depressed outpatients concluded that complex presentations, specifically the presence of personality disorder or avoidant and paranoid beliefs were related to a poorer outcome after a course of CB T as indicated by scores on the BDI - I I and the GAF. Although homework was not a focus of the study, the researchers observed that interference with homework tasks was potentially a key barrier to treatment response. The researchers also implemented a measure of therapist competence, the results of which suggested that well - trained, highly supervised and 72 competent cognitive therapists had better treatment outcomes than less competent therapists. However, Kuyken et al. (2001) reported that there was not enough variance in their pilot measure of therapist competence to come to a definitive conclusion. In a study of 26 patients undergoing a 20 - session protocol of CB T for major depression Bryant, Simons, and Thase (1999) investigated therapist competence and patient variables specifically in the context of homework. The researchers found that the number of previous depressive episodes reported by a patient was related to the degree of homework compliance on the Assignment Compliance Rating Scale (ACRS ; Primakoff, Epstein, & Covi, 1986). These findings support Kuyken et al.’s (2001) proposition that the complexity of client presentation affec ts homework compliance in turn influencing treatment outcome highlighting an interplay between therapist and client factors. Moreover, the study by Bryant, Simons, and Thase (1999) emphasises the need for therapists to display high levels of competence in integrating homework into treatment for complex patients as well providing support for the necessity of controlling for variations in therapist competence in assigning, designing and reviewing homework as a core change process in therapy (Kazantzis, Deane, Ronan & L’Abate, 1993). The results discussed again call attention to the requirement that psychometrically sound measures are utilized in conducting research on the process and mechanisms of change in CBT. 5.9 Summary Currently there is no existent empirically validated measure for directly assessing therapist competence in CC in CBT . Research discussed on therapist competence and the preliminary results indicate that therapist factors and in particular therapist competence, is important to identify mechanisms of change to achieve a 73 positive therapeutic outcome. In light of this discussion there is a need to develop standardized methods for measuring aspects of therapist competence to give feedback and improve the quality of treatment delivery ( Barbe r et al., 2007 ; Kuyken et al., 2001; Milne et al., 2001; Shaw & Dobson, 1988; Whisman, 1993). Furthermore, t he importance of monitoring adherence and competence to the manualized protocol used in research is emphasised as a requirement for methodologically sound research. In this way a ‘research paradox’ is formed where current research is unable to inform practice until appropriate research methods are utilized while it is still unclear the extent of which domain - specific competencies used within the process CBT impact various treatment outcomes or objectives. 5.10 Research Objectives for the Current Project In summary, the introductory chapters of the current project have sought to provide a discussion of therapist competence in the use of homework an d CC in the context of CBT for depression, and to provide a rationale for the development of a new measure of therapist competence in CC. While the use of homework has increasingly found empirical support in literature, the area of cognitive CC has not been widely researched. In general , there has been a paucity of research into therapeutic process and the mechanisms of change in psychotherapy making a new measure of therapist competence to facilitate research essential ( Barber et al., 2007; Milne et al., 2001 ). As discussed, theory and research have suggested a definitive need for a measure to facilitate exploration of t he relationship between homework or between session tasks and the use of cognitive CC (for an overview of methodological suggestions and guidelines see Perepletchikova & Kazdin, 2005). Thus , the current project aims to investigate therapist competence in constructing and utilizing 74 cognitive CCs and provide preliminary psychometric evidence in support of reliability and validity in the use of cognitive CC. Specifically the current projects seeks to further address the following research questions: 1. How do therapist ’s case conceptualization s evolve over the course of therapy? 2. Can cognitive therapists formulate reliable case conceptualization s? 3. Ho w are therapist’ s written case conceptualization s related to therapist competence in formulating case conceptualizations in - session? 4. Is there a relationship between therapist competence in case conceptualization, homework, and therapeutic outcome? 5.11 Specific Hypotheses Investigated With respect to the above research questions the followi ng hypotheses have been developed: Hypothesis One Hypothesis One states that CCs will become more complete over a course of CBT. This would be demonstrated by a greater quantity of information in overall written CCs as well as increased quality of CCs reflected in a greater emphasis on deeper- level CC components. Similarly, it is expected that more discussion will be generated about CC of the course of therapy and that greater discussion of deeper level CC will be had as therapy progresses reflecting a more complete CC. 75 Hypothesis Two Hypothesis Two states that there will be a high degree of coherence within different domains of therapist CC. This would be demonstrated by high levels of internal consistency across individual items of CC as measured on the CRS (e.g., patient behaviour, thought, and emotion) as well as agreement on demonstrations of therapist competence observed by independent raters of in- session discussions of CC between the therapist and the patients and the match between in- session content discussed and the content of written CCs. Such agreement relative to different domains (i.e., integration, importance, competence and fit / match) would also be demonstrated by high levels of reliability being observed for specific discrete sessions of therapy (e.g., sessions 1, 7 and 9 would be observed to have independently high reliability scores for all measurements taken at the particular session). Hypothesis Three Hypothesis Three states that there will be conco rdance between therapist written CC and in session use of CC. This would be demonstrated by high levels of “fit” of match between what therapists write in their CCs and what information they have generated during in- session discussions. This would reflect agreement between therapists and observers regarding the importance of information included in a written CC. This hypothesis is based on the assumption that therapists have attempted to record information in their written CCs that they think best represents discussions of CC undertaken with therapy sessions. In this way independent observers are able to independently make judgements to gauge the extent to which they agree or disagree that the written CC is a reflection of in- session CC discussion. 76 Hypothesis Four Hypothesis Four states that there will be a high concurrent validity between therapist case conceptualization s and patient personality beliefs. This is to say that for patients reporting greater beliefs associated with disordered personality traits CC would hold particular value for understanding patients with complex presentations often associated with the presence of personality disorders. However, preliminary research bearing on the relationship between therapist competence and the presence of personality beliefs has produced mixed results concerning the direction of the relationship between the presence or absence of personality beliefs and variations in levels of therapist competence. For e xample, although it may be important for a therapist to display competence in CC it may be that therapists display less competence as a result of a more challenging presentation associated with patients exhibiting personality disorders. In contrast, a ther apist may display more competence as a result of ongoing and increasing discussion of CC as the therapeutic relationship develops and the therapist is better able to explore CC at a deeper level. In each case, personality beliefs are expected to explain va riance related to a reduction in depressive symptoms necessary to provide a control for any additional variance explained by therapist competence in CC. Hypothesis Five Hypothesis Five states that the inclusion of beliefs about homework completion (or non - completion) in therapist case conceptualiza tions will be related to better patient outcomes. This would be demonstrated by an association between positive beliefs related to homework completion or non - completion and a reduction in patient depressive symptoms. Moreover, in order to fully assess the impact of 77 therapist competence in homework integration, it is hypothesised that patient beliefs about homework would be likely to explain variance associated with a reduction in depressive symptoms in order to provide a control for therapist competence in homework integration. For example, strong negative patient beliefs regarding homework would be likely to impact the extent to which therapist competence in homework is demonstrated to be effective. Hypothesis S ix Hypothesis Six states that therapist competence in CC will demonstrate particular utility with more complex patient presentations. Prior research in CBT for depression has shown that therapist competence is associated with improved outcomes regardless of co- morbidity, but that co - morbidity may moderate the effects of therapist competence on treatment outcomes ( Kuyken & Tsivrikos, 2009). Accordingly, it is expected that the level of patient symptom severity may account for a reduction in patient depressive symptoms. This reduction will be above that of measures of therapist competence in CC and must be controlled for to best determine the impact of therapist competence in CC on depressive symptomology. Hypothesis Seven Hy pothesis Seven states that therapist competence in CC will be associated with therapist competence in using homework. It remains an empirical question to be answered as to whether different domains of therapist competence are consistent across different domains of therapist competence or if in fact therapist competence is variable across different domains of therapist competence ( Barber et al, 2007). In order to investigate this hypothesis, the relative levels of therapist competence across 78 these two domains will provide a means to determine how therapist competence in CC and therapist competence in homework integration change relative to one another over the course of therapy. H ypothesis Eight Hypothesis Eight states that higher levels of therapist competence in CC will be related to greater patient outcomes. In the context of CBT for depression this would be demonstrated by an association between increased therapist competence in CC and a reduction in patient depressive symptoms. It would be expected that therapist competence in CC would explain a significant portion of variance related to a reduction in depressive symptoms after controlling for other variables that are likely to impact the relationship between therapist competence in CC and depressive symptom change. Hypothesis Nine Hypo thesis Nine states that taken together, therapist competence in integrating homework and therapist competence in CC will be related to greater patient outcomes. This would be demonstrated by therapist competence in CC and therapist competence in homework i ntegration each making a unique contribution to the explanation of depressive symptom change as the primary outcome variable of interest in CBT for depression, and a fundamental indicator of patient improvement monitored over the course of therapy. 79 CHAPTER VI The CBT Homework Project 6.1 Overview The CBT Homework Project The present research is set within the context of the CBT Homework Project or Depression Study. The CBT Homework Project was set up as a study on CBT to examine the homework as a key pr ocess in psychotherapy (Kazantzis et al., 2005) . The study protocol aimed to operationaliz e what was already implicit in Beckian CBT (A. T. Beck et al., 1979) , considered to be the standard CBT, by ensuring that therapists adhere to a structured and systematic use of homework assignments over the course of therapy. In order to enable the investigation of the relationship between homework and CC, therapist’s use of CC within the study protocol was integrated systematically into the study protocol (see Study 1 ). This chapter gives an overview of the CBT Homework Project sufficient to understand and replicate the current project. For further information about the CBT Homework Project refer to Kazantzis et al., (2005) . 80 6.2 Method Patients In total 28 patients received treatment in the CBT Homework Project. The sample consisted of 10 male and 18 female patients of average age 44.75 years (range from 20 to 62 years old). Twenty - four out of 28 (85.7% ) of the participants identified their ethnicity as Euro pean / Caucasian. The remaining four participants identified as New Zealand Indian, American, Scottish, and Australian. Each patient completed a full course of CBT up to 20 sessions (average of 17.79 sessions) with an additional two booster sessions at two months and six months following therapy during the CBT Homework Project. Four of the 28 patients completed both two and six month booster sessions. Only 11 of the 28 patients only completed two month booster sessions. In total, 16 of the patients did not complete booster sessions. Table 3 provides a summary of demographics for the total sample of patient data used in the present thesis. Therapists Seven female therapists of average age 36.86 years (range from 23 to 50 years) provided the course of CBT; al l were intern psychologists in their second to last or final year of training towards a DClinPsych. All of the therapists identified as New Zealand European / Caucasian. The therapist demographic questionnaire is attached in Appendix D. 81 Table 3 Total sample: Participant demographics, diagnosis, severity of depression and number of therapy sessions Variable N % M SD Gender Male 10 36% Female 18 64% Age (years) 44.75 11.51 Depression Severity* Mild 5 18% Moderate 1 0 36% Severe 8 28% No diagnosis 5 18% Number of therapy sessions 17.79 3.93 Number of follow up sessions Two month 11 39% Six month 4 14% * Note. Diagnoses and depression severity reported as measured on the CIDI. Following intake assessments all patients included in the CBT Homework Project received a primary diagnosis of MDD following further assessment. Patients ( N = 28) were initially assessed on the CIDI and some met additional diagnostic criteria for generalised anxiety disorder (57% ; n = 16), social phobia (36% ; n = 10), PTSD (21% ; n = 6), panic disorder (14% ; n = 4) , specific phobia (14% ; n = 4), OCD (10% ; n = 3) , brief psychotic disorder (10% ; n = 3) , alcohol dependence (7%; n = 2), alcohol abuse, (4% ; n = 1), agoraph obia (4%; n = 1), bulimia nervosa (4%; n = 1), conversion disorder (4% ; n = 1), hypochondriasis (4% ; n = 1) , pain disorder (4% ; n = 1) , nicotine dependence (4%; n = 1) , nicotine withdrawal (4%; n = 1) although were later assessed to meet a primary diagnosis of MDD without meeting any exclusionary criteria after further assessment. 82 Therapist Training Each therapist had undertaken two advanced courses in the theory and practice of CBT for depression entitled “Theory and Practice of Cognitive Behaviour Therapy” and “Cognitive Behaviour Therapy of Depression” at Massey University. Each course was a 5 - day intensive postgraduate applied training course involving role play demonstrations, role - plays and supervision in techniques, and other active learning activities. Therapists were also required attend a workshop on “Homework Assignments in Cognitive Behaviour Therapy”. These applied training workshops involved role- play demonstrations of the study protocol, therapist role - play of therapy sessions, and individu al feedback on a therapist protocol adherence scale. Training was delivered by Dr. Nikolaos Kazantzis, an experienced registered clinical psychologist and senior lecturer at Massey University (presently La Trobe University, Melbourne) and also primary investigator of the CBT Homework Project. Therapists were required to demonstrate competence in delivering Cognitive Behaviour Therapy for depression. Therapists recorded a role - play session for evaluated by an independent clinical supervisor Robyn Vertongen, a registered clinical psychologist with experience in clinical supervision in the practice of CBT, as a prerequisite for engaging in CBT with patients. The supervisor assessed whether the therapist is practicing therapy in a manner that reflected competent and safe practice. Role - plays were required to achieve a basic standard for competent delivery of CBT as defined by ratings on the “Cognitive Therapy Scale” (CTS: Young & Beck, 1980) , the international standard assessment scale for assessing competence in cognitive therapy (Kazantzis, 2003). Further to this, the therapists also attended weekly supervision sessions with Robyn Vertongen . 83 Treatment Fidelity Regular fidelity checks were carried out over the course of therapy by a senior registered clinical CBT psychologist. R atings were made based on DVD - R recorded therapy sessions for each therapist using the Cognitive Therapy Scale (CTS) to provide an indication of the level of therapist competence in the carrying out the treatment protocol as intended. Therapists were required to exceed the cut - off score of 39 (see Brosan et al., 2006; Sudak et al., 2003) in three CTS assessments or were required to demonstrate increased competence in further CTS assessments until this level of competence was attained. CTS scoring was carried out and reviewed by the therapist’s clinical supervisor, Robyn Vertongen. Protocol adherence and competence checks using the Homework Adherence and Competence Scale ( HAACS) were carried out by five independent raters who were employed research staff with postgraduate level qualifications in psychology who also received substantial training in administering the HAACS provided by the primary investigator . Also, competence pertaining to core skills relating to the integration and use of cas e conceptualization were carried out using the CRS rated by two postgraduate level independent raters. Length of Treatment, Booster Sessions, and Follow - up Data Treatment length modelled a standard 20 session course of CBT (A. T. Beck et al., 1979) plus additional 6 and 12 month follow up sessions. Patients attended therapy twice weekly for the first four weeks of therapy (i.e. the first eight active therapy sessions). The remainder of the therapy sessions were attended on a once weekly basis. For the tot al sample participants ( N = 28) the total number of sessions of CBT received ranged from 6 to 20 sessions. The average number of therapy sessions was 17.82 sessions . 84 6.3 Relevant Measures Intake Assessment Measures Relevant pre - treatment assessment measures included structured clinical interviews to determine the presence or absence of DSM - IV Axis I disorders (Composite International Diagnostic Interview; CIDI), and the Beck Depression Inventory - II (BDI - II) to assess depression severity. Demographic Quest ionnaires Demographic questionnaires were used to gather data on participant’s (patient, therapist, independent rater) characteristics such as age, gender, ethnicity, education or training, employment history, occupation, families and marital / relationsh ip status (see Appendices C and D ) . Cognitive Therapy Scale The Cognitive Therapy Scale (CTS ; Young & Beck, 1980 ) is a 13 - item supervisor- rated scale commonly used in clinical practice and training to assess therapist competence in dimensions pertinent to CBT. The CTS is rated on a scale from ‘ 0 ’ to ‘ 6 ’ with descriptive anchors for each item. The highest possible score is 66 although a generally established cut- off score or “red - line” of 40 is commonly used to indicate competence in CBT (Beck, Sudak, & W right, 2003; Young, Beck, & Budenz, 1983 ). The CTS has demonstrated intraclass correlation coefficients (ICC) ranging from .59 to .91 for the total measure (Dobson, Shaw, & Vallis, 1985 ; Vallis, Shaw, & Dobson, 1986). Research provides limited support for the correlation between the CTS and reduction of depressive symptomology (Shaw et al., 1999; Trepka et al., 2004). 85 Beck Depression Inventory - II The Beck Depression Inventory - II (BDI - II) is 21 - item patient self- report measure widely used to assess depression severity ( A. T. Beck, Rush, Shaw, & Emery, 1979; A. T. Beck, R. A. Steer, & G. K. Brown, 1996 ). Each item is rated on a ‘ 0 ’ to ‘ 3 ’ point scale with possible scores ranging from 0 to 63. The BDI - I I has been subjected to extensive research and is a common ly used measure in the treatment of depression ( Dozois, Dobson, & Ahnberg, 1998; Schotte, Maes, Cluydts, De Doncker, & Cosyns, 1997; Steer, Ball, Ranieri, & Beck, 1999 ). The BDI - I I has demonstrated excellent internal consistency ( α = .92 ) among depressed outpatients (A. T. Beck et al., 1996) . The BDI - I I has demonstrated good validity (i.e., content, concurrent, factorial composition), for example, moderate correlations between other well validated scales used to measure depressive symptomology such as the Hamilton Psychiatric Rating Scale for Depression (.71) and the Beck Hopelessness Scale (.68) have been found (Beck et al., 1996). Composite International Diagnostic Interview The Composite Diagnostic Interview (CIDI) Version 2.1 (WHO; 1997) is a structured computerised assessment tool designed to assist clinicians in providing diagnoses for a range of mental disorders consistent with the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders (DSM - IV). The CIDI was originally developed by the World Health Organisation (WHO) and the United States Alcohol, Drug Abuse, and Mental Health Administration (ADAMHA) now renamed the Substance Abuse and Mental Health Services Administration. Of particu lar relevance to the current study the CIDI is able to be used to generate a preliminary diagnosis and severity of first episode Major Depressive Disorder (MDD). Although this 86 does not constitute a formal diagnosis, the CIDI diagnostic output provides a tentative diagnosis and the information that resulted in this diagnosis for further discussion by a therapist in the intake session and has been designed to be administered by lay people with limited clinical training (R osenman, Levings, & Korten, 1997 ) . The CIDI Version 2.1 was selected for the current study having demonstrated good psychometric properties in a range of studies since the release of Version 2.1 . Wittchen et al. (1994) provided a review of psychometric research for Version 1.1 and across three studies the authors reported a test- retest kappa ( k) of .66 for first episode MDD and a test - retest percent agreement of 86%. Interrater agreement of diagnoses were 100% based on an aggregated sample ( n =575) for diagnoses concordant with the DSM - III - R. In a su bsequent review Andrews and Peters (1998) suggest that the CIDI Version 2.1 has been demonstrated to have adequate validity. Kessler et al., (2003) in a large sample ( N = 9090) door to door survey found a concordance (Cohen’s k = .40) between the CIDI and the structured clinical interview for DSM - IV (SCID) for those diagnosed with experiencing a major depressive episode over the last 12 months. Homework Adherence and Competence Scale The Homework Adherence and Competence Scale (HAACS; Kazantzis, Wedge, & Dobson, 2004) is an observer- rated 19 item measure rated on a 7 - point Likert scale with descriptive anchors related to skills and techniques used by therapists in the use of homework in session. The HAACS is designed to be rated by clinical supervisors and researchers trained in the use of the measure, based on video recordings of live therapy sessions. The HAACS is designed to measure adherence and competence in the review, design and assigning of homework tasks consistent with the treatment manual being used. For the present project the HAACS provides a means of measuring therapist adherence 87 and competence in homework use. The HAACS is intended for use alongside the treatment manual and guiding model for practice (Kazantzis, MacEwan, & Dattilio , 2005) representing an augmented protocol for homework use as intended in for use of CBT in the treatment of depression ( A. T. Beck et al., 1979). Preliminary psychometric testing has refined the HAACS items and suggested good psychometric properties. In two pilot studies, t otal adherence percent agreement has been demonstrated to range from 74% to 85% and competence scales have produced ICCs ranging from .79 to .83 (Wedge, 2005) . In a further study, total adherence percent agreement was 81% and produced an ICC of .8 1 for competence. The HAACS was also demonstrated concurrent validity 20% of shared variance with a therapist- completed measure of homework adherence and competence (Munro, 2006). Based on a sample of 76 ratings of DVD recorded therapy sessions of CBT for depression, the overall ICC for adherence and competence for .77 and .81 respectively. Internal consistency demonstrated to be excellent ( α = .80) for the competence domain (Kazantzis, Dobson, Munro, & Wedge, 2006). Homework Rating Scale - II The Hom ework Rating Scale (HRS - II ; Kaza ntzis, Deane, & Ronan, 2005 ) is a 12 - item questionnaire rated on a 5 - point Likert scale (ranging from ‘0’ = not at all to ‘4’ = extremely) used to rate patient’s beliefs about between - session activities (i.e., homework) over the course of therapy. Three versions were used in the CBT Homework Project , the therapist version, client version, and o bserver version. Initi al research on the early versions of the client version of the HRS demonstrated a Cronbach’ s Alpha of .77 (Jones, 2002). In a sample 56 patients, the total HRS - II has a demonstrated ICC of .82 while the majority of individual - item ICCs were greater than .6 0. The HRS - II has also demonstrated concurrent validity with 37% of shared variance with a client - completed 88 measure of homework compliance (Munro, 2006). Most recently, in a sample of 74 patients DVD recorded therapy sessions of CBT for depression, excelle nt rates of reliability were demonstrated with an overall ICC of .82 . Factorial validity or model fit for a four factor model was adequate (45, N = 104) = 101.6, p < .00 1 f or the client version of the HRS - II while a poor fit was demonstrated for the therap ist and independent observer versions (Kazantzis et al., 2006). Personality Beliefs Questionnaire The Personality Beliefs Questionnaire (PBQ) is a 126 - item self- report measure developed to assist in the development of cognitive CC, the assessment of dys functional beliefs and intervention for the treatment of personality disorders (Butler, Brown, Beck, & Grisham, 2002; Beck & Beck, 1991; Beck et al., 2001 ). The PBQ is rated on a 5 - point Likert scale from ‘0’ to ‘4’ (0 = “I don’t believe it at all”; 4 = “I believe it totally”) as they relate to patient’s individual endorsements of belief statements. Patient beliefs generate nine categories related to their personality beliefs including Avoidant, Dependent, Obsessive Compulsive, Histrionic, Passive - Aggressive, Narcissistic, Paranoid, Schizoid, and Antisocial. The PBQ categories provide an indication of the presence of personality disorders where a link between these categories and associated personality disorder diagnoses has been demonstrated (Beck et al., 2 001 ; Butler et al., 2002 ) . Cronbach’s alpha ( α) coefficients for individual scales on the PBQ range from .79 to .94 across two studies (Beck et al., 2001; Butler et al., 2002). Beck et al., (2001) reported t he median inter- correlations between scales are moderate ( 0.51 ) and Pearson test- retest correlations for the individual PBQ categories were adequate ranging from .57 to .93 . The PBQ was administered (session 8) as a means of informing therapist’s CC and the sharing of the CC with the patient (session 10 ) where changes in personality 89 beliefs have been suggested to be more measurable from session eight onwards (Beck & Freeman, 1990). Follow Up Questionnaire At 2 months and 6 months post - treatment participants are asked to complete a battery of questionn aires. Relevant measures to the current thesis include the BDI - II as well as a feedback questionnai re (s ee Appendix I ) . The follow up questionnaire comprises of six sections. ‘Sec tion IV – Understanding Your Problems’ was specifically designed for the current thesis to assess patient beliefs about different aspects of the CC as it was used in their therapy. Section IV includes eight items each rated on a 4 - point Likert scale (‘1’= strongly disagree, ‘2’ = disagree, ‘3’ = agree, ‘4’ = strongly agree). Patien ts were given the instruction “We would like you to rate the process of understanding my problems in therapy. Please indicate the extent to which you agree or disagree with the following phrases”. Items in Section IV were selected to complement the areas o f focus on the CRS. For example, ‘My therapist focused on my strengths in therapy’. 90 6.4 Procedure Recruitment of Participants Participants were recruited between mid 2007 and early 2010 from the wider Auckland community primarily by means of ad vertising (s ee Appendix J ). Advertisements were placed mostly in suburban newspapers local to the Centre for Psychology in Albany (e.g., the North Shore Times, Rodney Times) as well as in the Massey University Albany campus magazine. A pamphlet was designe d for distribution to University health centres and local General Practitioners offices ( see Appendix K ). Contact was made with Team Leaders from the Auckland and Waitamata District Health Board (DHB) community mental health teams. Advertisements asked that participants left an answer phone message and that they would be contacted by a Depression Study coordinator to arrange an initial telephone interview. There were three doctoral- level study coordinators, including the author, each were psychology grad uates with postgraduate research, statistical, and administrative experience necessary for the multiple tasks required. Study Coordinators acted as an initial point of contact between patients and therapists and carried out safety monitoring for the overall CBT Homework Project. 91 Figure 4. Participant recruitment, screening and final treatment sample Initial Contact Excluded, did not meet criteria ( n = 36) Withdrew from active therapy ( n = 1) Participants that made initial telephone contact with the CBT Depression Study n = 251 Participants that met preliminary MDD criteria and asked to attend intake assessment n = 65 Participants that made completed a full course of therapy in the CBT Depression Study n = 28 Excluded, did not meet criteria (e.g. using anti - depressants, previous diagnoses of MDD) n = 186 n = 37 92 Telephone Interview Telephone interviews were conducted by the author and another doctoral- level psychology graduate student with experience in conducting telephone interviews and specific training in assessing the studies inclusion criteria. A telephone screening tool including standard assessment questions based on DSM - I V - TR (APA, 2003) criteria was used to ascertain whether those interested in the study met criteria for involvement the CBT Homework Project. Telephone interviewers were trained by the primary investigator in the use of this measure and procedure for selection of participants which included a weekl y coordination and review meetings with the primary investigator to discuss the acceptance or referral of participants as well as the general day- to- day running of the study and managing issues of potential risk . Participants were informed that the telephone interview could take up to 45 minutes to complete. Telephone interviews involved an overview of the research, an explanation of the bounds of confidentiality, and a preliminary screening for symptoms of Major Depressive Disorder (M D D), Alcohol Dependenc e, Borderline Personality Disorder (BPD), and other major mental illnesses likely to influence the participants continued participation and suitability for involvement in the CBT Homework Project consistent with DSM - I V - TR (APA, 2003) criteria. A risk asse ssment was carried out at any point in the interview where participants expressed some degree of risk of harm to themselves or others and the conclusion of the telephone interview. At any point in the interview where it was assessed the participants did not meet the criteria for inclusion in the CBT Homework Project the telephone interview was discontinued. Participants that met preliminary criteria for involvement in the CBT Homework Project were invited to attend a computerised and pencil and paper intake assessment at the Centre for Psychology. At 93 this point a participant information sheet and confidentiality agreement was mailed to the participant. Participants deemed unsuitable were referred to the appropriate community mental health service or resource. Pencil and Paper / Computerised Intake Assessment In the second phase of screening participants were orientated around the Centre for Psychology and were asked to complete a number of pencil and paper measures (e.g., Demographic Questionnaire, BDI - I I ). Participants were then asked to complete the Composite International Diagnostic Interview (CIDI). At the conclusion of the interview participants were invited to attend a face- to- face intake assessment with a therapist in the study. Participants were informed that this was the final stage of screening before beginning therapy. Intake Assessment Therapists were then given the original completed telephone screening information, all pencil and paper measures completed by the participant and print outs of the CIDI interview prior to the final intake assessment to aid their decision in the final screening of the participant for continued involvement in the CBT Homework Project. If participants were deemed unsuitable or choose not to begin therapy at this point therapists provided appropriate referrals and resources. Upon beginning therapy participants were considered to be patients in the CBT Homework Project. Inclusion / Exclusion Criteria For inclusion in the CBT Homework Project participants were requi red to be experiencing a major depressive episode (DSM - I V - TR) for the first time. As a 94 minimum requirement participants were required to be between 18 and 65 years old, proficient in reading, writing, and conversing in English. Participants were not eligible if they were taking central nervous system (CNS) acting drugs (except an occasional hypnotic or the oral contraceptive) or if they met criteria for substance abuse, psychosis, or borderline personality disorder (BPD). Patients did not meet inclusion criteria if they were currently engaged in other psychological treatment. It was necessary that patients were able to be safety managed with outpatient psychotherapy (i.e., absence of risk to self or others). It was also explained to participants that there w as no provision for a culturally- specific service. In the event that individuals required or made a clear preference for a culturally specific service an appropriate referral was made. Setting Therapy sessions were conducted at the Massey University Centr e for Psychology in Albany Village located on the third floor of the Albany library building. Therapy took place in one of three identical rooms designed specifically for individual psychotherapy. Therapy sessions were conducted on Mondays, Tuesdays, Thur sdays, and Fridays between 9am and 4pm to ensure Centre for Psychology clinical staff were available in case of a clinical emergency. Observational independent ratings of CBT Homework Study data discussed in study three were conducted at two sites at Mass ey University in Auckland, New Zealand and La Trobe University in Melbourne , Australia . 95 Cultural Factors Specific to New Zealand During the initial telephone screening procedure participants were informed that a culturally- specific approach in the CBT Homework Project was not a primary focus during active therapy being offered. However, p articipants were informed that should they prefer a culturally specific approach to therapy, or should cultural issues arise for them that a suitable referral would be made with special provision for participants identifying as Maori. For example, the CBT Homework Project protocol names MO KO services, Maori Mental Health services at Waitamata district health board as a specific referral source. It is noteworthy that the vast majority on participants identified as European / Caucasian. None of the participants expressed an interest in being referred to a more culturally specific service. Ethics The CBT Homework Project and subsequent analyses of data collected have been reviewed and approved by the Northern X Regional Ethics Committee (NTX/06/ 08/ 085). Safety Monitoring In fulfilment of obligations outlined with the Northern X Regional Ethics Committee approval, ongoing monitoring of CBT Homework Project patient safety took place during active therapy from the 28.04.07 to 03.02.09. In order to ensure patient safety, it was required that patients were generally improving as a result of the therapy. This was defined by BDI - I I scores remaining lower than intake BD I - I I scores (i.e., BDI - I I change scores were positive). It was agreed that if more than 50% of 96 patients currently engaged in active therapy were observed not to be improving the therapy would be discontinued for the CBT Homework Project. Safety monitoring was carried out by the author by means of weekly safety updates to the primary investigator and the research group directly involved in the CBT Homework Project. Informed C onsent All participants were required to give informed consent. Participant information sheets and consent forms are attached in Appendices A and B . 97 CHAPTER VII Study One - The Development and Change of Case Conceptualizations throughout a Course of Cognitive Behavioural Therapy (CBT) for Depression 7.1 Outline and Aims The primary goals of Study 1 were to demonstrate how CCs develop and change over a course of CBT for depression and to inform the development and interpretation of the CRS in Studies 2 and 3 . This study addresses a research gap in existing case conceptualization literat ure in CBT. Current research fails to address how CCs change throughout a naturalistic course of CBT. Subsequently, although hypotheses were made about the way in which therapists use CCs in CBT, the present study was largely exploratory. Patient informat ion from J. Beck C ase Conceptualization Diagrams ( CCD; J. Beck, 1995) completed by therapists was able to be analysed for trends in the frequency (quantity) and type (quality) of individual patient emotion, thoughts, and behaviours at different levels of C C. A small sample design was adopted to report detailed qualitative data to illustrate the development and change of CCs over therapy sessions. A small N design was intended to capture any potential “unique” phenomena or occurrences should they arise (Gedo , 1999), that might obscure or inform quantitative analyses using the CRS in Studies 2 and 3 . This approach was consistent with the explorative nature of the current thesis given the absence of repeated measures empirically based research examining the cha nge of case conceptualizations over time in CBT. Furthermore, c ognitive- behavioural theory suggests that in early sessions of CBT primarily “surface” level components of CC are explored and discussed while “deeper” level components are not readily explor ed and discussed until later sessions 98 (A. T. Beck, 1967) . However, no research in CBT addresses this assumption. Existing research from other psychotherapeutic approaches suggests that CCs may become more complete over the course of therapy. In sum, Study 1 sought to answer the above research questions both to address a gap in the literature and to further inform the construction and development of the CRS. 99 Table 4 Study 1: Participant demographics, diagnosis, severity of depression and number of therapy sessions Variable N % M SD Gender Male 2 20% Female 8 80% Age (years) 44.10 11.45 Depression Severity* Mild 3 30% Moderate 1 10% Severe 5 50% No diagnosis 1 10% Number of therapy se ssions 17.10 4.84 Number of follow up sessions Two month 5 50% Six month 3 30% * Note. Diagnoses and depression severity reported as measured on the CIDI. Following intake assessments all patients included in the CBT Homework Project received a primary diagnosis of MDD following further assessment. Patients ( n = 10) were initially assessed on the CIDI and some met additional diagnostic criteria for generalised anxiety disorder (80%, n = 8), social phobia (50% , n = 5) , PTSD (50% , n = 5) , p anic disorder (40% , n = 4) , alcohol dependence (20% , n = 2), brief psychotic disorder (10% , n = 1) , OCD (10% , n = 1) , alcohol abuse, (10% , n = 1), bulimia nervosa (10%, n = 1), conversion disorder (10% , n = 1) although were later assessed to meet a primary diagnosis of MDD without meeting any exclusionary criteria after further assessment. 100 7.2 Method Participants Data from the first 10 patients who received treatments in the CBT Homework Project were included in the present study (see Table 4 ) . The sample consisted of two male and eight female patients of average age 41.1 years (rang e from 25 to 62 years old). Eight of the participants identified their ethnicity as European / Caucasian , one identified as American and one as Australian. Of the participants in the present study, each had completed a full course of CBT up to 20 sessions (average of 17. 1 sessions) with an additional two booster sessions being offered at two months and six months following therapy during the training phases of the CBT Homework Project. Four of the 10 patients completed both two and six month booster sessions. Five of the 10 patients completed two month follow up sessions but only three of these remaining patients attended six month follow up sessions . Five of the patients did not complete any follow up sessions. It must be reiterated that while all patients were initially assessed using the Composite International Diagnostic Interview (C IDI; see s ection 6 .3) , patients also took part in an initial telephone screening interview and a clinical intake interview to assess whether or not patients met the criteria for a prima ry diagnosis of first episode Major Depressive Disorder (MDD) . Table 4 illustrates different diagnoses and depression severity categories as indicated on the CIDI. F or example, although patient one patient did not meet the CIDI criteria for firs t episode MDD , they were later assessed to meet prerequisite criteria during their clinical intake assessment interview consistent with requirements of their participation in th e CBT Homework Project (see Chapter V I ). 101 Therapists Three female therapists of average age 43 years (range from 29 to 50 years) provided the course of CBT; all were intern psychologists in their second to last or final year of training towards a DC linPsych. All of the therapists identified as New Zealand European / Caucasian. Relevant therapist training and experience is previously outlined in Chapter V I . 7.3 Procedure The data were gathered during training phase of the CBT Homework Project or Depression Study. Therapists completed J. Beck CCDs (J. Beck, 1995) systematically at five sampling points (intake, sessions 3, 5, 8, & 10). Information included in the J. Beck CCD s was generated collaboratively with the patient during each session and the J. Beck CCDs were formally written up by therapists between sessions. All therapists had prior training in completing J. Beck CCDs, for example, were taught to complete the bottom half or surface- level portion of the J. Beck CCD before moving on to complete the top half or deeper- level underlying mechanisms portion of the J. Beck CCD consistent with best practice recommendations (J. Beck, 1995, Stenhouse & Van Kessel , 2002). Ideally, through a continual process of therapist exploration, reflection and feedback from the patient during collaborative discussions it is expected that the CCD become more refined over the course of therapy and that this would be reflected in the quantity and quality of J. Beck CCD completion. Therapists were instructed to share their overall hypotheses as related to the J. Beck CCD at session 10 to systematise and model appropriate timing for sharing of the overall case conceptualization with the patient in a naturalistic course of therapy (Pain et al., 2008; Persons, 1989) . The only exception to this was if 102 discussion in supervision suggested that it would be more helpful to share the case conceptualization with individual patients at a later stage in therapy. The present study was based on a sample of 53 J. Beck CCDs from the first 10 patients in the CBT Homework Project. Analysis A qualitative approach was taken using a descriptive analysis to observe trends through the graphical presentation of data using bar graphs for the total frequency of data units. Prior research investigating the content of written CCs has advocated that use of coding information into meaningful data units (Eells et al., 1998; Kuyken et al., 2005). Further research has utilized graphical display as a means of interpreting change CC data units over time (Mumma, 2004; Muran & Segal, 1992). However, prior research has not investigated the change of CBT specific CC individual CC components over time. Due to the modest patient sample ( n = 10) inferential statistics were not able to be used and would not have the appropriate power to yield meaningful results. Results for individual patients were displayed graphically in order to provide a visually simple demonstration of trends in CC information. In particular hypothesis one states that CCs will become more complete over the course of CB T for depression. Bar graphs were used to show fluctuations and potential trends in the amount of information contained in J. Beck CCD s. Coding of Qualitative Data Each piece of information recorded by the therapists was coded as an individual ‘ data unit’ . Data units were coded from raw data recorde d in J. Beck CCD s using content coding methodology ( Strauss & Corbin , 1998 ; Miles & Huberman, 103 1984). For example, in the ‘core beliefs’ component of the J. Beck CC D “I am a failure” or “peo ple are critical” would be coded as two individual data units. A frequency count for each data unit in each CC component was carried out. This descriptive information was then presented graphically to demonstrate trends in the amount of information therapists recorded for each major CC component. In addition a frequency count for information pertaining to ‘patient strengths’ was undertaken consistent with research suggesting that patient strengths are not often conceptualized with a tendency to focus on n egative patient features, dysfunction or weakness (Sacco & A. T. Beck, 1995). Furthermore, frequency counts of data unit s in the ‘core beliefs’ was further divided into beliefs about the self, world or others, and the future consistent with dimensions of Aaron T. Beck’s Negative Cognitive Triad (A. T. Beck, 1976) a disorder specific model used by the therapists to explain underlying mechanisms of depression. Case Conceptualization Diagram The J. Beck CCD (J. Bec k, 1995) was the primary coding tool used in this study. Although this format for written CCs is generally accepted to have strong clinical utility and is used as a standard clinical tool by cognitive therapists, there is little psychometric research to support the efficacy of this format (Kuyken, Fo thergill, Musa, & Chadwick, 2005). For the purposes of the current project t he J. Beck CCD was used both as a clinical tool and as a research tool for surveying the current use and change of written cognitive case conceptualizations in the early phase of C BT for depression. In total 1526 data unit s were coded based on the information in the J. Beck CCD s. 104 7.4 Results In general, more information tended to be recorded in overall written CCs in later sessions than in early sessions (s ee Figure 5 ). No outliers were apparent from the data. This observation was facilitated by the practical space constraint in the J. Beck CCD where by therapists were only able to include a limited amount of information, making the presence of outliers (i.e. unusually high frequency counts unlikely). Table 5 presents raw sums, means and standard deviations of data units coded based on J. Beck CCDs. Results show that while the amount of information contained within J. Beck CCDs tended to increase from early sessions to later sessions, greater standard deviations in later sessions reflected more variability in the number of data units of information coded in later sessions compared with earlier sessions (see Table 5 ). In particular session 10 showed the largest standard deviation (SD = 2.10) suggesting a greater variability in the amount of data units contained within CCDs compared with earlier sessions. Table 6 also shows the raw number of data units for ranged from 270 (intake) to 323 (session 10) which represents a qualitatively obser vable difference in the level of completion of CCD from the first CCD completed by therapists in the study to the final CCD completed at session 10. Furthermore, by taking an average to account for missing CCDs at sessions 8 and 10 for patient 164, it can be observed that therapists included on average 33% more information units in CCDs at session 10 than recorded at CCDs completed following intake. 105 Figure 5. Changes in mean amount of information in different components of J. Beck Case Conceptualization Dia grams across sessions Table 5 Study 1: Descriptive Statistics of Data Units Coded from J. Beck Case Conceptualization Diagrams Session Minimum Maximum Sum Mean S tandard D eviation Intake 0 9 27 0 3 .00 1. 63 3 0 7 272 3 .02 1. 24 5 0 9 298 3.31 1. 47 8 0 9 307 3. 79 1 .77 10 0 12 323 3. 99 2. 10 Note. Sessions represent the five assessment points where therapists completed J. Beck CCDs based on the data units of the nine information boxes contained in the J. Beck CCD . Sums represent the raw number of overall data units at each time point and the mean represents the average number of data units per category on the CCD. 106 In contrast to a consistent overall trend in the amount of information in J. Beck CCDs, difference s existed in how much information was recorded in different individual components of the J. Beck CCDs. Steady increases in the number of automatic thoughts (see Figure 6 ), the meaning of automatic thoughts (see Fig ure 7 ), emotion (see Figu re 8), behaviour (see Figure 9 ), condi tional assumptions (see Figure 10 ) , and compe nsatory strategies (see Figure 11 ) as well patient strengths (see Figure 12 ) in J. Beck CC Diagrams (1995) were observed. Generally at least two or more units of information were recorded for each of these categories from therapy sessions five to ten. The increase in number of units of information for emotion in particular were marked and also suggested g reater attention to a broader range of emotions over time (see Figure 8 ). It is likely that this was due to therapists assisting patients to identify a fuller range of emotion or simply that more information was collected over time. Figure 6. Changes in mean number of automatic thoughts in J. Beck Case Conceptualization Diagrams (average fr equency of data units for all ten participants) 107 Figure 7. Changes in mean number of meanings of automatic thoughts in J. Beck Case Conceptualization Diagrams (average frequency of data units for all ten participants) Figure 8. Changes in number of emotions recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participants) 108 Figure 9. Changes in number of behaviours recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all te n participants) Figure 10. Changes in number of conditional assumptions recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participants) 109 Figure 11. Changes in number of compensatory strategies recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participants) Figure 12. Changes in number of patient strengths recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participa nts) 110 Therapists consistently described situations in their CCD s (se e Figure 7 .9). Over the first ten sessions the majority (81.25 % ) of CCD s included all three situations completed on the CCD s. Generally between two and three situations were consistently filled out after the initial session CCD . This result demonstrates that therapists consistently using CCD s but more so after initial session. Relevant childhood data and important background information remained reasonably constant. This suggests good initial assessment by therapists and use of this information over the course of therapy. The number of units of information recorded for core beliefs was variable (see Figure 15) and dependent on the individual patient. In some cases more core beliefs were identified over the course of therapy and alternatively for other patients towards session ten few core beliefs were identified. Figure 13. Changes in mean amount of information for situations in J. Beck Case Conceptualization Diagrams (average frequen cy of data units for all ten participants) 111 Figure 14. Changes in number of relevant childhood information / background data recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participants) Figure 15. Changes in number of core beliefs recorded in J. Beck Case Conceptualization Diagrams (total frequency of data units for all ten participants) 112 It is likely that the sharing of the CCD at session 10 was likely to have impacted the number of information units recorded by therapists. For exam ple, from visual observation much of the CCD s were often incomplete at session ten. This impacted on the total number of units of information recorded on the overall CCD due to the CCD being filled out with the patient in- session when the CCD is shared, but not added to by the therapist at the completion of the session (see Figure 5 ). Additionally, conditional assumptions contained in the CCDs increased considerably after session ten when the CCD was shared with the patient (se e Figure 10 ). Also, it was observed that avoidance and withdrawal were most commonly conceptualized as a compensatory strategy. Only twenty - four units of information coded as patient strengths were recorded (see Figure 12 ). This suggests a f ocus on dysfunction by the therapists and a limited conceptualization of patient strengths. Patient strengths that were conceptualized tend ed to be integrated in later sessions. Lastly, the results suggested that therapists had a greater focus on self - schemas than the world / others or future with a considerably larger number of self- schema units of information recorded in CCDs . The circled potions of Figures 16 and 17 give some anecdotal examples , where the number of circles depicts the amount of core beliefs pertaining to ‘world / others’ and the ‘future’ identified by therapist in J. Beck CCDs completed by therapists in the CBT Homework Project. In comparison, Figure 18 demonstrates an increase in identified core beliefs pertaining to the ‘self’ denotable from the increase in circles shown around individual beliefs identified. Figure 19 shows a graphical display of overall information units divided into relative core beliefs about the ‘self’, ‘world / others’ and the ‘future’. 113 Figure 16. E xample of b eliefs about ‘future’ as recorded in J. Beck Case Conceptualization Diagrams Figure 17. E xample of b eliefs about ‘world / others’ as recorded in J. Beck Case Conceptualization Diagrams 114 Figure 18. E xample of beliefs about ‘self’ as recorded in J. Beck Case Conceptualization Diagrams Figure 19. Number of core beliefs related to ‘self’, ‘world / others’, and ‘future’ in J. Beck Case Conceptualization Diagrams a t corresponding therapy sessions 115 7.5 Discussion Conclusions from Study O ne CC is espoused as the foundation of CBT that ties together assessment and intervention sufficient to understand the patient. However, little is known about how therapists use CCs longitudinally over the course of CBT. The present study aimed to integrate CCs, specifically the J. Beck CCD written format systematically into the CBT treatment protocol to provide exploratory information on how therapists in the CBT Homework Project used CCs. In support of hypothesis one, Study 1 revealed that therapists tend to produce more complete written CCs over the course of therapy with an average of 33% more information being recorded in case conceptualizations from intake to session 10, but with differences in the amount of specific J. Beck CCD components that were observed. This finding refutes the notion that therapists are able to produce high quality CCs based on information from their initial intake session. Instead therapists tended to add information cumulatively to their CCDs. The results suggest that therapists gain a more complete knowledge of their patients during the early phase of a course of CBT. Further to the investigation of how written case conceptualizations change over the course of therapy, a s suggested by Sacco and Beck (1995) the results also suggest that therapists may not conceptualize patient strengths very often together with the patient. Furthermore, patient strengths are conceptualized to a greater extent towards later therapy sessions but still remain infrequent compared to other areas or components of CC. Thus, study one provides research support for the often lamented belief that CBT therapists do not place enough emphasis on patient strengths, but instead focus on dysfunction and weaknesses. 116 Of particular note, w hen conceptualizing core beliefs the therapists in Study 1 tended to focus on negative beliefs of the ‘ self’ more so than beliefs relating to the ‘ world or others’ and the ‘ future’ . A. T. Beck argued that d epression was triggered, perpetuated and maintained by an individual’s view of one’s self rather than the ‘depression’ accounting for negative beliefs of the self. Negative views of the world, others or environment and the future were suggested to support these negative self views (A. T. Beck, 1976). Although the results might indicate a neglect for core beliefs pertaining to the ‘world or others’ and the ‘future’, this focus on core beliefs relating to the ‘self’ found in Study One is consistent with A. T. Beck’s (1976) cognitive theory of depression which puts greater emphasis on negative beliefs related to the ‘self’ as the most important, where beliefs about the ‘world / others’ and ‘future’, stem from our own self beliefs or self perceptions. However, while the majority of J. Beck CCDs components became more complete over time, core beliefs showed variations between patients as to whether more or less core beliefs were generated. In this way for some patients therapists began with as few as one core belief and generated more core beliefs from their initial CCD to their final CCD, while for other patients therapists generated a range of core beliefs in the early sessions of therapy but presented a few, perhaps more encompassing core beliefs in their final CCD. This is consistent with Persons (1989) suggestion that core beliefs are combined into more specific beliefs or a particular belief is focused on that is considered to be of greatest importance. Here therapists maximise the use of the J. Beck CCD (199 5) by arriving at fewer numbers of core beliefs later in sessions as they hone in on the most underlying “core” beliefs that explain the most about the patient. Thus, it is possible that therapists who arrived at a 117 large number of core beliefs were yet to refine their understanding of patient core beliefs and focus their discussions of related CCD components in session. Limitations A number of limitations were apparent from the present study. In practice it is unlikely that therapists would complete five s eparate J. Beck CCDs (J. Beck, 1995). However, in consideration of the benefit of an initial CCD to guide treatment, and the highest amount of information being contained in later sessions, the results of study one might indicate that CCDs should ideally b e completed at intake as well as session 8 with a revision after the sharing CCD in session around session 10. Although not intended in the conception of the research, all of the therapists at times shared their CCs later than session 10 on advice from the ir clinical supervisor. Also, from observation, therapists in this study completed parts of the final (session 10) CCDs with patients during the sharing of the CCD and completed the remainder between sessions as was done for all previous CCDs. Subsequently the final CCD sometime s remained incomplete for some patients existing as a source of information that might best be used alongside previous CCDs where therapist perhaps did not see the utility in re- writing information from previous CCDs if it were consi stent with the in session information the patient had provided. The impact of this may have been to in fact underestimate the overall amount of information contained within therapist’s final CCDs. This is consistent with the larger standard deviations in d ata units at later sessions reflecting the more variable levels of completion of CCDs. A further limitation is that the therapists were still in training during the time that therapy was conducted. This bears on the ability to generalise results to the construction of written CCs in expert therapists, who for example, may through 118 experience be better able construct high quality, complete CCDs from the outset of therapy. Future research should investigate the use of expert therapist’s use of different written formats of CC in CBT. Finally, Study 1 centres solely on the role of the therapist in constructing CCs. However, by examining the raw output of written CCs, Study 1 provides a platform for understanding the collaborative exchange that takes place betwee n therapist and patient to produce written CCs and better understand the link between written and in - session eliciting and discussion of CC information using the CRS in Studies 2 and 3. Summary Taking these limita tions into consideration Study 1 has provided the first research providing information of the ‘shape’ of therapist’s use of a co mmonly used written CC format, the J. Beck CCD, as it evolves over the course of CBT. Also information on different components of CC consistent with cognitive- behavioural theory, have been observed. In partic ular, Study 1 has provided more information about how therapists generate core beliefs and the type of core beliefs generated beyond the assumptions of cognitive- behavioural theory. In sum , Study 1 provides a platform for the creation of a new measure of therapist competence in CC suitable for use in CBT. The inclusion of ‘ patient strengths’ and special attention to the therapist’s competence in formulating core beliefs is warranted consistent with recent recommendations (Kuyken et al, 2008) and discussed in the next chapter. 119 CHAPTER VIII Study Two - The Development of the Conceptualization Rating Scale (CRS) 8.1 Outline and Aims It has been said that the history of psychotherapy can be characterized as the search for the specific processes that reliably produce change (Duncan, Sparks, & Miller, 2005). The primary aim of the present study was to develop a reliable and useful measure of therapist competence in CBT to capture the process by which CC develops between therapists and patients over the course of active therapy. Although therapist competence in CC is a requirement under New Zealand and international professional psychological bodies, currently no appropriate methods or measures for assessing therapist competence in CC exist. Prior research has focused on written aspects or vignettes of CCs in CBT. The results of this research have yielded poor reliability scores on more inferential or “deeper” level components of CBT case conceptualization (i.e. core b eliefs, conditional assumptions). A recent focus on therapist competence in different skill - sets essential to the practice of empirically supported CBT has lead to the need to develop a measure of therapist competence in case conceptualization suitable for use in CBT. Study 2 provides a preliminary evaluation of the Conceptualization Rating Scale 1 (CRS; Easden & Kazantzis, 2008; 2009) measure as part of the development process. 1 Preliminary psychometric data for the Conceptualization R ating S cale (CRS) was presented at the 36 th and 37 th Annual Conferences for the British Association of Behavioural and Cognitive Therapies (BABCP) in Edinburgh, Scotland (2008) and Exete r, United Kingdom (2009). 120 In achieving this purpose Study 2 : - Provides a discussion of the rationale for item selection in the initial stages of construction of the CRS (section 8.2) - Describes the process that was undertaken for integrating expert feedback on the CRS (section 8.3) - Describes the training sample (section 8 .4 ) - Outlines the procedure for the t raining of independent raters (section 8 .5 ) - Provides a description of the CRS measure (section 8 .6 ) - Outlines statistical analyses used (section 8 .7 ) - Provides a discussion of final expert feedback on th e CRS final version (section 8.8 ) - Presents preliminary psychometric estimates of reliability scores for the four categories or questions of the CRS and an indication of overall reliability scores produced from the initial training sample (section 8 .9 ) - Provides the individual item and total item estimates of reliability scores as well session specific reliability estimates based on the total sample used in the present thesis (section 8 .10 ) - Provides preliminary conclusions and discussion on the psychometric properties of the CRS measure (section 8 .11 ) Study 2 resulted in a number of changes to the format and contents of the CRS measure. In turn, Study 2 facilitated a further psychometric evaluation bearing on the validity of the CRS measure in chapter eight. Expert feedback and recommendations for changes to the CRS are detai led in section 8.3 . 121 8.2 Item Selection and Measure Construction The CRS was designed to be an operationalization of in - session CC competencies in CBT (A. T. Beck et al., 1979). It was intended that the different items and domains on the CRS would reflect cognitive behavioural theory (A. T. Beck, 1967) and be acceptable within the framework of competent day - to- day practice of CBT. The purpose of the CRS was both as , 1) a supervision tool to provide feedback to therapists regarding their competence in case conceptualization in CBT and as , 2) a research tool hoped to provide empirical support for processes and outcomes related to case conceptualization in CBT. In turn, the J. Beck Case Conceptualization Diagram (CCD; J. Beck, 1995) was used as a framework for many of the items on the CRS. This decision was primarily made due to the cognitive behavioural theoretical foundations on which the CCD was created. Although other written formats for case conceptualizations in CBT exist (see Section 2 .4) the continued popul arity and common usage of the CCD in clinical practice after nearly 15 years of existence (Sudak et al., 2003) provides a sound rationale for investigating assumptions regarding the empirical utility of the CCD. In addition to the m ajority of items included in the CRS, some additional items labelled ‘alternative’ items were included sufficient for the CRS to investigate comprehensive CC in CBT. Recommendations from the literature and recent developments in CBT research since the crea tion of the CCD were considered for this purpose as outlined below. Furthermore, the selection of scale descriptors for which the level of competence was rated was decided upon based on current scales used in research on therapist competence and generally accepted aspects on case conceptualization discussion necessary in CBT. 122 The selection of descriptors for the different levels of therapist competence on the ‘CRS Therapist Competence Scale’ ( see Appendix F ) was require d for the production of a rating of ‘c ompetence’ in cognitive case conceptualization on the CRS . Core areas of therapist competence in CBT were targeted to generate a comprehensive scale for case conceptualization in CBT as discussed below. In particular Kuyken et al. (2008) identify three “m etacompetencies” (p. 250) of case conceptualization in CBT emphasised in their book Collaborative Case Conceptualization that are consistent with and have been incorporated into the CRS competence scale. These include 1) collaborative empiricism, 2) workin g at an appropriate level of conceptualization for a given patient, and 3) incorporating patient strengths to achieve a balance of problem- focused and resilience focused work. 1. Collaborative Empiricism (i.e., eliciting patient f eedback ) Eliciting patient f eedback to determine clinical relevance as well as patient acceptance and understanding of different components of case conceptualization has been identified as a key compet ency in the practice of CBT. Kuyken et al. (2008) highlight the need for collaborative empiricism and described the advanced competency of collaborative empiricism as “the ability to elicit collaboration and client engagement under challenging circumstances and to also collaboratively reconcile differences between empirically supported models and client observations”. For example, recent qualitative research has explored the importance of patient feedback specifically in the context of case conceptualization in CBT and the potential for inconsistency with patient symptom decrease. Their research identifies the potential for greater overall therapeutic benefit with regularly elicited patient feedback (Chadwick et al., 2003; Pain, 2008). 123 2. Depth of Discussion (i.e., level of cognitive / behavioural exploration / linking) In CBT, therapists in itially gather more “ surface” level information and over time gather more complex and “deeper level” information to gain a more holistic understanding of the patient. Moreover, at times it is often not desirable to activate deeper level cognitions such as core beliefs that might distress the patient in the early phases of therapy (J. Beck, 2005) . For example, it is suggested that gathering enough information of form and write a reasonable formulation is likely to take at least three sessions (Ingram, 2006). As the therapist’s understand ing of the patient and the patient’s understanding of themselves becomes deeper, the therapist and patient are able to discuss links between different aspects that make up the overall CC, to monitor cognitive patterns and refine hypotheses about a patient’s experience. The downward arrow technique ( A. T. Beck et al., 1979; Burns, 1980) is an example of a commonly used technique for eliciting deeper level information which illustrates aspects of the process by which therapists might collaboratively elicit deeper- level information about a patient (J. Beck, 1995) . In using the downward arrow technique, the therapist asks S ocratic questions exploring the importance of beliefs or statements to arrive at an underlying or deeper level belief. For instance , the therapist will repeatedly ask “ what does this mean to you?” , “what’s the worst thing about this?” or “and what if this were true?” . 3. Patient Strengths and R esilience A focus on patient strengths and resilience has also been identified as a core principal of CBT CC (Kuyken et al., 2008) . A useful case conceptualization is contingent on how it reflects understanding of both patient deficits and strengths (Greenberger & Padesky, 1996 ). However, it has been suggested that cognitive 124 therapists often neglect the exploration and assessment of patient strengths or resilience in favour of a disorder or problem focus (Sacco & Beck, 1995) . This lack of focus on patient strengths is observable both in- session and in gaps in formal written CCs (Groth - Marnat & Horvath, 2006). Also, “ignoring” patient strengths has been cited as a commons means of arriving at an incorrect assessment when using CCs (Ingram, 2006, p.471). For example, using a simplified medical example to illustrate the practical benefit of identifying strengths, for patient wanting to rehabilitate after a car crash, having been bedridden for a month it would be useful to note that the patient is a) physically fit, previously participated in marathons just prior to the accident, or: b) overweight, physically unable to engage in strenuous physical activity. By gauging both a patient’s weaknesses and strengths a case conceptualization can be translated into an appropriate treatment plan. In this case “ patient a)” may be recommended to begin by taking short walks and begin a light, restricted e xercise programme at the gym. “ Patient b)” may be better suited to minimum impact exercise such as regular walks around the house or swimming. Mental health is just one other dimension of overall health and well being. In turn, psychotherapy as the treatment of choice must make use of both debilitating and facilitating aspects of a patient in order to translate a case conceptualization into an optimal treatment plan. Furthermore, what is illustrated here is the clear relevance of homework use within the CC process which is investigated in detail in Study 3 of the present thesis. It has been suggested that conceptualization of patient strengths might operate automatically over the course of therapy. H owever, preliminary research supports the notion that therapist behaviour primed towards activation of patient strengths actively and positively influences therapy as a mechanism of change (Flückiger & Holtforth, 2008 ; Flückiger, Caspar, Holtforth & Willut zki, 2009 ). 125 Sacco and Beck (1995) identify three common failures when engaging in CBT with patients which further demonstrate the relevance of homework to the current project within the context of the concept ualization of patient strengths: a) f ailing to engage in potentially reinforcing activities, b) s ocially alienation behaviours, and c) covert disorder maintaining behaviours. In addition recognition of “mirror positive s” is identified that provide a strengths- based perspective on case conceptualization : a) engagement in various social activities, b) s ocially functional behaviours, c) p otential or current disorder combatant behaviours, and, d) p romotion of new behaviours. These “mirror positives” represent areas for potential engagement and for data gathering related to patient strengths. By incorporating a patient strength focus into the CRS, it was hoped to achieve a greater understanding of how patients’ deficits and strengths are conceptualized in the process of therapy as well as the written format of CC. Furthermore, in relation to homework it was intended that the relationships between CC and treatment planning could be explored where patient strengths are often built upon during the process of discussing and undergoing homework tasks as barriers to homework completion are overcome (Kazantzis & Shinkfield, 2007) . Figure 20 provides a conceptual model summarising the core features of the CRS. The final version of the CRS is included in Appendix E as well as the necessary resources required to produce ratings using the CRS ( see Appendix, F, G ; see Chapter V I ). 126 Figure 20. Diagram of Conceptualization Components and Core Therapist Competencies in Case Conceptualization Collaborative Empiricism / Patient feedback Building of links among different (i.e. acceptance, understanding) areas of conceptualization Appropriate depth of discussion Ascertaining Importance and integration of all components (i.e. clinical relevance) Disor “Deeper” level Conceptualization “Surface” level Conceptualization Patient Strengths Focus Disorder-specific conceptualization Sharing of Conceptualization 127 8.3 Expert Feedback Prior to the initial CRS train ing and after each training session feedback on the CRS measure was sought by clinicians and researchers considered to be experts in t he field of CC in CBT. Key consultants included Dr. Judith Beck, due to the present thesis utilizing her CCD (1995) , Dr. W illem Kuyken who has produced some of the most recent of the scarce research on CC in CBT, as well as collaborators in the CBT Homework Project with expertise and clinical experience in CC for depression including Dr. Keith Dobson, Dr. Beverly Haarhoff, an d Dr. Paul Salkovskis. The following includes some feedback of note which was used to modify the early drafts of the CRS measure. Initially the general feedback on the measure reflected some possible gaps in assessing a comprehensive case conceptualization . This resulted in the addition and revision of the “Alternative” components on the CRS measure necessary to fully assess a therapist’s comprehensive CC. S pecific feedback was provided on the use of the term “match” and suggested “fit” be used to in recog nising that a different array of diverse information conceptually should fit with the patient’s presentation while it may not always be a direct match. F eedback was also provided on the descriptive Likert scale used to assess competence; specifically it was suggested that the scale be sensitive to instances where it is inappropriate to assess core beliefs to allow therapists that appropriately choose not to explore aspects of core beliefs, thus demonstrating competence, to achieve a high competence score. Consistent feedback was received from experts in the field of CC and CBT reflecting a concern that the initial draft of the CRS did not allow for appropriate non- 128 intervention. For example , the following comments provide an illustration of possible concerns addressed in the first revision of the CRS : This model only works if every item should be demonstrated in every session. There are sometimes very good clinical reasons for not assigning homework, for example. In the case of the CRS, this problem is compounded...A therapist could have a very good case conceptualization of a given client, but for strategic reasons not demonstrate them in the session (it might be too early in therapy; the therapist might be gathering more information; a particular problem that requires problem - solving has emerged, for example) (K. Dobson, personal communication, December 2, 2008) . How about times when it would be inappropriate for the therapist to link an appropriately eli cited current...deeper level cognitions - e.g., t he therapist appropriately judges that it is inadvisable to activate a core belief (e.g. fragile patient, weak therapeutic alliance, early in treatment, activation of core belief might significantly increase patient’s depression or anger) (J. Beck, personal communication, April 3, 2008). These comments were consistent with recommendations for the creation of new measures of therapist competences suggesting that: “ competence, on the other hand, is always context dependent and requires a knowledge of “when” and “when not” to intervene ” (Barber et al., 2007) . A revision was redistributed asking for feedback following the addition of a standard statement in the CRS competence scale recognising therapist’ s appropriate non- intervention in particular components. N amely the CRS was modified to include 129 the statement: “ the therapist has appropriately recognised it is inadvisable to activate deeper level components (e.g., therapist displays good judgement in not activating core beliefs following a rupture in the alliance, etc) ” . Further feedback reflected that the addition of this statement was considered sufficient to prevent penalisation of therapist’s engaging in appropriate case conceptualization strategies. Following the training of raters and once changes to the C RS were incorporated after the analysis of initial reliability scores, a second round of expert feedback was sought. Feedback was again sought from the same experts who initially gave feedback in section 8.3. Subsequent feedback reflected a general agreeme nt that the CRS was a comprehensive case conceptualization measure appropriate for use in CBT. 130 8.4 Method Raters Raters were trained at two sites at Massey University (site one) in Auckland, New Zealand and La Trobe University (site two) in Melbourne , Australia. Ratings were conducted at the Massey University Centre for Psychology between February 2008 and September 2009. Ratings were conducted at the LaTrobe Psychology Clinic between March 2009 and September 2009. In total six raters were used over t he course of the present thesis. The rater demographic questionnaire is attached in Appendix D. Site One Four raters (three female, one male) attended the initial training session in early February 2008 at Massey University including the rating of a single CBT session recorded on DVD. The average age of the raters was 38 years (range 23 to 50 years). The raters were all postgraduate psychology students with a minimum of one to two years postgraduate experience. For the second rater training session and the rating of a second CBT session one rater dropped out leaving three raters (two female, one male) with an average age of 43 years (range 31 to 50 years). In the third and final rating training session a second rater dropped out leaving two female raters with an average age of 40.5 years (raters were aged 31 and 50 years). Site Two Two female raters attended the second training session in April 2009 at La Trobe University including the rating of three CBT session s recorded on DVD across 131 three days. The average age of the raters was 2 3 years ( raters were aged 22 and 24 years). The raters were both postgraduate psychology students with a minimum of one to two years postgraduate experience. 8.5 Procedure Initial Rater Training and Measure Development In the f irst round of training raters from Massey University (site one) attended three blocks of training sessions each followed by the rating of an entire DVD recorded therapy session. After the initial introductory training for raters from site one, changes were made to the measure after the consideration of rater feedback, expert feedback (see S ection 8 .7), and results of overall measure and item observation and analysis for each question on the CRS. Figure 21 provides a summary of the initial training and measure development process. In the initial training session raters attended a full day 8 hour training day conducted by the author and the primary investigator. The first objective of the training was to give the raters from site one sufficient theoretical and practical knowledge to rate therapy sessions using the CRS. A second objective was to pilot the CRS so that changes could be made to the content and structure if needed to improve the utility of the measure. The training was divided into three main stages. Firstly the raters were given a one hour introduction to CC in CBT. This included a brief overview of the “CBT Homework Project” or Depression Study as the context for the present study. This was done using a PowerPoint slideshow presentation ( see Appendix H ). Although during this time raters were encouraged to ask questions and discussion was facilitated. Raters also engaged in a brainstorming activity to assess their current knowledge on CC from prior study and experience. Raters were then 132 Figure 21. Diagram summarising the CRS rater training and measure development process CRS - 3rd Draft (Site 1 - Training Session #1): - 8 hours in duration - PowerPoint presentation - Brainstorming to assess current knowledge of raters - Rating of “snippets” from DVD recorded therapy session - 1st rating of entire DVD therapy session - Additional 1 hour discussion the following day CRS - 4th Draft (Site 1 - Training Session #2): - 2 hours in duration - 2nd rating of entire DVD therapy session CRS - 1st Draft: - Informed by current theory and research - 1st round of feedback obtained from key international and local experts in the field (e.g., Judith Beck, Willem Kuyken, Keith Dobson) CRS - 2st Draft: - Critique and suggestions integrated into measure - 2nd round of feedback obtained from key international and local experts in the field CRS - 5th Draft (Site 1 - Training Session #3): - 2 hours in duration - 3rd rating of entire DVD therapy session CRS - Final Draft - Used to rate therapy sessions in the CBT Homework Project CRS - Final Draft (Site 2) - Replication of initial training using final version of CRS 133 given a 15 minute break before commencing the next stage of training. In the second stage of training raters viewed three short 5 to 10 minute snippets of DVD recorded therapy from the training phase of the CBT Homework Project. Snippets were chosen with an agenda to provide a full range of observable therapist and patient interactions sufficient to make ratings for the full range of behaviours measured on the CRS (see Figure 22 ). The first snippet focused on “Surface” lev el questions (see Figure 23) on the CRS and the raters were asked to only rate questions 1 to 5 which pertained to this snippet. This was followed by a 10 to 15 minute discussion to ensure the raters understood how ratings were made for that section. The second snippet focused on “Deeper” level questions (see Figure 24) on the CRS and the raters were asked only to rate questions 6 to 9 which pertained to this snippet. This was followed by another 10 to 15 minute discussion about ratings. The final snippet focused on the “Alternative” questions (see Figure 25) required to assess a comprehensive CC and the raters were asked only to rate questions 10 to 12 which pertained to this snippet. This was followed by a final 10 to 15 minute discussion. For each snippet raters were asked to complete all items except for the fourth items for each question as it required the use of the J . Beck CCD completed for each session. At the completion of all snippets raters were given the appropriate conceptualizations and asked to complete the final item for the entire CRS. The raters then took a 45 minute break for lunch before returning for the final stage of the initial training. 134 Level of Conceptualization Inference Conceptualization Component “Surface” level components Situation Automatic Thought Meaning of automatic thought Emotion Behaviour “Deeper” level components Relevant Background Information Core Beliefs Conditional Beliefs Compensatory Strategies “Alternative” components Disorder Specific Model Patient Strengths Sharing of the Conceptualization Figure 22. D iagram of conceptualization component categorisa tion at conceptualization levels 135 Automatic Thoughts Meaning of Automatic Thoughts Behaviour Emotion Situations “Surface” Level CC Components Relevant Childhood / Background Information Compensatory Strategies Conditional Assumptions Core Beliefs “Deeper” Level CC Components Figure 23. “Surface ” le vel conceptualization components Figure 24. “ Deeper” level conceptualization components 136 Disorder Specific Model Sharing of the CC Patient Strengths “Alternative” CC Components Figure 25. “Alternative” conceptualization components I n the last stage of training raters used the CRS to rate an enti re therapy session from the CBT Homework Project. They were asked to rate the first three items for each question and leave item four until last. Item four requires that the raters turn over the therapist’s complete d J. Beck C CD for comparison to what was observed from in session content. J. Beck CC Ds were left face down until the viewing of the therapy session had ended so as not to influence ratings during the session. A final 10 to 15 minute discussion took place at the conclusion of the rating. The next day a further hour long discussion took place between the four raters and the author to clarify any misunderstandings about the measure and to discuss difficulties with the administration of the measure that could be improved. Following this discussion a number of changes were made to the CRS including the use of a separate colour- coded competence scale template ( see 137 Appendix F ) and a component identification guide ( see Appendix G ) or note page with specific spaces for information required from therapy s essions to complete the CRS. Also changes were made to the instructions of the CRS which included additional information to the introduction and definitions and examples of each conceptualization component above the associated item to assist ratings. The raters then had an additional two hour training session. One female rater from the previous training did not continue at this point and dropped out. Changes from the last training were explained and discussed during the first 15 minutes of training. The raters then rated a second DVD recorded therapy session. The final 15 minutes to an hour was spent discussing ratings made from the session and the utility of changes made to the CRS. A final training session was held to incorporate a final change including the additional guidelines for the rating of question four in the CRS. Raters were again given an opportunity to discuss these changes for the first 15 minutes, rated a DVD recorded therapy session, and discussed the ratings and changes for the remainder of the two hour training. A summary of the training process is outlined below in Figure 25 . In total, over the course of the initial training, site one raters used the CRS to rate three DVD recorded therapy sessions from the CBT Depression Study. In total 13 hours of group training was administered. The initial four participants received nine hours of training. The three participants that continued the rater training received a further four hours of training. In the second round of CRS training, two raters from La Trobe University (site two) were trained by replicating the initial training procedure in Melbourne. Raters also attended three blocks of training sessions. The training was provided solely by 138 the Dr. Nikolaos Kazantzis however in this second round of training the author was not present. Although raters were not required to provide feedback on the CRS measure for the purposes of changing the measure, they were able to provide feedback for discussion to improve their understanding of the CRS. In tot al six raters were trained in the use of the CRS for the rating of psychotherapy sessions as these ratings apply to CBT protocols. Four of these raters (two from site one, two from site two) continued beyond training to provide ratings to be included in the final data set of the current project in studies used in the later part of Study 2 and S tudy 3 . 139 8.6 Measure The CRS is a 12 - item measure with four domains for each item ( see Appendix E ). The first domain, the “integration” of CC into the therapy session adopts a dichotomous ‘yes or no’ format. The second domain , the “importance” also uses a dichotomous ‘yes o r no’ format. The third domain “competence” uses a 7 - point L ikert scale. The fourth domain the “fit” uses a 6 - point Likert scale. In order to make a complete rating the CRS must be used in conjunction with additional resources included in the CRS rating package. These include: - CRS - Competence Scale Template (see Appendix F) - CRS - Component Identification Guide ( see Appendix G ) - Photocopies of therapist’s completed conceptualizations, in the context of this data set, th e J. Beck CCDs based on the according rated therapy sessions 8.7 Statistical Analyses Treatment of Data D ue to the small sample of sessions used in the initial training of independent raters and the development of the CRS less inferential analyse s were able to be conducted than those conducted for the total sample of ratings. In light of this relative limitation inherent in using percentage agreement type statistics (La schinger, 1992) , reliability analyses of the training sample data were used as a guide for the development of the CRS rather than to draw any broad inferences about the ability of raters to produce consistent ratings using the CRS. Also, the complete sample also facilitated the analysis of reliability at each session in order to draw inferences about the relative reliability of ratings using the CRS over time across therapy sessions. 140 Furthermore, for the total sample Cronbach’s Alpha was used as a measure o n internal consistency (Cronbach, 1951) . It should be noted that ICC (3, k) reported for continuous data is equivalent to Cronbach’s Alpha coefficient (α) as a measure of internal consistency (Shrout & Fleiss, 1979) when applied to the consistency of items as opposed to that of pairs of raters. Treatment of Categorical Data In order to approximate clinical utility and to assess reliability for the purposes of the current study ratings of sensitivity and specificity were calculated based on a small sample of three DVD recorded therapy sessions carried out in the initial training at site one. This was carried out for the “importance” and “integration” categorical questions for all items on the CRS measure (Gerardi, Keane, & Penk, 1989) . This preliminary analysis was important for identifying problematic items on the CRS and providing a rationale for the thought process behind changes made to the final version of the CRS measure. For the total sample of ratings made using the CRS , reliability was able to be assessed using Cohen’s Kappa ( κ; Cohen, 1960). Cohen’ s Kappa is considered to provide a conservative estimate of reliability suitable for use with categorical data which provides a calculation incorporating “observed” and “chance” values to provide a more accurate estimate of reliability (Sim & Wright, 2005) . Treatment of Continuous Data In order to assess the overall reliability of “competence” and fit or “match” questions (continuous variables) for the CRS measure was calculated using SPSS version 17.0 for each DVD recorded therapy sessions rated. Shrout and Fleiss (1979) 141 identify a number of different forms of ICCs that can be used for analyses depending on the form the data takes and inferences the researcher intends to make based on the analysis. The IC C (3, k) was selected to reflect a limitation in making gross generalisations from pairs of raters, in contrast, to the possible use of numerous expert raters where results could be generalised further (Laschinger, 1992). In practice, however, comparisons of alternate forms of ICCs suggested minimal variations from what was reported in the total sample. For the training sample, i n order to assess reliability at the individual item level, visual analysis of data was undertaken to determine any systematic bi as due to the small amount of data used. For the total sample, individual item ICCs were calculated to determine rates of agreement among ratings of individual items on the CRS (McGraw & Wong, 1996 , 1999 ) . For example, where small numbers of observations were gathered (i.e., approximately 35 per training session) differences in individual items were apparent from direct observation. When the total sample of observations available was used (i.e., 225 in total) more inferential statistics were able to be used and interpreted more readily. Furthermore, a descriptive comparison between ‘surface’ level and ‘deeper’ level components of CC (e.g., competence in therapists discussion and eliciting of ‘surface’ level situations, beliefs and thoughts, versus, ‘deeper’ level core beliefs, conditional assumptions and compensatory strategies) was undertaken where previous research has found discrepancies or differences between more or less inferential components of CC (Kuyken et al., 2005). 142 8.8 Results – Training Sample The following section provides the interrater reliability for scores from the initial training of raters based on the first three therapy sessions rated. Discussions with raters and observations of estimates of reliability scores were used to guide the development of the final version of the CRS and justify the continued use of the final version of the CRS in the current project. Interrater Reliability Integration The “integration” question data were categorised to assess type I (false positive) and type II (false negative) error rates for each rater and as a means to make predictions about the reliability of the integration question and CRS measure in general. Where consensus was reached among the majority of raters (i.e., three out of four) it was a ssumed that this was the “true” score. The results of the initial pilot training ratings produced 47 “hits” or true values and one “m iss” or false value (see Table 6 ), reflecting a specificity of 98%. The results of the second pilot training ratings produced 36 “hits” or true values and zero “miss es” or false values (see Table 7 ), reflecting a specificity of 100%. DVD recorded therapy sessions were chosen where therapist integration of components was present and observable to optimise training by providing examples of what integration of particular techniques looked like in live therapy sessions. However, without the presence of true negatives in training session one and two the sensitivity of the CRS measure could not be assessed (i.e., correctly identif ied instances where case conceptualization components were not integrated into therapy) . To amend this, t he third and final pilot training DVD recorded therapy session included some example of 143 the absence or “non - integration” of conceptualization component s. The results of the third and final pilot training ratings produced 36 “hits” or true values and included 30 true positives and 6 true negatives (see Table 8 ), reflecting a specificity of 100% and a sensitivity of 100%. The data did not reflect any systematic “misses” for particular items across the three sessions rated in the training ratings. These scores reflect excellent rates of agreement for categorical answers pertaining to the appropriate integration of conceptualization components by therapists. Table 6 Integration of conceptualization component for training session one Rater – I Rater - II Rater – III Rater – IV False positive 0 0 0 0 False negative 1 0 0 0 True positive 11 12 12 12 True negative 0 0 0 0 Table 7 Integration of conceptualization component for training session two Rater – I Rater – II Rater – III False positive 0 0 0 False negative 0 0 0 True positive 12 12 12 True negative 0 0 0 144 Table 8 Integration of conceptualization component for training session three Rater – I Rater – II Rater – III False positive 0 0 0 False negative 0 0 0 True positive 10 10 10 True negative 2 2 2 Importance As conducted with “integration”, t he “importance” question data were categorised to assess type I (false positive) and type II (false negative) error rates for each rater and as a means to make predictions about the reliability of the importance questio n and the CRS measure . Again, where consensus was reached among the majority of raters (i.e., three out of four) it was assumed t hat this was the “true” score. The results of the initial pilot training ratings produced 45 “hits” or true values and three “misses” or false values (see Table 9 ), reflecting a specificity of 94%. The results of the second pilot training ratings produced 35 “hits” or true values and one “miss” or false value (see Table 10 ) , reflecting a specificity of 97%. DVD recorded therapy sessions were chosen where the integration of components was present and observable to optimise training by providing examples o f is considered important and how importance might be observed in live therapy sessions. As with the third training session used to assess ‘integration’ of CC components , t he third and final pilot training DVD recorded therapy session used to asses ‘import ance’ included some example s of the absence or “non - integration” of conceptualization components. The results of the third and final pilot training ratings produced 35 “hits” or true values 145 and one “miss” and included 32 true positives, one false negative and 3 true negatives (see Table 11 ) , reflecting a specificity of 100% and a sensitivity of 97 %. These scores reflect excellent rates of agreement for categorical answers pertaining to the assessment of importance of CC components by therapists. Table 9 Importance of the integration of conceptualization component for training session one Rater – I Rater – II Rater – III Rater – IV False positive 0 0 0 0 False negative 1 1 0 1 True positive 11 11 12 11 True negative 0 0 0 0 Table 10 Importance of the integration of conceptualization component for training session two Rater – I Rater – II Rater – III False positive 0 0 0 False negative 0 1 0 True positive 12 11 12 True negative 0 0 0 146 Table 11 Importance of the integration of conceptualization component for training session three Rater – I Rater – II Rater - III False positive 0 0 0 False negative 0 1 0 True positive 1 1 10 1 1 True negative 1 1 1 Therapist competence Ratings of competence for the overall CRS reflected a general inc rease in reliability scores (see Table 12 ) . Ratings of therapy session one yielded the lowest overall reliability score for “competence” ( ICC (3, 4) = .67). For the second session rated the reliability score increased (ICC (3, 3) = .79). The third session rated reflected the highest reliability score and a steady increase in of scores as more training was administered to therapists (ICC (3, 3) = .92). Confidence intervals for the ICCs became smaller from the initial training session to the final training session (See Table 12 ). Although initial confidence intervals were large (ICCs range from .19 to .89 ) the final ICC estimate produced a smaller confidence interval demonstrating that for the final rating it can be said with 95% confidence that estimates of scores for competence would produce ICCs between .78 and .97 . 147 Table 12 Intraclass Correlation Coefficients of competence for rated therapy sessions at each training session Therapy Session (Training) ICC (3, k) Lower Upper 1 (Initial) .67 .1 9 .89 2 (Second) .79 .44 .93 3 (Final) .92 .78 .97 Note. “Lower” and “Upper” refer to the boundaries of the 95% confidence interval Visual analysis of training session one at the item level suggested that raters had particular difficulty assessing competence for item 11 (patient strengths). Also items nine (compensatory strategies) and 12 (sharing of the conceptualization) suggested problems at the individual item level reflected in outlying ratings by individual raters. Further vi sual analysis of individual items in training rating sessions one, two and three suggested that there were no systemic problems at the item level with ratings tending to be within one point of other raters scores. Competence scores were on average in the moderate range ( Scores out of ‘ 7 ’ on ‘ 0 ’ - ‘ 6 ’ Likert scale). Competence scores tended to be higher in ‘surface’ level CC components and lower in ‘deeper’ level CC components (see Table 1 3 ). Scores show that whilst therapists are discussing important conceptualization components consistently, they do not consistently elicit patient feedback on CC information gathered (i.e., about clinical relevance AND / OR acceptance AND / OR understanding). Also whilst they consistently link ‘ surface’ level CC components, ‘ deeper’ leve l CC components are not consistently linked. 148 Table 13 Comparison of “surface” level and “deeper” level average competence scores Therapy Session (Training) Competence Average Score - “Surface” Level Components Competence Average Score - “D eeper” Level Components 1 (Initial) 4.35 3.85 2 (Second) 4.40 4.75 3 (Final) 3.66 2.99 Total 4.14 3.86 Note. Higher amounts represent a higher quantity and quality discussion on links between aspects of CC as well as increased amounts of feedback sought by the therapist as rated on a 6 - point Likert scale with descriptive anchors Fit / match between written and spoken case conceptualizations Ratings of fit or “match” for the overall CRS reflected fluctuations in reliability scores and identified a need for increased training and structure in this question (see Table 14 ). Ratings of therapy session one yielded an excellent reliability score (ICC (3, 4) = .81). However the large confidence interval asso ciated with this ICC (range .50 to .82 ) suggests that although the ICC estimate is very high it is possible that the true estimate for session one was considerably lower. For session two the reliability score was very poor (ICC (3, 3) = .20). Large confidence intervals associated with the session two ICC was also observed (range .44 to .93) . For session two J. Beck CCDs used included blank spaces, where as in training session one the J. Beck C CDs used was fully completed. Confusion among raters on how to rate the fit of incomplete or blank boxes in the conceptualization diagram was identified. After clarification and additional rater training the third session rated reflected the highest reliability score (ICC (3, 3) = .90). 149 While c onfidence intervals for the ICCs fluctuated from the initial training session to the final training session (See T able 14 ) , ICC estimate s for the final rating training session produced the smallest confidence interval suggesting that it can be said with 95% confidence that estimates of scores for co mpetence would produce ICCs between .74 and .97 . Of note, the ICC estimates produced in the second training session shows a large confidence interval (range - 1.12 to .75) which reflects poor reliability. Thus, prior to the final training session the reliab ility was poor between raters. Worthy of explanation, the negative lower bound of the confidence interval produced is difficult to interpret since negative ICCs are produced as a result of small sample size or due to “random disturbance” or “sampling error ” as reliability estimates approach zero and where sampling variability is high (Magnusson, 1966; Nichols, 1999). Table 14 Intraclass Correlation Coefficients of fit or “match” for rated therapy sessions at each training session Therapy Session (Training) ICC (3, k) Lowe r Uppe r 1 (Initial) .81 .50 .82 2 (Second) .20 - 1.12 .75 3 (Final) .90 .74 .97 Note. ” Lower” and “Upper” refer to the boundaries of the 95% confidence interval Visual analysis of training session one at the item level suggested that raters also had particular difficulty assessing for item 11 (patient strengths) where similar difficulty was had in assessing “competence”. Further visual analysis of individual 150 items in training rating sessions one, two and t hree suggested that there were no systemic problems at the item level with ratings tending to be within one point of other raters scores. Fit or “match” scores were in the moderate range ( Scores out of ‘ 6 ’ on ‘ 1 ’ – ‘ 6 ’ Likert scale). Fit or “match” scores tended to be consistent across ‘sur face’ level and ‘deeper’ level CC components (see Table 15 ). Scores reflect the level of completeness as well as quality of information contained in written J. Beck CCDs completed by therapists in relation to information generated and gathered in session. For illustration, the majority of the scores lie close to a ‘4’ rating. According to the guidelines “ If half or over just over half of the information is present then a ‘4’ rating would be selected”. These results suggest that whilst therapists are discussing important conceptualization c omponents consistently they are not necessarily taken written record of information considered to be important and worthy of discussion in session. Table 15 Comparison of “surface” level and “deeper” level average fit or “match” scores Therapy Session (Training) Fit / Match Average Score - “Surface” Level Components Fit / Match Average Score - “Deeper” Level Components 1 (Initial) 4.7 0 4.11 2 (Second) 3.67 3.67 3 (Final) 3.66 3.52 Total 4.01 3.77 Note. Higher amounts suggest a better “fit” between the content of therapist’s written and spoken CCs rated on a standard 6 - point Likert scale 151 8.9 Results – Total Sample The following results present estimates of interrater reliability for scores produced by the use of the CRS based on the total sample of patients ( N = 28; see Chapter V I ) and the total sample of ratings carried out of the course of the current project ( N = 225 ). Thus, t wo rater pairs scored a total of 225 sessions (159 from site one, 66 from site two; see Study 3 for a comparison of sites ). The larger sample used facilitated the examination of individual item reliability scores which were not deemed to be meaningful in the previous section due to the small training sample used. As introduced previously, kappa and ICC coefficients are used as the primary estimates of reliability in the present study (Shrout & Fliess, 1979). In order to interpret the magnitude of both kappa and ICC statistics C icchetti (1994) has proposed generally accepted “rules of thumb” or guidelines whereby coefficients of .75 or above are considered excellent, .60 and above good, and .40 and above fair. It should be noted that while kappa is applied to response- level variables the ICC is applied to dimensional variables at the level of the entire measure. In this way ‘average measure’ estimates are compu ted (Viglione & Meyer , 2007 ). In contrast, kappa coefficients target pairings of individual ratings where as average measure ICCs assess the overall number of ratings produced for each specific code. This can be compared to the means by which the reliability of parallel forms of a single test is assessed (Bakeman & Gottman, 1997). For this reason , although kappa and ICC magnitudes are reasonably comparable, it would be expected that total measure kappa statistics would be generally lower than ICCs given that a measure was consistent across domains for which kappa and ICCs are computed. 152 However, SPSS also calculates an ‘ind ividual- measure’ ICC (McG raw & Wong, 1996, 1999) . Bearing this in mind, b oth average- measure and single- measure total scale ICCs are reported for continuous data (i.e. competence and fit / match) since the interpretation can have potential implications for the generalisability of each coefficient. To gain more information on response- level or the agreement for individual items, a verage- measure ICCs are r eported for individual- items on the CRS . Single measure individual - item estimates reflect the degree to which an individual rater would be able to reliably produce a score for a particular item and bears on the ability to generalise scores of a single item. Average- measure individual- item estimates reflect the degree to which raters ‘on average’ would be able to produce reliable scores for an individual item. It is intended that the CRS be used by single supervisors or observers in practice. For these purposes the single measure ICCs should guide the interpretation of the overall total- scale scores. Ho wever, a verage- measure ICCs are sufficient to draw inferences about the reliability of individual items. Lastly, reliability estimates a reported for individual therapy sessions to providing an indication of possible changes in the reliability of the CRS at differ ent therapy sessions. Internal consistency Cronbach’s Alpha coefficients were computed to determine the internal consistency of each domain on the CRS to provide an indication of the full scale reliability of the overall measure. Cronbach’s Alpha coeffic ients for integration, importance, competence and fit / match were .72, .66, .79 and .87 respectively. High Cronbach’s Alpha scores suggested that total scale scores for each domain were a good indicator of individual items in each domain. 153 Interrater Reli ability In general each domain (i.e., integration, importance, competence, and fit / match) produced good to excellent reliability and was observed to be consistently reliable across items and across therapy sessions. However, the ‘importance’ domain generally had lower reliability than other domains. Ratings from site one (Massey University) and site two (La Trobe University) were screene d and observed to be comparable (see Study 3 for statistical comparison of sites). Table 16 displays inter - rater reliability across items on the CRS. Table 17 displays inter - rater reliability over time and across sessions which is discussed for each domain. Integration Cohen’s Kappa coefficients for total and individual- item estimates of scores were excellent (see Table 16 ). Indiv idual item ICCs ranged from .66 (item 11) to .87 (item 12) . The total estimate ICC for the integration domain was .83 . ICCs for ‘integration’ scores were demonstrated to be consistent across therapy sessions (see Table 1 7 ) ranging fro m .7 8 (sessi on 5 ) to .8 5 (session s 3 and 4 ). Importance Cohen’s Kappa coefficients for the total estimate of scores were adequate (see Table 16 ). However, individual - item Cohen’s Kappa coefficients were variable (range from .12 for item 11 to .66 for item 8 ) . In pa rticular, Cohen’s Kappa for items 4 (emotion; k = .17) , 10 (disorder - specific model; k = .15 ) and 11 (patient strengths / resilience; k = .1 2) were poor. The total estimate ICC fo r the importance domain was .65 . ICCs for ‘i mportance’ scores were demonstrat ed to be consistent across therapy 154 sessions (see Table 1 7 ) ranging from .63 (session 8 ) to .7 5 (session 6 ). However, the one exception to this was an ICC of .36 at session 10. Therapist Competence ICCs for total and individual- item estimates of scores were excellent (see Table 16 ). Individual item ICCs range d from .76 (item 9) to .99 (item 12) . The total estimate ICC for the ther apist competence domain was .93 . ICCs for ‘competence ’ scores were the highest of all the domains and demonstrated to be consistent across therapy sessions (see Table 1 7 ) ranging from .91 (session s 5 and 7 ) to .96 (session 1 ). However, t he one exception to this was an ICC of .64 for session 8. Fit / match ICCs for total and individual- item estimates of scores were excellent (se e Table 16 ). I ndividual item ICCs ranged from .68 (item 10) to .96 (item 12) . The total estimate ICC for the fit / match domain was .86 . ICCs for ‘fit / match ’ scores were demonstrated to be consistent across therapy sessions (see Table 1 7 ) ranging from .7 7 (session 9 ) to .9 0 (session 3 ). 155 Table 16 Individual item and total scale reliability estimates for the CRS Integration Importance Competence Fit / Match 1. Situation 0.79 0.32 0. 91 0. 87 2. Automatic thoughts 0 .82 0. 45 0.8 9 0.91 3. Meaning of ATs 0 .66 0. 46 0. 81 0.7 7 4. Emotion 0 .80 0.17 0. 88 0. 69 5. Behaviour 0 .73 0.2 9 0. 82 0.7 6 6. Relevant Childhood data 0 .80 0.5 9 0. 90 0 .85 7. Core beliefs 0 .7 1 0. 60 0. 91 0.8 5 8. Conditional Assumptions 0 .77 0. 66 0. 90 0. 77 9. Compensatory Strategies 0 .73 0. 42 0.76 0. 69 10. Disorder - specific model 0. 7 9 0.15 0. 94 0. 68 11. Resilience / Strengths 0. 66 0.12 0. 88 0.8 9 12. Sharing of overall CC 0.87 0.40 0.99 0.96 Total Estimate 0.8 3 0. 65 0.93 (0.87 ) 0 .86 (0.75 ) Note. Integration and Importance are reported as Cohen’s Kappa, Competence and Fit / match reported as ICC s. Individual item estimates are reported as average measure ICCs, total ICCs are reported as average measure (and single measure) estimates. 156 Table 17 Session specific reliability estimates for the CRS Session Number Integration Impor tance Competence Fit / Match 1 0.8 3 0.6 4 0.96 0.8 4 2 0.79 0.64 0.92 0.8 1 3 0.85 0. 70 0.94 0.9 0 4 0.8 5 0.6 5 0.94 0.8 4 5 0.7 8 0.71 0.91 0.87 6 0.81 0.7 5 0.94 0.8 5 7 0.83 0. 60 0.91 0.8 0 8 0. 80 0.6 3 0.64 0.8 5 9 0.8 3 0. 64 0.9 2 0. 77 10 0.8 3 0.36 0.93 0 .85 Note. Integration and Importance are reported as Cohen’s Kappa, Competence and Fit / match reported as ICC s. 157 8.10 Discussion Conclusions from Study Two The present study provides a description of the development of the first measure of therapist competence targeting CC suitable for use in CBT. There were two overarching goals of the rater training to a) provide training for the raters for future use of the CRS in and b) to study pilot test the CRS to provide preliminary data on the development of the measure. In particular it wa s hypothesised (hypothesis two ) that the preliminary data would reflect “coherence” of CC demonstrated by acceptable internal reliability and inter- rater reliability. It has been suggested that CC in CBT is “theoretically c oherent” where, for example, the J. B eck CCD provides a potential framework for experimental investigation given the structure it provides for links between different aspects of CC (Kuyken et al., 2005). It might be further hypothesised that this coherence would be demonstrated over time. In the present study the CRS provides a systematic method for assessing structured CC information consistent with the categories defined within the J. Beck CCD. The results from the present sample suggest that the CRS is i nternally consistent. While reliability scores were variable and improved over the course of training, a t the completion of training inter- rater reliability scores were excellent. The reliability scores obtained in the present study reflect the ability of independent raters to agree on aspects of therapist use of CC during therapy sessions and how in- session content is related to formal written CCs. Moreover, the present study provides empirical support for the theoretical coherence of CC based on live therapy sessions. However, agreement rates surrounding the “importance” of different aspec ts of CC were considerably lower than other domains. In particular, raters were observed to have the lowest rates of agreement on the importance of conceptualizing patie nt 158 strengths and resilience. It is possible the lack of clarity between raters reflects a general lack of understanding and emphasis on patient strengths in CBT for depression. Alternatively, raters may have benefited from additional training on the importance of conceptualizing patient strengths in CBT for depression. Furthermore, prior research has suggested that practitioners or raters reach higher rates of agreement on less inferential, descriptive or “surface” level CC components but are less able to agree on more inferential or “deeper” level components of CC (Bieling & Kuyken, 2003). However, no clearly discernable pattern emerged suggesting that raters were more or less able to reach agreement on particular levels of the CC across the four domains on the CRS. This suggests that the structure provided by the CRS is likely to facilitate agreement on overtly observable behaviour related to CC. A second facet of hypothesis two concerned the levels of reliability estimates for different domains over the course of therapy. Prior research examining the psychometric properties of measures of therapist competence and related domains has not generally investigated the variation in reliability estimates over time. Using the CRS, it was dem onstrated that raters were able to maintain high rates of agreement throughout the early and mid phase of CBT. This pattern of highly reliable estimates of therapist competence was observable across each of the four domains of the CRS . Generalization of Findings Prior research has identified that there is currently a paucity of across sites reliability data for measures of therapist competence (Barber et al., 2007). The current study provides data from independent raters at two sites: 1) Massey University in New Zealand and 2) La Trobe University in Aus tralia. The results of Study 2 provide 159 an example of multi - site research that has produced consistently high interrater reliability scores. This finding has implications for the use of the CRS suggesting that it is suitable for use both in New Zealand and Australia, and provides positive support for the use of the CRS internationally. Further research might address the suitability of the use of the CRS internationally and in different sites. Limitations The present study is limited by the modest number of patients included in the sample, only intended to provide preliminary indications of psychometric properties of the CRS measure for further development. In contrast, the number of individual session ratings ( N = 2 25 ) was comparable or greater than the number of ratings conducted in previous studies of independently rated measures of therapist competence ( Bennet & Parry, 2004; Blackburn et al., 2001; Connolly Gibbons, et al., 2003; Vallis, Shaw, & Dobson, 1986; Young & Beck, 1980). The present study is further limited due to the use of postgraduate- level independent observers. Research has demonstrated that expert therapists are generally more skilled in constructing CCs and perhaps better able to make judgement regarding what makes a quality CC than experienced or novice therapists (Eells et al., 2005). In contrast, the use of well- trained raters and a structured and systematic approach to CC has been advocated as a potential means to evaluate CC in CBT (Bieling & Kuyken, 2003; Kuyke n et al., 2005) as has been used implemented in the present study. Accordingly, the high rates of agreement in the present study suggest that through sufficient training independent observers considered to possess “novice” level experience are able to reac h agreement regarding different aspects of CC when using the CRS as a systematic and structured approach to CC in CBT. 160 Summary The present study describes the development of a reliable method for assessing the utility of both written and in- session CCs. In part, the purpose of developing a reliable measure of CC was to facilitate validity testing in Study 3. Further to this, t he results of Study 2 suggest that independent observers are able to agree on the extent that CBT therapists utilize CCs during liv e therapy sessions and how in- session CC content relates to formal written CC. The CRS was found to demonstrate excellent reliability both across different items, as well as over time (i.e., across therapy sessions). This might suggest that the CRS is able to be used at any point in the early to mid phase of therapy (i.e., first ten sessions) which were investigated in this sample. Generally h igh rates of agreement provide support for the further use of the CRS to gather data on the psychometric properties of the CRS based on further ratings using the CRS as well as use in clinical practice and supervision. Though the use of larger patient samples taken from more diverse patient populations, more robust conclusions about the psychometric properties of the CR S can be made. Preliminary results indicate that the CRS is a reliable indicator of CBT therapists use of CC. Study 3 builds provides psychometric evidence bearing on the validity of the CRS in light of reliable estimates obtained in the present study. In particular, Study 3 provides a detailed analysis of therapist competence in CC as defined in the CRS to better understand the relationship between therapist competence in CC, homework use and outcomes related to CBT for depression. 161 CHAPTER IX Study Three - The Relationship between Therapist Competence in Case Conceptualization and Therapist Competence in Homework in CBT for Depression 9.1 Outline and Aims Therapist competence is considered to possess a strong association with psychotherapy outcome. The fundamental premise of administering CBT for depression relies upon the reduction of depressive symptoms as a direct result of psychotherapeutic intervention. However, little is known about the relationship between different domains of therapist competence and their relative contribution to reduction in depressive symptoms in CBT for depression. Therapist competence in CC and in- session and between session treatment planning (i.e., homework) , are two central domains of therapist competence considered to impact psychotherapy outcome. In chapter two, it has been identified that research to date has focused on written CC and failed to find a relationship between CC and psychotherapeutic outcome. Such research has focus ed on the “quality” of CCs as a means to approximate “real life” in - session therapist competence in utilizing CC. Just like the “quality” of a therapist’s or patient’s written outline of proposed homework tasks is not likely to be synonymous with a therapist’ s competence in integrating homework collaboratively into a therapy session, nor are formal written CCs likely to fully capture that ability, skill or competence of a therapist in developing and utilizing CC over the course of therapy. Furthermore, r esearch investigating therapist competence in homework has advanced considerably but produced mixed results regarding the relationship between aspects of homework use and various psychother apeutic outcomes as an important mechanism of change in CBT for depression (Kazantzis et 162 al., 2005) . The present study extends prior research based on written CCs through the use of the Conceptualization R ating Scale 2 (CRS; Easden & Kazantzis, 2008; 2009) as a structured measure of in- session therapist competence used to model the utility of CC as a naturally occurring phenomenon. The present study has particular importance for psychologists practicing in Aotearoa / New Zealand where competence in the area of CC is required but methods suitable for the measurement of therapist competence for supervision and ongoing professional development do not exist. No study to date has assessed the magnitude of therapist competence in CC in CBT in relation to therapeutic outcome. In order to achieve this broad aim Study 3 begins by providing a brief overview of multilevel modelling as the primary means of statistical analysis used in the present study and provides a rationale for its use for data in the CBT Homework Project. Second, a model for prediction of therapist competence in CC and homework is specified for empirical testing. Third, the treatment and screening of data is described as well as the checking of model assumptions. Fourth, descriptive data bearing on the level of therapist competence across time and correlations are presented. Finally, the results of a multilevel m odel for change in depressive symptoms in relation to therapist competence in CC and homework use is provided and discussed. 2 Preliminary psychometric data for the Conceptualization R ating S cale (CRS) was presented at the 36 th and 37 th Annual Conferences for the British Association of Behavioural and Cognitive Therapies (BABCP) in Edinburgh, Scotland ( 2008) and Exeter , United Kingdom (2009). 163 9.2 Statistical Analyses S tructure of Data The CBT Homework Project data set involves a number of different units of analysis. For example, data can be examined at the level of the therapist, patient, or therapy sessions over time. Both within subject and between subject variation are of interest. For example, in examining therapist competence in case conceptualization both the level of competence displayed by an individual therapist and how this relates to an individual patient outcomes (e.g., change in depressive symptomology) and aggregate competence levels for all therapists in relation to overall patient outcomes. In this way, the CBT Homework Project data is “nested” or multilevel data ( Raudenbush & Bryk, 2002 ). More specifically, repeated observations (i.e., therapy sessions) are nested within persons (i.e. therapist and patient dyads). In order to determine which variables (i.e. the therapist, patient or sessions) account for hypothesised change this so called “causal entanglement” must be addressed (Elkin, 1999; Krause & Lutz, 2009). Thus, the data lends to a multilevel analytical strategy that simultaneously analyses relationships that occur within subject s and between subject s at different level of analysis. In contrast , traditional OLS approaches have been demonstrated to be inadequate compared to multilevel analyses ( Lambert, Doucette, & Bickman, 2001; Nich & Carroll, 1997; Hedeker, 2004). Traditionally psychotherapy data has been analysed using statistical methods that aggregate scores without consideration of within subject effects when individual therapist or patient trajectories of measurement scores are often the major point of interest beyond mean scores totalled across all subjects. In contrast, multilevel analysis requires the construction of “person period” dataset which analyses individual subject change trajectories (Singer & Willett, 2003). For example, Hed eker 164 (2004) provides a reanalysis of previous research examining individual patients’ response to antidepressant medications. Using a traditional MANOVA analysis not only reduced the sample size by a third did not reveal the difference between patient response to medication where an average patients responded well while some individual patients had poor responses to medications. Defining Multilevel Analysis Analysis of multilevel data takes on a range of different names in published literature. These include, for example, random - effects regression models, mixed regression, mixed effects, multilevel model ling, hierarchical linear mode lling, and Bayesian estimation for linear models. For the purposes of the current project analysis will be referred to under the general term of ‘multilevel mode lling’. Sample Size One advantage of multilevel modeling approaches is that the analysis is flexible enough to allow for unbalanced designs and missing data. The practical advantage in a psychotherapy data set is that ‘intent - to- treat’ samples can be analysed allowing the utilization of all cases and existing data rather than having to exclude cases. For example, in the current study follow - up data was able to be used despite only some of the patients completing follow- up sessions at 2 - months post- treatment and even less of the patient’ s completion follow- up sessions at 6 - months post- treatment. However, caution has been advised when using small sample sizes for multilevel modeling approaches (Hox, 2002; Kre ft et al., 19 96; Maas & Hox, 2 005 ). These recommendations suggest that for a two- level model such as that used in the 165 present thesis, a minimum of 30 groups are necessary with 30 observations per group. In samples with 30 groups for higher level groups such as in the p resent thesis it is important to give consideration to the size of standard errors of the estimates. In a simulation study, Maas and Hox (2002) found that with 30 groups, the standard errors are estimated about 15% too small, resulting in a non - coverage rate of almost 8.9%, compared to a rate of 5% for groups with closer to 100 participants . What is suggested is that the patient sample size for the current study ( N = 28 ) which represents the ‘group’ with 10 observations for each group (i.e., repeated measur es of therapy session at 10 time points) is considered a small, sample for multilevel modeling. However, recommendations do not preclude multilevel modeling from being carried out in the present study. Similar concerns regarding quantitative analysis of a small sample size remain for utilization of traditional G eneral L inear M odel (GLM) approaches. Thus, it can be concluded that the benefits of a multilevel approach outweigh the possible risks. Statistical Software A range of statistical software has been produced for the analysis of multilevel data change over time (Bryk & Raudenbush, 1987 ; Jennrich & Schluchter, 1986; Johnson et al., 1992; Jones & Boadi - Boateng, 1991; Rutter & Elashoff, 1994). For the purposes of the current project the program SPSS MI X E D Version 17 was chosen. SPSS MI XE D Version 7 was chosen because: 1) it is the most recent version of SPSS and likely to have the most accurate computations, 2) SPSS is a commonly used program in the social sciences likely to be more familiar to psycholog ical researchers than other popular MLM capable software programs and, 3) in order to run an analysis using SPSS MI XE D a single “person period” data set must be created. Unlike 166 some software programs, for example HLM ( Raudenbush, Bryk, Cheong, & Congdon, 2 002 ) it is not necessary to create individual ‘Level 1 ’ and ‘Level 2 ’ data files using an alternative statistical software package. In general, a lthough the intricacies of sorting the data files and running the analysis are different and perhaps more complex than traditional non - parametric approaches, a brief explanation is sufficient for social science researchers to understand the major results of the analyses (e.g., significance of within subject effects). Evaluating a Multilevel Modelling Approach It i s postulated that traditional fixed effect approaches such as ANOVA Multiple Regression , and methods based on general linear model (GLM) methods often utilized in psychotherapy research is done so out of convention, rather than methodological rigor (Singer & Willett, 2003) . In contrast, multilevel analyses estimate random effects in additional to fixed effects to enable the analysis of both average and person- specific trends (Hedeker, 2005). Multilevel analysis has a number of benefits relevant to psycho therapy research or clinical trials. A number of advantages of multilevel modeling have been advocated in research including use with longitudinal and missing data ( Hedeker & Gibbons, 1997; Nich & Carro ll, 1997) , inclusion of random effects and alternative variance and co- variance structures to better understand individual and group change across time (Hedeker, 2004; Kwok, et al, 2008). Moreover, it is possible to conduct post hoc “r eanalysis” of previously analysed nested data sets, such as the data set generated in the CBT Homework Project, using appropriate multilevel analyses over and above previously conducted standard General Linear Model (GLM) analyses such as Multiple Regression and 167 ANOVA methods . Multilevel methods have demonstrated the superiority over aggregated conventional analyses to detect previously undetected relationships among variables (Nich & Carro ll, 1997; Hedeker, 2004 ). For example, Nick and Carroll (1997) reanalysed a clinical trial on psychotherapy and medication for Cocaine abusers comparing tradition ANOVA analysis with multilevel analytical methods. The authors were able to use the full intent- to- treat sample for their multilevel analysis while only 20% of the sample was able to be analysed using traditional ANOVA analysis due to m issing data. 9.3 Specifying the Model Fixed and Random Effects The first consideration is to determine which variables will be modelled as “fixed” effects at Level 1 or as “random” effects at Level 2 . Essentially for the present data set, “fixed ” effects were obtained when coefficients for variables were constant (i.e., time - invariant, categorical data) for individual patients. “ R andom” effects were obtained when the coefficients for variables were allowed to vary over the course of therapy (i.e., time - variant, continuous data) sessions within patients (Bryk & Raudenbush, 1998; Snijd ers & Bosker, 1999 ) . For example, gender is modelled as a fixed effect given that the coefficient is held constant over therapy sessions where individual patients are either male or female over the course of therapy and is therefore considered a random, L evel 2 , variable. Conversely, therapist competence in case conceptualization in its raw form is continuous data that is variable over the course of therapy for individual patients and is therefore modelled as a random effect at L evel 1 . However, it could potential ly be modelled as a fixed effect at L evel 2 by recoding the data into high and low competence (i.e., ‘0’ and ‘1’) or any number of 168 different categorical divisions that might make the analysis more meaningful. However, to maximise the amount of variability in the model, therapist competence in both case conceptualization, and homework were modelled at as a random effect at L evel 1 . Does Therapist Competence Predict Outcom e? In order to examine the relationship between therapist competence in case conceptualization, therapist competence in homework use and outcome a multilevel analysis using SPSS MI XE D version 17 was conducted. A number of different measures were used as controls in investigating the bulk of the hypothesis in the present thesis towards testing of hypothesis nine culminating in the final model. To this ends, t he BDI - I I was used as the dependent variable to estimate reduction in patient depressive symptomology. Specifically, a n index of BDI - I I change was used generated by subtracting patients’ raw BDI - I I scores at each session from their initial BDI - I I score assessed at intake. It was important to use an index of change in contrast to a raw index of cumulating change to control for differences in initial patient symptom change. Each of the variables specified were integrated at different stages into the tentative MLM . The Unconditional Model The purpose of the unconditional model (Model 1 ) is to assess whether or not the baseline for the proposed MLM (i.e. BDI - I I scores) contain enough variance to justify an MLM analysis and to provide an indication of where this variance is best explained, either within patients or between patients . For the current project thi s involved modelling BDI - I I scores without the inclusion of any predictors. 169 The Unconditional Growth Model The unconditional growth model (Model 2 ) introduces time , as a L evel 1 covariate ( Session) , to model the trajectory for proposed MLM without any substantive predictors. In this way the growth trajectory for repeated measures ( Session) are modelled upon individual patient depression score ( BDI_Change) trajectories. The primary purpose of the unconditional growth model is to determine whether or not the proposed model has enough variance to justify an MLM analysis across time. The MLM growth model for the current project is given below: Level 1: yit = π0i + π1i (Sessioni) + eit Level 2: π0i = β00 + r0i π0i = β10 + r1i Combined: yit = β00 + β10 ( S essioni) + r0i + r1i (Sessioni) + eit The Conditional Growth Model The conditional growth model (Model 3 onwards) introduces substantive predictors hypothesised to model change of BDI - I I scores across therapy sessions. As stated Study 3 focuses on therapist competence in CC and hypothesises that at L evel 1 therapist competence in CC ( CRS_Comp) will independently predict reduction in depressive symptomology ( BDI_Change) across therapy sessions (hypothesis eight) . Model 4 introduces therapist competence in the integration of homework ta sk s ( HAACS_Comp) as a further predictor hypothesised to be related to relief of depressive symptomology. Any additional variance explained by therapist competence in the integration of homework bears on the relationship between 170 therapist competence in CC and homework being investigated in the current project (hypothesis nine). Furthermore, the fit / match between written and in- session discussion of CC as measured on the CRS was used as a predictor variable to determine whether the extent to which a link between what therapists report they wil l do as indicated by their written conceptualizations in addition to the relationship that is captured by independent observation of therapist competence in CC (hypothesis three) . At L evel 2, it is hypothesised a further reduction in depressive symptoms will be explained after controlling for depressive symptom severity ( Symp_Sev) and the complexity of patient presentations as indicated by individual patient beliefs associated with personality disorders ( PBQ_Complexity) . The Level 1 and L evel 2 equations for the current project are given below: Level 1: yit = π0i + π1i (Sessioni) + π2i (CRS_Comp1) + π3i (HAACS_Comp2 ) + π4i (HRS_Beliefs3 ) + π5i (CRS_Fit3 ) + eit Level 2: π0i = β00 + β01 (Symp_Sevi) + β02 (PBQ_Complexityi) + r0i π0i = β10 + β11 (Symp_Sevi) + β12 (PBQ_Complexityi) + r1i Combined: yit = β00 + β01 ( CRS_Compi) + β02 ( CRS_HAACSi) + β03 (HRS_Beliefs3) + β04 (CRS_Fit3 ) + β05 ( Symp_Sev i) + β06 (PBQ_Complexityi) + β10 ( Sessioni) + β11 ( CRS_Compi*Sessioni) + β12 ( HAACS_Compi*Sessioni) + β13 ( HRS_Beliefsi*Sessioni) + β14 ( CRS_Fiti*Sessioni) + β15 ( Symp_Sevi*Sessioni) + β16 ( PBQ_Complexityi*Sessioni) + r0i + r1i ( Sessioni) + eit Where yit represents BDI - I I scores as the dependent variable as change in single units as rated on the BDI - II compared with individual patient BDI - I I scores 171 recorded at initial intake assessments. Session represents therapy sessions where each measurement was taken . CRS_Comp, HAACS_Comp, HRS_Beliefs, and CRS_Fit , represent variables measured at each therapy session modelled as a continuous, time variant, function. Symp_Sev and PBQ_Complexity represent time- invariant variables that are constant across therapy sessions for each individual patient. 9.4 Treatment of Data Before multilevel analysis can take place it is necessary to screen or prepare the data and code or transform both the general data arrangement and the individual variables into an appropriate format for running the multilevel analysis. Data Coding Consideration was given to the coding of each variable included in the analysis for Study 3. For o bservational interrater data (i.e., the CRS and HAACS) where two scores were generated for each observation, the average score for each pair of ratings was selected to represent the true score. This was done in consideration of high interrater reliability score obtained in Study 2. All remaining data was treated as continuous time series data to maximise variation within the data set and to allow for the possibility of differences in the level of variables over time (i.e., L evel 1). The exception to this was the treatment of symptom severity (i.e., CIDI) and personality beliefs (i.e., PBQ) which were intended as controls were made to re main constant over time (i.e., L evel 2). Specifically, the CIDI generates four possibilities for indications of symptom severity associated with depression: “no diagnosis” and “mild” (coded as ‘0’) , “moderate” (coded as ‘1’) and “severe” (coded as ‘2’). The PBQ generates values associated with lesser or greater degrees of disordered 172 personality beliefs and patient complexity. Total scale persona lity beliefs scores were divided evenly between the lower and higher 50 th percentile (coded as ‘0’ and ‘1 ’ respectively) splitting patients into one of two categories (i.e., high or low personality beliefs. This splitting of the total scale PBQ score also coincided with recommendations for measuring observable differences in personality beliefs in patients with features of personality disordered beliefs (Beck et al., 2001). However, there is no formal cut- off score for diagnosing personality disorders using the PBQ. Centering In order to make multilevel output more easily interpretable it is common practice to adopt “ centering” or rescaling of variables (Peugh & Enders, 2005). However, a number of considerations must be made when choosing when and how to centre variables (Singer & Willet t, 1998) . For the present thesis, session data was coded starting from ‘0’ from the initial intake session by default. As a result ‘time’ was centred on ‘0’ and results are interpreted relative to the intake session representing the initial status in the MLM analysis conducted in Study 3 . D ata for the dependent variable and predictor variables were measured on reasonably comparable scales from the outset reducing the utility of centering data. For example, both HAACS and CRS scores representing therapist competence were measured on 7 - point scales with average scores of 49.08 and 24.75 a standard deviation of 14.37 and 9.87 respectively. The restriction and similarity in the range of the scales supports the plausibility of analysing the data as a natural metric due to them already being centred to each other in relative terms (Kreft, de Leeuw, & Aiken, 1995). Moreover , the use of non- linear time ( Session; see below) in the unconditional growth model changes the parameters for which to base interpretations. This restricts the utility of centering 173 confounding the interpretation of the magnitude of fixed effects based on raw data (Singer & Willet, 2003 ). Missing Data As introduced in section 9.2, MLM is able to accommodate missing data. In particular, in the analysis for the present study, this was done using Maximum Likelihood (ML) estimation. Simply put, ML estimation is a more computationally intensive alternative to OLS and related methods that utilises all available data to estimate relationships between variables based on probabilities ( Aldrich, 1997 ; Singer & Willet, 2003 ). Kwok et al., (2008) explain further: An advantage of MLM is that it can make use of all available data in the estimation of model parameters due to its flexible treatment of the time predictor. A research participant with only baseline data can be included in an analysis and contribute to the estimation of model parameters. The validity of using all available data does depend on whether missing data are missing completely at random (MCAR) (or missing at random (MAR) which is a less restrictive missing data assumption), and methods of assessing this requirement are available . (p.4) . None - the- less, for observational interrater data where a single rating was missing, the score generated by the remaining rater was used to make maximum use of all available data. This was done in consideration of high interrater reliability obtained in Study 2. In order to further ensure that the remaining missing data was not due to any systematic problems in the data collection process, the data was analysed 174 using a Missing Completely at Random (MCAR; Rubin , 1976; Little, 1988; Little & Rubin, 1987) analysis in SPSS missing values analysis based on a “trimmed” version of the data set that did not include BDI - I I , and session data after session 11 in order to run the analysis in the traditional manner. All variables used in the analysis were included in the analysis. H owever, the variable CRS_Fit was excluded from the total missing variables analysis due to known systematic difference in the collection of the data, namely that J. Beck CCDs (J. Beck, 1995) needed for ratings to be collected were systematically completed at specific sessions (see Chapter VII I) . This limitation in the inclusion of CRS_Fit in MLM analyses was considered and discussed in section 9.7. The MCAR analysis produced non - significant results. The non- significant result of the MCAR analysis ( 2 = 27.095, df = 28, p = .513) revealed that there was no systematic pattern of missing data in the dataset. In this way, missing data for the purposes of MLM analysis can be considered ignorable and meets the assumptions associated with missing data (Sch afer, 1997). Figure 26. Residual plot of depressive symptom change scores ( BDI change) 175 Treatment of Time Time was represented by therapy session in the current study. However, there were a number of different ways that therapy sessions could possibly be coded to most accurately model change in depressive symptomology. Table 28 (see Appendix L ) provides an analysis of the random effects produced for different methods of coding time as well as an indication of which model of time provides the best fit to the dependent variable (i.e., change in depressive symptomology). The most simple form for modelling time is to use the raw “wave” of data increasing in equal increments ranging from ‘0 ’ representing t he initial intake session to ‘10’ representin g the last session included in the analysis (i.e., session ten) . In this case the wave was equivalent to coding time as a linear metric (T1) in subsequent analyses including predictor variables. However, during the first four weeks of beginning active ther apy, sessions or dose were administered twice per week then continued at once a week from week five. In order account for this, it was considered that time could be modelled at shorter intervals during the first 8 waves of data (i.e., the first four weeks of therapy; T2 ). In this ‘0 ’ represented intake, ‘0.5’ session one, ‘1’ session two, ‘1.5’ session three, ‘2’ session four, etc, and from session five began to increase increments of one. A third possibility for modelling time would be to use a non- linear function. From visual analysis the dependent variable data ( i.e., BDI - II change scores) it was posited that the change in depressive symptom change could be curvilinear (see Figure 26 ). Subsequently, a curvilinear function was also considered as a potentia lly accurate means for coding time in subsequent analyses (T3). As in T2, an adjusted model for curvilinear time taking into account the increased frequency of sessions in the first four weeks of therapy was also considered (T4). 176 A fifth possibility (T 5 ) arose for modelling time by coding the precise time (days) since intake. Although, t reatment was intended to be delivered at systematic increments (i.e. twice per week for the first four weeks, then once per week until completion, then follow up sessions at two and six months) treatment often did not occur as intended. For example, patients would at times miss a week of therapy or come in for therapy more or less frequently than intended. To account for these individual patient differences in the timing of therapy, the precise days between one therapy session to the next were coded. For example intake session was coded as ‘0’. If a patient came in seven days later the second session would be coded as ‘7’. Having coded time in several distinct ways, the results of a random effects analysis revealed that M odel T1 explained 47% and T5 explained 46% of the variance in depressive change scores accounting for the most within patient variance on in BDI - I I change scores (see Table 28; Appendix L ) . Model T4 explained t he least within in patient variance (38% ). Between patient v ariance explained ranged from 50% (T4) to 72 % (T 2 and T3 ). However, while Model T4 explained modest amounts of variance in comparison to the other models, Model T4 obtained superior rates of significance relative to the amount of variance explained. Moreover, the lack of significance obtained for models in non- curvilinear conditions at L evel 2 might restrict the ability to introduce and interpret variables at L evel 2 using linear metrics of time. S imilarly, Model T4 had smaller standard errors relative to the amount of variance explained which would support the use Model T4 for modelling time. Furthermore, Model T5, although explaining a substantial amount of variance was considered from a theoretical point of view to introduce too many unknown sources of error regarding why patients did and did not attend sessions at particular times and thus was excluded for consideration as a coding of time. W hile the results are 177 comparable the magnitude of the result provides some evidence to suggest that T1 and T2 provide a better overall fit to the model than T4 (Raferty, 1995). Never - the- less, having been satisfied that the data can be observed to fit a fairly curvilinear structure (see Figure 31; Appendix L ), and in consideration of relatively smaller standard errors and higher levels of significance associated with the introduction of non- linear functions, a conservative choice was made in the modelling of time (Singer & Willet, 2003). Thus, a curvilinear func tion (T4) was selected for use in subsequent analyses to represent Session as the most accurate coding of time to model the dependent variable ( BDI change) in subsequent analyses. 9.5 Evaluating the Tenability of Model Assumptions The assumptions underlying the proposed “final model” were checked consistent with recommendations outlined by Singer and Willet (2003). Checking Functional Form Visually inspecting the shape of OLS - estimated individual change trajectories informed a) the suitability of diff erent models or coding of time ( Session) , and b) the feasibility of hypothesised relationships between Level 1 and L evel 2 predictors and the dependent variable. For the 28 patients who participated in Study 3, the hypothesised curvilinear structure of time ( Session) in relation to the BDI_Change was plausible and thus introduced into the model (see Fig ure 32 ; Appendix M ) . Furthermore, following screening of BDI_Change variable it was noted that outliers were present in patient 69 resulting in an irregula r data spread compared to other patients. The viewing of DVD recordings of patient 69 revealed an elevated 178 level of risk which impac ted the usual course of therapy and it was decided to exclude session 10 for patient 69 from the analyses in Study 3. Check ing Normality In order to check the assumptions of normality, the raw residual distributions were examined for each predictor (or control) and dependent variable used in Study 3 . For example, Figure 26 depicts the residual spread of data to indicate normality where a greater fit to the linear function greater normality. The normal probability plots for both Level 1 and L evel 2 residuals demonstrated a reasonable fit to the line and thus meet the assumption of normality. Checking Homoscedasticity In order t o check the assumptions associated with homoscedasticity, the raw residuals were plotted against predictors. The results of examining the plots reveals that each predictor variable has a relatively even residual spread around ‘0’ in relation to the dependent variable ( BDI_Change) representing an absence of heteroscedasticity. For example, residual plots for therapist competence in CC ( CRS_Comp) demonstrate a desirabl e residual spread (see Figure 27 ). In contrast, BDI_Change in relation to time reflected the potential presence of heteroscedasticity in the lower end of the graph representing a potential lack of variance in this range. However, due to the small sample size and expectation that data reflecting depressive symptom change will change initially then remain high, it is logical that plots would take this form. Thus, it can be considered that the assumptions surrounding homoscedasticity are met. 179 Figure 27. Residual plot of scores for therapist competence in case conceptualization ( CRS comp) in relation to scores for change in depressive symptoms ( BDI change) Treatment Fidelity An important consideration related to the fundamental assumption of the present thesis, was to ensure that therapists involved in the CBT Homework Project were carrying out a course of therapy consistent with the general CBT model (i.e. treatment fidelity, adherence to the CBT treatment manual; see sections 4.5 and 5.4). For this purpose the CTS was used to screen or ensure therapist’s general competence to the CBT mod el (see Chapter VI) , while the focus of the present study was therapist competence in CC and homework as particular areas of interest. Furthermore, the limited variation in CTS scores within and between therapists and limited number of observations made for each therapist prevented the CTS from being included as a meaningful variable in statistical analyses. 180 Treatment of Sites An independent t- test was conducted to compare site one and two for any systematic differences in therapist competence scores as the primary variable of interest measured on the CRS . The results did not reveal a significant difference between site one (M = 24.23 , SD = 10.65) and site two (M = 25.32, SD = 9.48) mean competence scores on the CRS ; t(227) = - 7.31, p = .46 . Similarly, t he results did not reveal a significant difference between site one (M = 45 .2 1 , SD = 13.86) and site two (M = 50.53, SD = 14.27 ) competence scores on the HAACS ; t( 224 ) = - 2.50 , p = . 13. Accordingly, competence scores from site one and two were combined for subsequent analyses in Study 3. 9.6 Results – Descriptive Statistics of Observed Data I t is important to examine the raw data over time. Of particular importance is the ‘level’ and ‘shape’ of therapist competence in CC as a focal point in the present thesis consistent with an approach taken by Svartberg (1999) in the investigation of raw therapist competence data. It has been suggested that little data is available on the level and shape of therapist competence across therapy sessions (Butler et al., 2 007). By examining the level of therapist competence in CC particular areas of therapist strength or deficit are able to be identified. By examining the shape of the data we are able to form preliminary hypotheses about the function of the data over time. For example, from visual observation of the graphs we are able to suggest that scores for therapist competence in CC have a linear or curvilinear function. Similarly, by observing the shape of the scores for change in depressive symptomology, the dependent variable of interest and therapist competence in homework use as a second 181 focal point of the present thesis it is possible to arrive at similar hypotheses regarding the function the variable takes. Levels of Therapist Competence in Case Conceptualizatio n Table 18 shows raw and average levels of therapist competence in CC demonstrated for each session for individual patients. Competence scores observed for each patient ranged from 0 to 55 ( M = 25.14 , SD = 9.78 ). On average, all therapists were observed to be highly competent in integrating components of CC into therapy sessions. On average t herapist competence in CC increased over the course of therapy with each individual patient (see Figure 28 ) . Figure 28. Linear regression of therapist competence scores in case conceptualization as measured on the CRS for individual patient by therapy session 182 Table 18 Mean scores and descriptive statistics for therapist competence scores in case conceptualization for each patient on the CRS by session Patient Session 1 2 3 4 5 6 7 8 9 10 57 17 .0 0 23.5 0 26 .0 0 30 .0 0 24.5 0 18 .0 0 26.5 0 27 .0 0 22 .0 0 23.5 0 63 19 .0 0 27 .0 0 30.5 0 33.5 0 29.5 0 13.5 0 5 .00 29 .0 0 40.5 0 21.5 0 66 2.5 0 15.5 0 1 .0 0 28 .0 0 32.5 0 40 .0 0 33 .00 19 .0 0 31 .0 0 32.5 0 69 4.5 0 18 .0 0 6.50 26.50 13.0 0 35.50 25.50 19.50 84 7 .0 0 22 .0 0 33 .0 0 24 .0 0 25 .0 0 31 .0 0 26 .00 28.5 0 23 .0 0 31 .0 0 106 3 .0 0 111 24.5 0 19.5 0 21 .0 0 27.5 0 26.5 0 26.5 0 31 .00 29 .0 0 33 .0 0 38.5 0 113 19.5 0 25.5 0 19.5 0 21.5 0 18 .0 0 15.5 0 28.5 0 28.5 0 35 .0 0 15.5 0 116 26 .0 0 14.5 0 18 .0 0 18 .0 0 24 .0 0 21.5 0 30.5 0 19.5 0 34 .0 0 22 .0 0 133 22 .0 0 24.5 0 0 .0 0 29.5 0 32 .0 0 34 .00 35.5 0 31 .0 0 27 .0 0 43.5 0 143 24.5 0 3.5 0 18.5 0 20 .0 0 30.5 0 35.5 0 26.5 0 36 .0 0 46 .0 0 29 .0 0 160 37 .0 0 164 3 .0 0 2 .0 0 21 .0 0 0 .0 0 20 .0 0 25.5 0 165 20 .0 0 23 .0 0 13.5 0 39.5 0 29 .0 0 33 .00 35 .0 0 30.5 0 18 .0 0 39 .0 0 169 16 .0 0 17.5 0 24 .0 0 29.5 0 26 .0 0 22 .00 43 .0 0 33 .0 0 37 .0 0 16 .0 0 188 40 .0 0 34 .0 0 37 .0 0 37 .0 0 34.5 0 23.5 0 31.5 0 50 .0 0 50.5 0 195 14.5 0 32 .0 0 206 25 .0 0 5.5 0 17.5 0 31.5 0 27.5 0 19 .00 27 .0 0 29.5 0 24.5 0 13 .0 0 218 22.5 0 22.5 0 24 .0 0 21.5 0 31.5 0 28 .0 0 30 .0 0 28 .0 0 43.5 0 220 18.5 0 0.5 0 28 .0 0 10.5 0 21.5 0 31.5 0 30.5 0 41.5 0 30 .0 0 44 .0 0 223 25.5 0 14.5 0 36 .0 0 236 29 .0 0 9 .0 0 13 .0 0 18 .0 0 25.5 0 24 .0 0 11 .0 0 28 .0 0 26 .0 0 244 18.5 0 24.5 0 20.5 0 8.5 0 26 .0 0 35 .00 23. 5 0 24 .0 0 29.5 0 26.5 0 247 0.5 0 19.5 0 262 31.5 0 23 .0 0 26 .0 0 29.5 0 25.5 0 16.5 0 20 .0 0 38 .0 0 22 .0 0 271 26.5 0 33.5 0 35.5 0 35 .0 0 30.5 0 35 .00 32 .0 0 34 .0 0 33.5 0 55 .0 0 273 26.5 0 16.5 0 21 .0 0 12.5 0 25 .0 0 25.5 0 17.5 0 33.5 0 25.5 0 20.5 0 295 13 .0 0 13 .0 0 11 .0 0 32 .0 0 34 .0 0 33.5 0 21 .0 0 24.5 0 28.5 0 28 .0 0 Mean 18.9 0 18.2 0 19.5 0 24.0 0 26.7 0 27.3 0 27.5 0 28.1 0 31.8 0 30.5 0 SD 9.8 0 9.1 0 9.6 0 10.4 0 5.1 0 7.7 0 7.8 0 6.7 0 7.7 0 11.9 0 Note. Missing data represent s sessions where competence scores were not gathered 183 Levels o f Therapist Competence in Homework Integration Table 19 shows average levels of competence in homework integration across the first ten therapy sessions for individual and total item scores on the HAACS. Total competence scores ranged from 18 to 80.5 ( M = 49.13 , SD = 14.38 ). Therapist competence in Homework In tegration was variable over the course of therapy for across different patients (see Figure 29 ). Figure 29. Linear regression of therapist competence scores in homework use as measured on the HAACS for individual patient by therapy session 184 Table 19 Mean scores and descriptive statistics for therapist competence scores in homework use for each patient on the HAACS by session Patient Session 1 2 3 4 5 6 7 8 9 10 57 17.5 0 45.5 0 48 .00 3 4.5 0 43 .00 32 .00 36.5 0 40.5 0 52.5 0 36 .00 63 55 .00 45 .00 43.5 0 72 .00 28 .00 54.5 0 68 .00 59 .00 58.5 0 30.5 0 66 22.5 0 49 .00 62.5 0 55 .00 54 .00 57.5 0 38 .00 48 .00 46 .00 53.5 0 69 42 .00 37.5 0 53 .00 48 .00 36.5 0 34.5 0 41 .00 18 .00 84 50.5 0 61.5 0 65 .00 65 .00 67.5 0 60.5 0 66.5 0 57 .00 64.5 0 67.5 0 106 45.5 0 111 35.5 0 51 .00 46.5 0 36.5 0 39.5 0 39.5 0 28.5 0 46.5 0 33.5 0 66 .00 113 31 .00 31 .00 33 .00 35.5 0 49 .00 35 .00 23.5 0 28.5 0 28 .00 35.5 0 116 43 .00 37 .00 45 .00 40.5 0 54.5 0 38.5 0 51 .00 54 .00 65 .00 36.5 0 133 44 .00 57 .00 63 .00 23 .00 73 .00 61.5 0 61.5 0 62.5 0 26.5 0 143 25 .00 34.5 0 27.5 0 35 .00 32 .00 46 .00 39 .00 38.5 0 31 .00 38 .00 160 46.5 0 164 42 .00 40.5 0 51 .00 55 .00 36.00 39.5 0 165 71.5 0 72.5 0 63.5 0 72 .00 72 .00 66 .00 60 .00 67 .00 69 .00 169 50 .00 49.5 0 67 .00 62.5 0 65 .00 65 .00 61.5 0 61 .00 67.5 0 56 .00 188 63 .00 63 .00 38 .00 56.5 0 52 .00 50 .00 54 .00 64 .00 45.5 0 195 41.5 0 60.5 0 206 72 .00 45.5 0 39 .00 61 .00 52 .00 43.5 0 41 .00 60 .00 30.5 0 40 .00 218 28.5 0 59 .00 59.5 0 40.5 0 54 .00 36 .00 51.5 0 58 .00 51. 5 0 220 62.5 0 55.5 0 68.5 0 66.5 0 54 .00 72 .00 61 .00 80.5 0 69 .00 70 .00 223 27.5 0 44 .00 22 .00 236 45 .00 37.5 0 32.5 0 60 .00 63 .00 46.5 0 62 .00 48 .00 60.5 0 244 42.5 0 55.5 0 41 .00 42 .00 55.5 0 38.5 0 37.5 0 20 .00 23.5 0 247 41.5 0 35.5 0 262 33 .00 50 .00 50.5 0 33.5 0 50.5 0 46.5 0 46 .00 39.5 0 34.5 0 271 68 .00 54.5 0 66.5 0 53 .00 78.5 0 76 .00 70.5 0 52.5 0 48 .00 78.5 0 273 74.5 0 68 .00 58 .00 67 .00 77.5 0 59.5 0 57.5 0 67 .00 39.5 0 57 .00 295 43 .00 52.5 0 44.5 0 24 .00 42 .00 46.5 0 36 .00 29 .00 21 .00 22 .00 Mean 45.2 0 49 .4 0 50.7 0 49.0 0 52.2 0 51.8 0 47.0 0 50.6 0 46.5 0 48.6 0 SD 16.6 0 10.5 0 11.9 0 15.5 0 15.1 0 12.5 0 14.3 0 14.4 0 16.2 0 16.6 0 Note. Missing data represent s sessions where competence scores were not gathered 185 Levels of Depressive Symptom Severity Depressive symptom s everity improved over the course of therapy for each individual patient. Figure 30 provides a graphical summary of individual linear regressions for each individual patient. Figure 31 , displayed previously, provides a graphical summary of overall mean traj ectories of depressive symptom change as measured on the BDI - I I in relation to therapist competence in CC (CRS) and therapist competence in homework use (HAACS). Figure 30. Linear regression s of BDI - I I change scores for individual patient by therapy session 186 Figure 31. Mean centred data trajectories for therapist competence in case conceptualization, homework use and depressive symptom change Levels of Fit / Match between Written and In - session Case Conceptualization In add ition the levels of fit between therapist written case conceptualizations using the J. Beck CCD format with in session content of CC was considered as measured on the CRS . Total fit / match scores ranged fr om 8.5 to 61.5 ( M = 29.58 , SD = 11.36 ). There was greater variation in the CRS fit / match than in the CRS competence scores for individual patients (see Figure 35 and 36; Appendix M). 187 Table 20 Intercorrelations among predictor, control and dependent variables 1 2 3 4 5 6 7 8 9 10 11 12 1 CRS int egration - 2 CRS importance .769** - 3 CRS competence .807** .699** - 4 CRS fit / match .271** .272** .455** - 5 HAACS adherence .041 - .032 .106 .131 - 6 HAACS competence .084 - .002 .186** .193** .803** - 7 PBQ - .061 - .064 - .025 .078 - .049 - .062 - 8 HRS - I I (c lient beliefs) - .043 .030 - .058 .000 - .057 - .031 -.255** - 9 CIDI (symptom s everity) - .049 - .0 1 5 - .059 - .043 - .008 .041 .176** .162* - 10 CTS .784 - .382 .349 .182 .408 .315 - .260 - .201 .060 - 11 BDI raw -.149* -.142* -.165* -.166* - .044 - .166 .335* .036** .240** .383 - 12 BDI change .138* .173** .142 * .067 .062 .075 - .061 - .080 .440** - .533 -.440** - Note. * p < .05, * * p < .01. Symptom Severity was measur ed on the CIDI. PBQ total scale ‘global’ scores were used as an indication of patient complexity and lev el of disordered personality beliefs. HRS - I I therapist version was used to measure patient beliefs. 1 88 Correlations among Predictor, Control and Dependent Variables Intercorrelations were calculated for different variables included in the present study. Table 20 shows intercorrelations calculated for predictor and control variables as well as the dependent variable of interest ( BDI - II ) in order to provide an indication of the direction (i.e., positive or negative association) of relationships hypothesised between variables in the present study. As expected the results showed a positive relationship between all the scales on the CRS (range r = .271 to r = .807, p < .01) and HAACS adherence and competence scales ( r = .803, p < .01). Also as expected, a negative albeit weak correlation (i.e., an increase in competence was associated with a decrease in BDI raw scores) was found between CRS competence scores and BDI raw scores ( r = .222, p < .01). Similarly, a positive weak correlation between CRS competence and BDI raw scores ( r = - .149, p < .01) and positive weak correlation between CRS competence scores and BDI change scores ( r = .138, p < .01). However, while HAACS competence scales were observed to follow a similar trend to CRS scales, the HAACS competence was not observed to have a significant correlation with BDI raw or change scores. Also, positive correlations were observed between CTS and both CRS competence ( r = .784, NS) and HAACS competence ( r = .31 5, NS) . Unfortunately the low numbers of observations used to generate correlations for CTS scores resulted in non- significant findings. Higher scores (i.e., more complex or severe patients) on both the PBQ and CIDI Symptom Severity scales respectively wer e associated with lower therapist competence on both the CRS and HAACS competence scales. PBQ scores had a positive correlation with BDI raw scores ( r = .345, p < 0.01). While CIDI Symptom Severity scores had a positive correlation with BDI raw scores ( r = .240, p < 0.01) and also had a positive correlation with BDI change scores ( r = .440, p < 0.01) 189 where BDI change has also controlled for the initial symptom severity. HRS - I I scores were negatively associated with PBQ scores ( r = - .255, p < 0.01) and CIDI Symptom Severity scores ( r = .162, p < 0.05). However, PBQ and CIDI Symptom Severity scores were also positively correlated ( r = .176, p < 0.01). Finally, BDI Change scores (i.e., an index of positive symptom change or patient improvement after controllin g for initial symptom severity; see Chapter VI for an explanation ) had a negative correlation with BDI Raw scores (i.e., an index of negative symptom change or decreasing depression levels) consistent with what might be expected ( r = - .440, p < 0.01). Table 21 Descriptive statistics for the individual growth parameters obtained by fitting separate within-person OLS regression models for BDI_Change as a function of curvilinear time Initial Status (intercept) Rate of Change (slope) Mean 8.47 0.16 Sta ndard Error 1.06 0.04 Bivariate Correlation 0. 34* Note. * p < .05, ** p < .01. 190 Exploring the Heterogeneous Growth Model to Formulate Questions about Change Having explored raw cross correlations between all the variables in the sample it is important to consider the influence of time and the rate of change (i.e., acceleration or deceleration of scores over time) on primary variables of interest. By examining the a) intercept and b) slope of individual patients’ growth trajectories we are able to better formulate questions about change to be tested in the MLM. Initially, OLS estimates of intercepts and slopes were calculated at Level 1 for BDI_Change, the dependent variable and key indicator of out come in the sample (see Table 21 ). This provides an unbiased estimate of rates of change in depressive symptom change for individual patients in the sample as well as providing a sample mean showing the average change trajectory of all patients (Singer & Willett, 2003). The results of OLS estimates for the present sample produces an “initial status” which showed that patients had an initial change in depressive symptoms of 8.47 points which increased by .16 points each session. For example, the average patient had a depressive symptom change of 8.47. On ave rage, by session 10 BDI - I I scores improved by 10.07 (10 × .16). While this is a modest increase in BDI - I I scores, these results show that improvement in depressive symptom change is maintained as increases over the course of therapy for the average patient. Also, the correlation coefficient of p = .34 revealed a positive relationship between time (session) and depressive symptom change. Second, the impact of therapist competence in case conceptualization ( CRS Comp) and therapist competence in homework use ( HAACS Comp) were examined as the two primary variables of interest in Study 3. By exploring patterns of change in both therapist competence in CC and homework use a model for therapist competence in CBT for depression can be tested based on two central ar eas encompassing a 191 therapist’s demonstration of competence in formulation in - session and promoting between session activities across time. Table 22 provides bivariate correlation coefficients between intercepts and slopes produced by running OLS regression s between parings of BDI Change scores, time, and substantive predictor variables. The results produced a negative association between fitted initial status and fitted slope (rate of change) of CRS Comp ( r = - .82, p < .01) and Session. This suggests that therapists with higher initial competence tend to become more competent less rapidly over time (i.e., therapists with high competence remained competent over time with little increase in competence while therapists with a lower initial status in competence had a greater rate of change in competence over time). HAACS Comp ( r = - .17, p = NS) did not produce a significant association with time ( Session) . A strong negative relationship was produced between the fitted initial status and fitted slope between both CRS Comp ( r = - .96, p < .01) , HAACS Comp ( r = - .70, p < .01) and BDI Change. The negative correlations show that therapists with less change in competence (e.g., therapists that maintained high levels of competence across therapy sessions) had less change (or more consistent) amounts of change in depression scores . Again, for example, Figure 28 demonstrates that none of the therapists in the sample had trends showing a reduction in raw CRS Comp scores over time. This indicates that for this sample therapists maintaining high levels of competence maintained consistent depression change scores demonstrating maintenance of depressive symptom reduction over time while competence for therapists with initially low competence scores with greater change (i .e., accel eration over time) were related to greater rate of change in BDI Change scores. A strong positive relationship was found between the intercept and slope of CRS Comp and HAACS Comp ( r = .96, p < .01). This suggests that therapists with high rates of change in CRS Comp are likely to have greater rates 192 of change in HAACS Comp scores over time. The results suggest that a combined model including CRS Comp and HAACS Comp would provide and interesting avenue for investigating how two variables that potentially interact to increase the rate of change impact change in depressive symptoms over time. However, the results also suggest that individually CRS Comp and HAACS Comp are unlikely to be strongly associated with the rate of change of depressive symptom change due to the maintenance of depressive symptom change that takes place when competence remains stable as was often the case in this particular sample. Table 22 Correlations between intercepts and slopes for substantive predictors, BDI-II changes scores and time Constant Session CRS Comp HAACS Comp Constant - Session .34* - CRS Comp -.96** -.82** - HAACS Comp -.70** - .17 .96** - Note. p < .05, ** p < .01. 193 Order of Entry of Substantive Variables Having chosen to develop a combined mod el of therapist competence in CBT for depression, CRS Comp and HAACS Comp were selected as the first variables to be entered into the MLM (see Chapter V for a discussion on therapist competence). Second, beliefs about homework ( HRS Beliefs) are considered to greatly impact on the use of homework use even when therapists demonstrate high levels of competence in design, assign and review of homework (Kazantzis & Shinkfield, 2007) and were thus entered third into the MLM. The ability of the therapist to plan t herapy interventions within and across session based on collaborative CCs developed within and between sessions (i.e., written CCs) is considered to bear greatly on extent to which the therapist is able to demonstrate competence in CC and afford positive change in CBT for depression (Sudak , Beck, & Wright, 2003). Thus, t he fit or match between the written and spoken CC between patient and therapist in - session ( CRS Fit / Match) was introduced fourth to ensure discussion related to CC was integrated into therapy as intended by what was documented in written CCs. The use of the CRS Fit / Match also provides data to monitor how therapists gradually develop CCs over the course of therapy and control for the consistency of CC use over time. Finally the patient symptom severity ( CIDI Severity) and personality beliefs ( PBQ Beliefs) were entered into the model fifth and sixth as two crucial variables necessary to control for patient complexity which can also impact the extent to which a therapist competence can promote positive change across therapy. 194 9.7 Results – Development of a Multilevel Model for Change Change in Depression Tables 23 through 27 display the results of model building. Model 1 presents the unconditional means model. It is considered the ‘nul l’ model and acts as a baseline for depressive symptom changes since it does not include any predictors. Model 2 presents the unconditional growth model incorporating time (i.e., therapy session) into the baseline model for which to compare with predictors in subsequent models. A positive improvement in depressive symptoms ( BDI Change) was observed over the course of therapy for all patients. Improvement was defined as a reduction in depressive symptoms from the baseline total BDI - I I score assessed at intak e for each patient. Fro m the analysis of fixed effects in Model 1 it is estimated that patients had a non- zero pos itive BDI - I I score at intake ( 1 0 .78 , p > 0.01 ) and the introduction of fixed effects of time in Model 2 shows that BDI - II scores continued to have a non- zero positive change over time ( 8 .47 , p > 0.01 ) the significant but negligible rate of change (0.16 , p > 0.01) suggests that treatment gains were slightly more pronounced or accelerated over time. Although negligible, the significant finding mig ht suggest that further investigation into rates of change is warranted. After the inclusion of time, the analysis of random effects shows that a quarter of within patient variance in depressive symptoms is explained by time ( R2e = 25 % , p > 0.0 0 1 ) while the inclusion of time explains almost half of between patient variance ( R20 = 45 % , p > 0.0 0 1 ) . The significant results suggest that further variance is able to be explained by the addition of further predictor variables to the model. 195 Table 23 Estimates of fixed effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms Fixed Effects Parameter Model 1 Model 2 Model 3 Model 4 Model 5 Initial Status π0i Intercept γoo 10.78*** 8.47*** 5.73*** 9.65*** 7.40*** 1.33 1.06 1.42 3.05 2.48 CRS Competence γ01 0.11* 0.12* 0.05 0.06 HAACS Competence γ02 - 0.03 - 0.40 0.04 0. 42 Rate of Change π1i Intercept γ10 0.16* ** 0.28 *** 0.07 0.18~ 0.04 0.07 0.09 0.09 CRS Competence γ11 - 0. 00* - 0.00* 0.00 0.00 HAACS Competence γ12 0.00 0.00~ 0.00 0.00 Note. Model 1 is an unconditional means model. Model 2 is an unconditional growth mo del. Model 3 adds CRS Competence to the total model. Model 4 removes CRS Competence and adds HAACS Competence to the total model. Model 5 includes CRS a nd HAACS Competence. 196 Table 24 Estimates of random effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms Variance Components Parameter Model 1 Model 2 Model 3 Model 4 Model 5 Level 1 Within person σ2e 37.91 *** 28.60 *** 16.77 *** 28.41*** 16.66 *** 3.78 3.02 1.81 3.03 1.81 Level 2 In initial status σ20 42.65 *** 23.39 *** 23.52 *** 23.84*** 24 .20 *** 12.89 8.35 7.71 8.51 7.93 In rate of change σ21 0.61*** 0.51** 0.58*** 0.48** 0.20 0.20 0.21 0.20 Covariance σe 0.00* 0.02* 0.02~ 0.02~ 0.01 0.01 0.01 0.01 Psuedo R2 Statistics and Goodness of Fit R2e 0.2 5 0.41 0.00 0.42 R20 0.45 0.00 - 0.02 - 0.03 R21 0.16 0.11 0.11 Deviance 1537.64 1483.91 1349.76 1465.32 1330.55 AIC 1543.64 1495.98 1365.76 1481.32 1350.55 BIC 1553 .93 1516.56 1392.99 150 8.65 1384.44 Δ AIC 47.02 123.57 7.91 132.12 197 Therapist Competence in Case Conceptualization and Depressive Symptoms Model 3 presents the conditional growth model and introduces therapist competence in CC ( CRS Comp) into the mode l. An analysis of fixed effects after the introduction of therapist competence in CC revealed that for every one unit change on the BDI - I I, therapist in CC is estimated to change by 0.1 1 units ( p > 0.01 ) . An analysis of random effects revealed that inclusion of therapist competence in CC accounted for a 41 % reduction in within patient variance . Therapist competence in CC produced a negligible negative association ( - 0.00, p > 0. 05 ) with the rate of change of depressive symptom improvement. While a negative value might sugges t that impact of therapist competence in CC becomes less pronounced over time, the small value obtained from the present sample would not provide strong support for this notion. Therapist Competence in Homework and Depressive Symptoms Model 4 adds to the conditional growth model by further introducing therapist competence in homework integration ( HAACS Comp) into the model. Analysis of fixed effects revealed no direct significant association between therapist competence in homework use and positive change in depressive symptoms. For example, it is noted that although a small negative value was obtained ( - 0.03 , NS) the standard errors were larger than the obtained value (i.e., 0.04). However, analysis of random effects reveals that after controlling for time no reduction in within patient or between patient variance was observed. Therapist competence in homework use produced a negligible association with the rate of change of depressive symptom improvement. Although therapist competence in homework did not a ppear to be a strong predictor of positive change in depressive symptoms, the reduction in the Akaike Information 198 Criterion ( Δ AIC = 7.91 ) from Model 3 to M odel 4 suggested that therapist competence in homework use provided a better overa ll fit for in the prediction of positive change in depressive symptoms and should be retained in the subsequent models. A Model for Therapist Co mpetence and Depressive Symptom Change Model 5 represents the combination of therapist competence in CC and in homework use investigated individually in Models 3 and 4 . Subsequent variables (i.e., HRS Beliefs, CRS Fit / Match, CIDI Symptom Severity, and PBQ Complexity; see Section 9.3 ) introduced in to the model provide additional controls and are entered in simultaneously with the effects of the “combined ” model of therapist competence in CC and homework use (see Appendix M for a more detailed post hoc analysis of individual predictor impact on symptom change) . Analysis of fixed effects showed that in Model 5 it is estimated that patients had a non- zero positive BDI - I I score at intake which remained significant (7.40, p > 0.01 ) suggesting that the inclusion of additional predictors may be useful in further explaining variance in the final model. Analysis of fixed effect s reveals that the inclusion of therapist competence in homework in the model increases the magnitude of therapist competence in CC (0.12, p > 0.01 ) . Analysis of random effects explained more variance at Level 1 than previous models ( R2e = 41% , p > 0.01 ). Overall, the reduction in the Akaike Information Criterion ( Δ AIC =132.12 ) in Model 5 produced the largest reduction in comparison to previous models suggesting that a combined model of therapist competence in CC and homework use should be retained for further models. 199 Table 25 Estimates of fixed effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after the introduction of salient variables Fixed Effects Parameter Model 6 Model 7 Model 8 Model 9 Model 10 Initial Status π0i Intercept γoo 3.42 2.39 1.58 2.27 6.16 3.43 3.91 3.92 5.56 4.60 CRS Competence γ01 0.15** 0.20** 0.16** 0.18** 0.13** 0.06 0.07 0.06 0.07 0.05 HAACS Competence γ02 - 0.05 - 0.03 - 0.03 - 0.03 - 0.04 0.04 0.04 0.04 0.04 0.03 HRS Beliefs γ03 0.14 0.15 0.03 0.06 - 0.01 0.08 0.09 0.08 0.10 0.07 CRS Match / Fit γ04 - 0.03 - 0.02 - 0.02 0.06 0.05 0.06 CIDI Symptom Severity γ05 2.27~ 3.00 2.49~ 1.22 1.25 1.22 PBQ Complexity γ06 - 2.28 - 2.32 2.07 1.94 200 Table 26 Estimates of fixed effects for rate of change towards a final model from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after the introduction of salient variables Fixed Effects Parameter Model 5 Model 7 Model 8 Model 9 Model 10 Rate of Change π1i Intercept γ10 0.05 0.05 - 0.08 - 0.368 - 0.25 0.15 0.16 0.14 0.24 0.17 CRS Competence γ11 - 0.01** - 0.00* - 0.01 0.01 - 0.01** 0.00 0.00 0.00 0.00 0.00 HAACS Competence γ12 0.00 * 0.00 0.00~ 0.00~ 0.00* 0.00 0.00 0.00 0.00 0.00 HRS Beliefs γ13 0.00 0.00 0.00 0.00 0.00 0.0 0.00 0.00 0.00 0.00 CRS Match / Fit γ14 - 0.00 0.00 - 0.00 0.00 0.00 0.00 CIDI Symptom Severity γ15 0.12** 0.09~ 0.09* 0.04 0.05 0.04 PBQ Complexity γ16 0.09 0.06 201 Table 27 Estimates of random effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after the introduction of salient variables Variance Components Parameter Model 6 Model 7 Model 8 Model 9 Model 10 Level 1 Withi n person σ2e 16.61*** 17.31*** 17.40*** 29.54*** 17.09*** 1.82 2.08 2.09 3.48 1.90 Level 2 In initial status σ20 24.26** * 24.82*** 20.69** 15.88* 18.99** 7.94 8.44 7.52 7.16 6.93 In rate of change σ21 0.49* 0.49* 0.26~ 0 .59* 0.36* 0.19 0.21 0.16 0.29 0.14 Covariance σe 0.02* 0.02* 0.01** 0.02 0.01 0.01 0.02 0.00 0.00 0.01 Psuedo R2 Statistics and Goodness of Fit R2e 0.42 0.39 0.38 - 0.03 0.40 R20 - 0.04 - 0.06 - 0.03 0.32 0.19 R21 0.20 0.20 0.13 0.03 0.20 Deviance 1322.78 1158.33 1123.87 1165.80 113 2.13 AIC 1346.78 1186.33 1155.87 1189.80 1157.13 BIC 1387.39 1231.71 1207.31 1288.76 1208.66 Δ AIC 149.17 309.65 340.11 306.18 181.81 202 Homework Beliefs Model 6 adds patient beliefs about homework ( HRS Beliefs) at Level 1 into the overall model in relationship to change in depressive symptomology ( BDI Change) and as a control in relationship to therapist competence in CC and therapist competence in homework use . Analysis of fixed effect s revealed no significant relationship between patient beliefs about homework and initial status of depressive symptom severity. Patient beliefs about homework did not reveal a significant association with the rate of change of depressive symptom severity. Analysis of random effects showed that at Level 1 revealed that patient beliefs about homework explained 3% of within patient variance ( p > 0.01 ) . Analysis of fixed effects showed that the introduction of patient beliefs about homework as a c ontrol for therapist competence increased the predictive ability of therapist competence in CC (0.15 , p > 0.01) in relationship to initial status of de pressive symptom severity. No significant fixed effects of patient beliefs about homework were revealed. Analysis of random effects shows at Level 1 therapist competence in CC and homework after controlling for patient beliefs about homework explains 42% ( p > 0.0 0 1 ) of with patient variance associated with change in patient depressive symptom severity. Conceptualization Fit / Match between Therapist Spoken and Written CC Similarly, Model 7 continues to build upon the model by introducing the fit or match of the written and in session CC ( CRS Fit / Match) at Level 1 . Analysis of fixed effects showed that the introduction of CC fit / match as a control fo r therapist competence increased the predictive ability of therapist competence in CC (0.20 , p > 0.01) i n relationship to initial status of depressive symptom severity. However, when CC fit / match was introduced as an additional predictor as well as therapist 203 competence in CC and homework, only 39% of the variance associated with change in symptom severity suggesting that CC fit / match may not add substantively to the model as a predictor. This could perhaps be explained by the moderate significant correlation between CRS fit / match and CRS Comp ( 0.48, p > 0.01) . Symptom Severity Model 8 introduces CIDI s ymptom severity ( CIDI Symp Sev) at Level 2 into the model to control for patient complexity . An analysis of fixed effects after the introduction of patient symptom severity revealed that for every one unit change on the BDI - I I, patient symptom severity as measured on the CIDI is estimated to change by 2.27 units ( p > 0.10 ). Analysis of random effects revealed that inclusion of patient symptom severity accounted for a 29% reduction in between patient variance ( p > 0.05) . The amount of variance explained at L evel 2 suggests that CIDI Symp Sev should be retained in the final model. Personality Beliefs In order to control further for patient complexity, Model 9 introduces patient personality beliefs ( PBQ Beliefs) at Level 2 into the model. An analysis of fixe d effects revealed no significant effect. Analysis of random effects revealed that inclusion of patient personality beliefs accounted for a 6% reduction in between patient variance ( p > 0.01). The amount of variance explained at Level 2 suggests that PBQ Beliefs should be retained in the final model. Model 9 shows the results of including both therapist competence in CC and homework as well as controlling for patient personality beliefs and patient symptom severity. In total, Model 9 explains the most variance of previous models at Level 1 ( R2e = 40%, p > 0.01 ) and Level 2 204 ( R20 = 18% , p > 0.0 0 1 ) suggesting that patient personality beliefs and symptom severity should be retained in the model as a control for patient complexity. The Final Model Having ente red each variable systematically towards a ‘final model’ it was decided to retain the two major variables of interest in the present study, therapist competence in both CC ( CRS Comp) an d homework use ( HAACS Comp). Although CRS Fit / Match explained a consi derable amount of variance at Level 1, it did not appear to explain any variance beyond CRS Comp. Also, CRS Fit / Match presented potential confounds for inclusion in the model due to less observations of data compared to other variables which might have lead to an artificial inflation of variance explained and thus was not retained in the final model. Homework beliefs (HRS Beliefs) were retained as were Personality beliefs (PBQ Complexity) and symptom severity (CIDI Symp Severity) all of which provided significant reductions in variance associated with a relationship with change in depressive symptoms (BDI Change). Thus, analysis of fixed effects of the final model (Model 10 ) demonstrates that taken together, after controlling for salient variables that impact the ability of a therapist to display competence in CC and homework use during therapy, that for every one unit change on the BDI - I I, therapist in C C is estimated to change by 0.13 units ( p > 0.01 ; see Appendix L for further post hoc analysis of indiv idual variable affects). Therapist competence in CC ( CRS Comp) revealed a negative and significant rate of change, albeit producing a negligible magnitude ( - 0.01, p > 0.01). While symptom severity as measured on the CIDI ( CIDI Symp Sev) revealed a positive and significant rate of change (0.09, p > 0.05) demonstrating an acceleration of symptom improvement from initial status in relation to change in depressive symptoms. Model 205 10 explained the greatest amount of variance at Level 1 ( R2e = 40%, p > 0.01 ) and Level 2 ( R20 = 19% , p > 0.0 0 1 ). The reduction in the Akaike Information Criterion ( Δ AIC = 181.8 1 ) from the ‘baseline’ or unconditional growth model (Model 3) to Model 10 represents a “very strong” overall fit of the final model. 206 9.8 Discussion Conclusions from Study Three It has been espoused that “the core principle of case formula tion [CC] is that the treatment plans are a match for the specific client. That means that there is a connection between data, hypotheses, and plans” (Ingram, 2006 , p. 469 ). Ultimately, the primary goal of Study 3 was to assess the impact of this process in two central domains of therapist competence (i.e., CC and homework) on positive im provements depressive symptoms. However, prior research has suggested that a number of variables are likely to influence the relationship between therapist competence and outcomes in CBT for depression ( Kuyken & Tsivrikos, 2008 ) . More specifically, a number of variables are also likely to impact therapist competence in CC (e.g., personality beliefs that contribute to patient complexity; Beck, 2005) and therapist competence in homework (e.g., patient beliefs about homework; Kazantzis et al., 2005). Taking this into consideration, Study 3 cons idered the influence of a range of variables ensuring that variables that explained a substantial amount of variance in depressive symptom change were retained as controls in the final model. A second goal of Study 3 was to provide an indication of the level of t herapist competence across multiple therapy sessions (i.e., time series data) . The longitudinal MLM analysis conducted in Study 3 facilitated this goal where prior research has failed to investigate the relationship between therapist competence and time (Barber et al., 2007). Psychotherapy is generally undertaken within therapist - patient or therapeutic dyads ( Holmes, 1998; Swift & Callahan, 2009 ). In CBT the broad process of patient skill acquisition is facilitated by the CBT therapist. However, the CBT therapist does not presume to be the expert of the patient’s life. Consistent with this notion, the CRS 207 therapist competence scale incorporates ongoing feedback as a process associated with high therapist competence in CC (i.e., specifically eliciting information about “clinical relevance” to the patient, the patient’s “understanding”, and “acceptance”). Subsequently , Study 3 support s the notion that a therapist’s competence is not defined by their ability to didactically administer CBT, but rather by their ability to demonstrate competence by collaboratively engaging with a range of patients with different presentations. Further to this, Study 3 provided a robust analysis based on therapy from 10 time points of up to 20 sessions per patient. This enabled the use of a longitudinal MLM providing more meaningful results than traditional analysis. The variation in therapist and patient change trajectories suggest that a therapist who demonstrates high levels of competence with one patient, for example, a compliant patient with a single diagnosis of first episode MDD , does not necessarily demonstrate high levels of competence with, for exa mple, a complex patient that has held negative beliefs related to homework and subsequently not completed between session tasks. Another primary feature of the CRS therapist competence scale was the therapist’ s ability to discuss links between different areas of CC at depth with a patient (e.g., linking core beliefs with triggering situations and automatic thoughts). Once again, as a key feature of the CRS therapist competence scale, Study 3 pro vides empirical support for the process of fostering links between different aspects of the CC as a mechanism associated with positive symptom change in CBT for depression. Overall, the present study found that therapist competence investigated in relation to each therapist- patient dyad, was related to improvement s in individual patient’s depressive symptoms. 208 Depressive Symptom Change In order to carry out the primary analysis in Study 3, baseline scores for change in depressive symptoms for individual patient trajectories were mapped for comparison with predictor and control variables. Study 3 revealed patterns of depressive symptom change consistent with prior research using MLM statistical analyses using concurrent measures of depressive symptomology ( Connolly Gibbons, et al., 2003; Hayes et al., 2005; Hedeker , 2005). Of note, the inclusions of the passage of time (therapy sessions) in the analysis was shown to be an important predictor of depressive symptom change necessary for inclusion to best form a baseline for further comparisons using subsequent substant ive predictors (e.g., therapist competence). The results revealed that patients in the study generally improved over the course of therapy and also that there was significant variation between individual patients. The Fit / M atch of between Written and I n- session Case Conceptualization Consistent with hypothesis three, the MLM analysis found a positive relationship between high levels of fit or match (i.e., the link between written CCs and in session use of CC) and positive change in depressive symptomology. Also, based on the analysis of basic correlations a relationship was found between the fit / m atch of the written CC and therapist competence in CC. These are consistent with and extend p rior research that has suggested that the quality of written CCs are related to therapist experience and are likely to be important in promoting positive change in psychotherapy (Eells et al., 2005; Kuyken e t al., 2003 ). In turn, in assuming that quality written CCs are directly related to therapist competence in using the CC in- session it can be hypothesised that high levels of therapist competence in CC should fit or match with the content to written CCs (i.e., high in - session competence should 209 reflect high quality, or competent written CCs) . However, only a weak relationship was found between the content of therapist’s written CC and therapist competence in CC demonstrated in- session. I t has also been suggested that “competence” in producing a written CC does not always reflect therapist competence in the integration of the content of written CCs into therapy sessions. Rather than reject the importance or validity of the quality of written CC, the results might suggest that the quality of CC, although not directly related to change in depressive symptomology, moderate s the relationship between therapist competence in CC and change in depressive symptomology. Alternatively, i t is also possible that although therapist competence influences the quality of CCs that the quality of CCs does not have the same influence of therapist competence in CC. This is a question for further investigation possible through the use of the CRS in conjunction measures of CC quality (e.g., the Quality of Cognitive - Behavioural Case Formulation Rating Scale; Fothergill & Kuyken, 2002). Belief s about Homework, Personality and Symptom Severity In order to best explain the role of aspects of CC and therapist competence during psychotherapy, Study 3 examined the role of patient beliefs about homework as well as patient personality beliefs and symptom severity as a means to consider the influence of patient complexity. It has been suggested that CC and patient complexity (e.g., comorbidity, symptom severity, personality beliefs) are related, as CC is useful in optimising treatment for complex patie nts, and that increased patient complexity demands greater individualisation of CC (J. Beck, 2005; Persons, 2008). However, little research has considered the role of patient complexity in relation to CC in CBT. Contrary to hypothesis four, no direct relat ionship was found between the complexity 210 of patient personality beliefs and different areas of CC. Although, the results did reveal that the inclusion of patient personality beliefs increased the explanatory power of the final model. Consistent with hypothesis five, the inclusion of beliefs about homework completion in therapist case conceptualizations were shown to be related to greater patient outcomes. Furthermore , inclusion of patient beliefs about homework increased the explanatory power of the final m odel. Based on this finding the decision to focus on patient ratings of beliefs about homework was demonstrated to be useful. This is consistent with a recent meta- analysis by Mausbach et al., (2010) demonstrating that patient rating of different aspects of homework compliance ( r = .32) had a larger effect size in comparison to therapist ( r = .24) or objective ( r = .16) ratings. However, client and therapist ratings ( r = .35) were demonstrated to have a greater still effect size emphasising the benefit of collaborative ratings of homework use and beliefs in - session. Patient beliefs about homework were also negatively and significantly correlated with personality beliefs while there was a positive correlation between patient symptom severity and beliefs about homework . None - the- less, personality beliefs and symptom severity remained positively correlated. Taken together, these findings suggest that patients with greater levels of disordered personality beliefs also reported less positive beliefs about homework . However, patients with high levels of depression may still maintain positive beliefs pertaining to homework tasks when administered systematically as carried out in the CBT Homework Project protocol. This is able to be reconciled with the idea that patients with higher levels of symptom severity also have higher levels of disordered personality beliefs as suggested in Study 3. Consistent with hypothesis seven, the inclusion of patient symptom severity in the final model was shown to be related to greater patient outcomes. Prior research in 211 CBT for depression has suggested that therapist competence is associated with improved outcomes regardless of co- morbidity, but that co - morbidity may moderate the effects of therapist competence on treatment outcomes (K uyken & Tsivrikos, 2009). Accordingly, the level of patient symptom severity accounted for a reduction in patient depressive symptoms above that of measures of therapist competence in CC. Therapist Competence in Case Conceptualization and H omework use The primary research question of interest in Study 3 was ‘ is there a relationship between case conceptualization, homework, and therapeutic outcome? ’. In order to begin to answer this question Study 3 first hypothesised that therapist competence in CC will be positively associated with therapist competence in using homework (hypotheses seven) . Although there was no direct positive relationship observed between therapist competence as observed on the CRS and HAACS in raw correlations, the correlations between slopes and intercepts in rates of change suggest that as therapists in the present study demonstrate more or less competence focusing on either discussing homework or forming a conceptualization with a patient that this results in greater changes in the one area of competence. Put another way, as competence in CC increases or decreases at a more or less rapid rate, so too does therapist competence in homework. This suggests that as therapists in the present study, for example, demonstrate an acceleration in the rate at which they are improving their competence in one area that they are likely to increase their competence in another area. This finding, however, relates to the speed at which therapists in the present study change in their levels of competence rather than the magnitude of competence displayed. Furthermore, therapist competence in homework 212 use was observed to increase in the early phase of therapy and peak at session five before dropping steadily towards session ten. It has been suggested that CB T therapists are likely to demonstrate a certain level of “flexibility” in delivering treatment interventions (e.g., homework tasks, discussing aspects of conceptualization) depending on, for example, the level of depression severity a patient is experienc ing. In this way, more depressed patients may require deeper exploration and discussion of their experiences than less depressed patients. In relationships to homework use, where initially in therapy more depressed patients may require a higher level of th erapist competence, in particular therapist - driven clinical discussion exploring on the specifics of unfamiliar or challenging homework tasks, later in therapy as depression levels begin to drop and patients become more familiar with structured homework us e, less discussion may be required. Moreover, the patient begins to take a greater lead in the process as the process of skill acquisition takes place (Anderson, 1982; Carroll et al., 2005; Hollon, 2003). Consistent with hypothesis eight, therapist compete nce on the CRS was demonstrated to be related to a reduction in depressive symptom severity. Connolly Gibbons, et al. (2003) hypothesised that CBT therapists would use more statements to facilitate patient learning about their “patterns” with less depresse d patients than with patients presenting with higher levels of depression severity. However, depression was not found to predict the frequency of these learning statements but did influence the use of other therapeutic interventions. These findings bear on the level of therapist competence expected for more or less depressed patients. It is important to distinguish between initial depressive symptom severity and depression levels at discrete points over the course of therapy. 213 Finally, the combined impact of therapist competence in both CC and homework use was investigated (hypothesis nine), and after controlling for variables likely to influence the capacity of therapist’s to demonstrate competence, support was found for a relationship between therapist competence in homework, CC, and outcomes as measured on the BDI - II. Prior research has suggested that a complex relationship is likely to exist among different variables associated with domains of therapist competence (Barber et al., 2007). Consistent with these findings, while Study 3 failed to find direct relationships between therapist competence in CC and therapist competence in homework use, taken together as a combined model, therapist competence was related to a positive change in depressive symptom severity. Limitations Study 3 must be considered in the context of a number of limitation s presented by the choice and use of the variables used in the analysis. In working towards a model of therapist competence for predicting therapy outcomes in CBT, the use of a self- report measure of depressive symptom severity (i.e., the BDI - I I) , while commonly used in outcome research surrounding depression presented a limitation to the generalisability of the results. Also, some of the measures used as predictors wer e also self- report (i.e., HRS - I I, CIDI, PBQ) which may have inflated variance explained due to a shared method of observation. However, the importance of investigating and controlling for these variables, consistent with research recommendations, outweighe d the disadvantages for this research. Similarly, the inclusion of two variables taken from the CRS (i.e., therapist competence and fit / match) would be likely to inflate variance explained; this also contributed to the decision to remove the ‘fit / match ’ from the final model. 214 A second limitation lies in the decision to measure the first ten sessions of the twenty session CBT protocol. While this decision was made based on financial constraints and practical time constraints, research highlights the impo rtance of the early phase or first half of a course of CBT ( Busch, Kanter, Landes, & Kohlenberg, 2006; Kelly , Robert, Cielsa, 2005 ; Tang & DeRubeis, 1999 ) . Further research could investigate the level of therapist competence in homework use which might be expected to continue to drop, as well as therapist competence in CC where results of the current study might suggest that therapist competence in CC could either continue to increase, or could drop off as patients continue to discuss aspects of their deepe r level individualised CCs with less need for therapists to demonstrate competence by eliciting new information. A third limitation is found in that while Study 3 has considered the relationship between CC and homework as two specific limited- domain areas of therapist competence limited conclusions can be made regarding how these areas of limited- domain competence relates to the global competence of CBT therapists. This study used the CTS as a measure of global therapist competence to the CBT. However, the CTS was administered primarily as part of therapist training to ensure treatment fidelity, and thus was not used administered systematically but used discretely until therapist met a minimum level of competence required adequately provide a course of CBT c onsistent with how the protocol was intended to be delivered as measured on the CTS. It is likely the more meaningful results would have been obtained, and levels required to obtain significance would have been reached with greater numbers of observations using the CTS. Summary 215 The present study provides evidence which empirically supports the relationship between CC and symptom change in CBT for depression. More specifically, therapist c ompetence in CC and homework use , were found to be related to a positive change in depressive symptoms for patients treated as part of the CBT Homework Project. Study 3 demonstrates two very different patterns of therapist competence when comparing CC and homework use. This finding highlights the importance of therapist competence, not just in CC and homework use, but also in relation to broader implications as they might apply to other specific domains of therapist competence, as well as bearing on the continued use of global measures of therapist competence that group the effects of different domains of therapist competence and possess limited specificity. None - the- less, Study 3 provided evidence towards an empirically supported model of therapist competence in CBT, as well as providing empirical support for the importance of therapist competence in CC in CBT. The next chapter extends the discussion provided for Study 3, to consider the combined findings from Studies 1, 2 and 3, as well as to discus s the results in the context of cognitive- behavioural theory. 216 217 CHAPTER X General Discussion 10.1 Summary This thesis has identified a paucity of research supporting the reliability and validity of processes and products related to CC in CBT. E xisting research has focused on the “accuracy” or “cor rectness” of therapists’ wr itten CC rather than the process by which CCs are constructed and the clinical utility of resultant CCs. Prior research has often focused on the ability of different therapists to construct consistent CCs based on the same patient producing modest reliability scores in “deeper - level” aspects of CC, as well as a failure to find a relationship between CC and improved outcomes in CBT. The focus of this thesis has been on investigating the empirical basis for the relationship between CC in relation to in- session and between session tasks (i.e., homework use) in CBT for depression using data from the CBT Homework Project ( Kazantzis, et al., 2005) . At the outset, the review of literature guided the present thesis towards an investigation of therapist competence to best explore this relationship based on in- session observations of therapist behaviour. Specifically , the present thesis investigated four primary research questions: 1) H ow do therapist’s case conceptualizations evolve over the course of therapy? 2) Can cognitive therapists formulate reliable case conceptualizations? 3) H ow are therapist’s written case conceptualization s related to therapist competence in formulating case conceptualizations in - session? 4) I s there a relationship between therapist competence in case conceptualization, homework, and therapeutic outcome? 218 In order to achieve the primary o bjectives of the present thesis a new measure of therapist competence in CC, the Conceptualization Rating Scale (CRS; Easden & Kazantzis, 2 008) , was created and psychometric data gathered and presented on the reliability and validity for this purpose. However, many areas of therapist competence exist in CBT. In turn the current project has sought to investigate the relationship between two fundamental areas of therapist competence, CC and homework , in order to provide a fuller picture of how different but related areas of therapist competence, both in- session and between session, relate to psychotherapy outcomes (i.e., positive change or relief of depressive symptomology). This chapter considers the results of three interrelated studies presented in the present thesis addressing the use of CCs in the context of CBT for depression. Study 1 investigated how therapists use a commonly used format for written CCs in CBT (i.e., the J. Beck CC Diagram). Study 1 found that therapist’s use of CC evolves over the course of therapy, in particular that therapists obtain more information pertinent to CC over the course of therapy. Furthermore, therapists we re found to conceptualize core beliefs pertaining to the “self” more frequently that beliefs about the “world / others” or “future” consistent with A.T. Beck’s (1976) theory of depression. Study 1 suggested that further time- series investigation surrounding the evolution of CC over a course of CBT , and the construction of a measure suitable for use across multiple therapy sessions would be useful. Study 2, presented the development of the CRS based on theory and expert feedback, then focusing on the reliability between domains, between raters, and across time. The results of Study 2 provided scores suggesting good reliability which was demonstrated to be stable across time. Study 3 extended the investigation of the psychometric properties of the CRS. The val idity of the CRS was examined focusing on therapist competence as the main point of interest 219 in Study 3. Study 3 produced the first empirical support for a positive relationship between therapist competence in CC and primary therapeutic outcomes in CBT (i. e., change in depressive symptoms) supporting the validity of both the CRS scores and processes involved in competent CC as a specific mechanisms for change in CBT. The following chapter provides a general discussion on the combined findings from studies 1, 2 and 3 further to the review of literature and conclusions presented for each study and also synthesises relevant findings from new and existing research in the topic area. 10.2 The Evolution of Case Conceptualization Therapists are encouraged to conceptualize and re - conceptualize with the patients to arrive at a tentative CC that will facilitate therapy. The CC is changed and developed over the course of therapy to inform treatment planning, and is in turn re - informed by between- session patient activities (i.e., homework tasks) and in - session discussion. However, little data exists on therapist’s written or in - session use of CC across therapy session in CBT. Existing research can at times rely on “part or all” of a therapy session recording or vignette and makes conclusions based on discrete observations or single points in therapy (Chadwick et al., 2003; Persons, 1989; Persons et al., 1995; Kuyken et al., 2005). The use of written CCs in naturalistic research can be further problematic due to the fluidity of the CC where, for example, it is expected that therapists might modify or add information to CCs written in pencil during therapy sessions. Studies 1, 2 and 3 provided information about the ‘consistency’, ‘shape’ and ‘level’ of therapist’s use of CC in CBT in the sample considered in the present thesis. The present thesis also incorporated a structured approach to the timing of writing therapist CCs using the J. Beck CCD as well as 220 measuring the aspects of in- session CC considered to demonstrate therapist competence in the context of CBT . Historically, there has been contention surrounding w hether therapist competence is likely to be a “state” or “trait” (Kazantzis, 2005; Shaw & Dobson, 1989). In relation to a course of therapy this contention has to do with the consistency or inconsistency of therapist competence across the course of therapy (i.e., the degree to which therapist competence scores remain stable over time or vary). Svartberg (1999) concluded in a sample of 13 patients, that therapist co mpetence in Short Term Anxiety - Provoking Psychotherapy (STAPP) was time-invariant with “small variation” in competence ratings over the course of therapy sessions. In fact, the highest average competence was observed in the initial therapy session. However , in Study 3, therapist competence tended to increase across therapy sessions and was observed to be time-varying across therapy sessions with large variations observed within and between therapists. Also, the results of Study 3 show that therapists with initially high levels of competence are likely to continue to demonstrate competence over the course of therapy resulting in maintenance of depressive symptom change. In contrast, therapists with lower initial levels of competence are likely to increase in competence over the course of therapy at a more rapid rate than those with initially high competence resulting also in a greater rate of change in depressive symptoms. This suggests that therapists are able to still bring about positive change in depressive symptoms even after initially demonstrating less than optimal levels of competence in case conceptualization while therapists that demonstrated greater consistency tended to maintain treatment gains across therapy better than therapists that did not demonstrate the same consistency. The results of Study 3 were consistent with patterns found in Study 1 showing a greater amount of information being included in 221 CCs. In sum, the results of the present thesis provides support for the notion that therapist competence can be considered a “state” as therapists in the present sample produced different levels of competence across sessions. Differences in competence levels were also observable within therapists between patients. Therefore, one potential implication, contrary to Svartberg (1999) is that a single estimation of therapist competence at any given therapy session, or for any single patient, may not be representative of therapist competence across an entire course of psychotherapy depending on the particular therapist being assessed and the particular presentation of the patient, unless we can be confident that a therapist demonstrated and will continue to demonstrate consistency in their level of competence across therapy sessions (e.g., perhaps based on a range of prior competence measurements for a particular therapist) . In practice it may be the case that some therapists, perhaps expert therapists, are able to demonstrate consistency in their levels of competence while other therapists, such as trainee or novice therapist are less able to maintain a consistent level of competence. This is an empirical question for further research. Moreover, therapist’s use of CC was shown to evolve over the course of therapy both in the amount and type of information used and integrated. For example, this was especially the case for core beliefs and other deeper- level patient information which was discussed at greater length and depth in later therapy sessions. This finding is consistent with guidelines for the practice of standard CBT (A. T. Beck et al., 1979) where it is expected that deeper level information, such as core beliefs, are discussed in greater depth towards later therapy sessions, and when increasingly greater amounts of information is synthesised during therapy sessions, together with the patient. 222 10.3 The Reliability of Case Conceptualization Research has produced poor rates of agreement in between CBT therapists in identifying the hypothesised underlying cognitive mechanisms for change (Kuyken et al, 2005; Persons & Bertagnolli, 1999 ; Persons et al ., 1995). It has been suggested that a more systematic or structured approach to constructing and measuring CC could be requi red to increase rates of agreement between CBT therapists or independent judges particularly in “deeper” level, more inferential aspects of CC (Bieling & Kuyken, 2003; Persons & Bertagnolli, 1999). Further to this, failure to focus on the process by which CCs are constructed, collaboratively with the patient, may reflect a misapplication of clinically relevant measurement of reliability and validity as it relates to CC. The current thesis focused on the construction and testing of the CRS, a structured measure primarily aimed at measuring the competence of CBT therapists in using CC. The results of Study 2 suggest that independent observers, given limited training in how to administer the CRS are able to reach high rates of agreement, particularly regarding how CBT therapist integrate and use CC in vivo. In order to reach agreement independent observers would be required to first identify the aspect or particular component of the CC being discussed during the therapy session. Furthermore, rates of agreement were consistently high across more “surface” level CC components and “deeper” level hypot hesised underlying cognitive mechanisms (i.e., core beliefs, conditional assumptions). In light of the findings of Study 2 suggesting that the CRS was a reliable measure of cognitive CC, Study 3 enabled the investigation of the validity of CC in CBT. Also, the relationship between homework use and CC as two central processes in CBT , were able to be examined. 223 10.4 The Validity of Case Conceptualization CBT therapists depend on individualised CCs to make sense o f their patients and to exist as a written record of their understanding of a patient’s problems as well as to explain what maintains their problems. To suggest that the process of CC is not a valid one bears on the competence of the therapist to make accurate and meaningful clinical judgements. Ho wever, the process of CC often remains implicit and considered to be as much an art as a science. This project has sought to make the process of CC as it is used during therapy sessions explicit. This was done by identifying core therapist competencies that are a part of gathering CC information considered to help the patient better understand themselves and to work towards skill acquisition through structured homework (e.g., through the use of thought records, discussion of the specifics of when, where, an d how long will be spent on homework, the importance and difficultly of the task, etc). This thesis developed the CRS a means of operationalising CBT therapist’s use of CC in session to investigate whether the mechanisms identified and framed in the CRS ba sed on the review of literature and expert clinical feedback related to positive psychotherapeutic outcomes. Written Case C onceptualization and Clinical P ractice The written CC provides the primary means of formalizing and making explicit the therapist s understanding of the patient. Study 1 showed that in general therapists tend to incorporate more information in written CCs from session one to session 10 where the amount of information in the formal written format used (i.e., the J. Beck CCD; 1995) beg an to plateau. However, Study 1 did not consider whether or not the additional information being recorded provided any treatment utility. For example, as well as adding additional information to better understand the patient, it is 224 also important to integrate and simplify complex information about patient to provide a succinct written account of the patient’s hypothesised underlying mechanisms and maintaining factors (Sim et al., 2005 ; Eells, 1997a; 2007 ). Study 2 and 3 utilized the CRS to investigate the f it / match between written CC and what a therapist actually discussed in session. In this way , information was able to be gathered regarding whether or not therapists formal written CCs are consistent with what is undertaken during CBT. Study 2 revealed th at independent observers were able to agree on the fit / match between CC content of written CCs and what was discussed in session. In this way, Study 3 provided a reliable indication of the level of consistency between written and in- session CCs. Although there was a relationship between fit / match, therapist competence in CC and positive change in depressive symptoms, the observed levels of fit / match between written and in - session content of CC were low to moderate. This might suggest that therapists written CCs are not as linked to their clinical practice as might be expected. I t has been suggested that the primary measure of the validity of a CC would be indicated by the “fit” of the CC to the patient, as judged by patient acceptance and agreement (Bi eling & Kuyken, 2003). However, there is little empirical support for this assertion. For example, an anecdotal account reflects the issue at hand: A student wrote a very competent case formulation, discussing the client’s difficulties maintaining intimate relationship in terms of her mother’s example as a role model. However when I reviewed a tape of the session, I discovered that the client had talked at length about her father’s infidelity. This information was never mentioned in the database. (Ingram, 2006, p. 454) . 225 Further to this, the positive relationship demonstrated in Study 3 between the fit / match of the written CC and in - session content with change in depression symptoms might be strengthened if therapists were to record more accurately what was done in session, or to put a greater emphasis on the written CC as a guide to subsequent session discussion. The combined findings of Studies 1, 2 and 3 suggest that the link between therapist formal written CC and in - session use of the CC calls for further investigation. Case Conceptualization and Therapeutic O utcomes Bieling and Kuyken (2003) identified that no research had produced empirical support for a relationship between the content of CC and psychotherapeutic benefits for individual patients. Since then a small amount of re search has begun to address the validity of cognitive CCs. O nly a small body of research provides research towards the operationalization of an empirically validated CC method, measure or format outside of the current projec t (see Mumma, 2004; Mumma & Mooney, 2007a, b) . For example, it has been suggested that cognitive CC does not necessarily have to be reliable to be valid. It has been suggested that less specific and less reliable CCs may lead to more idiosyncratic (and val id) CCs (Persons, Mooney, & Padesky, 1995). That is, two separate CCs for a single patient may focus on different clinical data and lead to equally efficacious treatment outcomes (Kuyken et al., 2005). In turn, the quality of CCs has been the focus of CC research in CT that addresses issues of validity (Bieling & Kuyken, 2003; Eells, Lombart, Kendjelic, Turner, & Lucas, 2005; Kuyken et al., 2005). However, a number of other domains of validity remain unexplored. 226 10.5 Limitations While, t his thesis builds upon current research and makes a valuable contribution to CC literature a number of limitations exist . The following section considers additional, more general limitation not already discussed in relation to Studies 1, 2 and 3. Firstly, only a small numbe r of therapists and patients were able to take part in the CBT Homework Project. This limits the generalisability of the results restricting findings largely only to the patients and therapists in this particular sample. However, the large amount of observations based on DVD recordings live - therapy sessions provided a rich data source on which to draw conclusions and guide future research. Second, no comparison group was used in the present thesis. In the context of CC research providing a comparison group that is useful as a control is difficult considering that generating information and demonstrating some level of competence in CC is fundamental to the practice of CBT (i.e., not attempting to conceptualize and provide records of a patient’s treatment wou ld be considered unethical) . Recent research by Jacqueline Person’s and colleagues CC compares the effect sizes of “case - driven” conceptualization with the effect sizes for existing research demonstrating positive change in psychopathological symptoms (Per sons, 2006; Persons et al., 2006; Persons, Bostrom, & Bertagnolli, 1999 ). This methodology provides a possible means to demonstrate the relative treatment utility of CBT incorporating highly systematised CC and guided by structured CC training for therapists versus CBT carried out where therapists were not involved in some type of targeted intervention towards improving therapist competence and quality of CC. For the present thesis, in lieu of a comparison group, Study 3 investigated the fixed and random ef fects (e.g., relationships between the dependent variable and initial status, and changes in variance explained) for 227 regressions of individual patient trajectories foll owed for the different variables analysed (i.e., depressive symptom c hange for individual patients). In contrast the longitudinal MLM provided a relative strength as a nalyses were carried out based on observations of multiple therapy sessions to provide a better indication of how varying levels of, for example, therapist competence, was likel y to predict changes in depressive symptomology. 10.6 Recommendations for Future Research Future research should continue to investigate CC in CBT. In particular, the measurement of therapist competence in CC has gained validation as a potential avenue for further investigating the empirical basis for CC in CBT. Further to this, the present thesis has demonstrated and provided: 1) a measure suitable for investigating CC that has demonstrated good psychometric properties (i.e., the CRS), and 2) a statistical method suitable for investigating CC and other limited domain competencies using MLM to better understand the relationship among complex variables inherent in CBT. Existing research has not focused on therapist competence in CC but rather on written CC themselves in discrete observations. In light of acceptable psychometric results research should seek to answer ‘what competencies, skill sets, or specific tasks are necessary to produce reliable and valid cognitive CC?’ Future research should aim to investigate the triangulation of agreement - or disagreement - among therapist, supervisor (observer) and patient CCs over time, and in relation to therap eutic outcomes such as reduction in target symptom severity or comorbidity (e.g., depressive symptom severity). 228 10.7 Implications CBT is generally considered to be an empirically based treatment. Recently the question has been asked in relation to CC, ‘is the emperor clothed?’ to which the conclusion has been “very sparsely” ( Kuyken, 2006). Positive implica tions of a well constructed and integrated CC are that they facilitate the linking of theory, research and practice, normalise problems and increase empathy, organise large amounts of complex information, and enable high quality supervision among other ben efits. CC is also suggested to guide treatment intervention by selecting, focusing and sequencing the order of treatment intervention, facilitating a collaborative approach, pre - empting possible therapy interfering behaviours, and enabling the simplest mos t cost effective interventions. While other psychotherapeutic orientations have made headway into empirical demonstration that CCs enhance therapeutic outcomes within their respective paradigm, the importance of CC in CBT remains unclear. The vast majorit y of CC research in CBT has focused on written CCs based on discrete portions of therapy. The broad implication is that there is no empirical basis for the clinical utility of CCs as they apply to a naturalistic course of CBT; particularly components perti nent to cognitive specificity which sets CB T apart from other therapeutic orientations. The present thesis presents several implications relevant to the practice of CBT. Firstly, this thesis highlights therapist competence in CC as playing an important role in affording positive outcomes. This includes the potential for stark contrasts between different areas of therapist competence, and the potential benefit for monitoring therapist competence with greater specificity. Further to this, the potential for individual therapist competence to vary over time has implications for when (i.e., what therapy session/s) and how often therapist competence should be measured. It 229 could be suggested that taking one measurement of therapist competence is not enough to provide an accurate bearing on overall therapist competence based on the findings of the present thesis. A second implication applies to the use and implementation of standardised treatment manuals. The varying levels of therapist competence observed between therapists in two central components of CBT, CC and homework use, support the notion that regardless of the validity of the treatment manual that the ability of the therapist to demonstrate competence in using that manual will directly impact the resultant treatment utility of the manual for an individual patient. Regarding the role of therapist competence in psychotherapy it has been asserted that “ the interventionist does not always equal the intervention” (Nezu & Nezu, 2005). Thus, the present thesis supports the need to monitor and promote treatment integrity. In practice, feedback can be provided to therapists regarding their levels of adherence and competence to different components of treatment based on live observation or ratings of therapy sessions (e.g., using observational measures such as the CTS, CRS, and HAACS in the context of CBT). Further research should be conducted to best determine when and how often the checks should be carried out. It has been suggested that systematic “spot checks” and regular supervisory feedback be provided to therapists in consideration of the cost of detailed and time consuming adherence and competence ratings. Alternatively, clinicians have suggested methods of systematic and ongoing self- monitoring of an individual therapist’s own performance (Duncan & Miller, 1994; Duncan et al. , 2006 ; Miller, Duncan, Brown, Sparks, & Claud, 2003; Miller, Dun can, Johnson, 2000 ; Miller, Duncan, Sorrell, & Brown, 2005 ) or suggested the examination of individual therapist’s levels o f performance at the level of the clinical practice ( Okiishi, 2003; Persons, 2006; Persons et al., 2006; Persons, Bostrom, & 230 Bertagnolli, 1999 ). Again, the broad implication is measures suitable for use in CBT must be developed before they can be integrated in practice. Perhaps most importantly, the present research has identified particular areas of therapist competence in CC hypothesised to account for change in CBT for depression towards a greater understanding of mechanisms of change. Table 28 outlines common features of therapist competence in CC that the present research demonstrated to have clinical utility. These areas have been demonstrated to relate to depressive symptom reduction and key indicator s of depressive symptom reduction. The features are intended to strengthen empirically, and provide guidelines for the process of CC as intended in standard CBT for depression (A. T. Beck, 1976; A. T. Beck et al., 1979). For example, attention has been drawn to the importance of developing an individualiz ed CC in CBT (J. Beck, 1995; Needleman, 1999; Persons, 2001). The guidelines in Table 28 begin to provide some structure towards achieving an individualised and clinical valid CC. For instance, the findings of the present research support the systematic integration of each different aspects of CC as outlined on the J. Beck Case Conceptualization Diagram (CCD; 1995) written format. Put another way, the research emphasises the need to discuss each part of the CC with the patient systematically, but also to do this as the patient is able to cope with increasingly challenging or upsetting cognitions. Furthermore, discussing the most important or clinically relevant content at depth, as agreed upon by both the therapist and patient, and helping to patient to see how different parts or their CC are linked to both individual weaknesses and strengths routinely is potentially related to improved treatment outcomes. These common features are areas that warrant further research both as combined indicators of general competence in CC and individually to isolate the more or less important areas of therapist competence in CBT. 231 Table 28 Common features of in-session therapist competence in CBT case conceptualization 1. The therapist should consistently discuss in-depth all aspects of a CBT case conceptualization (e.g., automatic thoughts, core beliefs) 2. The therapist should also make judgements about the timing of discussions of “deeper - level” more inferential (e.g., or potentially traumatic or difficult) aspects of the conceptualization and gradually increase the depth of discussion 3. The therapist should foster discussions to link different areas of case conceptualization 4. The therapist should seek regular patient feedback on the tentative overall conceptualization as well as mini- conceptualizations or aspects of the conceptualization during the session in regards to the patient’s: a) Acceptance of the conceptualization b) Understanding of the conceptualization c) Assessment of clinical relevance (i.e., importance) of t he conceptualiza tion to the resolution of presenting problems and building of patient strengths 5. The therapist should conceptualize patient strengths as well as weakness as routine practice Note. Summarised from key features of the therapist competence scale of the Conceptualization Rating Scale (CRS ; Easden & Kazantzis, 2008; 2009). Thus, the CRS developed in the current thesis provides a potential means to measure therapist competence in CBT with preliminary psychometric support produced in the context of CBT for dep ression. In provided empirical support for the CRS, empirical support has been provided for the continued use of the J. Beck CCD and Beckian cognitive behavioural theory on which the J. Beck CCD was based. 232 10.8 Final Conclusions It has been posed that “the value of therapist competency evaluations will be demonstrated by their ability to predict treatment outcome” (Shaw & Dobson, 1989, p. 670). The present thesis was the first both in New Zealand and internationally to provide empirical support for the relationship between therapist competence in case conceptualization and change in depressive symptoms in CBT. Further to this, the Conceptualization Rating Scale (CRS ; Easden & Kazantzis, 2008) provides an empirically supported operationalization of core domai ns necessary for therapists to demonstrate competence in case conceptualization in CBT. In general, therapists utilized CCs increasingly over the course of therapy both in the amount of content and in their competence in integrating aspects of case conceptualization into therapy. However, greater focus is needed on protective factors or patient strengths that might promote greater well- being over a course of CBT for depression. The present thesis has highlighted the importance of the individual therapist as important to treatment over and above the level of existent empirical for any particular treatment manual. The differences in between therapist competence in CC and homework use suggest that greater specificity is required for measuring therapist competence in CBT than existing measures are able to provide. The relationship between CC and homework itself is complex suggesting that while patient beliefs about homework continue to predict positive change in depressive symptoms over time, that a therapist’s n eed to demonstrate competence in integrating homework seems to reduce across sessions, perhaps as patients learn what is required to successfully carry out homework tasks and require less input from therapists as therapy progresses. Ironically, at times wh en therapists demonstrate high competence in homework, less competence seems to be demonstrated in CC while therapist 233 competence in CC increased across sessions. Thus, i t may be the case also that as therapists develop a greater understanding about their patients that homework is better tailored to the patient and less “competence” is needed to be d emonstrated by a therapist. In turn, the present thesis has drawn attention to the way in which competence in defined and understood. In conclusion , the presen t thesis describes the rationale and development for the CRS as valid and reliable measure of assessing different aspects of CC in CBT. The use of the CRS has facilitated research into the relationship between CC and homework use and provides a means for f uture research in CC in CBT. This research is the first to investigate how therapist competence in CC relates to depressive symptom change using longitudinal data, and isolates and operationalizes CC as a measurable and valid area of central therapist competencies in CBT. Using previously developed measures of homework use the importance of homework use alongside CC has been shown to contribute to a reduction in patient depressive symptomology while demonstrating that differences exist between therapists in different areas of therapist competence. As well as examining the relationship between CC and homework use, this thesis considered the influence of pertinent variables that influence the course of CBT demonstrating that the influence of patient beliefs and patient complexity influence the ability of the therapist to demonstrate competence. In sum, the CRS developed in this thesis is an empirically tested measure that: 1) provides a valid and reliable structure for CC in CBT during therapist sessions, 2) is suitable for use as a training instrument n the practice of CC in CBT, and 3) is based on cognitive - behavioural theory and through the development and validation process the CRS has provided empirical support for both cognitive behavioural- theory establishing CC as a central therapeutic process in CBT. 234 235 REFERENCES Albani, C., Pokorny , D., &. Kächele , H. (199 8). The connected central rela tionship patterns ( CCRP ): A structural version of the CCRT . Psychotherapy Research, 8 ( 4) , 408 - 425. Aldrich, J. (1997). R. A. Fisher and the making of maximum likelihood 1912 - 1922. Statistical Science, 12( 3) , 162 - 176. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text revision). Washingt on, DC: Author. Anastasi, A. , & Urbina, S. (1997). Psychological Testing (7 th ed.). New Jersey: Prentice Hall. Addis, M., & Jacobson, N. (2000). A closer look at the treatment rationale and homework compliance in cognitive behavioural therapy for depress ion. Cognitive Therapy and Research, 24( 3), 313 - 326. Alford, B. A., & Beck, A. T. (1997). The integrative power of cognitive therapy. New York: Guilford Press. Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological Review, 89, 369 - 403. Andrews, G., & Peters, L. (1998). The psychometric properties of the composite international diagnostic interview. Social Psychiatry and Psychiatric Epidemiology, 33. 80 - 88. Barber, J., & DeRubeis, R. (1989). On second thought: where the action is in cognitive therapy for depression. Cognitive Therapy and Research, 13(5), 441 - 457 . 236 Barber, J., & DeRubeis, R. (2001). Change in compensatory skills in cognitive therapy for depression. Journal of Psychotherapy, Practice and Research, 10 ( 1) , 8 - 13. Barb er, J . P., Liese , B . S ., & Abrams, M . J. (2003). Development of the Cognitive Therapy Adherence and Competence Scale. Psychotherapy Research, 13 , 205 – 221. Barber, J. P., Sharpless, B. A., Klostermann, S., & McCarthy, K. S. (2007). Assessing intervention competence and its relation to therapy outcome: A selected review derived from the outcome literature. Professional Psychology: Research and Practice, 38( 5) , 493 - 500. Beck, A. T. (1976). Cognitive therapy and the emotional disorders. New York: Internatio nal Universities Press. Beck, A. T. (2005). The current state of cognitive therapy. Archives of General Psychiatry, 62, 953 - 959. Beck, J. (1995). Cognitive Therapy: Basics and beyond. New York: Guilford Press. Beck, J. (2005). Cognitive Therapy for challenging problems: What to do when the basics don’t work. New York: Guilford Press. Beck, A. T., & Beck, J. S. (1991). The Personality Belief Questionnaire. Unpublished manuscript, University of Pennsylvania. Beck, A. T., Butler, A. T., Brown, G. K., Dahlsg aard, K. K., Newman, C. F., & Beck, J. S. (2001). Dysfunctional beliefs associated with specific personality disorders. Behavior Research and Therapy, 39, 1213 – 1225. Beck, A. T., Emery, G., & Greenberg, R. (1985). Anxiety disorders and phobias: A cognitive perspective. New York: Basic. 237 Beck, A. T., & Freeman, A. (1990). Cognitive therapy of personality disorders. New York: Guilford. Beck, A. T., Rush, A., Shaw, B., & Emery, G. (1979). Cognitive therapy of depression. New York: The Guilford Press. Beck, A.T., Steer, R.A., & Brown, G.K. (1996). Beck Depression Inventory manual ( 2 nd ed.). San Antonio. TX: Psychological Corporation. Beckham , E. E., Leber , W. R., Watkins , J. T., Boyer , J. L., & Cook , J. B. ( 1986 ). Development of an instrument to measure Be ck's cognitive triad: The cognitive triad Inventory. Journal of Consulting and Clinical Psychology, 54, 566 - 567. Bennet, D., & Parry, G. (2004). A measure of psychotherapeutic competence derived from cognitive- analytic therapy. Psychotherapy Research, 14( 2) , 176 - 192. Beutler, L., Castonguay, L., & Follette, W. (2006). Therapeutic factors in Dysphoric Disorders. Journal of Clinical Psychology, 62( 6) , 639 - 647. Bieling, P., & Kuyken, W. (2003). Is cognitive case formulation science or science fiction? Clinical Psychology Science and Practice, 10( 1), 52 - 69. Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis (2 nd ed.). New York: Cambridge University Press. Blackburn, I., James, I. A., Milne, D. L., Baker, C., Standart, S., Garland, A., Reichelt , F. K. (2001). The revised Cognitive Therapy Scale (CTS - R) : Psychometric properties. Behavioural and Cognitive Psychotherapy, 29, 431 - 446. Blacker , C. V. R., & Mortimore , C. ( 1996 ). Randomized controlled trials and naturalistic data: Time for change? Human Psychopharmacology, 11, 353 - 363. 238 Bond, F. (1998). Utilising case formulations in manual - based treatments. In M. Bruch, & F. Bond (Eds.), Beyond Diagnosis: case formulation approaches in CBT. Chichester: Wiley. Bro wn, R. A., Evans, D. M., Miller, I. W., Burgess, E. S., & Mueller, T. I. (1997). Cognitive- Behavioral Treatment for Depression in Alcoholism. Journal of Consulting and Clinical Psychology, 65( 5), 715 - 726. Bryant, M., Simons, A., & Thase, M. (1999). The rapist skill and patient variables in homework compliance: controlling the uncontrolled variable in cognitive therapy outcome research. Cognitive Therapy and Research, 23( 4) , 381 - 399. Bruch, M. (1998). The development of case formulation approaches. In M . Bruch, & F. Bond (Eds.), Beyond Diagnosis: case formulation approaches in CBT. (pp. 1 - 18). Chichester: Wiley. Burns, D. (1980). Feeling good: The new mood therapy. New York: Avon Books. Burns, D. D., & Auerbach, A. H. (1992).Does homework compliance enhance recovery from depression? Psychiatric Annals, 22, 464 – 469. Burns, D. D. , & Eidelson, R. (1998). Why are depression and anxiety c orrelated? A test of the tripartite model. Journal of Consulting and Clinical Psychology, 66 ( 3) , 461 - 473. Burns, D. D. , & Nolen - Hoeksema, S. (1991). Coping st yles, homework compliance, and the effectiveness of cognitive- behavioural therapy. Journal of Consulting and Clinical Psychology, 59( 2), 305 - 311. Burns , D. D., & Spangler , D. L. (200 0 ) . Does psychotherapy homework lead to improvements in depression in Cognitive- Behavioural Therapy or does improvement lead to increased homework compliance? Journal of Consulting and Clinical Psychology, 68( 1), 46 - 56. 239 Busch, A. M., Kanter, J. W., Landes, S. J., & Kohlenberg, R. J. (2006). Sudden gains and outcome: A broader temporal analysis of Cognitive Therapy for depression. Behavior Therapy, 37, 61 - 68. Butler, A. C., Brown, G. K., Beck, A. T., Grisham, J. R. (2002). Assessment of dysfunctional beliefs in borderline personality disorder. Behaviour Research and Therapy, 40, 1231 - 1240. Butler, A. C., Chapman, J., Forman, E., & Beck, A. T. (2 006). The empirical status of Cognitive- Behavioural Therapy: A review of meta - analyses. Clinical Psychology Review, 26, 17 - 31. Butler, G., Fennell, M., Robson, P., & Gelder, M. (1991). Comparison of behavior therapy and cognitive behavior therapy in the treatment of generalized anxiety disorder. Journal of Consulting and Clinical Psychology, 59, 167 - 175. Carroll, K., Nich, C., & Ball, S. (2005). Practice makes progress? Homework assignments and outcome in treatment of cocaine dependence. Journal of Consulting and Clinical Psychology, 73( 4), 749 - 755. Caspar, F. (1995). Plan analysis: Toward optimizing psychotherapy. Seattle, WA: Hogr efe & Huber. Chadwick, P., Williams, C., & Mackenzie, J. (2003). Impact of case formulation in cognitive behaviour therapy for psychosis. Behaviour Research and Therapy, 41, 671 – 680. Chambless, D. L., & Gillis, M. M. (1993). Cognitive therapy of anxiety d isorders. Journal of Consulting and Clinical Psychology, 61, 248 - 260. Chambless, D. L. , & Hollon, S. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66( 1) , 7 - 18. 240 Cicchetti, D. V. (1994). Guidelines, criter ia, and rules of thumb for evaluating normed and standardized assessment instruments in psychology, Psychological Assessment, 6, 284 - 290. Clarkin, J. F. (2006). Conceptualization and treatment of personality disorder. Psychotherapy Research, 16( 1) : 1 - 11 . Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, pp. 37 - 46. Connolly Gibbons, M., Crits - Christoph, P., Levinson, J., & Barber, J. (2003). Flexibility in manual- based psychotherapies: predi ctors of therapist interventions in interpersonal and cognitive- behavioral therapy, Psychotherapy Research, 13 ( 2), 169-185. Crits- Christoph, P., Cooper, A., & Luborsky, L. (1988). The accuracy of therapists’ interpretations and the outcome of dynamic psychotherapy. Journal of Consulting and Clinical Psychology, 56, 490 - 495. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297 - 334. Curtis, J., & Silberschatz, G., Sampson, H., & Weiss, J. (1994). The plan fo rmulation method. Psychotherapy Research, 4( 3&4) , 197 - 207. Curtis, J., & Silberschatz, G. (1997). The plan formulation method. In T. Eells (Eds.), Handbook of psychotherapy case formulation. (pp. 116 - 136). New York: The Guilford Press. Davidson, K., Sc ott, J., Schmidt, U., Tata, P., Thornton, S., & Tyrer, P. (2004). Therapist competence and clinical outcome in the prevention of parasuicide by manual assisted cognitive behavioural therapy trial: T he POPMACT study. Psychological Medicine, 34, 855 - 863. 241 DeMaat, S., Dekker, J., Schoevers, R., & DeJonghe, F. (2006) Relative efficacy of psychotherapy and pharmacotherapy in the treatment of depression: A meta - analysis. Psychotherapy Research, 16( 5), 566 - 578. DeRubeis, R., Hollon , S., Amsterdam, J., Shelton, R., Young, P., Salomon, R., O'Reardon, J., Lovett, M., Gladis, M., Brown, L., & Gallop, R. (2005). Cognitive therapy vs medications in the treatment of moderate to severe depression. Archives of General Psychiatry, 62( 4), 409 - 416. Detweiler - Beddell, J . B., & Whisman, M. A. (2005). A lesson in assigning homework: Therapist, client, and task characteristics in cognitive therapy for depression. Professional Psychology: Research and Practice, 36( 2) , 219 - 223. Dobson, K. S. ( 2001 ). Handbook of cognitive-behavioral therapies (2nd ed.). New York: The Guilford Press. Dozois, D., Covin, R., & Brinker, J. (2003). Normative data on cognitive measures of depression, Journal of Consulting and Clinical Psychology, 71( 1), 71 - 80. Dozois , D. J. A., Dobson , K. S., & Ahnberg, J. L. ( 1998 ). A psychometric evaluation of the Beck Depression Inventory - II. Psychological assessment, 10, 83 - 89 . Duncan, B. L., Sparks, J., & Miller, S. D. (2006). Client, not theory, directed: Integrating approaches one client at a time. In G. Stricker & J. Gold (Eds.), A casebook of psychotherapy integration (pp. 225 - 240). Washington D. C.: APA Press. Easden, M. H., & Kazantzis, N. (2008, July). In P. Salkovskis (Chair), Recent developments in cognitive behavioural case conceptualization. Case conceptualization in CBT for depression: The development of a comprehensive case conceptualization rating scale [ Field note data summary and slide presentation] . Symposium conducted at the 36 th Annual Conference 242 of the British Association of Cognitive and Behavioural Cognitive Psychotherapies (BABCP), Edinburgh, S cotland. Easden, M. H., & Kazantzis, N. (2009, July). In M. Zivor (Chair), An emerging empirical basis for the use of formulation (pt. 1). The Conceptualizati on Rating Scale (CRS): Further psyc hometric data and links between therapist competence and treatment planning [ Field note data summary and s lide presentation]. Sym posium conducted at the 37 th Annual Conference of the British Association of Cognitive and Behavioural Cognitive Psychotherapie s (BABCP), Exeter, UK. Eells, T., Kendjelic, E., & Lucas, C. (1998). What’s in a case formulation? Development and use of a content coding manual. Journal of Psychotherapy Practice and Research, 7( 2) , 144 - 153. Eells, T., Kendjelic, E., Lucas, C. , & Lomba rt, K. G. (1 998). Manual for case formulation and treatment plan coding. Unpublished manuscript. Eells, T. (1997a). Handbook of psychotherapy case formulation. New York: Guilford Press. Eells, T. (1997b). Psychotherapy case formulation: History and curr ent status. In T. Eells (Ed.), Handbook of psychotherapy case formulation (pp. 1 – 25). New York: Guilford Press. Eells, T. (2007). Handbook of psychotherapy case formulation ( 2 nd ed.). New York: Guilford Press. Eells, T. Lombart, K., Kendjelic, E., Turne r, L., & Lucas, C. (2005). The quality of psychotherapy case formulations: a comparison of expert, experienced, and novice cognitive– behavioral and psychodynamic therapists, Journal of Consulting and Clinical Psychology, 73( 4), 579 - 589. 243 Elkin, I. (1999) . A major dilemma in psychotherapy outcome research: Disentangling therapists from therapies. Clinical Psychology: Science and Practice, 6 , 10 – 32. Field, C.E., Nash, H.M. , Handwerk, M.L., & Friman, P.C. (2004). Functional assessment and experimental fun ctional analysis as a guide for individualizing treatment within a residential care setting. Clinical Case Studies. 3, 25 - 26. Flitcroft, A., James, I. A., Freeston, M., & Wood - Mitchell, A. ( 2007 ). Determining what is important in a good formulation. Behavioural and Cognitive Psychotherapy, 35, 325 - 333. Flückiger, C., Caspar, F. Holtforth , M. G., & Willutzki, U. ( 2009 ). Working with patients’ strengths: A microprocess approach. Psychotherapy Research, 19 ( 2) , 205 - 212. Flückiger , C. & Holtforth, M. G. ( 2008 ). Focusing the therapist’s attention on the patient’s strengths: A preliminary study to foster a mechanism of change in outpatient psychotherapy. Journal of Clinical Psychology, 64( 7) , 876 - 890. Fothergill , C. D., & Kuyken, W. (2002) . The Quality of Cogn itive Case Conceptualization Rating Scale. Unpublished Manuscript. Freeley, M., DeRubeis, R., & Gelfand, L. (1999). The temporal relation of adherence and alliance to symptom change in cognitive therapy for depression. Journal of Consulting and Clinical Psychology, 67( 4), 578 - 582. Garland, A., & Scott, J. (2007). The obstacle is the path: Overcoming blocks to homework assignments in a complex presentation of depression. Cognitive and Behavioral Practice, 14( 3), 278 - 288. 244 Garret, G., Ingram, R. E., Ran d, K. L., & Sawalani, G. (2007). Cognitive processes in cognitive therapy: Evaluation of mechanisms of change in the treatment of depression. Journal of Clinical Psychology: Science and Practice, 14, 224 - 239. Gedo , P. M. ( 1999 ). Single case studies in p sychotherapy research. Psychoanalytic Psychology, 16, 274 - 280. Geller, J. D., Norcross, J. C., & Orlinsky, D. E. (2005). The Psychotherapist’s own psychotherapy: Patient and clinician perspectives. New York: Oxford University Press. Gerardi, R . Keane, T. M., Pe nk, W. ( 1989 ). Utility, sensitivity and specificity in developmental diagnostic tests of combat- related post- traumatic stress disorder (PTSD). Journal of Clinical Psychology, 45, 691 - 701. Goldfried, M. R. (1999). The pursuit of consensus in psychot herapy research and practice. Clinical Psychology: Science and Practice, 6(4), 462 - 466. Goldfried, M. R. (2000). Consensus in psychotherapy research and practice: Where have all the findings gone? Psychotherapy Research, 1(1), 1 - 16. Gonzalez - Prendes, A. A. (2007). Cognitive - behaviour treatment of men and anger: Three single case studies. Cognitive and Behavioral Practice, 14, 185 - 197. Gorenstein, E. E., Tager, F. A., Shapiro, P. A., Monk, C., & Sloan, R. P. (2007). Cognitive- behavior therapy for reduction of persistent anger. Cognitive and Behavioral Practice, 41, 168 - 184. Gould, R. A., Mueser, K. T., Bolton, M. A., Mays, V. K., & Goff, D. C. (2001). Cognitive therapy for psychosis in schizophrenia: an effect size analysis. Schizophrenia Research, 48, 335 – 342. 245 Greenb erg, L. ( 1991 ). Research on the process of change. Psychotherapy Research, 1, 3 - 16. Greenberger , D. , & Padesky , C. A. (1995). Mind over mood: Change how you feel by changing the way you think. New York: Guilford Press. Groth - Marnat, G., & Horvath, L. S. (2006). The psychological report: A review of current controversies. Journal of Clinical Psychology, 62( 1) , 73 - 81. Haubert, L. & Dobson, K. S. ( 2007).Treatment of depression and mechanisms of change: Strengthening the links among theory, research and practice. Clinical Psychology: Science and Practice, 14(3), 247 - 251. Hayes, A. M., Beevers, C. G., Feldman, G. C., Laurenceau, J., Perlman, C. (2005). Avoidance and processing as predictors of symptom change and positive growth in an integrative therapy for depression. International Journal of Behavioral Medicine, 12( 2), 111 - 122. Health Practitioners Competence Assurance Act, No. 48, New Zealand Statutes. (2003). Retrieved February 20 , 2008 , from: http://www.legislation.co.nz. Hedeker, D. ( 2004). An introduction to growth modeling. In D. Kaplan (Ed.), Quantitative Methodology for the Social Sciences. Thousand Hedeker, D. (2005). Generalized linear mixed models. In B. Everitt & D. Howell (Eds.), Encyclopedia of Statistics in Behavioral S cience. Wiley, New York. Oaks CA: Sage Publications. Hedeker, D., & Gibbons, R.D. (1997). Application of random- effects pattern- mixture models for missing data in longitudinal studies. Psychological Methods, 2, 64 - 78. Hedeker, D. & Gibbons, R.D. (2006). Longitudinal data analysis. New York: Wiley. 246 Henry, W. P., Schacht, T. E., & Strupp, H. H. (1986). Structural analysis of social behavior: Application to a study of interpersonal process in differential psychotherapeutic outcome. Journal of Consulting and Clinical Psychology, 54, 27 – 31. Henry, L., & Williams, R. (1997). Problems in conceptualization within cognitive therapy: an illustrative case study, Clinical Psychology and Psychotherapy, 4 ( 3) , 201 - 213. Hollon, S. (2003). Does cognitive therapy have an enduring effect? Cognitive Therapy and Research, 27( 1), 71 - 75. Hollon, S., DeRubeis, R., Shelton, R., Amsterdam, A., Salomon, R., O’Reardon, J., Lovett, M., Young, P., Haman, K., Freeman, B., Gallop, R. (2005). Prevention of relapse following cognitive therapy vs medications in moderate to severe depression. Archives of General Psychiatry, 62, 417 - 422. Holmes, C. A. V. (1998). There is no such thing as a therapist: An introduction to the therapeutic process. London: H. Karnac (Books) Ltd. Horowitz , L., & Rosenberg, S. (1994). The Consensual Response Psychodynamic: Part One: Method and research results, Psychotherapy Research, 4( 3 & 4), 222 - 233. Horowitz, L. M., Inouye, D., & Siegelman, E. Y. (1979). On averaging judge’s ratings to increase their correlation with an external criterion. Journal of Consulting and Clinical Psychology, 47(3), 453 - 458. Hox J.J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Lawrence Erlbaum Associates. 247 Ilardi , S. S., & Craighead , W. E. ( 199 4 ). The role of nonspecific factors in cognitive therapy for depression. Clinical Psychology: Science and Practice, 1, 138 - 156. Ingram, B. L. (2006). Clinical case formulations: Matching the integrative treatment plan to the client. New Jersey: Wiley. J acobson, N., Dobson, K., Truax, P., Addis, M., Koerner, K., Gollan, J., Gortner, E., & Prince, S. (1996). A component analysis of cognitive behavioural treatment for depression. Journal of Consulting and Clinical Psychology, 64 (2) , 295 - 304. James , A. A. C. J ., Soler , A., & Weatherall , R . R . W. (2005). Cognitive behavioural therapy for anxiety disorders in children and adolescents. Cochrane Database of Systematic Reviews 2005, 4, 1 - 34. Jarrett, R, B., Schaffer, M., McIntire, D., Witt - Browder, A., Kraft , D., Risser, R, C. (1999) . Treatment of atypical depression with cognitive therapy or phenelzine: a double- blind, placebo - controlled trial. Archives of General Psychiatry, 56 ( 5) , 431 - 437. Johnson, M.E., Popp, C., Schacht, T.E., Mellon, J., & Strupp, H.H . (1989). Converging evidence for identification of recurrent relationship themes: Comparison of two methods. Psychiatry, 52, 275 - 288. Jones, D. T. (2002). Client attitudes towards homework administration in cognitive behavioural therapy: Relationship t o compliance and treatment outcome. Unpublished master’s thesis, Massey University, Albany, New Zealand. Jose, A., & Goldfried, M. (2008) . A transtheoretical approach to case formulation. Cognitive and Behavioral Practice, 15, 212 – 222. 248 Kazantzis, N. (2 003). Therapist competence in co gnitive- behavioural therapies: A review of the contemporary empirical evidence. Behaviour Change, 20( 1), 1 - 12. Kazantzis, N., Bjornholdt, A., Munro, M., Dobson, K., Merrick, P., Fetcher, R., & Jones, D. (2006). Development of the Homework Rating Scale: A measure of patients’ beliefs about homework in Cognitive Therapy. Poster presented at the 40 th Annual Convention of the Association for Behavioural and Cognitive Therapies (ABCT ), Chicago, USA . Kaz antzis, N., & Deane. F. P. (1998 ). Theoretical Orientations of New Zealand Psychologists: An international comparison. Journal of Psychotherapy Integration, 8( 2) , 97 - 113. Kazantzis, N., Deane, F. P., & Ronan, K. R. (2000). Homework assignments and therapy outcome: A metaanaly sis. Clinical Psychology: Science and Practice, 79, 189 - 202. Kazantzis, N., De ane, F. P., & Ronan, K. R. (2005 ). Homework Rating Scale II (HRS- II). Kazantzis, N., Deane, F. P., Ronan, K. R., & L’Abate, L. (Eds.). (2005). Using homework assignments in cognitive behavior therapy. New York: Routledge . Kazantzis, N., Dobson, K., Munro, M., & Wedge, P. (2006). Development of the Homework Adherence and Competence Scale: A measure of therapist competence. Poster presented at the 40 th Annual Convention of the Association for Behavioural and Cognitive Therapies (ABCT), Chicago, USA. 249 Kazantzis, N., MacE wan, J., & Dattilio, F. M. ( 2005 ). A guiding model for practice. In N. Kazantzis, F. P. Dean, K. R. Ronan, & L, L’Abate (Eds.), Using homework assignments in cognitive behavioral therapy. New York: Routledge . Kazantzis, N., & Ronan, K. R. (2006). Can between session activities (homework) be considered a common factor in psychotherapy? Journal of Psychotherapy Integration, 16( 2) , 115 127. Kazantzis, N., & Shin kfield, G. (2007). Conceptualizing patient barriers to adherence with homework assignments. Cognitive and behavioral practice, 14, 317 - 324. Kazantzis, N., Wedge, P., & Dobson, K. (2005). Homework Adherence and Competence Scale (HAACS). Kazantzis, N., Whi ttington, C., & Dattilio, F. (in press). Meta - analysis of homework effects in Cognitive Behavioral Therapy: A replication and Extension. Manuscript submitted for publication. Kazdin, A. (2006). Mechanisms of change in psychotherapy: Advances, breakthrou ghs, and cutting - edge research (do not yet e xist) . In R. R. Bootzin & Patrick, E. (Eds.), Strengthening Research Methodology : Psychology. Washington, DC: American Psychological Association. Kelly, G. A. (1955) . The psychology of personal constructs. New Yo rk: Norton. Kelly , P., Deane, F., Kazantzis, N., Crowe, T., & Oades, L. (2006). Use of homework by mental health case managers in the rehabilitation of persistent and recurring psychiatric disability. Journal of Mental Health, 15( 1) , 95 - 101. Kelly, M. A . R., Robert, J. E., & Cielsa, J. A. (2005). Sudden gains in cognitive behavioural treatment for depression: When do they occur and do they matter? Behaviour Research and Therapy, 43, 703 - 714 250 Krause, M. S. & Lutz, W. (2009). Process transforms inputs to d etermine outcomes: Therapists are responsible for managing processes. Clinical Psychology: Science and Practice, 16( 1) , 73 - 81. Kreft, I. G., de Leeuw, J., & Aiken, L. S. (1995). The effect of different forms of centering in hierarchical linear models. Multivariate Behavioural Research, 30(1) , 1 - 21. Kuyken, W. (2006). Evidence - based case formulation: Is the emperor clothed? In N.Tarrier (ed.) Case formulation in cognitive behaviour therapy: The treatment of challenging and complex clinical cases (pp. 12 - 35). London: Brunner - Routledge. Kuyken, W., Dudley, R., & Padesky, C. (2008). The science and practice of case conceptualization. Behavioural and Cognitive Psychotherapy, 36, 757 - 768. Kuyken, W., Fothergill, C., Musa., M., & Chadwick , P. (2005). The rel iability and quality of cognitive case formulation . Behaviour and Research Therapy, 43, 1187 - 1201. Kuyken, W., Kurzer, N., DeRubeis, R., Beck, A., & Brown, G. (2001). Response to cognitive therapy in depression: the role of maladaptive beliefs and pers onality disorders. Journal of Consulting and Clinical Psychology, 69( 3) , 560 - 566. Kuyken, W., Padesky, C. and Dudley, R. (2009). Collaborative case conceptualization: Working effectively with clients in cognitive-behavior therapy. New York: The Guilford P ress. Kuyken , W. , & Tsivrikos, D. (2009). Therapist competence, co - morbidity and cognitive- behavioral therapy for depression. Psychotherapy and Psychosomatics , 78, 42 - 48. 251 Kwok, O., Underhill, A. T., Berry, J. W., Luo, W., Elliot, T. R., & Yoon, M. (200 8). Analyzing longitudinal data with multilevel models: An example with individuals living with lower extremity intra - articular fractures. Rehabilitation Psychology, 53 ( 3) , 370 - 386. Lam, D. H., Watkins, E. R., Hayward, P., Bright, J., Wright, K., Kerr, N., Parr - Davis, G., & Sham, P. (2003). A randomized controlled study of cognitive therapy for relapse prevention for bipolar affective disorder: Outcome of the First Year. Archives of General Psychiatry, 60( 2) , 145 - 152. Lambert, W. E., Douchette, A., & Bickman, L. (2001). Measuring mental health outcomes. Journal of Behavioural Health Services and Research, 28( 3) , 273 - 286. Laschinger, H. K. (1992). Intraclass correlations as estimates of interrater reliability in nursing research. Western Journal of Nursing Research, 14, 246. Levenson, H. (1995). Time-Limited Dynamic Psychotherapy: A Guide to Clinical Practice. New York: Basic Books. Levenson, H., & Strupp, H.H. (2007). Cyclical maladaptive patterns: Case formulation in time- limited dynamic psychotherapy. In T. D. Eells (Ed.), Handbook of psychotherapy case formulation (pp. 164 - 197). New York: Guilford. Linehan, M. M. (1980). Content validity: Its relevance to behavioral assessment. Behavioral Assessment, 2, 147 - 159. Linehan, M. M. (1993). Cognitive Behavioral Treatment of Borderline Personality Disorder. New York: Guilford Press. 252 Little, R. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83, 1198 - 120 2. Little, R. J.A., & Rubin, D.B. (1987) . Statistical Analysis with Missing Data. New York: Wiley . Luborsky, L. (1997). The core conflictual relationship theme: A basic case formulation method. In T. Eells (Ed.). Handbook of psychotherapy case formulation. (pp. 58 - 83). New York: Guilford Press. Luborsky, L., & Crits - Christoph, P. (1998). Understanding transference: The Core Conflictual Relationship Theme method (2nd ed.). Washington, DC: American Psychological Association. Maas, C. J. M., & Hox, J. (2 005). Sufficient sample sizes for multilevel modelling. Methodology, 1( 3) , 85 - 91. Macdonald , G ., Higgins J . P. T., & Ramchandani , P. (2006). Cognitive- behavioural interventions for children who have been sexually abused. Cochrane Database of Systematic Reviews, 4 , 1 - 51. Magnusson, D. (1966). Test theory. Reading, MA: Addison - Wesley. March, J., Silva, S., Petrycki, S., Curry. J ., Wells, K., Fairbank. J ., Burns, B., Domino, M., McNulty, S., Vitiello, B., Severe, J; Treatment for Adolescents With Depress ion Study (TADS) Team. (2004). Fluoxetine, cognitive - behavioral therapy, and their combination for adolescents with depress ion. Treatment for Adolescents with Depression Study (TADS) randomized controlle d trial. Journal of the American Medical Association, 292, 807 - 820 . Manring , J., Beitman , B. D., & Dewan, M. J. (2003). Evaluating competence in psychotherapy. Academic Psychiatry, 27( 3) , 136 - 144. 253 Mansell, W. (2008). The seven C’s of CBT: A consideration of the future challenges for Cognitive Behaviour The rapy. Behavioural and Cognitive Psychotherapy, 36, 641 - 649. Mausbach, B . T., Moore, R., Roes ch, S., Cardenas, V., & Patterson, T. L. (2010). The relationship between homework compliance and therapy outcomes: An updated meta- analysis. Cognitive Therapy and Research, Published online on Feb 6 2010. McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients, Psychological Methods, 1(1) , 30 - 46. Mc Graw, K. O., & Wong, S. P. (1999 ). Confidence intervals and F tests f or intraclass correlations based on three- way random effects models, Educational and Psychological Measurement, 59 , 270 - 288 . Meier, S. T. (1999). Training the practi tioner- scientist: Bridging case conceptualization, assessment, and intervention. The Counseling Psychologist, 27, 846 - 869. Miles M. B. , & Huberman A. M. , (1984) . Qualitative data analysis: A sourcebook of new methods. Newbury Park, CA: Sage. Miller, S., Duncan, B., Brown, J., Sparks, J., & Claud, D. (2003). The Outcome Rating Scale: A prelim inary study of reliability, validity, and feasibility of a brief visual analogue measure. Journal of Brief Therapy, 2, 91 - 100. Miller, S. D., Duncan, B. L., & Johnson, L. D. (2000). The Session Rating Scale. Available at http://www.talkingcure.com Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005 ). Partners for change outcome management session. Journal of Clinical Psychology, 61(2), 199 - 208. 254 Milne, D., Claydon, T., Blackburn, I., James, I., & Sheikh, A. (2001). Rationale for a new measure of competence in therapy. Behavioural and Cognitive Psychotherapy, 29, 21 - 23. Morasco, B. J., Weinstock, J., Ledgerwood, D. M., & Petry, N. M. (2007). Psychological factors that promote and inhibit problem gambling. Cognitive and Behavior Practice, 14, 2 08 - 217. Mumma, G. (1998). Improving cognitive case formulations and treatment planning in clinical practice and research, Journal of Cognitive Psychology: An International Quarterly, 12( 3) , 251 - 274. Mumma, G. (2004). Validation of Idiosyncratic Cogniti ve Schema in Cognitive Case Formulations: An Intraindividual Idiographic Approach. Psychological Assessment, 16(3), 211 - 230. Mumma, G. & Mooney, S. R. (2007 a). Comparing the validity of alternative cognitive case formulations: A latent variable, multivar iate time series approach. Cognitive Therapy and Research, 31(4), 451 - 481. Mumma, G. & Mooney, S. R. (2007b). Incremental validity of cognitions in a clinical case formulation: An intraindividual test in a case example. Journal of Psychopathology and Behavioural Assessment, 29, 17 - 28. Mumma, G., & Smith, J. (2001). Cognitive– Behavioral - Interpersonal Scenarios: Interformulator reliability and convergent validity. Journal of Psychopathology and Behavioral Assessment, 23(4), 203 - 221. Muran, J. C., Samstag, L. W., Segal, Z. V., & Winston, A. (1998). Interpersonal scenarios: A measure of self - schemas. Psychotherapy Research, 8(3), 321 - 333. 255 Muran, J. C., Samstag, L. W., Ventur, E. D., Segal, Z. V., & Winston, A. (2001). A cognitive- interpersonal case study of self. Journal of Clinical Psychology, 57(3), 307 - 330. Muran, J. C., Segal, Z. V., & Samstag, L. W. (1994). Self - scenarios as a repeated measures outcome measurement of self- schemas in short- term cognitive therapy. Behavior Therapy, 25, 255 - 274. Munro, M. D. (2006). Client and therapist variability with psychotherapy homework: A preliminary psychometric evaluation of two scales. Unpublished master’s thesis, Massey University, Albany, New Zealand. National Institute for Heal th and Clinical E xcellence ( 2009) . Depression: The treatment and management of depression in adults. L ondon, United Kingdom: National Institute for Heal th and Clinical E xcellence. Needleman, L. ( 1999). Cognitive case conceptualization: A guidebook for practitioners. Mahwah, NJ, La wrence Erlbaum Associates. New Zealand Psychological Society. (2002). Code of ethics for psychologists working in Aotearoa /New Zealand. Wellington, New Zealand: New Zealand Psychological Society. Nezu, A. M. & Nezu, C. M. (2005). Comments on "Evidence - based behavioral medicine: What is it and how do we achieve it?": The interventionist d oes not always equal the intervention - The role of therapist competence. Annals of Behavioral Medicine, 29(1), 80 Nezu, A. M. & Nezu, C. M. (Eds.). (2007 ). Evide nce- based outcome research: A practical guide to conducting randomized controlled trials for psychosocial interventions. New York: O xford University press. 256 Nezu, A. M. & Nezu, C. M., Lombardo, E. (2004). Cognitive-Behavioural case formulation and treatment design: A problem solving approach. New York: Springer Publishing Company. Nich, C. & Carroll, K. (1997). Now you see it, now you don’t: A comparison of Traditional versus random- effects regression models in the analysis of longitudinal follow- up data from a clinical trial. Journal of Consulting and Clinical Psychology, 65 , 252 - 261. Nichols, D.P. (1999) , My coefficient alpha is negative! , SPSS Keywords, 68 , available at: http://www.ats.ucla.edu/stat/SPSS/library/negalpha.htm . O kiishi, J., Lambert, M. J ., Eggett, D., Nielsen, L., Dayton, D. D., & Vermeersch, D. A. (2006). An analysis of therapist effects: Toward providing feedback to individual therapists on their clients’ psychotherapy outcome. Journal of Clinical Psychology, 62( 9) , 1157 - 1172. Okiishi , J., Lambert, M. J., Nielsen, S. L., & Ogles, B. M. (2003). Waiting for supershrink: an empirical analysis of therapist effects. Clinical Psychology and Psychotherapy, 10, 361 - 373. Pain, C. M., Chadwick, P. & Abba, N. (2008). Clients’ experience of cas e formulation in cognitive behaviour therapy for psychosis. British Journal of Clinical Psychology, 47, 127 – 138. Padesky, C. , & Mooney, K. (1990). Clinical tip: Presenting the cognitive model to clients. International Cognitive Therapy Newsletter, 6, 13 - 14. Paykel , E., Scott, J., Cornwall, P., Abbott, R., Crane, C., Pope, M., & Johnson, L. (2005). Duration of relapse prevention after cognitive therapy in residual depression: follow - up of controlled trial, Psychological Medicine, 35( 1) , 59 - 68. 257 Perepletchikova, F., & Kazdin, A. E. Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12( 4) , 365 - 383. Perry, J. (1994). Assessing psychodynamic patterns using the Idiographic Conflict F ormulation method, Psychotherapy Research, 4( 3 & 4) , 239 - 252. Persons, J. (1989). Cognitive therapy in practice: a case formulation approach. New York: Norton. Persons, J. (2001). Individualized formulation and treatment planning. In J. Persons, J. David son, & M. Tompkins (Eds.). Essential components of Cognitive- Behavior Therapy for depression, Washington, DC: American Psychological Association. Persons, J. (2005). Empiricism, mechanism and the practice of cognitive - behavioural therapy, Behaviour Therapy, 36, 107 - 118. Persons, J. (2008). A Case formulation approach to Cognitive-Behavior Therapy. New York: Guilford. Persons, J., & Berta gonolli, A. (1999). Inter - rater reliability of cognitive- behavioural case formulations of depression: a replication. Cognitive Therapy and Research, 23( 3) , 271 - 283. Persons, J., Mooney, K., & Padesky, C. (1995). Interrater reliability of cognitive - behavioural case formulations. Cognitive Therapy and Research, 19( 1) , 21 - 34. Persons, J., Roberts, N., Zalecki. C., & Brec hwald, W. (2006). Naturalistic outcome of case- formulation- driven cognitive- behavior therapy for anxious depressed outpatients, Behaviour, Research and Therapy, 44, 1041 - 1051. 258 Persons, J. , & Tompkins, M. (1997). Cognitive - behavioral case formulation. In T. Eells (Ed.), Handbook of psychotherapy case formulation. New York: Guilford. Persons, J. (2006). Case formulation - driven psychotherapy. Clinical Psychology: Science and Practice, 13( 2) , 167 - 170. Peugh, J. L., & Enders, C. K. (2005). Using the SPSS Mi xed procedure to fit cross - sectional and longitudinal multilevel models. Educational and Psychological Measurement, 65, 717 - 741. Pike , K. M., Walsh B. T., Vitousek. K., Wilson G. T., & Bauer, J. (2003). Cognitive behavior therapy in the posthospitaliza tion treatment of anorexia nervosa. American Journal of Psychiatry, 160, 2046 - 2049. Primakoff, L., Epstein, N., & Covi, L. (1986). Homework compliance: An uncontrolled variable in cognitive therapy outcome research. Behavior Therapy, 17, 433 - 446. Prochaska, J. O., & DiClemente, C. C. (2005). The transtheoretical approach. In J. C. Norcross & M. R. Goldfried (Eds.), Handbook of psychotherapy integration ( 2nd ed., pp. 147 - 171). New York: Oxford. Raudenbush , S. W., & Bryk , A. S. ( 2002 ). Hierarchical Linear Models: Applications and data analysis methods (2 nd ed.). Thousand Oaks, CA: Sage Publications. Raudenbush, S. W., & Bryk, A. S., Cheong , Y., & Congdon, R. T. (2000). HLM 5: Hierarchical linear and nonlinear modeling. Chicago: Scientific Software Inte rnational. Rector, N. A. (2007). Homework use in cognitive therapy for psychosis: A case formulation approach. Cognitive and Behavioural Practice, 14, 303 - 316. 259 Rector, N. A., Seeman, M. V., & Segal, Z.V. (2003). Cognitive therapy for schizophrenia: A preliminary randomized controlled trial . Schizophrenia Research, 63, 1 – 11. Rees, C., McEvoy, P., & Nathan, P. (2005). Relationship between homework completion and outcome in cognitive behaviour therapy. Cognitive Behavioural Therapy, 34( 4) , 242 - 247. Ro gers, G., Reinecke, M., & Curry, J. (2005). Case formulation in TADS CBT. Cognitive and Behavioral Practice, 12, 198 - 208. Rosenberg, S. E., Silberschatz, G., Curtis, J. T., Sampson, H., & Weiss, J. (1986). A method for establishing reliability of statements from psychodynamic case formulations. The American Journal of Psychiatry, 143(11), 1454 - 1456. Rosenman, S. J., Levings, C. T., & Korten, A. E. (1997). Clinical u tility and patient acceptance of the computerized composite international d iagnostic interview. Psychiatric Services, 48( 6) , 815 - 820. Rubin, D.B. (1976) . Inference and Missing Data. Biometrika, 63 , 581 - 592 . Sacco, W., & Beck, A. T. (1995). Cognitive Theory and Therapy. In E.E. Beckham & W.R. Leber (Eds.), Handbook of Depression ( 2 nd ed., pp. 329 - 351). New York: The Guilford Press. Sa nderson, W. (2006). Evaluating adherence and flexibility in the use of a manual in clinical practice. Pragmatic Case Studies in Psychotherapy, 1( 2) , 1 - 5. Schafer, J.L. (1997) . The Analysis of Incomplete Multivariate Data. London: Chapman & Hall . 260 Scheel, M. J., Hanson , W. E., & Razzhavaikina , T. L. ( 2004 ). The process of recommending homework in psychotherapy: A review of the therapist delivery methods. Psychotherapy: Theory, Research, Practice, Training, 41 ( 1) , 38 - 55. Schotte, C. K. W., Maes, M., Cluydts, R., De Doncker, D., & Cosyns, P. ( 1997 ). Construct validity of the Beck Depression Inventory in a depressive population. Journal of Affective Disorders, 46, 115 - 125. Schr öder, T., Wiseman, H., & Orlinsk y, D. ( 2009 ). “You were always on my mind”: Therapists’ intersession experiences in relation to their therapeutic practice , professional characteristics, and quality of life. Scot, M. J., & Sembi, S. (2006). Cognitive behaviour treatment failures in prac tice: The neglected role of diagnostic inaccuracy. Behavioural and Cognitive Psychotherapy, 34, 491 - 495. Sensky, T., Turkington, D., Kingdon, D., Scott, J., Siddle, R., O’Carroll, M., & Barnes, T. R. (2000). A randomized controlled trial of cognitive - behavioral therapy for persistent symptoms in schizophrenia resistant to medication. Archives of General Psychiatry, 57, 165 −172. Shaw, B., & Dobson, K. (1988 ). Competency judg ments in training and evaluation of psychotherapists. Journal of consulting and clinical psychology, 56( 6), 837 - 846. Shaw, B., Olmsted, M., Dobson, K., Sotsky, S., Elkin, I., Yamaguchi, J., Vallis, T., Lowery, A., Watkins, J., & Imber, S. (1999). T herapist competence ratings in relation to clinical outcome in cognitive therapy of depression. Journal of consulting and clinical psychology, 67(6), 837 - 846. 261 Shelton, J. L., & Ackerman, J. M. (1974). Homework in counselling and psychotherapy: Examples of systematic assignments for therapeutic use by mental health professionals. Springf ield, IL: Charles C. Thomas. Shelton, J. L., & Levy, R. L. (1981). Behavioural assignments and treatment compliance: A handbook of clinical strategies. Champaign, IL: Research Press. Shrout , P. E., & Fleiss , J. L. ( 1979 ). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420 - 428. Silberschatz , G. Fretter , P. & Curtis, J. ( 1986 ), How do interpretations influence the process of psychotherapy? Journal of Consulting and Clinical Psychology, 54, 646 - 652. Sim, K., G wee, K. P., & Bateman, A. (2005). Case formulation in psychotherapy: revitalizing its usefulness as a clinical tool . Academic Psychiatry, 29( 3) , 289 - 292. Sim, J. & Wright, C. C. (2005).The Kappa statistic in reliability s tudies: Use, interpretation, and sam ple size r equirements, Physical Therapy. 85, 257 - 268. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis. New York: Oxford University Press. Snijders, T. & Bosker, R. (1999 ). Multilevel Analysis: An introduction to basic and advanced multilevel modeling. London: Sage. Sperry, L., & Maniacci, M. (1992). An integration of DSM - I I I - R Diagnoses and Alderian case formulations. Individual Psychology, 48( 2) , 175 - 181. 262 Startup, M., Jackson, M., Pearce, E. (2002). Assessing therapist adher ence to cognitive- behaviour therapy for psychosis. Behavioural and Cognitive Psychotherapy, 30, 329 - 339. Strauss, A. & Corbin, J. (1998). Basics of Qualitative Research Techniques and Procedures for Developing Grounded Theory ( 2 nd ed.) . Sage Publication s: London. Steer, R. A., Ball, R., Ranieri, W., & Beck, A. T. ( 1999). Dimensions of the Beck Depression Inventory - II in clinically depressed outpatients. Journal of Clinical Psychology, 55( 1) , 117 - 128. Strupp , H. H. & Binder , J. L. ( 1984 ) , Psychotherap y in a New Key: A Guide to Time - Limited Dynamic Psychotherapy. New York: Basic Books. Stenhouse, L., & Van Kessel, K. (2002). Cognitive Therapy and Dialectical Behaviour Therapy: An integrative approach to the conceptualization of Borderline Personality Disorder. New Zealand Journal of Psychology, 31( 2), 87 - 92. Sudak, D. M., Beck, J. S., & Wright, J. (2005). Cognitive Behavioral Therapy: A blueprint for attaining and assessing psychiatric resident competency. Academic Psychiatry, 27( 3) , 154 - 159. Svartb erg, M. (1999). Therapist competence: Its temporal course, t emporal stability, and determinants in short- term anxiety - provoking p sychotherapy, Journal of Clinical Psychology, 55, 1313 - 1319. Svartberg , M., Stiles, T. C., & Seltzer, M. (2004). Randomised, c ontrolled trial of the effectiveness of short- term dynamic psychotherapy and cognitive therapy for cluster C personality disorders. American Journal of Psychiatry, 161, 810 - 817. 263 Swift, J. & Callahan, J. (2009). Early psychotherapy processes : An examinati on of client and trainee clinician perspective convergence. Clinical Psychology and Psychotherapy, 16, 228 - 236. Tang, T., & DeRubeis, R. (1999). Reconsidering rapid early response for cognitive behavioural therapy of depression. Clinical Psychology: Science and Practice, 6 ( 3) , 283 - 288. Thase, M. E., & Callan, J. A. (2006). The role of homework in cognitive behaviour therapy of depression. Journal of Psychotherapy Integration, 16( 2), 162 - 177. Tomkins, M. A. (2004). Using homework in psychotherapy. New Y ork: The Guilford Press. Trepka, C., Rees, A., Shapiro, D., Hardy G., & Barkham, M. (2004). Therapist competence and outcome of cognitive therapy for depression. Cognitive Therapy and Research, 28( 2), 143 - 157. Turkat, I. (1987). Invited case transcri pt: the initial clinical hypothesis, Journal of Behaviour, Therapy and Experimental Psychiatry, 18( 4) , 349 - 356. Vallis, T., Shaw, B., & Dobson, K. (1986). The Cognitive Therapy Scale: Psychometric Properties, Journal of Consulting and Clinical Psychology, 54 ( 3) , 381 - 385. Viglione, D. J., & M eyer, G. J. ( 2007 ). An overview of Rorschach psychometric s for forensic practice. In C. B. Gacono , & F. Barton Evans (Ed s.), Handbook of forensic Rorschach assessment (pp. 3 – 20). New York: Taylor & Francis Group . Wa ltz, J. & Linehan, M. M. (1999). Functional analysis of borderline behavioral criterion patterns: Links to treatment. In J. Derksen & H. Groen (Eds.), Treatment of personality disorders. NY: Plenum Press. 264 Wedge, P. (2005). The development of the Homework Adherence and Competence Scale (HAACS ): A measure for assessing therapist adherence and competence in administering homework assignments within cognitive behaviour therapy. Unpublished master’s thesis, Massey University, Albany, New Zealand. Weerasekera, P. (1996). Multiperspective case formulation: A step towards treatment integration. Melbourne, Fla: Krieger Publishing . Westen , D . (1998) Case formulation and personality diagnosis: two processes or one? In J. Barron (Ed. ) , Making diagnosis meaningful: Enhancing evaluation and treatment of psychological disorders ( pp. 111 - 138 ). Washington, DC: American Psychological Association. Whisman, M. (1993). Mediators and moderators of change in the cognitive therapy of depression. Psychological Bulletin, 114( 2) , 248 - 265. Wilson, G. (1996). Manual - based treatments: the clinical application of research findings. Behaviour Research and Therapy, 34( 4) , 295 - 314. Young, J., & Beck, A. (1980). Cognitive Therapy Scale rating manual: Revised draft August 22, 1980. Unpu blished manuscript, University of Pennsylvania, Philadelphia. Available from www.academyofct.org. Young, J., Beck, A. T., & Budenz, D. (1983). Assessment of compliance in cognitive therapy. Unpublished manuscript, University of Pennsylvania, Philadelphia. Yovel, I ., & Safren , S . A. (2007). Measuring homework utility in psychotherapy: CBT for adult ADHD as an example. Cognitive Therapy and Research, 31, 385 - 399. 265 APPENDIX A Participant Information Sheet 266 267 APPENDIX B Participant Consent Form 268 APPENDIX C Participant Demographic Questionnaire 269 APPENDIX D Therapist / Rater Demographic Questionnaire 270 APPENDIX E CRS – Final Draft 271 272 27 3 274 275 276 277 278 279 280 APPENDIX F CRS – Therapist Competence Scale 281 APPENDIX G CRS - Component Identification Guide 282 APPENDIX H Training Material Conceptualization Rating Scale (CRS) Training Pack Michael Easden Nikolaos Kazantzis, PhD February 2008 Overview of Today TRAINING ACTIVITY TIME Introduction to using the Conceptualization Rat ing Scale (CRS) 9.00am -10.00am Rating of 1s t snippet + discussion 10.00am - 10.30am MORNING TEA 10.30am - 10.45pm Rating of 2nd and 3rd snippets + discussion 10.45pm - 12.30pm LUNCH BREAK 12.30pm - 1.30pm Rating of 1s t therapy session 1.30pm - 2.15pm Discussion of 1st rating 2.15pm - 2.45pm AFTERNOON TEA 2.45pm - 3.00pm Rating of 2nd therapy session 3.00pm - 4.15pm Discussion of 2nd therapy session 4.15pm - 4.45pm Final discussion and wrap up 4.45pm - 5.00pm 283 Presentation Overview Context: – The CBT Homework Project aka “Depression Study” – Conceptualization and homework – Thesis objectives and method Practice exercise: What we know about case conceptualization What is case conceptualization? What is cognitive case conceptualization? Therapist use of case conceptualization Conceptualization Diagram Conceptualization Rating Scale (CRS): Overview, terminology and general rating considerations Practice exercise: Using the CRS Discussion, wrap up, and next steps Context: The CBT Homework Project To remind you…the CBT Homework project or “Depression Study” has a broad aim to increase understanding of the effect of homework as a mechanism of change in psychotherapy Separately conceptualization and homework have received much attention However, cognitive case conceptualization has had little attention Few studies have investigated the relationship between case conceptualization and homework or whether one or the other makes a greater contribution to outcome 284 Context: Conceptualization and Homework Three broad goals a therapist has over the course of psychotherapy are to: Understand the client and their presenting problems Apply a treatment plan promoting skill acquisition in session Promote generalization of these skills into areas of deficit in a client’s life Case Conceptualization Treatment Plan / Standard Treatment Protocol Homework Context: Thesis Objectives Empirical support for CBT case conceptualization based on existing theory Develop a new measure Test the psychometric properties Investigate how therapist conceptualizations change over time Investigate the relationship between homework, conceptualization and outcome Discuss findings and implications for future research 285 Context: Thesis Method Develop measure – 1st draft Expert feedback Training Study #1 (training ratings) Analysis – Intra-class correlation coefficient (ICC), ra ter feedback, expert feedback Revise measure – 2nd draft Study #2 (systematic session ratings) Analysis – Intra-class correlation coefficient (ICC), ra ter feedback, expert feedback Revise measure – final draft Today: Training Purpose and Procedure  Raters gain a knowledge of cognitive case conceptualization adequate to score therapist competence on the CRS  The training phase will continue until reliability between raters is acceptable. Ratings will be made independently after and during the viewing of a recorded therapy session  Recalibration will take place following at the completion of the training phase where upon a further session will be rated on the CRS and discussed 286 Practice Exercise: Case Conceptualization in CBT An exercise to activate your existing knowledge! Collaboratively draw a mind- map EVERYTHING you know about case conceptualization Use others’ ideas to stimulate new ideas Use colour Use pictures Be creative HAVE FUN Conceptualization Behaviour Emotion Thoughts understanding What is Case Conceptualization? A case conceptualization should be a working, evolving hypothesis about a client with the goal of integrating a client’s presenting problems into a justifiable, individualized and consistent description accounting for a client’s psychological, interpersonal and behavioural deficits Used by the therapist to guide the treatment plan and subsequent interventions Intervention re-informs the evolving conceptualization process over the course of therapy 287 What is Cognitive Case Conceptualization? Cognitive Case Conceptualization • In cognitive case conceptualizations (CCC), cognitive- behavioural theory guides explanatory inferences and exists as a framework for understanding the client’s presenting problems and informs intervention • In standard CBT a CCC is used as standard and best practice • Specific formats used have not been evaluated thoroughly References: Eells, 1997, 2007; Ee lls, Lombart, Kendje lic, Turner, & Lucas, 2005; Curtis & Si lbershatz, 1997; Bieling & Kuyken, 2003; Kuyken, Fothergill, Musa, & Chadwick, 2005; Needleman, 1999; Persons, 1989, 2001 Therapist Use of Case Conceptualization Case Conceptualization is a fundamental feature of CBT Early phase of therapy – Therapist begins data gathering and hypothesis testing, focus on “surface“ level information (i.e., thoughts, emotions, behaviours) Middle Phase of therapy – Therapist begins to focus more on “deeper” level information (i .e., core beliefs, conditional assumptions, childhood data, compensatory strategies) – Therapist usually shares the entire conceptualization with the patient around session 10 Late Phase of Therapy – Consolidation of conceptualization, generalization of understanding of underlying mechanisms (e.g., common core beliefs/s, AT’s, emotions in depressionistic cycle) 288 Figure 1. J. Beck Cognit ive Case Conceptualizati on Diagram (1995) Emotion Conditional Assumptions / Beliefs / Rules Compensatory Strategies BehaviourBehaviour Behaviour EmotionEmotion Automatic ThoughtAutomatic ThoughtAutomatic Thought Situation 3Situation 2Situation 1 Core Beliefs(s) Relevant Childhood Data / Background Information Meaning of the AT Meaning of the AT Meaning of the AT Other people are critical I am not good enough If I don’t respect others then I am selfish If I don’t work hard / comply then others wi ll not respect me If I don’t succeed in things then I am a failure Does things asked of her / suggested to her Avoids decisions / too much responsibility. Talking to self. Puts others fi rst If I don’t do this, I’m not he lping myse lf I should be compliant Others are critical Disappointed, anxious Difficulty making decision about shifting I might make the wrong decision I’m no t good enough Sad, anxious, stressed Wakes up at night, can’t sleep, won’t read Need to sleep, but don’t want to disturb husband I should respect others Sad, anxious Critical Father / high expectations High achiever of family Mother depressed and drinking throughout teenage years Got out of bed and tried strategies Talking to self Avoid making decisions Put others needs first Difficult ly forcing self out of bed to try sleep routine CRS: Overview The measure is conceptually driven Items are modelled on the J. Beck Case Conceptualization Diagram Measures therapist behaviours / interventions in regards to case conceptualization Each item is rated based on four key domains: a) Integration of the conceptualization component b) Importanceof the in tegration of the conceptual ization component c) Competence in the integration of the conceptual ization component d) Fit of the conceptual ization component 289 a) Integration of the conceptualization component Integration is rated dichotomously (i.e., “yes” or “no”) Refers to the presence of therapist integration of components of the case conceptualization during the therapy session (i.e., did the therapist discuss the conceptualization component in the session or not). These ratings are made irrespective of how competently the conceptualization component is delivered. a) Integration of the conceptualization component Key Features: – Therapist asks about the conceptualization component – Therapist follows up a patient ‘s comment regarding the conceptualization component – The therapist may not specifically mention / fol low up a conceptualization component that a patient mentions as long as it is clear from the session that the therapist has prompted / been involved in the discussion in some way (i .e., discussion is not relevant due to previous discussions the patient / therapist allude to) 290 b) Importance of the integration of the conceptualization component Importance is rated dichotomously (i.e., “yes” or “no”) This rating refers to the importance of the integration of the conceptualization component into the session (i.e. was it appropriate for the therapist to leave out this part of the conceptualization or was the component neglected in this session). b) Importance of the integration of the conceptualization component Key Features: – “Surface” level conceptualization components should ordinarily be discussed at EVERY session – “Deeper” level conceptualization component MAY NOT be discussed during the early phase of therapy (e.g., sessions 1 or 3) but SHOULD be discussed with increasing importance during middle to late phases of therapy (e.g., session s 5, 8, 10) – Sharing of the conceptualization usually only discussed at session 10 – Disorder –specific cognitive model should be discussed at EVERY session – Client resilience / strengths should be discussed at EVERY session – There are exceptions! (i.e., crisis / safety situations, very complex / difficult patients) 291 c) Competence in the integration of the conceptualization component Competence is rated on a 7-point Likert scale (from 0 = ‘low competence’ to 6 = ‘high competence’), with descriptive anchors which build in complexity. The competence refers to the skill of the therapist in sharing information gathered or contained within the conceptualization, and incorporating information from the case conceptualization into therapy. This includes use of conceptualization as part of integrating homework tasks and hypothesis testing of important information from the therapy session. If several items seem to apply equally well, record the lowest number (e.g., if considering recording “3-4”, record it as a “3”). d) Fit of the conceptualization component Fit is rated on a 6-point Likert scale (from 1 = ‘strongly disagree’ to 6 = ‘strongly agree’), on a more global rating. Match or coherence between in session integration of conceptualization and written conceptualization The fit of case conceptualization should be based on the clinical material obtained this session and any other information available (i.e., standardized measures), and whether the written material presents a coherent formulation of the etiology and maintenance of the patient’s diff iculties. If several items seem to apply equally well, record the lowest number (e.g., if considering recording “3-4”, record it as a “3”). 292 CRS: General Rating Considerations Some ideas which may help: – Please try and rate each item on its own merit – for competence items, please try and use the full range of the Likert scale (i.e., use the descriptive anchors as your guide) – Remember that sometimes it will not be necessary, or may be better for a therapist NOT to integrate some conceptualization components into a session (e.g., core bel iefs may not be discussed until later sessions in therapy, the entire conceptualization is usually not shared until around session 10) – Similarly, although some therapists may discuss one some components well they may not discuss others with the same level of competence (e.g., a therapist might assume that a cl ient feels a certain way but have sought feedback regarding automatic thoughts and how they link with other aspects of the conceptualization) Practice: Rating a CBT clip During the following clip practice using the CRS to rate “surface” level components (items 1 - 5) Lets discuss! 293 Practice: Rating another CBT clip During the following clip practice using the CRS to rate “deeper” level components (items 6 - 9) Lets discuss! Practice: Rating another CBT clip During the following clip practice using the CRS to rate “different”, “alternative” or “broader” level items (items 10 - 12) Lets discuss! 294 Practice: Rating a CBT session During the following CBT session practice rating the entire session using the entire CRS (items 1 - 12) Lets discuss! 295 APPENDIX I Depression Study Feedback Questionnaire 296 297 298 299 300 301 APPENDIX J Depression Study Advertisement Example 302 APPENDIX K Depression Study Advertisement Pamphlet 303 APPENDIX L Treatment of time model Table 29 Estimates of random effects from a series of individual growth models of time for predicting change in depressive symptoms Variance Components Parameter Model T0 Model T1 Model T2 Model T3 Model T4 Model T5 Level 1 Within person σ2e 39.91*** 21.15*** 22.69*** 23.94*** 27.02*** 21.55*** 3.58 2.02 2.16 2.28 2.57 2.08 Level 2 In initial status σ20 37.76*** 10.76~ 10.61* 15.85*** 19.06*** 12.66* 11.18 5.63 5.38 5.82 6.60 7.54 In rate of change σ21 0.16 1.59 0.21** 0.74*** 0.00 0.99 1.59 0.09 0.26 0.01 Covariance σe 0.97*** 2.61*** 0.00*** 0.04** 4.55E - 5 0.33 0.93 0.00 0.02 1.94E - 5 Psuedo R2 Statistics and Goodness of Fit R2e 0.47 0.43 0.40 0.38 0.46 R20 0.72 0.72 0.58 0.50 0.66 Deviance 1729.72 1743.87 1755.29 1779.49 1742.63 AIC 1741.72 1755.87 1767.29 1791.49 1754.63 BIC 1763.44 1777.59 1789.01 1813.21 1776.35 304 APPENDIX M Graphs of linear regressions Figure 32. Linear regressions for change in depression severity as measured by BDI - I I change scores for individual patients by therapy session 305 Figure 33. Linear regressions of HAACS competence scores for individual patient by therapy session Figure 34. Linear regressions of CRS competence scores for individual patient by therapy session 306 Figure 35. Linear regressions of CRS fit / match scores for individual patient by therapy session Figure 36. L inear regressions for change in the fit of written and in- session content of CCs as measured by CRS fit / match scores for individual patients by therapy session 307 Table 30 Estimates of fixed effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms Fixed Effects Parameter Model 1 Model 2 Model 3 Model 4 Model 5 Initial Status π0i Intercept γoo 10.78*** 8.47*** 5.73*** 9.65*** 7 .40*** 1.33 1.06 1.42 3.05 2.48 CRS Competence γ01 0.11* 0.12* 0.05 0.06 HAACS Competence γ02 - 0.03 - 0.40 0.04 0.42 Rate of Change π1i Intercept γ10 0.16* ** 0.28 ** * 0.07 0.18~ 0.04 0.07 0.09 0.09 CRS Competence γ11 - 0. 00* - 0.00* 0.00 0.00 HAACS Competence γ12 0.00 0.00~ 0.00 0.00 Note. Model 1 is an unconditional means model. Model 2 is an unconditional growth mo del. Model 3 adds CRS Competence to the total model. Model 4 removes CRS Competence and adds HAACS Competence to the total model. Model 5 includes CRS and HAACS Competence. 308 Table 31 Estimates of random effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms Variance Components Parameter Model 1 Model 2 Model 3 Model 4 Model 5 Level 1 Within p erson σ2e 37.91 *** 28.60 *** 16.77 *** 28.41*** 16.66 *** 3.78 3.02 1.81 3.03 1.81 Level 2 In initial status σ20 42.65 *** 23.39 *** 23.52 *** 23.84*** 24 .20 *** 12.89 8.35 7.71 8.51 7.93 In rate of change σ21 0.61*** 0.51** 0.58*** 0.48** 0.20 0.20 0.21 0.20 Covariance σe 0.00* 0.02* 0.02~ 0.02~ 0.01 0.01 0.01 0.01 Psuedo R2 Statistics and Goodness of Fit R2e 0.25 0.41 0.00 0.42 R20 0.45 0.00 - 0.02 - 0.03 R21 0.16 0.11 0.11 Deviance 1537.64 1483.91 1349.76 1465.32 1330.55 AIC 1543.64 1495.98 1365.76 1481.32 1350.55 BIC 1553 .93 1516.56 1392.99 1508.65 1384.44 Δ AIC 47.02 123.57 7. 91 132.12 309 Table 32 Estimates of fixed effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after introducing beliefs about homework and the match / fit of therapist case conceptualizations Fixed Effects Parameter Model 6 Model 7 Model 8 Model 9 Model 10 Initial Status π0i Intercept γoo 4.76~ 7.73*** 3.42 6.78** 2.39 2.72 0.03 3.43 2.80 3.91 CRS Competence γ01 0.15** 0.20** 0.20** 0.06 0.07 0.07 HAACS Competence γ02 - 0 .05 - 0.03 - 0.03 0.04 0.05 0.04 HRS Beliefs γ03 0.13 0.14 0.15 0.08 0.08 0.09 CRS Match / Fit γ04 0.03 - 0.03 - 0.03 0.05 0.06 0.06 Rate of Change π1i Intercept γ10 0.04 0.22*** 0.05 0.18 0.05 0.12 0.08 0.15 0.10 0.16 CRS Competence γ11 - 0.01 ** - 0.00~ - 0.00* 0.00 0.00 0.00 HAACS Competence γ12 0.00* 0.00 0.00 0.00 0. 00 0.00 HRS Beliefs γ13 0.00 0.00 0.00 0.00 0.00 0.00 CRS Match / Fit γ14 - 0.00 0.00 - 0.00 0.00 0.00 0.00 310 Table 33 Estimates of random effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after introducing beliefs about homework and the match / fit of therapist case conceptualizations Variance Components Parameter Model 6 Model 7 Model 8 Model 9 Model 10 Level 1 Within person σ2e 27.67** 18.56*** 16.61*** 17.32*** 17.31*** 2.94 2.18 1.82 2.07 2.08 Level 2 In initial status σ20 23.36 * 23.22 * 24.26*** 24.91** * 24.82*** 22.2 7.89 7.94 8.46 8.44 In rate of change σ21 0.63** 0.46** 0.49* 0.46* 0.49* 0.21 0.19 0.19 0.21 0.21 Covariance σe 0.02~ 0.02* 0.02* 0.02* 0.02* 0.01 0.02 0.01 0.01 0.02 Psuedo R2 Statistics and Goodness of Fit R2e 0.03 0.35 0.42 0.39 0.39 R20 0.00 0.00 - 0.04 - 0.06 - 0.06 R21 - 0.03 0.25 0.20 0.25 0.20 Deviance 1470.52 1192.94 1322.78 1165.80 1158.33 AIC 1486.52 1208.94 1346.78 1189.80 1186.33 BIC 1513.92 1235.04 1387.39 1288.76 1231.71 Δ AIC 9.46 287.04 149.17 306.18 309.65 311 Table 34 Estimates of fixed effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after controlling for depressive symptom severity and personality beliefs Fixed Effects Parameter Model 11 Model 12 Model 13 Model 14 Model 15 Initial Status π0i Intercept γoo 3.02 12.00*** 3.08 9.69* 5.91 2.34 3.27 3.01 3.73 3.94 CRS Competence γ01 0.12* 0.13** 0.13** 0.05 0.05 0.05 HAACS Competence γ02 - 0.04 - 0.04 - 0.04 0.04 0.00 0.04 CIDI Symptom Severity γ03 2.95* 2.33~ 2.42~ 1.16 1.18 1.22 PBQ Complexity γ04 - 0.26 - 1.89 - 2.26 2.1 2 2.08 1.93 Rate of Change π1i Intercept γ10 - 0.07 0.07 - 0.05 0.00 - 0.13 0.07 0.12 0.10 0.13 0.13 CRS Competence γ11 - 0.01* - 0.01** - 0.00*** 0.00 0.00 0.00 HAACS Competence γ12 0.00~ 0.00* 0.00* 0.00 0.00 0.00 CIDI Symptom Severity γ13 0.13*** 0.12*** 0.10*** 0.04 0.04 0.04 PBQ Complexity γ14 0.06 0.10 0.07 0.08 0.06 0.06 312 Table 35 Estimates of random effects from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after controlling for depressive symptom severity and personality beliefs Variance Components Parameter Model 11 Model 12 Model 13 Model 14 Model 15 Level 1 Within person σ2e 28.65*** 29.03*** 16.72*** 17.09*** 17.13*** 3.03 3.10 1.82 1.89 1.90 Level 2 In initial status σ20 16.72* 21.99** 20.18** 23.65** 19.28** 6.63 8.15 6.99 7.99 6.96 In rate of change σ21 0.35 0.73** 0.25~ 0.52** 0.32* 0.15 0.38 0.14 0.19 0.15 Covariance σe 0.13** 0.02 0.00~ 0.01* 0.01 0.04 0.02 0.00 0.01 0 .01 Psuedo R2 Statistics and Goodness of Fit R2e - 0.00 - 0.02 0.42 0.40 0.40 R20 0.29 0.06 0.14 - 0.01 0.18 R21 0.43 - 0.20 0.59 0.15 0.48 Deviance 14 71.58 1451.90 1319.75 1297.75 1289.85 AIC 1487.58 1467.90 1343.75 1321.75 1317.85 BIC 1515.02 1495.13 1384.42 1362.14 1364.98 Δ AIC 8.4 28.08 152.23 174.23 177.43 313 Table 36 Estimates of fixed effects of initial status towards a final model from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after the introduction of salient patient variables Fixed Effects Parameter Model 16 Model 17 Model 18 Model 19 Initial Status π0i Intercept γoo 1.58 8.36~ 6.16 4.24* 3.92 4.86 4.60 4.23 CRS Competence γ01 0.16** 0.16** 0.13** 0.16** 0.06 0.06 0.05 0.05 HAACS Competence γ02 - 0.03 - 0.03 - 0.04 - 0.03 0.04 0.04 0.03 0.04 HRS Beliefs γ03 0.03 - 0.02 - 0.01 0.08 0.08 0.07 CRS Fit / Match γ04 - 0.02 - 0.01 - 0.01 0.05 0.05 0.05 CIDI Symptom Severity γ08 2.27~ 2.49~ 2.28~ 1.22 1 .22 1.28 PBQ Complexity γ09 - 1.14 - 2.32 - 1.62 2.19 1.94 2.06 314 Table 37 Estimates of fixed effects for rate of change towards a final model from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after the introduction of salient patient variables Fixed Effects Parameter Model 16 Model 17 Model 18 Model 19 Rate of Change π1i Intercept γ10 - 0.08 - 0.23 - 0.25 - 0.17* 0.14 0.18 0.17 4.23 CRS Competence γ11 - 0.01* - 0.01* - 0.01* * - 0.01* 0.00 0.00 0.00 0.05 HAACS Competence γ12 0.00~ 0.00* 0.00* 0.00* 0.00 0.00 0.00 0.00 HRS Beliefs γ13 0.00 0.00 0.00 0.00 0.00 0.00 CRS Fit / Match γ14 0.00 - 0.00 - 0.00 0.00 0.00 0.00 CIDI Symptom Severity γ18 0.12** 0.09 * 0.10 ** 0.04 0.04 0.04 PBQ Complexity γ19 0.16* 0.09 0.09 0.08 0.06 0.06 315 Table 38 Estimates of random effects towards a final model from a series of individual growth models in which therapist competence in case conceptualization and homework data predicts change in depressive symptoms after the introduction of salient patient variables Variance Components Parameter Model 16 Model 17 Model 18 Model 19 Level 1 Within person σ2e 17.40*** 17.80*** 17.09*** 17.89*** 2.09 2.16 1.90 2.17 Level 2 In initial status σ20 20.69** 24.14** 18.99** 20.39** 7.52 8.44 6.93 7.64 In rate of change σ21 0.26~ 0.53** 0.36* 0.30* 0.16 0.19 0.14 0.15 Covariance σe 0.01** 0.01~ 0.01 0.01 0.00 0.01 0.01 0.01 Psuedo R2 Statistics and Goodness of Fit R2e 0.39 0.38 0.40 0.37 R20 0.12 - 0.03 0.19 0.13 R21 0.57 0.13 0.41 0.51 Deviance 1148.73 1123.87 1282.17 1125.13 AIC 1180.73 1155.87 1314.17 1157.13 BIC 1232.60 1207.31 1367.95 1208.66 Δ AIC 315.25 340.11 181 .81 338. 85