This is to acknowledge the hard work and dedication of all the people who contributed to the progress made on the Oklahoma Co-Occurring State Incentive Grant (OK-COSIG) project in this second quarter. The work over the last three months has been both challenging and rewarding. The COSIG staff at ODMHSAS and those who have been working partners in organizing and presenting the yearly conference, the program planning meetings, and the first stage of training on the CCISC model for systemic integration to improve services for people with a co-occurring disorder was substantial. It is important as well to recognize SAMHSA’s Co-Occurring Center for Excellence and their investment of time and considerable professional skill to support the work of the OK-COSIG project. I also wish to recognize the people behind the scene that have also given their time, energy, and formidable talent to the work done on the OK-COSIG. The enthusiasm of the professionals and consumers for the work to be done on the COSIG project continues at a high level. This enthusiasm will go a long way in helping the OK-COSIG project reach its goals and accomplish its objectives.

How this quarterly evaluation report is organized

This second quarterly report will begin with a brief overview of the work accomplished by the COSIG Departmental staff and the OK-COSIG evaluation team in the last three months (January 1 through March, 2005). This section is followed by the latest iteration in the development of the evaluation plan. A new matrix of systems concept is presented and will be presented to the OK-COSIG team as a way to organize part of the evaluation project. Next, the progress made on the COSIG project by timeline is presented. A discussion on emerging themes ends the narrative.

Editorial note: The Quarterly Reports produced during the year of this project will be data for the year-end reports. The year-end reports will be the data used for the five year report. These reports will also form the basis for the final report on the Process Evaluation. To maintain the highest level of accuracy, corrections will be made on quarterly reports as errors are identified or clarifications are needed. These changes will be issued as new pages that will replace the pages with errors. The new pages will retain the old text, but the old text will appear with a ~~strikethrough~~ to indicate that it was changed. The new added text will be underlined. All revision dates will appear at the bottom of the page.

Disclaimer:

This project was supported by funding awarded by the ODMHSAS and SAMHSA. Points of view in this document are those of the author and do not necessarily represent the official position or policies of ODMHSAS.

Executive Summary

In this second quarter of the OK-COSIG Project, there were six major events. These events were designed to facilitate the process necessary to affect statewide systematic infrastructure change. Additionally, contacts with the OU Evaluator and the ZiaLogic group were finalized. These events supported and informed the overall OK-COSIG project.

Over the last three months, the driving force for these events has been the enthusiasm of those involved at all levels of the Project. Several major changes have also occurred in the context of the ODMHSAS in relationship to the desire to develop an integrative model of providing services to those with a co-occurring disorder. Because there are several parallel initiatives that have complementary goals, not only will such integrated services be available to those with co-occurring disorders but in addition such services should be available to all people served by the Department. Combining these initiatives has many advantages but also presents many challenges, one of which is complexity. Some sense of this complexity can be realized when examining the OK-COSIG Systems Evaluation Matrix—A Horizontal and Vertical Analysis of interfacing systems on page 21 to 23 in this report. There are 63 cells that identify the points of system interface in this conventionalization of the COSIG project. Each added initiative will exponentially increase the number of the points of system interface. Adding two similar initiatives could increase the number of points of interface to a quarter of a million cells.

In other areas of work and accomplishments, there is still a great deal of work needed to develop a model of treatment at the community and program level. At least two programs at two pilot sights do not seem to have a demonstrable program that is designed to treat people with a co-occurring disorder.

In the remaining six months there is still a great deal of work to do to complete the planning for the screening and assessment protocols, to identify the data to be collected by ICIS, and to delineate the details of the evaluation plan. The objectives and activities that need to be accomplished to meet the goals of the OK-COSIG project are clear and specific. However, because of multiple tasks that must be undertaken at the same time, and because these tasks are very labor intensive it is questionable whether the tasks set out for this funding year can be completed in a reasonably careful and thorough way during the remainder of this planning year. Even so, the OK-COSIG team is working to meet the first year goals and objectives within the proposed time-line for this project.

Second Quarter Overview

This is the second quarterly report on the OK-COSIG project to improve Treatment of persons with Co-Occurring Substance Related and Mental Disorders in Oklahoma. The overarching goal of the OK-COSIG project is to improve the delivery of state-funded services for people in Oklahoma with a co-occurring disorder. The project will use two interventions to promote systemic infrastructure change: 1) a standard protocol for the screening and assessment of mental health and substance abuse problems will be developed, evaluated, and field tested, and 2) a model of integrated treatment that is accessible, culturally competent, and grounded in evidence-based practices.

Over the last three months a great deal has been accomplished toward these two goals. Progress has been facilitated by the following major activities:

1. ODMHSAS Leadership and Departmental representatives participated in The COCE Policy Academy workshop.

2. The 17^th Annual Substance Abuse conference provided a statewide platform of alerting the substance abuse and mental health communities about COSIG.

3. OK-COSIG staff participated in four SAMHSA’S COCE conference calls.

4. OK-COSIG staff that are focused on the evaluation of COSIG participated in the first SAMHSA Evaluators Conference call.

5. There were four Program Planning Meetings, two in Norman and two in Tulsa.

6. The first training on Minkoff and Cline’s integrative model, Comprehensive Continuous Integrated System of Care (CCISC), was completed.

7. There were three Advisory Group meetings and the name, focus, and task for the committee changed during this quarter.

8. The leadership workgroup met three times.

9. Additionally, work was accomplished on:

a. Screening & Assessment.

b. ZiaLogic educational tools.

c. Training curricula.

d. All but one OK-COSIG staff member has been hired.

e. Personnel were hired to staff the OU Evaluation team.

The focus continues on infrastructure development that will be needed to sustain system wide changes that will support an integrated treatment model for people with a co-occurring disorder. The specifics of work accomplished in each of these areas is presented next.

Implementation Activities and Events:

January 1, 2005 through March 31, 2005

After six months of this five year project, the work necessary to integrate the departments of Substance Abuse, Mental Health, and Domestic Violence to support the provision of services to people with a co-occurring disorder is becoming more clearly defined. Even so, a good deal more work is needed to organize the tasks so that the goals and objectives can be accomplished. The following is a day by day list of major activities from which data was collected and analyzed.

==============================================================

January Activities

2^nd Annual National Policy Academy

January 11^th—14^th This Co-occurring Policy Academy Meeting focused on developing an action plan to integrate services among the three divisions of substance abuse, mental health, and domestic violence. The Vision Statement that was developed at the academy is: A Healthy Oklahoma: All people with or at risk for co-occurring disorders have access to a recovery-oriented, consumer-driven system of care.

§§§§§§§§§§§§§§§§§

COD Leadership Workgroup Committee

January 19^th in Oklahoma City. This was the second meeting of the Leadership Committee.

Attending: John Hudgens, Sheila Tillery, Randy May, Dr. David Wright, Todd Crawford, L. D. Barney, Jan Savage and Andrew Cherry via telephone.

The discussion at this meeting centered around the work of the 2^nd Annual National Policy Academy. The integration of COSIG process and Policy Academy Goals, and the functions and duties of the COSIG Advisory Committee was also discussed.

§§§§§§§§§§§§§§§§§

COSIG Advisory Committee Meetings

January 21st in Oklahoma City. The second COSIG Advisory Meeting was held. Some 17 people attended this COSIG Advisory Group meeting.

Attending:

Jennifer Freeman – Family and Children’s Services, Tulsa

Cathy Bates – Consumer and Advocate

Kimberly Cox – NAIC, Norman

Ruby Gilbreath – 12&12, Inc, Tulsa

Patricia Turner – OCARTA, OKC

Mary Dillon – OU Evaluation Team member

Dr. Andrew Cherry – COSIG Evaluator

Mary McCormick – Mental Health Consumer Council, OKC

Luis Guerrero – DBSA, OKC

Charles Miller – Consumer, Norman

Cindy Honeycutt – COCMHC, Norman

Didi Herron – ACT, Tulsa

Dr. Robert Powitzky – DOC, OKC

Kathy Otis – GMH, OKC

Melody Riefer, ODMHSAS – Consumer Affairs Director

Tom Orr – GLCMHC, Vinita

Todd Crawford – ODMHSAS, COSIG Manager

At this meeting, the work of the Department’s staff at the National Policy Academy and the State Action Plan was reviewed with group members. The idea of the Advisory Group becoming a driving force to guide the State Action Plan was explored with the group. Other State initiatives that could be folded into the COSIG project were identified as the Recovery Collaborative, and the Children’s Behavioral Health Initiative. The group agreed to change its focus in part because the COSIG project was seen as mirroring many of the State Action Plan goals and objectives.

§§§§§§§§§§§§§§§§§

The 17^th Annual Substance Abuse conference

January 25^th—28^th in Tulsa. The 17^th Annual Substance Abuse conference in Oklahoma provided a statewide platform of alerting the substance abuse and mental health communities about COSIG. The featured speakers on co-occurring disorders at the conference were Kenneth Minkoff, M.D. and Christy Cline, M.D. Their presentation opened the conference with an extensive presentation called Integrated Model for Treatment of Dual Diagnosis. Afterwards there were two breakout sessions, one for administrators, and one for providers that dealt with issues of treatment and support at the administrative and clinical levels.

During this conference, another component, a peer support model, called Double Trouble in Recovery (DTR) was introduced at several workshops (Recovery Challenges for Dual Diagnosis). As a result of this series of presentations, the OK-COSIG staff, in consultation with Departmental leadership and SAMHSA, are developing a contract with Howard Vogel of Brooklyn NY to train consumers in Oklahoma on developing DTR groups. There are several functioning groups currently in Oklahoma. The training that is planned will focus on establishing DTR groups at the three pilot program sites.

§§§§§§§§§§§§§§§§§

An Evaluation planning meeting

January 26^th in Tulsa.

Attended: Todd Crawford, L. D. Barney, Khepra Khem, Howard Vogel, and Andrew Cherry.

This meeting with COSIG staff reviewed possible evaluation models that would fit with the goals and objectives of the OK-COSIG. Screening and assessment measures were also discussed.

==============================================================

February Activities

COSIG Leadership Telephone Conference.

February 3^rdin Tulsa.

Attended from Oklahoma: Todd Crawford, L. D. Barney, Khepra Khem, and Andrew Cherry. Other COSIG states were also represented by staff.

The progress of the COCE led COSIG workgroup was reviewed. Issues related to assessment instruments, consensus based practices (which were almost finished), and workforce development were discussed.

§§§§§§§§§§§§§§§§§

Telephone conference with Howard Vogel of DTR.

February 4^th in Tulsa.

Issues related to implementing DTR at the Oklahoma pilot sites and across Oklahoma were discussed.

§§§§§§§§§§§§§§§§§

Evaluation planning meeting with COSIG staff.

February 15^th in Tulsa.

Attending: Todd Crawford, L. D. Barney, Mary Dillon, and Andrew Cherry.

Went over the progress to date and plans for how the evaluation could best support the COSIG project. The process evaluation will be designed as an applied evaluation. A feedback loop will exist between the evaluation team, COSIG staff, the Department, and the programs at the pilot sites. Feedback from the OU Evaluation Team will be provided as collected documents and observations used to monitor progress during this planning year are analyzed. In this way, if problems develop during the project development and they are identified by the evaluation team, OK-COSIG participants will be alerted and informed of the situation and implications.

§§§§§§§§§§§§§§§§§

OISI Leadership Workgroup Committee (formally known as the COD Leadership Workgroup Committee)

February 16^th in Oklahoma City. This was the second meeting of the Leadership Committee.

Attending: Todd Crawford, L. D. Barney, Khepra Khem, Nancy Warren, Shelia Tillery, Richard Bowden, Andrea Vigil, and Andrew Cherry via telephone.

At this meeting, the name of this committee was changed to the Oklahoma Integrated Systems Initiative (OISI) Leadership Group.

The Levels of Program Capability diagram was explained. The basic outline of the diagram is similar to the TIP 42 description. These levels are described as basic, intermediate, and advanced care. A list of competencies is being developed with the intent to outline the capabilities of each facility. The letter to SAMHSA requesting permission to expand DTR’s role in the COSIG Project was also reviewed.

§§§§§§§§§§§§§§§§§

ISI Advisory Group Meeting (formally called the COSIG Advisory Committee)

February 16^th in Oklahoma City. The Third COSIG Advisory Meeting, now called the ISI Advisory Group was held. Some 21 people attended this COSIG Advisory Group meeting.

Attending:

Jennifer Freeman – Family and Children’s Services, Tulsa

Cathy Bates – Consumer and Advocate

Kimberly Cox – NAIC, Norman

Steve Stewart – 12&12, Inc, Tulsa

Patricia Turner – OCARTA, OKC

Nichole Burland – OHCA, OKC

Charles Miller – Consumer, Norman

Suzie Seymour – Consumer

Cindy Honeycutt – COCMHC, Norman

Didi Herron – ACT, Tulsa

Dr. Robert Powitzky – DOC, OKC

Tom Orr – GLCMHC, Vinita

Janet Hackworth – GLCMHC, Vinita

Carol Baxter – ODMHSAS, Finance

Penny Malone – OCCY, OKC

Gary Nunley – Choctaw Nation Behavioral Health

Valerie Millheim – Gateway to Prevention Recovery

Valerie Christopher – Red Rock PACT

Todd Crawford – ODMHSAS, COSIG Manager

Khepra Khem – ODMHSAS, Co-Occurring Training Specialist

L.D. Barney – ODMHSAS, Co-Occurring Program Specialist

The group approved the use of the Modified MINI and the Modified DALI, in addition to the OK-COSIG proposed short screening questionnaire. If acceptable, these tools are to be tested in the Norman, Tulsa, and Vinita pilot program sites. The conceptual diagram for state systems of care for co-occurring disorders was presented.

Copies of the Co-Occurring Training Outline were disbursed among the group. The current plan for training was presented by Dr. Khepra Khem.

A draft for the Oklahoma Integrated Services Initiative Plan was also passed out to the group.

§§§§§§§§§§§§§§§§§

==============================================================

March Activities

COSIG Leadership Telephone Conference.

March 3^rd from NYC.

Attended from Oklahoma: Todd Crawford, L. D. Barney, Khepra Khem, and Andrew Cherry. Other COSIG states were also represented by staff.

The Assessment Instrument review was discussed. It was predicated on the SAMHSA COCE Assessment Matrix, which was reviewed by four states (see Appendix).

§§§§§§§§§§§§§§§§§

1^st Program planning meeting.

March 10^th in Norman.

Attending: Sign in sheets lists 14 attendees in the morning and 12 attendees in the afternoon session.

The first two program planning meetings to help design the COSIG evaluation was conducted with pilot site administrators in the morning and clinicians in the afternoon. The data will be analyzed when all of the six program planning meetings are completed (tentatively in May).

§§§§§§§§§§§§§§§§§

OISI Leadership Workgroup Committee (formally known as the COD Leadership Workgroup Committee)

March 15^th in Oklahoma City. This was the third meeting of the Leadership Committee.

Attending: Todd Crawford, L. D. Barney, Khepra Khem, Richard Bowden, Shelia Tillery, Andrea Vigil, and Andrew Cherry via telephone.

The nature of the change in the acronym COSIG to ISI – Integrated Services Initiative was explained. Each member of the ISI leadership group was asked to assign themselves to one of the subcommittees.

Screening Assessment: Todd Crawford and L. D. Barney

Outcomes/Evaluations: David Wright

Training: Randy May

Finance: Richard Bowden and members of the Health Care Committee

Workforce Development: Shelia Tillery and Khepra Khem

Systems Integration: Todd Crawford and John Hudgens

The Evaluation team leader Andrew Cherry will be involved at some level in all of the subcommittees.

§§§§§§§§§§§§§§§§§

ISI Advisory Group Meeting (formally called the COSIG Advisory Committee)

March 18^th in Oklahoma City. The fourth ISI Advisory Group Meeting was held. Some 22 people attended this COSIG Advisory Group meeting.

The Integrated Services Initiative was discussed in more depth with group members. Afterwards breakout sessions by subcommittees began the work of organizing themselves and reviewing their individual tasks. The OISI members who had assigned themselves to a subcommittees led the group discussions.

The State Action Plan including the committee role was shared with the participants. The final document of the State Action Plan was made available to the participants.

§§§§§§§§§§§§§§§§§

2^nd Program planning meeting.

March 23^rd in Tulsa.

Attending: Sign in sheets lists four attendees in the morning session and six in the afternoon session.

The second two program planning meetings to help design the COSIG evaluation was conducted with pilot site administrators in the morning and clinicians in the afternoon. The data will be analyzed when all of the six program planning meeting are completed. The tentative date in May has been set for the program planning meeting to be held with Vinita administrators and clinicians.

§§§§§§§§§§§§§§§§§

COSIG Evaluators Telephone Conference.

March 24^th from NYC.

Attended from Oklahoma: Steve Davis, David Wright, and Andrew Cherry. Evaluators from nine other COSIG states were also represented by staff.

The evaluators from each of nine COSIG states gave an overview of the evaluation they are conducting on their states COSIG project. The Pennsylvania COSIG project is the closest to the model of integration that Oklahoma is planning.

§§§§§§§§§§§§§§§§§

Regional planning and preparation for persons with a co-occurring disorder using the CCISC model for integration.

March 28^th in Oklahoma City. This meeting was led by Kenneth Minkoff and Christy Cline of ZiaLogic.

Attending: Members of the OISI Leadership Group, and Members of the ISI Advisory Group.

This was a ZiaLogic planning day with the ISI Advisory Group members and others in the Department. The thee days of training in Woodward, Lawton, and Oklahoma City was discussed. The three screening tools were also made available to the Evaluator at this meeting. The three assessment tools are: the Co-occurring Disorder Educational Competency Assessment (CODECAT), the CCISC Outcome Fidelity and Implementation (COFIT), and the Agency Self-Survey (COMPASS). These scales will be incorporated into the evaluation process.

March 29^th in Woodward.

Attending: Sign in sheets lists 29 attendees.

At this meeting an overview of the rationale for integrating services and the use of the CCISC model were presented. There was a discussion about barriers that need to be expected and implementation strategies.

March 30^th in Lawton.

Attending: Sign in sheets lists 20 attendees.

At this meeting an overview of the rationale the integrating services and the use of the CCISC model were presented. There was a discussion about barriers that need to be expected and implementation strategies.

March 31^st in Oklahoma City.

Attending: Sign in sheets lists 89 attendees.

At this meeting an overview of the rationale for the integrating services and the use of the CCISC model were presented. There was a discussion of barriers that need to be expected and implementation strategies.

==============================================================

Six Major Shareholder Groups were Active

There were six of the seven major shareholder groups active in this first quarter. Shareholders from these six groups were involved to some degree at all levels of planning the COSIG project. Continued collaboration with these and other shareholders will provide guidance in shaping the services provided for people with a co-occurring disorder in Oklahoma, and will contribute and help shape the design of the evaluation plan. The shareholder groups were:

1. The ISI Leadership Workgroup ─ ODMHSAS

2. ISI Advisory Group

3. Providers of SA & MH services

4. Practitioners providing treatment in SA & MH agencies.

5. Consumers

6. The ISI (previously called COSIG) Subcommittee on Screening and Assessment.

7. Professional State Licensure bodies were not active this quarter.

==============================================================

Overview of COSIG Evaluation plan

Fidelity Assessment

The Fidelity Assessment will determine the extent to which the OK-COSIG project has implemented the components needed to provide services and treatment to people with a co-occurring disorder. The Fidelity Assessment will document deviations and will record the reasons for the changes and any consequences that result from the changes. The Fidelity Assessments will be conducted at the provider agencies that will be participating in the pilot project.

Progress: The three assessment tools developed by Minkoff and Cline of ZiaLogic will be incorporated in the evaluation of the integrative initiative. These tools are called: the Co-occurring Disorder Educational Competency Assessment (CODECAT), the CCISC Outcome Fidelity and Implementation (COFIT), and the Agency Self-Survey (COMPASS). These scales will be incorporated into the evaluation process.

Process Evaluation

The process evaluation will record the composition of the OISI Leadership Group and the ISI Advisory Group, and the progress made during this first year of planning and preparing for the evaluation at the pilot sites.

Progress: Organizing the process by which data is collected from OK-COSIG staff has been evolving and is becoming an effective conduit for data. The cooperation of the staff has been noteworthy. There continues to be a need to refine the process but these are technical not organizational issues.

Training. Professionals who will be providing services and treatment to people with a co-occurring disorder will be trained at two levels – in the use of the screening and assessment protocol, and in the use of the integrated treatment model. In relation to both types of training the Evaluator will assess who has been trained, the extent of the training, the number or proportion of training sessions attended, the level of learning that takes place, the trainees perception of the training environment, and satisfaction with the training experience. The Evaluator will also assess the trainees’ beliefs and attitudes regarding mental illness and substance use disorders and their knowledge of the screening and assessment protocols and the integrated treatment model. Self-report instruments to assess these areas will be developed over the course of the first funding year with consultation from ZiaLogic, COCE, SAMHSA COSIG and with input from the OK-COSIG Advisory Group. The instruments will be piloted at the initial training provided by ZiaLogic to the OK-COSIG Advisory Group and, based on the pilot, the instruments will be adjusted and approved for use by the OK-COSIG Project Manager and the OK-COSIG Advisory Group. Assessments will be conducted at the various service pilot sites at the beginning of training and at the end of training. Data will be analyzed using appropriate statistical tests.

Progress: Two of ZiaLogic assessment tools will be incorporated into this level of the evaluation. The tools are called: the Co-occurring Disorder Educational Competency Assessment (CODECAT), and the CCISC Outcome Fidelity and Implementation (COFIT).

Grant Related Activities and Events. A relational database of grant funded events and activities, including planning meetings, OK-COSIG Advisory Group meetings, consultation meetings between ODMHSAS and ZiaLogic staff, trainings, site visits by trainers and evaluation staff, and other relevant activities will be created by the Evaluator. The number and identities of persons attending each event will be recorded along with the agenda and meeting minutes of each event. A summary report of activities and events will be produced every quarter and distributed to the Governor’s liaison and the OK-COSIG Advisory Group.

Progress: Summary reports have been written on each related meeting on which data has been collected. Attendance lists have been compiled.

Pilot Project Services. Information about services provided to recipients, staff providing services, and service costs are regularly recorded in the ODMHSAS, ICIS system, as described above. The Evaluator will work with Decision Support Services Division to extract this information for each of the services pilot sites on an annual basis for each year of funding. Data from the first planning year will provide a baseline against which data from the pilot sites at years two and three can be assessed. Annual reports will be developed by the Evaluator and submitted to the OK-COSIG Project Manager, the OK-COSIG Advisory Group, and the Governor’s liaison.

Progress: Several informal discussions between the evaluator and the Director of Decision Support Services has been useful in helping shape this part of the evaluation. The decision on what data the Oklahoma Integrated Client Information System (ICSI) will collect and what data will be collected by the Evaluation Team has not been made but progress has been made related to this task.

OK-COSIG Systems Evaluation Matrix—A Horizontal and Vertical Analysis

The OK-COSIG project is by far more ambitious than projects in other COSIG states. As a result, it is more complicated and complex. This complexity can result in disproportionate progress between systems and even within systems. To deal with this complexity over the next quarter the following conceptualization will be explored.

In this concept the systems are identified as horizontal and vertical systems that interact at system interface points. In this case, the systems will be organized by function at the Macro, Mezzo, and Micro levels. This set of systems is conceptualized as the horizontal. The Marco systems are ODMHSAS, SAMHSA/COSIG/COCE, and Other State Systems. The Mezzo systems are: the MH Community, MH Programs, the SA Community, and SA Programs. The Micro systems are the Individuals receiving treatment and their families.

The vertical systems are represented by the areas that have been assigned to the ISI Advisory Group subcommittees and adding ZiaLogic. These are: ZiaLogic, System Integration, Outcomes/Evaluations, Screening Assessment, Training, Finance, Workforce Development and Credentialing, and DTR-peer support groups.

The OK-COSIG Systems Evaluation Matrix below illustrates this conceptualization. Where the systems cross, the cells are considered the system interface point for those two systems. The cells provide a point of focus on the tasks that must be accomplished by the interfacing systems for the OK-COSIG project to reach its goals and objectives.

One additional step will complete this evaluation tool. Using the logic of Chi Square, each cell can be used to examine and compare the ‘expected’ and the ‘observed’ related to that cell. In this conceptualization, the cells represent a point of systems interface and become a source of four kinds of data about the systemic change that takes place between and within the interfacing systems. The four kinds of data are:

1. What was happening between the two systems before the Integrative Initiative started?

2. What is expected to occur between the two systems if the Integrative Initiative is successful?

3. What methods of observation should be used to document change between these two systems?

4. What are the observed changes between and within systems at the end of the evaluation process?

Using this Evaluation tool, ISI Advisory Group subcommittees will be able to define the expected changes that will be needed for overall systemic integration to occur. Given the expected interaction between and within the vertical systems as they interface with the horizontal systems at the Macro, Mezzo, and Micro levels, each cell can be studied. Moreover, using this methodological approach supports the use of standardized measures across cells and mixed methods within the cells. The following matrix design is an illustration of the evaluation tool being developed by the OU Evaluation Team for the COSIG project.

OK-COSIG Systems Evaluation Matrix

OK-COSIG Evaluation Matrix—MACRO SYSTEMS

ZiaLogic

System Integration

Outcome

Evaluation

Screening & Assessment

Training

Workforce

Credentialing

DTR Groups

Peer Support

MACRO

SYSTEMS

ODMHSAS

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

SAMHSA/

COSIG/

COCE

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Other State Systems

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

OK-COSIG Evaluation Matrix—MEZZO SYSTEMS

ZiaLogic

System Integration

Outcome

Evaluation

Screening & Assessment

Training

Workforce

Credentialing

DTR Groups

Peer Support

MEZZO

SYSTEMS

MH Community

Systems

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Programs

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

SA Community

Systems

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Programs

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

OK-COSIG Evaluation Matrix—MICRO SYSTEMS

ZiaLogic

System Integration

Outcome

Evaluation

Screening & Assessment

Training

Workforce

Credentialing

DTR Groups

Peer Support

MICRO

SYSTEMS

Individual

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Family

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Pre ISI:

Expect from ISI:

Methods of observation:

Observed:

Outcome Evaluation

Service Coordination and Networking. The assessment of coordination and networking will be strictly qualitative and based on a combination of key informant interviews with program administrators at the State, regional, and local levels and focus groups with provider staff at the pilot sites during the second and third years of funding, and at a random sample of seventeen regional provider sites, half at mental health facilities and half at substance abuse facilities, during the fourth and fifth funding years. Sites for this assessment will be selected to provide equal coverage of all service areas within the State. The semi-structured interview guide to be used in this assessment will be developed during the first funding year with input from the Project Implementation Team.

Service Recipient Outcomes for Services Pilot: The Project Evaluator, with the assistance of the ODMHSAS Decision Support Services Division, will track clients and arrange for reassessments at three and six month observation periods post-intake. The Co-occurring Disorders Specialist will be responsible for coordinating the reassessments with an emphasis on the independent verification of current levels of service recipient functioning. Service recipients, who are assumed to be transient, will be tracked with a combination of procedures. First, service recipients will be asked at intake and each reassessment to provide information on their current residence, plans to move, and the name of a family member and/or friend who will be likely to know their whereabouts over the next year. The service recipient will also be asked to provide written permission to the Evaluator to contact family members. Second, service recipients included in the outcome evaluation will be contacted by either mail or phone on a monthly basis to verify their address and other contact information. These procedures have become a standard method for locating participants in longitudinal research (van Kammen & Stouthamer-Loeber, 1998; Sullivan, et al., 1996).

Three measures of outcome will be employed: The assessment instrument developed by the ODMHSAS working group and approved by the OK-COSIG Advisory Group will be repeated at three and six month observation periods. Currently, the Client Assessment Record (CAR) is in use with mental health service recipients and, unless changed by the screening and assessment working group, will be the assessment instrument used with persons with co-occurring disorders identified by mental health service providers. This instrument assesses mood/affect, thinking processes, substance use, medical/physical issues, medications, family involvement and problems, interpersonal relationships, role performance, socio-legal involvement, self-care/basic needs, and communication ability. Each item is rated for past and present functioning and results will be recorded in a numeric score. For substance abuse service recipients the Addiction Severity Index (ASI) is currently being used as the primary assessment tool. The ASI is a semi-structured interview that covers seven domains: medical, employment/support, drug use, alcohol use, legal, family/social relationships, and psychiatric status. The ASI has established reliability and validity, both predictive and construct (McClelland et al., 1992).

Cultural Appropriateness of Evaluation. The evaluation process will institute several steps to insure cultural competency. First, staff hired to assist with the assessment process that is integral to the evaluation will be recruited to reflect the current cultural groups likely to be served: American Indian, Hispanic, and African American. Second, the OK-COSIG Advisory Group will have members deemed to reflect the cultural diversity of the State and this group will be charged with approving all instruments and processes within the evaluation, as well as, the cultural sensitivity of the interpretation and reporting of evaluation results. Third, all instruments will be cross translated for members of ‘language minority’ communities. Fourth, all members of the implementation team, all mental health and substance abuse service providers, and all members of the evaluation team will receive periodic training through ODMHSAS in cultural sensitivity and competence.

Plans for Using the Findings of the Evaluation. The evaluation findings will be distributed to members of the OK-COSIG Advisory Group, to the Governor’s liaison and to members of the eight Regional Advisory Boards, and to the mental health and substance abuse service providers at the services pilot sites. After approval is obtained from the OK-COSIG Advisory Group and the Governor’s Office, the findings will be posted on the evaluator’s web site and the ODMHSAS web site. As reports are prepared, the Project Evaluator, Program Specialist, and Project Manager will meet with the OK-COSIG Implementation Team and the OK-COSIG Advisory Group to review findings and then make recommendations to ensure continuous quality improvement will occur. As recommendations and plans for corrections are made, the Project Evaluator and the Program Specialist will be responsible for monitoring the impact of such changes.

Involvement of Members of the Target Population in the Design and Implementation of the Evaluation. The OK-COSIG Advisory Group will have a primary role in examining and approving all training and evaluation protocols being planned for use in the project. The OK-COSIG Advisory Group will have both service recipients and advocacy groups, forming approximately 40% of the membership, from the services pilot sites and other service regions within the State. All training materials, data collection protocols, including informed consent processes, will be piloted with the OK-COSIG Advisory Group and, based on this experience, will either be approved for use in the project or recommended for modification.

===============================================================

Methodology Used to Develop the Second Quarterly Report

The methodology that was used to collect the materials and make observations to produce this second quarterly report continues to be largely qualitative. Relevant documents were collected from committee meetings, the State 17^th Conference, and from four ZiaLogic trainings around the State. A great deal of qualitative data was also collected from four program planning meetings in Norman and Tulsa with administrators and clinicians who will be working with programs that will make up the sights where the pilot programs will be carried out. As well, the weekly and monthly reports by the COSIG Project Manager are also collected and used as supporting documents of the primary sources on which this report is based. These documents and data as a whole will be used to provide a description of events, activities, accomplishments, and tasks yet to be finished.

===============================================================

Source of Field Notes

1) Material from the 2^nd Annual National Policy Academy—January 11^th & 12^th2005.

2) Handouts from the 17^th Annual Substance Abuse Conference in Tulsa.

3) Documents collected from the Department COSIG staff.

4) Documents collected from SAMHSA COCE.

5) Documents from the Regional planning and preparation for persons with a co-occurring disorder using the CCISC model for integration.

=========================================================

OK-COSIG Objectives by Timeline for Year One

2 Months

· In the first two months of funding, a co-occurring disorder training specialist will be hired (Activity 1.2.1).

Status: Completed. Dr. Khepra Khem was hired as the training specialist on January 6, 2005. Andrea Vigil was hired January 24, 2005.

3 Months

In the first three months after the award date, a contract will be executed with Drs. Cline and Minkoff to provide technical consultation (Activity 1.1.1.)

Status: Completed in February and extended to be a two year contract. The contract is in the final stage of being concluded.

In the first three months after the award date, the State’s licensure bodies and the ODMHSAS will complete their review of the licensure/credentialing requirements (Activity 2.2.1).

Status: This work continues. The COCE is also working on licensure/credentialing requirements. The Department has moved this task over to the Workforce subcommittee of the ISI (previously called COSIG) Advisory Committee.

In the first three months after the award date, the ODMHSAS and the Contracts Division will complete their review of contracting procedures and rules, and issue a report of their findings.

Status: The task is being moved to the Finance subcommittee of the ISI (previously called COSIG) Advisory Committee.

In the first three months after the award date, ODMHSAS Contracts Division will have reviewed existing contracting procedures and will produce a report and a set of recommended changes.

Status: In progress.

A Quarterly report will be produced each quarter. The first quarterly report is due January 10, 2005.

Status: The first quarterly report was submitted on January 10, 2005. The second quarterly report was e-mailed to the OK-COSIG staff on 4-10-5 and a hard copy mailed overnight on Monday, April 11, 2005.

The Quarterly Report is a summary report of activities and events for the quarter and is distributed to:

1) The Governor’s liaison and oversight committee,

2) The oversight committee, and

3) The COSIG Advisory Group.

Objectives by Timeline for the next Six months.

6 Months

First six months after the award date, the OK-COSIG Advisory Group will approve of a screening and assessment protocol for implementation in the selected pilot sites (Activity 1.1.2 thru Activity 1.1.4).

Status: This task has been moved to the ISI (previously called COSIG) Advisory group sub-committee on Screening and Assessment. Discussions about the screening instrument has been ongoing through this quarter. There has been discussion with Support Services at the Department about how the ICIS will interface in terms of data collection. The Screening and Assessment Subcommittee met for the first time on March 18, 2005. The next scheduled meeting will be in Tulsa on April 21, 2005 to continue working on the type of screening tool that will be used by Department funded agencies.

First six months after the award date, the OK-COSIG Advisory Group will receive an overview and training in the integrated treatment model (Activities 2.1.1 & 2.1.2).

Status: The ISI (previously called COSIG) Advisory group has been briefed on the integrated treatment model and a number of the advisory group members attended a training on the integrated model on March 28, 2005.

First six months after the award date, the [licensure bodies] and the ODMHSAS will modify and approve changes to the licensure/credentialing requirements (Activity 2.2.2).

Status: This task has been shifted to the workforce subcommittee and is in the early stages of development.

First six months after the award date, ODMHSAS and the Oklahoma Health Care Authority (OHCA) will complete their review of rules defining billable services and will issue a report of their findings (Activity 2.2.3).

Status: There has been no discernable work between ODMHSAS and the OHCA to review rules defining billable services. This is still in the discussion stages.

First six months after the award date, the ODMHSAS and the Contracts Division will publish the changes to the contracting procedures used to fund mental health and substance abuse treatment providers.

Status: The work on contractual procedures used to fund mental health and substance abuse treatment is still in its earliest phase. No discernable progress has been made on this task.

9 Months

Within 9 months, the training specialist and 10 staff members will have received train-the-trainer training from the consultants (Activity 1.2.2).
First nine months after the award date, the OK-COSIG Advisory Group will approve an integrated treatment model for the pilot sites (Activities 2.1.3 & 2.1.4).
Within nine months of the award date, the ODMHSAS contracting division will modify existing rules and procedures.

12 Months

Within 12 months, 80 percent of clinical staff at the selected pilot program sites will have received training in the use of the screening and assessment protocol (Activity 1.2.3).
Within the first year after the award date, the co-occurring disorders training specialist and selected staff will have been trained in the integrated treatment model (Activity 2.4.1).
First year after the award date, the ODMHSAS and OHCA will publish changes to the rules covering billable services that include substance abuse treatment, group counseling and case management services (Activity 2.2.4).
The necessary modifications to the ICIS system to collect additional measures will be done during the first year of funding.
Self-report instruments to assess these areas will be developed over the course of the first funding year with consultation from ZiaLogic and with input from the OK-COSIG Advisory Group.
The Evaluator will work with Decision Support Services Division to extract this information for each of the services pilot sites on an annual basis for each year of funding. Data from the first planning year will provide a baseline against which data from the pilot sites at years two and three can be assessed.
The semi-structured interview guide to be used in this assessment will be developed during the first funding year with input from the Project Implementation Team.

==============================================================

Progress on Project Goals and Objectives

Goal 1.

Develop, implement, and evaluate a standard protocol for the screening and assessment of mental health and substance abuse treatment service recipients in all State funded programs.

Status:

The ISI (previously called COSIG) Advisory subcommittee on Screening and Assessment has been formed and has begun to revise the Dali in terms of adding current drug names to the DALI. The MINI is more acceptable as it stands. There is also a draft of a Screening Questionnaire developed by COSIG staff. If it is decided that this could be the screening tool to use, there is a plan to determine its Cronbach’s Alpha reliability before it is used in practice. To test its reliability and factorial validity, pilot programs will administer the scale to 200 people seeking admission. The data from completed forms will be analyzed using the SPSS statistical program. This will be developed in more detail at the next Screening and Assessment Subcommittee meeting April 21, 2005.

Dartmouth Assessment of Lifestyle (DALI)

Two-page 21-item version of the DALI, screening for substance abuse symptoms (alcohol, marijuana, cocaine, and heroin) only. Administration time: 5-10 minutes.

Developed for use in mental health settings; can be used with SMI individuals.

Only 18 items are used to sum a total score; ≥ 3 indicates a moderate likelihood of a diagnosable substance abuse problem. Scoring time: <5 minutes.

Can be administered by interviewer with minimal training or be self-administered.

The Modified DALI has been used in combination with the modified MINI in a four quadrant system in New York State.

M.I.N.I. Screen Modified

Four page, 22 item version for screening mental health symptoms only (Mood, Anxiety, Trauma (exposure, symptoms), and Psychotic Disorders). Administration time: 5-10 minutes.

Adapted for use in substance abuse settings.

Contains a screen (1 question) for risk of self-injury.

Can be administered by interviewer with minimal training or be self-administered.

Instrument is divided into 3 sections; a summary score is used to determine the likelihood of mental illness. Scoring time <5 minutes.

Resources:

COCE has compiled a set of diagnostic tools that are available and can be used on this project.

==============================================================
DRAFT 2-4-2005

SCREENING QUESTIONS

MENTAL HEALTH

1. Within the last 30 days have you had a significant period (that was not a direct result of drug/alcohol use) in which you have:

Experienced depression FORMCHECKBOX Yes FORMCHECKBOX No

Experienced anxiety or tension FORMCHECKBOX Yes FORMCHECKBOX No

Experienced hallucinations FORMCHECKBOX Yes FORMCHECKBOX No

(Heard or seen things that others don’t hear or see)

Experienced thoughts of harming another person FORMCHECKBOX Yes FORMCHECKBOX No

Experiences thoughts of harming yourself FORMCHECKBOX Yes FORMCHECKBOX No

Attempted suicide FORMCHECKBOX Yes FORMCHECKBOX No

Been prescribed medication for any psychological FORMCHECKBOX Yes FORMCHECKBOX No

emotional problem

DOMESTIC VIOLENCE

1. Have you ever been afraid of your partner or anyone else? FORMCHECKBOX Yes FORMCHECKBOX No

If yes, who? ______________________________________________________

2. Have you ever been hit, slapped, kicked, emotionally or sexually

hurt, or threatened by someone? FORMCHECKBOX Yes FORMCHECKBOX No

If yes, by whom? __________________________________________________

When? __________________________________________________________

3. If you answered yes to any of the questions above, is the

person(s) listed still a part of your life? FORMCHECKBOX Yes FORMCHECKBOX No

SUBSTANCE ABUSE

1. Within the last year, have you gotten into trouble at work/lost a

job or gotten in trouble at school/been suspended or expelled because

of your drinking or drug use? FORMCHECKBOX Yes FORMCHECKBOX No

2. Do you think the use of alcohol or other drugs has caused

problems for your family or has your spouse, significant other, parent or other close relative voiced concerns or complained about your

use of alcohol or other drugs during the last year? FORMCHECKBOX Yes FORMCHECKBOX No

3. During the past year, have you always been able to stop your use

of alcohol and/or other drugs when you want? FORMCHECKBOX Yes FORMCHECKBOX No

==============================================================

Objective 1.1 –

Develop consensus among providers, service recipients, consumer advocates and other interested parties on a standard screening and assessment protocol for use in mental health and substance abuse treatment settings.

Status:

Discussions of screening and assessment tools have been a central theme at committee meetings, the 17^th Annual Substance Abuse Conference in Tulsa, the planning meetings, and during the ZiaLogic trainings in March.

Resources:

Other States have spent a year struggling with the issue of a standard screening and assessment protocol. Their experiences will be invaluable for meeting this objective.

Objective 1.2 –

Train all mental health and substance abuse treatment providers in the screening and assessment protocol.

Status:

This work is pending. When the screening and assessment protocol has been determined, the training will begin.

Resources:

COCE and other COSIG states have begun this process and we will utilize the lessons they have learned so far.

Goal 2.

Develop, implement and evaluate an integrated treatment model for persons with co-occurring disorders that is accessible, culturally competent, and grounded in evidence-based practices.

Status:

The work to develop an Oklahoma model of integrated treatment for people with a co-occurring disorder in Oklahoma has been the focus of all the work groups that met in this Second Quarter. It was also a focus of the 17^th Annual Substance Abuse conference, the four program planning meetings, and the first round of ZiaLogic trainings.

Resources:

The National Policy Academy on co-occurring mental and substance abuse disorders that met in January 2005 with the ODMHSAS leadership to help develop strategies that will facilitate COSIG efforts to meet this goal. Additionally, COCE and seven other COSIG states have had a year’s experience trying to implement a model of integrated treatment. Their experience will be invaluable in the planning to meet this goal.

Objective 2.1 –

Develop consensus among providers, service recipients, consumer advocates, and other interested parties on the elements of an integrated treatment model for persons with co-occurring disorders.

Status:

During this Second Quarter, the Department COSIG staff worked hard and held a number of formal and informal meetings on the SAMHSA diagram to adapt it to meet the Oklahoma environment of what an integrated system would look like using the basic SAMHSA diagram and adding Domestic Violence (Trauma Informed) as an integrated component.

Resources:

The National Policy Academy work done in January, SAMHSA-COCE, and ZiaLogic.

Objective 2.2 –

Establish joint licensure/certification and funding processes for both mental health and substance abuse staff.

Status:

Establishing joint licensure/certification is in the early discussion stage. This will be an issue that the ISI Workforce Subcommittee will work on during the next quarter.

Resources:

There is some information becoming available on strategies that are being used in other COSIG states. These models may be useful in our own licensure efforts.

Objective 2.3 –

Develop contracting procedures that create strategic incentives for the implementation of integrated treatment systems at the provider level.

Status:

Work on this objective is in the early planning stage.

Resources:

There is some information on using strategic incentives coming out from the other COSIG states that will be useful in accomplishing this objective.

Objective 2.4 –

Train all mental health and substance abuse treatment providers in the use of a comprehensive, integrated system of care model for persons with co-occurring disorders.

Status:

A model that could be used to train and cross-train mental health and substance abuse treatment providers is in the early discussion phase.

Resources:

COCE has just released Tip 42 which is titled: TIP 42: Substance Abuse Treatment for Persons With Co-Occurring Disorders. The material in this manual will be included in the training curricula. The Treatment Improvement Protocols (TIPs), was developed by the Center for Substance Abuse Treatment (CSAT), part of SAMHSA which is within the U.S. Department of Health and Human Services (DHHS). This manual provides information on the empirically based best-practice guidelines for the treatment of substance use disorders.

The TIPs manuals are the product of the CSAT’s Knowledge Application Program (KAP) Expert Panel. This is a group of experts on substance use disorders and professionals in primary care, mental health, and social services. They work with the State Alcohol and Drug Abuse Directors to generate topics for the TIPs series. After selecting a topic, CSAT invites staff from pertinent Federal agencies and national organizations to a Resource Panel that recommends specific areas of focus as well as resources that should be considered in developing the content for the TIP. Afterwards, recommendations are communicated to a Consensus Panel composed of experts on the topic. This Panel participates in a series of discussions; the information and recommendations on which they reach consensus form the foundation of the TIP. This manual is a compilation of material on empirically based best-practices and professional consensus on treatment and services for people with a co-occurring disorder.

The TIP 42 manual can be downloaded from the internet and it is free. The internet address is: http://media.shs.net/prevline/pdfs/bkd515.pdf

==============================================================

Emerging Themes

During this second quarter, the focus of the work was laying the ground work for the pilot projects and the statewide screening and assessment initiative. The work centered around building consensus for the screening and assessment protocol, and the beginning of the training and orientation of providers and practitioners to the Comprehensive Continuous Integrated System of Care (CCISC) model.

Enthusiasm at the Departmental, community, and provider levels

‘Stages of change’ theory can also be applied to agencies and in individuals who are involved in the systemic change that is occurring at the Department, the provider, and the clinical practitioner levels. Members of the Advisory Group have been energized by the tasks and responsibility they have been asked to assume.

The involvement and enthusiasm of OK-COSIG participants at the 17^th Annual Conference was a very good sign. The responses among participants would suggest that most people who were at the conference, both shareholders and non-shareholders alike, could see the need for developing a model for providing effective treatment and services for people with co-occurring disorders.

The empowerment of consumers

The Department’s effort to empower consumers is paying off. The Department’s effort has provided a platform for consumers to voice their views, give direction to professionals, and monitor programming and services. The involvement of the consumers on the ISI Advisory Committee is dynamic and makes a major contribution to the OK-COSIG project. The voice of people who are the consumers of our professional services is needed if we are going to deliver services that are effective and respectful of the individuals that we serve.

Combining several initiatives into the Integrative Systems Initiative (ISI)

There are good reasons for several ongoing initiatives to be rolled into one effort at all levels. This model could produce a more organized change process and less fragmentation. It could be less piecemeal and less disruptive. It is intuitive and seems logical.

Given our knowledge about the organizational dynamics, however, we need to realize that combining initiatives will increase the complexity exponentially. For example, coordinating the changes required for the OK-COSIG project (see pages 21-24 in this report) are almost unmanageable. The systems involved boggle the mind. Adding another initiative increases the level of difficulty of tasks required to achieve the ‘goals’ of each initiative involved in the larger integration effort. When the complexity of a task is increased, the question becomes, “Are resources available to deal with the increase in task complexity?” In many cases, because of the duplication of personnel across programs (personnel needed to manage the independent projects), combining initiatives can often provide the additional resources needed to offset the increased complexity.

Even so, an effort must be made to guard against individual initiatives such as the OK-COSIG initiative and its goals from being lost in the larger Departmental initiative. One approach already in place is to continue to evaluate each individual initiative independently. These independent evaluations are needed because of the assumption underlying such a merger of initiatives. The expectation is that this overarching model will increase the chances of success among the individual initiatives. Leastwise, it will not reduce the chances of success; the chance of reaching its Objectives and Goals while functioning as a member of the larger Department initiative. As the structure of the larger initiative is assembled, there will need to be an accompanying plan for managing and evaluating this overall effort.

Pilot site program issues

Based on interviews and planning meetings, the indications are that there is a need for programming implementation and training of existing personnel at several of the pilot sites. These programs staff or the OK-COSIG staff needs to decide on a treatment model that the program will use to treat people with a co-occurring disorder during the pilot test. Neither staff nor clinical programming is in place in at least two important programs that are to be pilots in the OK-COSIG project.

==============================================================

Postscript

During this second quarter, the activities related to the COSIG project were numerous at all levels. The work needed to reach the goals and objectives continues to move ahead at a rapid pace. The State Action Plan was complete and COSIG plays a prominent role in that vision. Program planning meeting, and training in the CCISC integrative model has begun.

In the next quarter, the analysis of the planning group meetings will be completed. An evaluation of the CCISC training will be conducted. The utility of the OK-COSIG Systems Evaluation Matrix will be assessed by the COSIG staff, key informants, and relevant committees. Additional work will focus on the data to be collected by the ICIS, and the degree of work necessary to append additional fields for COSIG data.

==============================================================

APPENDIX

The document in this Appendix was useful in the work completed this quarter.

COCE Full Matrix for Evaluating Screening Instruments
INSTRUMENT NAME	MHSF-111	DALI	DALI-Modified	MINI	MINI-SHORT	MINI-Modified	SSI-SA	ASI
Purpose		Screen SA in MH settings	Screen SA in MH Settings	A brief structured interview for Axis I of DSM-IV	A quick screen to determine if the MINI should be administered	Screen MH in SA settings
Screening	Yes	yes	yes	No	Yes	Yes	Yes	Yes
Assessment	No	no	No	Yes	No	No	No	Yes
Clinical utility: Diagnosis	I.D. of problem for further diag.	Poor	No	Yes	No	No	No	No
Clinical utility: Placement	Yes- If the Placement is COD capable	Poor	No	Yes, if intent is to place in a dual disorder program	No	No	No	Yes
Clinical utility: Treatment Planning	No	Poor	No	No	No	No	No	Yes
Clinical Utility: Outcome	No	Yes	No	No	No	No	No	Yes
Severity Measure	No	OK	No, but there is a score	No	No	No, but there is a score	No, but there is a score	Yes
Reporting	Yes	No	No	No	No	No	No	?
Mental Health Focus	No	No	No	Yes	Yes	Yes	No	No
Substance Abuse Focus	No	yes	Yes	Yes, two modules devoted to alcohol and other substance abuse and dependence	No, but includes two screening questions for substance abuse and dependence	No	Yes	Yes
COD Focus		No	No	No	No	No	No	No

INSTRUMENT NAME	MSF-111	DALI	DALI-Modified	MINI	MINI-SHORT	MINI-Modified	SSI-SA
Methodological Considerations
Norms available	Yes	Yes	?	Yes	yes	?	No
Reliability on re-administration	Unknown	Yes	Yes	Yes	Yes	Yes	Unknown
Internal consistency (with alpha)	Yes	Yes	?	?	?	?	Unknown
Validity	Yes	OK	Yes	Yes	Yes	Yes	Unknown
Overall accuracy	Yes	N/A	Good	Yes	Yes	good	Unknown
Effects of demographics or background on validity	Unknown	OK	Can be used with SMI	Yes	Yes	?	Unknown
Setting Applicability	SA Treatment seeking populations	Yes	MH treatment settings	Yes	Yes	SA treatment settings	Yes
Cut-off score specified	Yes	Yes	Yes	No	No	No	Yes

Administration
Tech. support available and free	N/A	No	No	Probably	NA	No	None needed
Tech. support available at minimal cost	N/A	No	No	Maybe	NA	No	None needed
Computer admin.	No	No	No	Under development	No, but could be easily adapted for computer based administration	No	No, but could easily be adapted
Computer scoring	No	No	No	Under development	See above	No	No
Interpretive	Yes	20min	No			No	No
Time taken for admin.	10-15 mins.	Easy:5-10 min.	5-10 min.	<20 minutes on average	5-10 minutes	5-10 min.	5 Min.