Topical Content
The mental health sector is increasingly being encouraged to adopt evidence-based practices, including empirically supported treatment and assessment methods, and to achieve a balance with a large demand for service. The sector faces important challenges, such as reducing the translation time between research and practice, determining how best to implement evidence-based practices, increasing readiness for change, and addressing the clinical utility and efficiency of these practices. These challenges provided the impetus for this conference.

In view of the recent positive events mentioned above, Ontario has in place key building blocks for improving the quality of service and promoting the use of evidence-based practices across the system. Although these changes can help us to move toward evidence-based practice in mental health, we must consider the need for change in related areas in order to realize the potential of this investment. Our experience in implementing standardized outcome measurement across 107 children’s mental health organizations over the past 5 years has taught us that merely training clinicians to use new practices does not lead to uptake. Nor is the addition of new monies, strengthened commitment, and policy planning sufficient in and of themselves to bring about the changes required. Research suggests that successful adoption of evidence-based practices for community settings requires greater attention to the contextual factors that may facilitate or impede adoption. Such factors include readiness for change on the part of practitioners and organizations, building learning organizations and a culture of continuing professional development in the social services, building efficiencies and knowledge regarding quality assurance and the role it plays in service provision, and developing clear direction and support from the Ministry. The conference theme was guided by a belief that it is not sufficient to transfer new knowledge in the absence of understanding what is needed to prepare the Ministry, children’s services organizations, and practitioners to receive and implement new knowledge and change their approach to more integrated care.

Figure 1
Issues addressed

Four distinguished speakers were invited with the express purpose of addressing the factors identified as posing significant barriers in our capacity to evolve as a sector. A brief bibliography of each speaker is provided, along with highlights and “main messages” captured from their presentations.

Making Professional Development Meaningful
Ivan Silver, MD, FRCD(C)
Director of the Centre for Faculty Development
Faculty of Medicine, University of Toronto at St. Michael’s Hospital

Dr. Silver is Professor in the Department of Psychiatry and the University of Toronto. He is a 1975 graduate of Dalhousie medical school and subsequently specialized in Psychiatry at the University of Toronto, Faculty of Medicine. He joined this faculty in 1979 and later completed a Master Degree in Education at the Ontario Institute for Studies in Education at the University of Toronto. He is a member of the Wilson Centre's Executive Committee. His academic career has focused on medical education scholarship and the development of new pedagogy especially the use of games within teaching. He has focused on undergraduate and postgraduate education in the first part of his career; in the past nine years he has concentrated on continuing education and professional development and most recently on faculty development. In his specialty area of psychiatry, he has developed local and national strategies for delivering continuing education to mental health professionals in Toronto and in Canada. He has worked with the Royal College of Physicians and Surgeons of Canada to help develop the Maintenance of Certification program. In October 2002, he was appointed the Director of Faculty Development in the Faculty of Medicine at the University of Toronto. Since then he has initiated several programs to enhance the professional development of faculty. He has academic interests in how students of all ages learn, self-directed learning, how to engage students in learning interactively and developing a culture among faculty that can lead to creative and meaningful career development.

Dr. Silver’s presentation focused on professional development for individual practitioners and for the organization and community more broadly, and he provided a framework for how one can conceptualize this important area.

Professional development can be defined as “the systematic maintenance, improvement and broadening of knowledge and skill together with the development of personal qualities necessary for the execution of professional tasks and duties.” Personal qualities are very important in terms of executing professional duties in care. Self-awareness with respect to how one’s goals are actually changing goes beyond a person’s day to day work. It is important to pay attention to and consider what your professional goals are.

Dr. Silver elaborated on key characteristics of effective professional development:

		It is evidence based;
		It must be collaborative and emotionally engaging;
		It is both content and context specific; relevant to the world you are in as well as specific to the learner and attuned to the learner’s needs. “Gone is the speaker who doesn’t know who the audience is or what they are doing”;
		It is grounded in knowledge about teaching and learning;
		Risk-taking is essential, thus, it is important to create an environment that encourages inquiry, reflection and experimentation;
		It is sustained, ongoing, intensive and supported by modeling, coaching, and collective problem solving. “You are as powerful as you are a group, as powerful as your leader.”
		It helps identify learning gaps and addresses the barriers to implementing new knowledge and skills. Knowledge can be metamorphosed if it is something that has been identified as being a gap. Those gaps can be addressed and learners can then work on them during conferences. “The hot ticket is knowing the gap”;.
		It must be supported by employers and their institutions and organizations in order to give it meaning;
		To provide evidence that it is working, professional development must be evaluated.

Domains of practice
In considering what is “a professional”, a much wider scope of thinking is required. It is not enough to be a specialist in medicine for we need to develop communication skills and learn how to work with other professionals, how to manage others, how to be an advocate and to consider our role as scholars. All these skill sets are important to look at as legitimate foci for professional development.

The driving forces
The profession is now part of the social contract of a professional. The public is requiring to know what you are doing to retain your competency as a professional, as a manager, etc. The regulatory bodies require written confirmation that you are continuing professional development.

Methods of continuing professional development

Group learning activities are among the traditional methods of professional development and include:
		conferences, seminars, rounds
		study groups
Individual learning activities include:
		reading (books, journals, internet)
		personal learning projects
		reflective journaling
		practice audit (with feedback) – this method, which involves monitoring how you are doing and is usually done with a colleague, is the most powerful tool for change.
		self assessment tests – these can be done on your own time and are based on contemporary practice
		supervision – this is a very effective one-on-one type of learning
Other learning activities include:
		opinion leaders – this method involves identifying leaders through surveys of practitioners in the area, providing them with a training program and letting the information fly..
		academic detailing – consists of a practitioner going to another practitioner’s place of practice and addresses their learning needs (eg. a pharmacist would go to an MD’s office and discuss how doctors can best utilize medications in their practice) While this is a very evidence-based method, it is very time consuming and expensive.

Review of outcome studies

Grimshaw et al. 2002, JCEHP 22(4) 237-242
		Passive dissemination is unlikely to change behaviour when used alone. However there is a place for it as part of a package to effect change;
		Active approaches which engage the audience in participation whether formally or informally are more effective;
		Audit with feedback and use of opinion leaders had mixed results with some working well and others not;
		Reminders were the most powerful professional development tool (eg. stickies onclients’ files or computer reminders after a conference with references and tools);
		Multi-faceted approaches are more effective than single interventions since they address a wide variety of learning styles – “Addressing different learning styles are powerful vehicles for change;”
Davis et al. 1999, JAMA 282(9) 867-874
		Longitudinal and multiple approaches for learning are more powerful than sitting in a lecture hall;
		Giving people time to interact with one another is important - “Don’t do away with lectures but give audiences a chance to use, apply or reflect the knowledge”
		Mixed methods address the learner more correctly.
Thompson, O’brien et al. 2004 (Cochrane Review)
		Review of trials of continuing education meetings and workshops found that interactivity makes a difference – giving audiences plenty of opportunity to participate.

What make a difference in professional development?

Several factors are important considerations for professional development, including:
		Needs assessment based learning activities – “Do a needs assessment!”
		Active learning;
		Interactivity;
		Learning from your practice (audit with feedback);
		Learning from colleagues (opinion leaders);
		Personalized learning (academic detailing, personal learning projects);
		Longitudinal learning activities – study groups where learning needs are assessed and ways of addressing them through pooled resources are a very effective self-driven learning activity;
		Patient mediated learning activities – involves educating consumers on how to educate health practitioners about new health information, about their own health, and other information that the practitioner should be aware of, in ways that are not offensive;
		Commitment to change exercises;
		Reminders.

What does not make a difference in professional development?
Lectures with little interactivity;
Single stand-alone continuing education events.

Recommendations to the field

Planners of continuing education (CE) need to consider the following recommendations:
		Do a needs assessment;
		Develop collaborative planning committees with consumers;
		Engage in and support Interactive contextually relevant teaching methods;
		Address barriers to implementation of new knowledge such as giving people more time to address behaviours;
		Engage in commitment to change exercises;
		Evaluate the impact of education interventions.
For individual learners, Dr. Silver’s recommendations include:
		In thinking of where you need to develop your knowledge, begin with patient/client care and select problems and questions that come from your practice;
		Consider a practice audit;
		Keep a professional portfolio/reflective journal;
		Form and participate in study groups;
		Consider the advantages and professional development from supervision.

Dr. Silver engaged the conference participants in examining potential barriers to continuing professional development. Audience members offered the following barriers:

		Time, money and a didactic culture;
		Linking outcomes to objectives other than accountability;
		Constant lack of focus due to multiple priorities;
		Shift or lack of responsibility of support for continuing professional development;
		Conflict between needs of agencies and practitioners;
		Unperceived needs of learners;
		Distinguishing between needs assessment versus wants assessment (objective versus subjective needs) – Need to bring both sides to the table;
		Synchronicity of learning – learning is lost because time hasn’t been devoted to it, therefore need a collaborative learning organization with planned, synchronized and relevant education;
		Non-relevant new information is disseminated when the speakers do not know the audience;
		Continuing education is undervalued by funders.
		Speaking/lecturing is also a good educational/learning opportunity.

If you trust your audience, your audience will know more than you do.

Dr. Silver defined personal learning projects (PLPs) as being personally planned and learner centered. Ideally, they are also practice specific – learning activities stimulated by any aspect of an individual’s professional practice. PLPs should focus on questions or problems that address gaps in knowledge, skills, attitudes or performance; enhance understanding or insight; review current practice standards.

The steps recommended for developing a personal learning project (PLP) were reviewed:

Step 1		Identify a question or describe an issue in practice – this issue or question may come from a client
Step 2		Determine the stimulus or trigger prompting learning to occur – e.g., ask yourself why you are interested in this issue? Frame the question – it may be a simple one hour exercise or something that needs to be researched.
Step 3		Select the resources for learning – usually involves speaking to a colleague
Step 4		Determine the outcome of learning for practice which involves documenting what you’ve done to answer the questions and reflecting on it. Taking the time to write down your learning moments is very important. Documentation cannot be over-emphasized. If anyone wants to know about your professional development, you have a professional portfolio to which you can refer.

The potential benefits of personal learning projects for the individual learner include:

		Fostering skills of reflection and critical inquiry;
		Promoting self-management of learning;
		Promoting the development of a personal knowledge management system to facilitate the transfer of new knowledge into practice;
		Facilitating application of new knowledge across content domains.

If we hope to continue professional development for others, we need to do it ourselves.

Community Readiness: A Successful model for change
Barbara Plested, Ph.D.
Research Scientist
Tri Ethnic Center for Prevention Research at Colorado State University
Ft. Collins, Colorado, USA

Dr. Plested has worked extensively in the provision of direct services to special populations including American Indian, Native Alaskan, child and adolescent, female, and jail-based programs. She has twenty years of experience, serving both as an administrator as well as a therapist in the fields of mental health and substance abuse. She serves as an evaluator and grant writer for several Native American programs and is one of the developers of the Community Readiness model. She has conducted research using the model on a variety of issues: intimate partner violence, HIV/AIDS prevention, methamphetamine prevention, drug and alcohol prevention and environmental trauma. The Community Readiness Model has been used successfully in urban areas, Alaskan villages and Native reservation areas throughout the United States as well as internationally to effect community change. Barbara has published extensively and has served on Roslyn Carter's panel on intergenerational caregiving.

Dr. Plested was part of the research team that developed the community readiness model
which has been used in various organizations, primarily for implementing prevention programs. Most of her work has been with First Nations and rural communities, although the model has been applied in a variety of contexts. For purposes of clarity, Dr. Plested defined a community is a group of people, whether they be within an organization or a neighbourhood.

The Tri-Ethnic Centre is a Colorado State University Centre of Excellence which began as a project thirty-seven years ago. The centre functions in such as way as to ensure that research results transfer back to the community where they can be utilized. The centre employs 40 full time scientists who conduct research on social and organizational concerns including:

Alcohol, tobacco and other drug use
Cultural competency
Implementation of prevention programming
Partner violence
Delinquency and dropout
HIV/AIDS

“Readiness is the first essential step to change” said Dr. Plested. For change to be effective we must know the level of readiness to determine the type of interventions that may be needed to move a community along a continuum of change. Communication and collaboration from all stakeholders is very important, including input from consumers. The community readiness model is very issue-specific and uses a measurable tool (pre/post) to assess where we are when we start this process and where we will be when it is completed.

What is community readiness?
Community readiness is a model based on science and theory, not just on good ideas. It is an issue specific and community specific model of intervention. Nine stages guide facilitation of community-directed change and provide a clear map for change. The community readiness model is useful in that it initiates action and instills ownership, both of which are critical in an era of change. Also, the model engages all people within the organization (e.g., talkers, contemplators, etc.) and this is necessary for change to occur. The model utilizes existing resources, thus no extra money or assistance is needed. The model and manual are accessible on the internet (http://triethniccenter.colostate.edu). Lastly, the model is conducive to the development of culturally-appropriate change strategies.

It is not a program but rather a vision which takes time to create and sustain.

The community readiness model has been used in program evaluation, research, community-based prevention and organizational analysis. The U.S Substance Abuse and Mental Health Services Administration (SAMHSA) now requires a description of community readiness in grant applications.

Stages of community readiness

		1. No awareness: The issue is not generally recognized by the community or organization as aconcern.
		2. Denial/resistance: Some recognition by some community members or staff members that the issue is a concern but mainly there is a feeling that nothing needs to be done.
		3. Vague awareness: General feeling by at least some that there is a local or organizational concern and that something should be done about it. No immediate motivation or identifiable leadership.
		4. Preplanning: Clear recognition by some in the community that there is a local concern about the issue and change should be implemented. Some discussion, but no real planning. Must allow time to inform and educate.
		5. Preparation: Planning with key individuals/stakeholders/staff members is going on and focuses on details. Preparation is huge – very important to get information from staff, board of directors, etc.
		6. Initiation: Activity and action is underway, but still viewed as a new effort. Program is being tried out.
		7. Stabilization: One or two efforts are running, supported by administrators, staff or community members. Model will not make conflict go away, but it will provide information. Evaluation is important.
		8. Confirmation/expansion: Standard efforts are now in place and leadership supports expanding and improving services. New efforts are being developed, resources are being sought to serve more groups. Important to connect with others versus getting grant money for yourself when there may be someone in the community already doing the work.
		9. High level of community ownership: Detailed and sophisticated knowledge of prevalence, risk factors, and causes of the issue exist. Important to tell the community what you are doing, services that are being provided. By building a high level of community ownership the community will see you as a valuable resource and if you go away, they will call somebody. You need allies in your community. You need to let them know that you are doing a good job.

The process of change
Change takes time, and it is important to lay out realistic time goals. It is Important to
communicate the mission - Why are we doing this? How is it going to benefit our
clients/organization and our objectives? Inform people in multiple ways through email, in
person, telephone, etc. It is necessary to gather input from individuals affected by the changes and use that information to form the plan.

Dimensions of community readiness
Look at what is currently going on in the community – strengths, weaknesses, barriers. Howknowledgeable is your organization/community about those efforts? What can you learn by examining the leadership; leaders’ attitudes and behaviours are important. Know your organizational climate – Is it friendly? What does your organization/community know about change, about the issue? What resources related to the issue are available, is there some place to meet, etc.?

Process for using the community readiness model
Identifying the issue is the beginning of the process. The community must be defined with
respect to geography, occupations, systems, etc. Key respondent interviews are then
conducted and scored to determine readiness level. Strategies are then developed that can assist the community to move along the change continuum. And, in the end, community change is achievable.

Dr. Plested encouraged all to remember that strategies of intervention for prevention efforts must be appropriate for the organization’s or community’s stage of readiness!

The community readiness model can help to identify resources and obstacles, and provide an assessment of how ready the organization or community is with respect to accepting a given issue or change as something that needs doing. It also identifies types of efforts that are appropriate to initiate, depending on the stage of readiness. The model cannot make people do things in which they do not believe. It is not a coercive model. Nor will the model specify exactly what you should do to accomplish intended objectives.

Powering Information - Getting to the Point of Human Services
David Raney, MD
CEO, Nuventive
Chariman of the Board, Esteam

Dr. Raney presented a model for substantially improving service quality and increasing productivity for organizations in the human services sector – one burdened with complex government reporting and documentation requirements, which take workers and clinicians away from their primary focus of serving clients. Dr. Raney has served as the CEO of Nuventive, LLC from its inception in December 2001. Nuventive has grown from an unfunded start-up to a market leader in institutional assessment and electronic portfolios in higher education. Dr. Raney also co-founded Esteam, LLC and currently serves as the Board Chairman. Esteam provides software and services to the human services market offering a broad based electronic record as well as software to manage program improvement and outcomes measurement. Dr. Raney was also a founder of two innovative programs at the Children’s Hospital of Denvershortly after leaving his residency. As a faculty member at the University of Pittsburgh Medical Center numerous new programs/business units were created under his leadership. Dr. Raney received his MD degree from Vanderbilt University and his BA in Psychology from the University of California, Davis.

Dr. Raney’s message was that although tools exist today that can significantly improve staff productivity and quality monitoring, organizations have demonstrated a tendency to collect large amounts of data that are infrequently used to inform service delivery. Raney encouraged us to close this loop and make quality monitoring something more than data collection and he illustrated how this is approached at Pressley Ridge. According to Dr. Raney, Information is powerful by itself, but there are many things in society which can de-power information. His presentation was focused on how to get information to people in a way in which it can be applied. He acknowledged that there is a lot of frustration in getting a system going and technology is central to helping remove some of the barriers.

As in Canada, it is very challenging to have an effective system of care within the United States, where the system is so refracted and silo-based, and information does not flow well among silos. Scientific information is difficult to disseminate into the community. For example, many people did not know about the benefits of cognitive behavior therapy for some time.

Dr. Raney referred to the Institute of Medicine Review which revealed that there is much room for improvement within the system; that it needs a systems approach and that IT is underutilized (http://www.iom.edu/reports.asp). Technology provides an opportunity in knowledge sharing in that it supports process improvement and institutional self examination. However, although technology has greatly evolved, this is Iess evident in medical care where it is not being used as effectively as it could be.

Data data everywhere: The business of terabytes
The sheer quantity of data is overwhelming to health service organizations and practitioners. Yet, there is little accurate, real time data available to the decision maker at the time and place of a decision being taken or required. Dr. Raney suggested that it is important to have accurate data at the time of decision making. We are in need of internal data that is client specific and addresses available resources, and we need information on drug interactions, links to best practice, guidelines, etc. Although a lot of data can be automated, healthcare and mental healthcare organizations have not been fully leveraging the available technology. As a result, they do not have the data they need to manage effectively.

Data must be available to the field in a way that supports self examination and quality improvement. We need to be asking, “How can you use data to make our programs better?” There are two key elements that speak to this need: (1) a culture of improvement, and (2) methods for sustaining this culture. We can achieve this change by obtaining more data and better analytics, and by focusing on effectively using the data we do collect.

“Knowing is not enough; we must apply. Willing is not enough; we must do.” Goethe

Quality improvement processes
Typically, quality cycles involve a lot of planning, less data collection, less data analysis (which takes time and money), less utilization of data and the follow-up gets very small. The more desirable process would consist of the focus being placed in the opposite direction where the data that is available is maximized.

Dr. Raney visited some core assumptions with the audience, including that the primary purpose of planning and assessment activity is improvement. For a quality assessment program to be effective, all phases of the assessment process must be addressed and must have a circular record. These include:

Planning
Data collection
Data analysis
Data utilization
Follow-up

Quality assessment programs often fail to fulfill their purpose due to a lack of understanding and the absence of an efficient, economical method of managing the assessment process.

Data management tools
Data management tools, specifically TOTAL:Quality, works to facilitate involvement by providing:

structure and process definition;
a uniform graphic interface for the entire assessment and planning cycle    so that everyone is looking at it in the same way;
a rapid, motivating feedback.

Planning is facilitated because the tool focuses on institutionally relevant goals and objectives and ensures that all organizational levels are consistent with each other and with external requirements. The tool goes further to ensure that this works efficiently by not creating additional time burdens. Data collection is facilitated by the use of successive approximations and formal and informal qualitative and quantitative data. Having data accessible from one application facilitates utilization by generating immediate reports in concise and uniform formats.

David Rumberger, Director of Sales and Marketing for Esteam, described a case study of a non-profit organization (Presley Ridge) based in Pittsburgh which had an extensive history of use of information, tracking the functional outcomes of clients. Pressley Ridge is multi-serviced, spreading across six states and serving 1,500 children any day of the week. In the early ‘80’s Pressley developed the PRIDE model, an empirically based foster care model. Over time there was concern that the model was being altered and having a negative impact on outcomes. The organization went through a solution process which involved ratifying the model. The most important thing the organization did was to collaboratively define success. Some observations resulting from the Presley Ridge experience include the following:

		Quality really does matter; it goes beyond press releases and marketing;
		Quality starts at the top – leaders must be serious about quality. Within the U.S., because there is no built-in motivation for quality, quality must come from the passion of the CEO and become the organization’s mission;
		Defining quality for the organization really matters;
		Important to understand what quality means, what are the parameters?
		We must be able to measure quality.
		Questions that need to be addressed include who should be involved in quality management? What resources need to be assigned to it and what investments need to occur?

Properly implemented, IT solutions can do a lot to make quality improvement better.

It’s not what you do, but rather what you do that doesn’t work which is important.

Therefore, it is important to have a way to record and access data that is not overly time
consuming. Speed is about leadership - either you see results early in the process or never. Technology tools exist today that can help make things go faster and smoother in a way that won’t discourage people from using them. In the end, what is needed is a system that’s really, really simple or people won’t use it.

Putting Research to Practice
Sonja Schoenwald, Ph.D.
Associate Professor of Psychiatry & Behavioral Sciences
Family Services Research Center, Medical University of South Carolina

Dr. Schoenwald’s current research focuses on the development, empirical validation, and dissemination of clinically and cost-effective mental health services for youth with complex
clinical problems and their families. Dr. Schoenwald has taken a leadership role in developing the clinical training and consultation protocols used to transport Multisystemic Therapy (MST) to communities throughout the United States and in several other countries, and in the development of research to investigate the transportability and dissemination of evidence-based practices for children and families. She is Principal Investigator of a 45-site NIMH-funded study of the transportability of MST and of an Annie E. Casey Foundation funded randomized trial of an MST-Based Continuum of Care in Philadelphia. Dr. Schoenwald is a founding member and co-investigator of the MacArthur Foundation Network on Youth Mental Health, and co-investigator of an NIMH-funded research network on schools as a context for youth mental health and of a randomized trial examining the effects of MST and organizational interventions on youth outcomes in rural Appalachia. Dr. Schoenwald has co-authored two books and authored or co- authored 50 other publications, and consults and collaborates with investigators and government groups pursuing the implementation and evaluation of evidence-based mental health practices in usual care settings.

According to Dr. Schoenwald, the field is now trying to take mental health treatments to scale and address the barriers that exist in their implementation. The concept of evidence-based practice (EBP) is, she said, a fairly recent development. The swell of EBP could vanish if researchers/practitioners don’t collectively overcome some of the challenges of what it means to have science and practice informing one another. We must work to ensure that the vision does not die as a result of missteps and a lack of science in support of specific treatments and how they can best be implemented.

Some of the challenges before us
There continues to be ongoing debate about the definition of “evidence” because of
differences in what goes on in the field versus in the lab. As such, we have much to learn about how to transfer soft technology. More is required in the area of practice research, to inform us of real world treatment effects on child populations.

Dr. Schoenwald spoke of variables that may differentiate “lab” and “usual care” settings. Treatments studied in randomized trials are very well defined, whereas these same treatments in the real world settings become fairly diffuse because therapists must juggle various issues. There is variability in the degree of treatment specification, ranging from treatment manuals having a great deal of specification, to more general manuals, to an absence of manuals for certain treatments. We must consider that the variety of practitioners, including social workers, psychologists, receive different types of training. Lastly, a vast array of service delivery issues give rise to a range of treatment duration or dose.

How do we speed the progression from development to deployment?
We have taken a big step forward in recognizing that we need to verify treatment effectiveness in the real world. We need to show that we are not doing harm by virtue of the treatment delivered. Safety of child and youth mental health care is the first imperative. In the U.S. there is explicit recognition that research must demonstrate the transportability of a practice as well as its sustainability and dissemination.

MST research and dissemination: An Example
MST is a time limited intensive treatment model geared to juvenile offenders. There is now twenty years of science behind the development, efficacy and implementation of this model. The MST Institute is a training organization which focuses on quality assurance and outcome tracking. Dr. Schoenwald and her colleagues have spent six years helping practitioners in the community achieve the same results (i.e., effect sizes) typically seen in controlled lab settings. In Canada, results have been mixed, with some sites showing good effects and others not.

MST program development has taught us that it is important to determine which elements of a treatment are important. Moreover, one must consider who are the appropriate target populations, who are the realistic referral sources (i.e., are they really going to send the child to you?), and how communities will approach ongoing program evaluation.

Stakeholder influences are important considerations for treatment deployment. Funding structures need to be in place, and the lead must be taken by the MST therapist. Facilitators include a clear understanding of MST and full commitment to implement MST within the organization. We have learned that the professional degree and work experience of the therapist does not predict outcome. Rather, therapist characteristics such as a strong work ethic, flexibility, intelligence, creativity, etc. are important for client outcome.

MST Quality Assurance Program
The purpose of quality assurance in MST is to increase the likelihood of achieving positive outcomes through identifying and removing barriers to effective implementation of the MST treatment model. Manuals are in place for therapist, supervisor, and consultant. A therapist adherence measure is reported on by parent report and therapist report (parent report is more accurate.) A supervision adherence measure is also used.

Key organizational context variables
In human services research, productivity is affected by the following organizational factors:

		Climate and structure are key and measurable (eg. How does it feel to be at work?)
		Structure: How hierarchical are you? How much participation in decision making is there? What matters is the fit between the structure of the place and the kind of work you are doing.
		Culture: this encompasses the values and mission. It is not what is stated as the mission statement but rather what is known (eg. “We do things this way.”)

Effectiveness of MST
Across 45 sites, outcomes revealed that changes in children were at the same magnitude as at the clinical level. Discharge decisions taken in the field were very different than those taken in randomized control trials. In the field, discharge decisions were made by the therapist and family versus an external entity. Organizational structure and climate were not associated with adherence scores, yet organizational factors were associated with youth outcomes.

Our work with MST suggests that we need to understand more about (1) the criteria for advancement and reward used in mental health provider organizations; (2) whether and how inclusion of adherence and outcomes criteria would improve child outcomes; and (3) how organizational hierarchy and procedures interfere with adherence to a specific evidence-based practice, and how to better align these to support the practice.

In the end, we have learned that “Treatments do work, paying attention helps.”

The traditional conference format is not highly successful at encouraging the uptake and use of new knowledge but it does offer an efficient format for exposing and disseminating new information. This conference sought to generate dialogue that would create a sense of excitement for the future of children’s mental health, as well as an appreciation for the capacities and initiative required to implement change in the children’s service sector. Anecdotal and evaluation feedback leads us to believe this was accomplished.

Evaluation of the day
Participants feedback about their experience of the day was very positive. Comments were overwhelming “good” to “excellent” and reflected the importance and value the day held for those in attendance. The conference venue (i.e., uncomfortable chairs, some poorly positioned seating) could be improved upon in future, however, the speaker presentations, format and registration processes, and format of the day were very well regarded by the majority of participants.

Future developments
An important outcome was the interest generated in TOTAL:Quality, the continuous quality improvement software presented by Dr. Raney and Mr. Rumberger. Several participants expressed a desire to see the software in action. As a result, all conference participants were invited to take part in a teleconference and webcast scheduled in early February 2005. There was significant interest, resulting in an audience of 11 service provider organizations and over 20 individuals - two organizations involved their entire management teams. There was also significant interest for pursuing TOTAL: Quality from a service provider in Winnipeg. The webcast led to the identification of several children’s mental health providers in Ontario that were interested in participating in a trial demonstration of the software tool. As a result of this interest, a research proposal was developed for the Ontario Centre of Excellence for Child and Youth Mental Health for their February competition. Results of this competition will be available in mid May 2005.

Transfer of Conference Knowledge
The conference report will be distributed in electronic form to participants and others in the child services sector (i.e., mental health, health, education, corrections, recreation, child welfare, ministry policy departments) using the dissemination vehicles of our sponsoring organizations (i.e., list serves, mailing lists, bulletin boards, web sites).

The conference would not have been possible but for the funding provided by the Provincial Centres of Excellence for Child and Youth Mental Health, the Hospital for Sick Children Foundation, and the Ontario Mental Health Foundation. Children’s Mental Health Ontario, The Sparrow Lake Alliance, and Parents for Children’s Mental Health are warmly acknowledged for their support in advertising the conference to their members, and for assisting in the dissemination of the conference report.

Several people assisted on the conference day in important ways. Appreciation