The Ecology of Sustainable Implementation

Phase 0: Exploration and Adoption

The case history began with a discussion of possible effective prevention efforts; this topic was raised between managers in child and adolescent psychiatry and adult psychiatry at a local hospital. This discussion concluded with selecting children of adult psychiatry patients as a target for selective prevention.

After internal surveys, a literature search, study visits, and seminars, the steering groups decided to offer an extensive educational program about “children at risk” including supervision groups, to municipal professionals and psychiatry personnel. The participant feedback was positive but called for training in how to use their newly gained knowledge and understanding in establishing preventive services. Thus, the investment in increased knowledge resulted in little practice change, as demonstrated in other studies (Klein & Knight, 2005). Training specific behavioral capacities were considered as the necessary future strategy by the purveyors.

Faced with this challenge to establish preventive services, adult psychiatry decided to limit their involvement and focus on improvements in patient record structure and for child visitors within their own department. They saw prevention as outside their clinical priority and mandate. This setback illustrates the dependence on solid purveyor commitment and stability when attempting to launch implementation efforts (Grol, 2013).

This setback challenged the remaining purveyor group to clarify their priorities. It was concluded that prevention services should be based in community services, which are more accessible and familiar to the population than hospital psychiatry departments. However, it was considered unrealistic to expect that community services in the area would be capable of building this competence. Community service personnel were not accustomed to manual-based interventions and experience with group-formats was rare.

Furthermore, the prevailing therapeutic focus among professionals was observed to be partly incompatible with a preventive approach and philosophy. Present community practices were focused on early detection of developmental problems or neglect, and offering information, support, and encouragement to all parents. Prevention would require a focus on risk rather than symptoms, and strengthening specific resilience capacities among selected children and parents, rather than a general supportive approach.

The child psychiatry department offered municipalities to collaborate with them to implement prevention services. Purveyor personnel resources were considered necessary to assist municipalities in their development process. Thus, a part-time coordinator position was designated and placed in the family section of the department. Furthermore, these situation analyses instigated a focus among the purveyors on values and philosophy, and the need to shape practitioners into prevention workers. This exemplifies the need for extensive preparations including organizational culture and values, and personnel selection and training when there are significant differences between current and intended practices (Barnoski, 2002).

Legally, prevention was encouraged but not mandatory for the specified risk groups. The top management in the hospital primarily focused on clinical productivity and wanted to cut secondary activities. Economically, the reimbursement system did not support prevention services or supervision across service levels. However, municipalities and the hospital could apply to state authorities for psychiatric service improvement grants, which were done.

The purveyors expected to encounter obstacles at several levels. The major concern was for the effort to only result in short-lived service improvements, which could occur from lack of local management ownership and service quality decay. This concern was based on experience with time-limited local experimental projects. The concern was not based on the implementation literature, which was largely unknown, and this literature had not yet clearly addressed sustainability as a separate implementation challenge (Fixsen et al., 2005).

Phase 1: Program Installation

The municipality piloting implementation entered into a close partnership with the purveyor group to plan and start prevention services. Later implementation sites were selected based on municipality readiness and motivation, only offering implementation to municipalities that actively asked for inclusion. Readiness was primarily focused on their willingness to elect, preferably at a political level, establishment of a preventive mental health service and allocate permanent, yet small, resources. Services should not be defined as a project depending on time-limited external funding. Furthermore, the required preparations included to assign management responsibility, appoint a local service coordinator, and select the first practitioners for training.

To further strengthen commitment, intervention manuals and intervention materials were lent to the municipalities that affirmed continual quality assurance and a long-term participation in a network organization. The municipalities were expected to pay a small fee, which would be allocated to cost of printing brochures, manuals, and intervention materials. Initial and future training and supervision would be provided for free. This combined controlling and rewarding approach was intended to outweigh the lack of legal obligations and economic incentives using a combination of reinforcement, social interaction, and potential coercive strategies (Grol, 2013).

Analysis and use of active strategies related to readiness and obstacles at the organizational level have been recommended by Chase (1979) and many others (e.g., Fixsen et al., 2005; Grol, Bosch, Hulscher, Eccles, & Wensing, 2007). In the present case, the previous investment in a comprehensive educational effort most likely resulted in good municipality and child psychiatry preparedness for the following implementation initiative. Thus, community readiness was both influenced actively and evaluated as recommended by several authors (e.g., Adelman & Taylor, 2003) However, no instruments were used to evaluate readiness (e.g., The Community Readiness model; Edwards, Jumper-Thurman, Plested, Oetting, & Swanson, 2000) and no structured planning tool was used (e.g., Intervention Mapping (Bartholomew, 2006) or The Hexagon Tool (Blase, Kiser, & Van Dyke, 2013). Such tools could have eased the planning process considerably.

Given the relatively generous purveyor terms to support the establishment of prevention service, municipalities queued for implementation during the first 3 years. The implementation organization had to be temporarily scaled up with additional supervisors recruited in psychiatry and addiction departments. During the 10 year implementation history, only five sites reversed their implementation intent prior to installation. These five sites were reluctant to enter the quality assurance regime. They were also all located some distance from the main purveyor site and close to distinguished professional centers that did not participate in the implementation effort. This confirms research indicating that distance is a risk for dropping out of implementation (Wolf, Kirigin, Fixsen, Blase, & Braukmann, 1995).

Although one community service in each municipality was formally responsible for prevention service, personnel recruitment was encouraged across services based on aptitude and motivation.

Phase 2: Initial Implementation

Purveyors anticipated a need for close monitoring of training outcomes and possible quality decay among practitioners during subsequent years in established practices. Based on empirical studies of prerequisites for learning new behaviors (e.g., Dancer et al., 1978; Maloney, Fixsen, & Phillips, 1981), the purveyors chose these principles for training practitioners: Following an initial two-day introductory course, practitioners were offered “in-practice” training with manuals. This training started with observing and assisting supervisors conducting the actual intervention with clients. Subsequently, the practitioners would take over in conducting the intervention, assisted and coached by supervisors during operation. This observational learning and coaching continued until acceptable mastery of behavior elements (as detailed by the manual) and core attitudes (as detailed by purveyors and supervisors) was achieved. Thus, any new practitioner would be observed and coached during at least one complete intervention series. This training model is well supported by later publication about quality control methods (Garland & Schoenwald, 2013).

No opposition to the use of intervention manual was encountered, and the first intervention manual was well accepted. These manuals had been developed and tested in practical use during a 10-year process at development sites in other countries, and had been adapted to culture and service system during a translation process. This feasibility testing and refinement probably contributed to manual acceptability in the eyes of the practitioners (McColl, Smith, White, & Field, 1998; Mihalic & Irwin, 2003).

Practitioners at two implementation sites found the training process to be too lengthy and gradual. In response to this criticism, accelerated progression was allowed routinely when practitioners felt ready for increasing challenges and was considered ready by supervisors. This illustrates the possibility of increased effectiveness in stable purveyor groups, which allows learning from the experiences at consecutive implementation sites (Fixsen, Blase, Timbers, & Wolf, 2001; Winter & Szulanski, 2001).

The implementation was piloted at a site with favorable conditions as recommended by Gendreau, Goggin, and Smith (1999) (strong leadership support, suitable and motivated personnel available). Although this approach might be wise for inexperienced purveyors, such strategies may create the risk of underestimating challenges when later sites are considerably more difficult. However, in the present case, purveyors put priority on letting pioneer supervisors become accustomed with the intervention and the training model, as supervisors had to start conducting the intervention without prior experience with it. This also required careful selection of the first supervisors as for suitability and competence.

Phase 3: Full Operation

Five feedback strategies were used to assure quality during full operation. First, colleague feedback on fidelity was done in a routine evaluation meeting after each session, related to the session objectives as detailed by the manual. Second, the manual included evaluations by children and parents within the last sessions of the intervention. Third, all practitioners were invited to network meetings twice a year. The use of network meetings as a quality strategy is detailed later.

Personnel turnover totaled less than 5% each year, which is well below the 25% maximum turnover rate recommended by Gendreau et al. (1999). In cases of personnel turnover, colleague practitioners well acquainted with the intervention and approved by a supervisor to do so, trained new practitioners in using the manual during normal operation. This was done using the same modeling/coaching procedure they had experienced themselves with a supervisor during training. Finally, new practitioners’ skills should be observed and approved by an experienced supervisor during a live intervention session. However, up to 2010, many new practitioners at old sites missed the two-day introductory course and the supervisor approval. This situation apparently occurred due to lack of written guidelines for personnel training.

One site had a complete practitioner turnover and pause in services twice during the case history. This site had a nonstandard organization of local services. Furthermore, services were reorganized and managers changed on several occasions during the implementation period. Local practitioners indicated that these factors resulted in a weaker local management ownership of the intervention and looser integration in the service organization. However, the network ensured training and motivational support for new practitioners despite the unstable local context.

The procedure of discussing readiness and mutual obligations with municipality management and politicians was not followed for some years during the middle period of the implementation effort. This failure partially resulted from a lack of implementation routine documentation and the inclusion of new supervisors. No municipality or implementation site ceased participation due to this error. However, the error identified a weakness in the implementation effort. The implementation process itself may require manuals, checklists, evaluation, or feedback procedures in similar ways as interventions (Grol, 2013).

At network meetings purveyors recurrently solicited initiatives for positive local media exposure during full operation. Practitioners were invited to share their media exposure, which was distributed throughout the network. Forty local media exposures were registered related to almost 200 completed intervention series during the first 10 years. One purpose of this was to strengthen practitioner confidence and proudness. However, the primary intention was to influence public service acceptability and reputation, a challenge often neglected in implementation, but systematically addressed in social marketing theory (Grol, 2013).

In the early full operation phase, supervisors identified few fidelity problems, and user evaluations and practitioner feedback were dominated by positive experiences. Practitioner evaluations ascribed this initial success to the combination of quality manuals, the training model, and coaching quality. The balance between quality and fidelity assurance and aiming for competence and independence was seen as attractive. Practitioner evaluations indicated that the network meetings inspired confidence in their own abilities and supported fellowship across service sites.

Practitioner feedback highlighted another important weakness of the implementation. The intervention was well manualized, but the manuals did not include guidelines for the often challenging process of recruiting and motivating families to participate. This later led to the distribution of a separate manual for such talks, but its use is not yet fully implemented.

Phase 4: Innovation

During network meetings, practitioners were encouraged to share their successes and positive experiences and discuss problems related to the intervention manual and the recruitment processes. Purveyors recommended that any substantial adaptation and change should be discussed with supervisors or presented for discussions in a network meeting. Such incidents increased with time and experience. Acceptable innovative alternatives and improvements gleaned from practice can be valuable contributions to the standard model (Winter & Szulanski, 2001) and important to improve practitioners experience of intervention feasibility (Bierman et al., 2002). However, innovation may represent problematic drift that should be corrected (Adams, 1994). Routines to evaluate and approve or reject innovation are needed.

In the present case, a positive attitude toward innovations as potentially valuable and important to share was repeatedly stressed, whereas the possibility for problematic drift was less focused. This was done primarily to stimulate mutual confidence and trust between practitioners, supervisors, and purveyors, as a prerequisite for open presentation of innovations to others. Secondly, a more strict focus on negative drift could have suppressed reports and discussions of positive innovation as well as negative drift.

Some incidents of “secretive” innovation that contradicted intervention principles were discovered, where manual changes had been done without any prior or subsequent supervisor consultations. In network meeting discussions this phenomenon was termed as “prevention ‘adolescent’ emancipation” to interpret this as a normal and expected but potentially problematic by-product of the desired acquisition of competence and independence.

Evaluations from parents and children did not reveal any occurrences of negative drift, but could have done so if asking specifically about intervention components and conduct, such as in the evaluation routines of MST (Henggeler, Schoenwald, Liao, Letourneau, & Edwards, 2002).

These network processes resulted in some improvements integrated into the intervention manuals, mostly as flexibility alternatives, or in other materials. However, most of these discussions raised issues not covered by the intervention manuals. The recurring themes were challenges related to parent/family recruitment efforts, introductory home visitations, child and parent motivation, and media strategies. These challenges were sources of insecurity and frustration for practitioners that stimulated innovation and trial-and-error experimenting, sharing ideas with colleagues, and discussing strategies with supervisors. However, these initiatives illustrate the need for supplementary manual parts or supportive documentation for such necessary support activities, not only the core intervention (Grol, 2013).

One major innovative initiative, which arose from the network activities, called for legal obligations to address child needs in situations where parents are burdened. This idea, which arose from several sources including this implementation network, was communicated by the purveyors to government agencies and channeled through national NGOs. In 2010, these initiatives resulted in a set of law revisions (Barn som pårørende [Children as next of kin], 2010) to secure child needs related to parental illness, and regulations that allow mutual family health information in child and adult patient records (Solantaus & Puras, 2010).

After five years, coverage expanded beyond the initial 200 km radius and included several sites at a distance from 250 to 450 km. Practitioners at remote sites gradually failed to attend some network meetings. As a counterstrategy, a secondary network hub was established to arrange a parallel network meeting once a year. Another secondary network hub has been planned for 2014. These changes successfully increased attendance at network meetings and stimulated local loyalty and fellowship by reducing travel distance as suggested by other implementation efforts (Wolf et al., 1995).

Practitioners were given two types of internal career opportunities. Motivated practitioners were offered training on a second and third intervention. The most skilled practitioners were allowed to train colleagues and later to serve as supervisors. Selected supervisors were coached to lead network meetings in secondary network hubs. Such career opportunities are rarely addressed in the literature, but must be common in implementation practice. Such opportunities may be vital to retain skillful and talented practitioners within an implementation organization.

Phase 5: Sustainability

During the 10-year period, local coordinators reported several local budget cuts that threatened to terminate the prevention services. However, termination has not happened at any site, in contrast to the five sites that dropped out before the initial implementation. Coordinators have reported that low cost, purveyor support, the network organization, as well as positive practitioner and user satisfaction were decisive arguments that stopped proposed cuts.

Later case history reports highlighted a threatening weakness in the implementation strategies. Practice statistics (see Figure 1 ) indicate an emerging problem by showing an increasing trend toward pauses in services. According to reports from local coordinators this happened due to failing recruitment of appropriate families although inclusion criteria were clear. Supervisors raised the question whether this emerging problem reflects the lack of procedures and manualization and training in recruitment strategies. However, they also reported little recruitment through adult health, psychiatry, and addiction services. These services would logistically be the most relevant service points to “intercept” relevant families based on parent service use.

According to personnel working with implementing the law revision passed in 2010 (Barn som pårørende [Children as next of kin], 2010), adult service professional are still somewhat reluctant to discuss children and parenting with their clients despite their new legal obligations to ensure support for client children (Solantaus & Puras, 2010). Prevention practitioners are therefore faced with two difficult alternatives: Attempt to find more appropriate families personally or engage in activities to stimulate and support implementation of new practices in reluctant adult services.

Practitioners that were asked to identify what they have experienced as vital to sustainability identified several aspects: The intrinsic motivation gained during the interventions, including trust, growth, and positive feedback from children and parents. They experienced the intervention tasks as meaningful and energizing, despite being confronted with neglect and suffering as well as difficult child experiences and feelings. Additionally, practitioners identified the availability of supplementary supervision, the fellowship established during network meetings, and the long-term bidirectional commitment between purveyors and municipalities. This stability, commitment, and positive focus ensured that difficulties, challenges, and setbacks could be transformed into increased competence and coping.

Many practitioners stated that sustainability depends on the strength of “a preventive culture” as expressed in values, attitudes, and behaviors. These were detailed as the belief in competence, potentials, and coping capabilities despite difficulties. This preventive culture was experienced throughout all activities and levels of the network organization: In interventions, personnel training, supervision, evaluation discussions, network meetings, and attitudes toward children and parents. Network meetings, supervision, and coaching were considered the most important instrument to sustain this culture and value set and to maintain practitioner motivation and continued engagement.

Implementation reviews systematically state that a renewal-oriented culture as well as value compatibility between old and new practices are important prerequisites in implementation (Fixsen et al., 2005; Grol, 2013). However, the implementation literature does not discuss how the values and philosophy of new practices can be conveyed convincingly. This absence is peculiar in contrast to the evidence-based program definition that includes “… Clear philosophy, beliefs and value ….” as one of five characteristics (The Dissemination Working Group, 1999, cited by Fixsen et al., 2005). Historically, psychosocial interventions possibly had too great of a reliance on ideology and conviction rather than the practical and empirical bases. Nevertheless, practitioners that understand the “heart and soul” of an intervention may be better equipped to adapt and innovate interventions and avoid drift that negates the core effects. Therefore, I consider it vital to establish recurring arenas like network meetings and other channels that may allow repeated presentations and discussions regarding core elements and attitudes in interventions.

Case History – Suggestions for Local Improvements

From the viewpoint of this author, purveyors and their network still face a set of serious challenges, leaving the services with a continual reliance on the implementation purveyors to maintain sustainability. Many municipality services are still not capable of sustaining quality assurance and competence renewal in prevention based in their own culture and capacities. Some of the reasons for this continued situation are in my view found in the small size of most municipality administrations, and the lack of culture for routine monitoring of service quality.

Despite the successful establishment of services and the good intervention manuals, the case description shows several weaknesses. Many strategies planned and utilized in the early years were later forgotten or disregarded due to a lack of implementation manualization. These events included occurrences of missing formal and informal contracts with municipalities, delayed implementation of practitioner and supervisor certificates, failure to include “replacement” practitioners in introductory courses, lack of systematic client recruitment routines, and nonsystematic use of structured colleague practitioner evaluations. Other challenges are more difficult to resolve, such as the adult professionals’ reluctance to address parenthood with clients and to recruit families for prevention services. Furthermore, guidelines and training for client recruitment are lacking.

Thus, a number of important support activities and implementation procedures are not documented in manuals, guidelines, or checklists. This is the most important suggestion for future improvement of the present case is development of such tools.

Speculation suggests that this network would be in danger of slowly crumbling if the purveyor group was phased out without any replacement for their coordinating functions. However, it is possible to envision sustainability with a minimum of continual regular network meetings, agreements to train and approve new practitioners at neighbor sites, and phone support provision by experienced colleagues in the network. Such a minimal remnant of the implementation organization could be sustained within ordinary services based on agreements across the network to exchange support between sites. Some support could also be mobilized by activating the hospital obligations to supervise municipalities. Such future perspectives should be discussed in the network and between purveyors before they occur.

This case history suggests an increased focus in implementation science and practice on the following subjects:

• •
  Documented procedures for implementation and support activities;
• •
  In-practice training with observational learning and coaching as training method;
• •
  Arenas and channels to repeatedly focus intervention “heart and soul”;
• •
  Positively focused procedures to detect and evaluate innovation and drift;
• •
  Renewal and maintenance of practitioner skills and confidence;
• •
  Structures for challenge and setback support and supervision;
• •
  Practice renewal and internal career opportunities;
• •
  Long-term sustainability perspectives before initiating implementation;
• •
  Strategies for sustainability without dependence on purveyors;
• •
  Development of quality assuring and self-renewing service organizations.

Limitations

Case history descriptions and viewpoints presented in this case study are based almost exclusively on sources among purveyors and practitioners involved in the implementation effort. Because of this any suggestions and conclusions must be treated with hermeneutic caution. Practitioners and supervisors that left the network did not contribute and few external or independent information sources were available. Most suggestions from this case study coincide with the implementation literature, but a case study only suggests and illustrates hypotheses and perspectives which must be tested with other designs.

Conclusion

This case history suggests that implementation success requires multiple broad strategies that span all seven implementation approaches suggested by Grol (2013): cognitive, motivational, marketing, reinforcement, social interaction, management, and control/compulsion. However, this conclusion may only hold true when many basic requirements for renewal and evidence-based practice are lacking. Such requirements must be evaluated in strategic surveys and analysis prior to planning for implementation efforts.

Another suggestion that emerges from this case history is that implementation plans require evaluations not only of present obstacles and resources, but also expected setbacks, quality decay, and sustainability challenges beyond the initial implementation project. Implementation strategies must consider all these aspects to live up to the real challenges of implementation and reduce its expectable risks (Panzano & Roth, 2006).

Future evidence-based practice requirements and renewals should be built into the education system and practitioner, supervisor, and manager role expectations, as well as integrated into our organizational cultures. This integration would simplify implementation and allow renewals to focus on replacing or improving core elements or support elements (Embry, 2004) rather than reinventing entire service organizations.