introduction activities publications events (just in Hungarian) our staff

library library database links contact us

magyarul

Timing of the evaluation of the National Drug Strategy, questions, recommended algorithm

Premises

Mid-term evaluation of the strategy was carried out in 2003-2004 by the experts of the Dutch Trimbos Institute. Evaluation particularly examined if strategy:

• is a consistent document,
• how does it fit within the EU Drug Strategy,
• if the national and local key parts of the implementation of the strategy feel strategy objectives successfully fulfilled,
• how national and local mechanisms are working and how they help to implement the strategy.

Quantity of available relevant data and their availability at the starting point of the strategy did not allow to use a method focusing on outputs and outcomes during the mid-term evaluation. Therefore analysis was focusing on procedures, key experts, persons working in the coordination in main positions and the following documents:

• the strategy itself
• government decrees related to the implementation of the strategy.

Summary of the evaluation:

1. The strategy is a very ambitious public policy document based on modern considerations.
2. It suits well with the EU Drug Strategy considering approaches, objectives and planned actions.
3. Inner consistence of the strategy is not always clear: objectives and planned actions are not always organized linear.
4. Action plans do not fit within the requirements of real action plans: there are too much »continuous« deadlines, carrier of responsibility is not always obvious and there are not obvious implementation indicators for certain activities.
5. Judgement of operation of coordination mechanisms is Janus-faced: most of the interviewed persons appreciated the multidisciplinarity of the Coordination Committee on Drug Affairs (KKB) but they disapprove, that KKB-decisions can not be enforced and decision making process can not be affected on proper level by its expertise. Local coordination has similar difficulties, there it was even more emphasized that local administration is unwilling to give more financial support for activities to combat drug problem and to prove modern approaches in drug related decision making.
6. Interviews with key persons pointed out that the realization of progressive objectives is embarrassed because of the lack of coordinated assertion, the position of drug problem on the political agenda and the lack of funds.
7. The national strategy is a very complex aggregation of different professional institutes and organizations of executive power. Therefore critics and difficulties show up generally in all kinds of programmes related to the strategy.

Later the government was able to enforce some actions have been pointed out by the lesson of evaluation:

• the Coordination Committee on Drug Affairs (KKB) got restructured to validate more decision preparation and adviser tasks;
• in the new government decree accepted in 2007 already definite responsibilities, deadlines, implementation indicators and funds have been assigned to every single action;

Final evaluation of the National Drug Strategy
As the drug strategy aims tasks to carry out until the end of 2009, it is needed and reasonable to conduct outcome evaluation also.

External evaluation of the „National Strategy to combat drug problem” (accepted by the Hungarian Parliament in 2000) has been ordered by the ministerial unit for drug coordination in 2009.
By NDI (National Institute for Drug Prevention) the HealthMonitor Research and Consulting Non-profit Ltd. (EgészségMonitor Kutató és Tanácsadó Nonprofit Közhasznú Kft.) has been charged with conducting research to evaluate the implementation of the strategy. Primary goal of the evaluation was to find out what conclusions can be drawn by analysing the Strategy implementation. The research was seeking to answer following questions:

• Changes observed in national drug scene are in line with Strategy objectives?
• Do activities that are related to Strategy correlate to the observed changes?
• According to experiences and available, up to date knowledge on the field, how do Strategy goals, assigned resources and related activities match?

During the evaluation, two different types of methodological approaches were used: on the one part descriptive method, i.e. they get information by observations and on the other hand test method, because new knowledge is established by information analysis.
Evaluation methodology was dual also by the participants: on the one part persons independent from drafting and implementing of the Strategy, on the other hand persons who took part in these activities.
The following methods were used in the evaluation:

• document analysis;
• deep interviews with decision makers and professionals;
• focus groups about the first findings of the evaluation;
• problem tree analysis.

Summary about findings of the four evaluation processes
Document analysis on activities related to 90 identified long-term Strategy objectives pointed out that 123 activities were identified as clearly related activities to the goals of the Strategy. 17 of them were connected to more goals. Among objectives, 14 of them were not assigned to any activities, in case of 5 goals most likely because of sweeping statement (too general drafting). Among identified activities 8 of them (all are from the field of „Treatment, services”) were contrary to the objectives.

In the four pillars of the Strategy considering all time frames, fulfilment of altogether 153 identified objectives were analysed by this method (document analysis). Completion of 153 identified objectives was split unequal by pillars and by time frame as well. Pillar contained the least goals was the „Sensitive community” (26), and the most objectives were identified in the pillar of „Treatment, services” (55). Regarding time frame: on short time 24, on mid term 54, on long time 75 goals to be completed were identified in the Strategy. On the whole Strategy – i.e. regarding all four pillars and all three time frames – 30% of the objectives fully, another 30% partially were completed, and more than 40% were not completed at all. Pillar of „Communities” shows a bit different distribution: rate of fully completed objectives is close to 40%, while every third objective was not completed at all. In case of „Prevention” we can find the most completed and the least not completed objectives together: 43% and 16%. Among goals of „Treatment, services” were fully completed only 22% and 64% of them were not at all. Score of „Supply reduction” pillar was very similar: 18% and 54%. We can sum up that – not considering time frames – objectives from pillars of „Treatment, services” and „Supply reduction” were the least and from pillars of „Prevention” the most completed. If we analyse only long-term goals, we can say that those mostly were not completed in any pillars (49%, 33%, 75%, 67%), contracting all pillars only less than half of the objectives were completed partially or fully.
We can see from above mentioned data that contrary to the ambitious objectives established during half time evaluation, smaller or bigger positive changes are perceived on all fields of the Strategy. Passing the common expression of half empty or half full glass, valid evaluation experiences help us to establish the new Drug Strategy.
Above evaluation experiences far reaching were used during drafting the new Drug Strategy. Therefore, new Strategy drafts clear, realistic objectives, where indicator system for each fields was developed to be able to measure implementation more exact. For these indicators specific goal rates will be also defined in action plans, as it already has become practice in the last phase of existing Drug Strategy.
Most of the participants of deep interviews and focus groups considered national drug strategy (accepted in 2000) as a very important step, because significant development relating drafted objectives was reached by the document. Primarily we have to mention establishment of the coordination institutions on drug problem, from national (KKB - Coordination Committee on Drug Affairs) to local (KEFs – Coordination Forums on Drug Affairs) level. Institutional background of related research and monitoring activities (National Institute for Drug Prevention, National Institute of Addictology, National Focal Point) also significantly contributed to the completion of objectives.  Pledges of implementation of the Strategy were budget, which was higher than financial resources for other social aims, and the human resources including qualified professionals trained in different trainee programs in the frame of the Strategy. Increasing resources supported to widen prevention activities and to build up almost full spectrum of treatment services. Operational conditions were ensured by promulgation of some new, very important legal regulations. Finally – maybe because of these above – social acceptation also has changed into favourable direction and different judgements of drug problem became „visible” in the Hungarian society.  

According to concordant opinions, effectiveness of Strategy was restricted not only by lack of resources but also by short collaboration willingness from the side of related sectors. They also say, changes in this field can be expected by the regulation and development of coordination and more effective communication, which obtains decision makers much better. Therefore, new Drug Strategy more emphasizes further development of coordination mechanism and institutions.
Accord was shown also regarding social acceptance of drug problem: despite favourable changes it is far from desired, importance of it is mostly not recognized. At reviewing reasons of the problem and solution options, it is mostly left out of consideration that drug consumption and other harmful health behaviours have the same roots. Necessity of change of approach was emphasized by most of interviewed persons; they drew attention also to change media behaviour.  
Policy makers, professionals involved into the evaluation judge prevention strongly to be revised regarding both coverage of target population and scenes, and its efficiency. Among reasons most of interviewed persons mentioned lack of accreditation and lack of resources for this purpose. Except need for enhancement of professionalism, they did not agree on agenda. Some of them suggested getting a special legal regulation for this activity through and therefore resources for prevention and health promotion could be increased.  
The new Drug Strategy drafts prevention objectives along more effective linking between scene and target population and draws objective for introduction of accreditation.
Common opinion in judgement of treatment and services are that harm reduction is positively appreciated and that lack of experts is problematic. Otherwise, judgement of the field in almost all other questions was contradicted.  
One of the most important experiences of the evaluation might were extremely different opinions occasional appeared regarding the Strategy and its completion. The very different judgement of problems and possible answers is explicable with partial knowledge about facts, interviewed persons´ different role in Strategy implementation and contradictory views in Hungarian society. Therefore, establishment of wider professional consensus and obtaining social support are priority tasks for new Drug Strategy drafters.
In all phases of strategy drafting extensive conciliation process were carried out with participation of concerned civil and professional organizations. A webpage (www.drogstrategia.hu) has started to work for wider publicity, where everybody can get information on concept being drafted and observations of evaluation.
There was understanding about some further suggestions related to new Strategy. Special attention should be paid to extension of implementation monitoring for all fields, and therefore efficient use of narrow resources can be ensured. Steady claim established to new Drug Strategy that future activities should improve all risk target groups´ situation and approach should be complex with wide interpretation of health promotion. There is general opinion also about content of new Drug Strategy: beside hierarchical goal setting system it should also include system of priorizated activities for achieving these. Finally, appropriate short-term action programs built on priority system of interventions with assigned necessary resources must be carried out for successful realization. New Drug Strategy wishes to comply all the expectations drafted in evaluation. Its professional innovation a three-dimensional structural solution allowing priorization of interventions developed along complex goal setting system.
By review of results coming from all methods applied in the research, we can summarize that on Strategy accepted in 2000 a very extensive development was perceived on drug field. Almost all items of the best international practice tested for solution of this problem have been appeared in Hungary and by favour of Strategy, their systematic use has been started. The task of new Drug Strategy might be intensive development of this system: geographical expansion and quality improvement of these services and ensuring the reach to all target groups as required.