The aim of this review is to outline the main features of the studies carried out in Spain and to indicate the essential conditions of this type of study and the strategies necessary for achieving them.

For the purposes of this work, we shall only consider as "treatment evaluation studies" those that identify this as their explicit aim, and that possess certain minimum methodological characteristics, as described below. Consequently, we shall not include follow-up studies whose sole aim is to monitor the evolution of a group of patients over a given period of time. Also excluded are those studies that evaluate treatment with agonists, since this type of programme pursues different therapeutic objectives, and therefore requires different evaluation methodology and criteria. For example, a maintenance programme with methadone may lack well-defined time limits, being judged fundamentally for its achievement of a high rate of retention and a significant reduction in criminal behaviour (Roca, Gómez and Paños, 1989). In such conditions, it is more appropriate to state that the object of the evaluation is an addictive course with a legal substance (substituting heroin) than a therapeutic procedure with an established temporal framework that seeks, among other things, "drug-free" behaviour.

The pioneering and reference studies in this field are from the United States. The DARP (Drug Abuse Reporting Program) and TOPS (Treatment Outcome Prospective Study) programmes, costly and wide-ranging, are classic examples of this type of work. Financed initially by the NIMH (National Institute of Mental Health), the DARP programme (Simpson and Sells, 1983) recorded data up to 1973. In the course of its work it examined 44,000 people that had been subjected to one or more of the following treatments: methadone, therapeutic community, outpatient disintoxication programmes and day centres with drug-free treatments, in a total of 52 different public centres. The results of patients that completed the treatment were compared with those of patients that had dropped out. Also analysed were the differences between the different treatment modes. The TOPS programme (Hubbard, Marsden, Rachal, Hardwood, Carvanaugh and Ginzburg, 1989) began collecting data in 1979, from a total of 11,000 patients, to whom a follow-up was applied, involving interviews at 3, 6 and 12 months after discharge. Results of the treatments, as in the case of the DARP, were compared with one another and with those of the control group. The data from both TOPS and DARP appear to support the idea that all treatments work well, giving results superior to those of drop-outs; the most effective programmes are those that succeed in maintaining patients within the programme for the longest periods.

In Spain, few studies have been carried out with the aim of evaluating the effectiveness of treatments, despite the fact that, since the 1970s, the public funds made available for this purpose have increased considerably. In this context, the remarks of Emiliano Martín, Assistant Director of the National Anti-Drugs Plan, are revealing: "We have passed through a phase of initiation and growth of programmes, especially assistance programmes, and we have now reached a stage of consolidation of such programmes and of the improvement of their quality and effectiveness" (Matellanes Matellanes, 1998, p. 40). Quite probably, the need to channel the available resources into satisfying the growing treatment needs of a burgeoning and increasingly demanding population leaves little room for research. In any case, the scarcity of studies carried out and their limitations are notable. Lack of resources and methodological problems with serious design deficiencies are the most common blights.

Below we outline the methodological characteristics of a typical study for the evaluation of drug-dependence treatments and the most common limitations and methodological problems. This is followed by an analysis of the different studies published in Spain.

In general, this type of study is designed according to the following scheme:

Despite the apparent simplicity of the design, there are several difficulties involved in carrying out studies for the evaluation of drug-dependence treatments. Melgarejo, Sanahuja, Masferrer, Sala, Pérez and Piña (1988, p. 17) list some of the most relevant: "a) Use of retrospective instead of prospective designs. Assignment of users to different treatment groups; b) Control of the variance of the different treatment groups; c) Use of indicators with poor sensitivity in the evaluation of results; d) Difficulties in locating patients; e) Difficulty in evaluating the reliability and validity of the data".

Sánchez-Carbonell and Camí (1986), in a previous article, pointed to similar design difficulties, among them: a) Lack of a control or comparison group; b) Bias in the sample studied, data being collected only from the cases that present the best evolution; c) Validity and reliability of the information; d) Disparity in the comparison periods from one study to another, with time intervals ranging from one year to ten; e) Disparity between studies in the parameters used for measuring characteristic variables, such as drug consumption, criminal behaviour, etc.; f) Markedly different criteria of success or cure from one study to another.

These difficulties would hinder comparative interpretations of results and impede, in some cases, the drawing of minimally acceptable conclusions.

There are some important reasons why these methodological problems are common in the field of the evaluation of drug-dependence treatments. Among these, three should be highlighted. First, the need for long evaluation processes with high research costs, which should ideally be the responsibility of institutions independent of those studied, and with no short-term economic or political interests, so common in the field of drug addiction. Second, the peculiar features of the population studied, frequently characterised by social rootlessness, broken homes, chronic illness, etc., which renders extraordinarily costly and difficult any research involving prolonged follow-up. Finally, the characteristics of the treatment programmes themselves, widely disparate in both basic philosophy and method, making problematic any process of standardisation of criteria or assessment instruments.

As pointed out in the previous section, a study for the evaluation of drug-dependence treatments should have, as part of its design, pre- and post-treatment data, measures of both addictive behaviour and work and criminal behaviour, a control or comparison group, and comparative analyses between groups. On the basis of these minimum requirements, we shall discuss briefly the most important methodological difficulties in terms of the effect they have on the validity of the treatment evaluation (Table 1).

In this section we shall review the studies that have been published in Spain, identifying their principal characteristics and limitations, in the light of that which has been discussed above.

The first observation researchers will make is the extreme scarcity of studies in this field in Spain. There are a considerable number of follow-up studies (Guardia Serecigni and González, 1983; Sánchez-Carbonell, Brigos and Camí, 1989; GID, 1992; García Martín, Roldán and Comas, 1995; Melús Moreno and Gutiérrez, 1996; Marina, Vázquez, Jiménez and Erkoreka, 1996; Comás, García and Roldán, 1996), which are usually referred to in the sections on evaluation of the programmes, but which will not be included in this review for two reasons. The first reason is that follow-up studies are considered to have different objectives from those of treatment evaluations. In the former, the priority is to obtain information on patients’ evolution and its relationship to different pre-, intra- and post-treatment variables; in the latter the aim is to determine whether the treatment under study is preferable to non-treatment (although comparisons may also be made with other treatments, including, in theory, placebo), in terms of their effects on the modification of certain variables already mentioned above. The second reason for excluding follow-up studies is that their different objectives require different methodologies, such as the use of, at least, a control or comparison group. These differences mean that any joint evaluation of the two types of study would be confused and of little informative value.

Thus, if we eliminate follow-up studies, to our knowledge there remain only two that should concern us here. The first is the study for the evaluation of the Dross programme in Cataluña (Melgarejo et al., 1988) and the second is that carried out on the treatment at the Provincial Drug-dependency Centre (Centro Provincial de Drogodependencias, CPD) in Cádiz, southern Spain (Girón García, 1997; Girón García and Martínez, 1998). Some of the characteristics of these two studies are shown in Table 2.

In both cases it is clearly stated that the study aims to assess a treatment programme, and control groups are used that have received no treatment or that have been treated for a period insufficient to be considered effective. The follow-up period in these two studies differs greatly: while in the case of Melgarejo et al. (1988) it is one year, in that of Girón García (1997) it is six years. This difference would appear to be reflected in the percentage of subjects with favourable evolution, which is far higher one year after treatment. In both cases there is a notable difference in the number of subjects included in the control groups, which are very small, compared to the numbers in the groups of discharged patients (see Table 2), and this can lead to certain difficulties in interpreting the results. In either study there appears to be a differential effect of treatment on the diverse variables analysed, as compared to non-treatment, which would indicate the effectiveness of the therapeutic services studied.

However, the two studies have some important shortcomings. If we analyse the results in terms of the biases described in the previous section, we find that both that of Melgarejo et al. (1988) and that of Girón García (1997) are subject to some of the problems affecting internal and external validity.

In the study by Melgarejo et al. (1988), data-collection was entrusted to a group of researchers that had to administer a questionnaire to patients, either in person or by telephone, without any type of cross-validation measure (urine analysis, use of external records, family information, etc.). In the study by Girón García (1997), the instrument for collecting data in the interviews was constructed on the basis of some test items, such as the ASI (Addiction Severity Index), the results referring to recent consumption being compared with those from a urine analysis carried out after the interview. None of the information relating to other variables analysed (family, employment, sociolegal situation, etc.) was checked against external records.

Statistical mortality was in neither case very high, if we take into account the results of other research in the field, such as the DARP (Simpson, 1993), with 39% of patients lost, and the TOPS (Hubbard et al., 1989), with losses of between 42 and 29 percent. Even so, losses were above the level recommended by authors such as Gould and Lukoff (1977), who set a maximum acceptable figure of 20%.

In neither of the two studies is any mention made of the possible effect on results of the mechanisms of selection for and admission to therapy. Nor is there any reference to the potential effect of the evaluation procedure on the sample studied, and how this may influence the results of the evaluation.

Another shortcoming of these studies is the lack of analysis of the active principles of the treatment that explain its differential effect compared to non-treatment. The mere confirmation that "the longer the time the subject has remained on the programme the lower the probability of relapses" explains very little if there is no control of the mechanisms of admission and exit from the programme. The explanation becomes completely circular: they are abstinent and they fit the programme’s criteria so they remain in it for longer, and because they remain in it for longer they stay abstinent and fit more closely the improvement criteria.

There is a need in the reports of the two studies for a more detailed description of the therapeutic procedures assessed. In particular, it would be appropriate to include a more detailed account of the mechanisms for selecting patients and of the therapy or therapies used in each case, with a view to assessing improvement effects within this framework.

As can be deduced from that which has been discussed up to now, very few studies for the evaluation of drug-dependency treatment programmes have been carried out in Spain; moreover, those that have been implemented have been hindered by certain problems that have made their interpretation difficult and reduced their general utility. Any study carried out in the field of treatment assessment must comply with the methodological criteria of this area of research (Fernández Ballesteros, 1995). Some of these have already been mentioned, such as the need for external evaluation, for the assessed programmes to be described in detail, and for the clear specification of the objectives of the assessment carried out.

Furthermore, there are some specific recommendations that should be taken into account by researchers in the field of drug-dependency, related to general planning of the research and precise methodological issues. In the following section we shall review some of these.

The majority of research carried out both for the study of the natural evolution of addiction and for the assessment of treatment has been of a retrospective nature. In no case has provision been made for an assessment process within the treatment programme itself. This is an important drawback, as it constitutes an obstacle to the connection between therapeutic operations and results, and to the corresponding adjustment of the assessment process to the different stages of treatment. In the conditions in which this type of assessment is usually made, only a prospective study could provide information on the therapeutic operations that maximise effectiveness of the intervention and its appropriateness for the profile of each patient. Retrospective studies (given that they evaluate the results of a complex problem) allow, at the most, to report on the effectiveness of a service and of a treatment programme as a whole, excluding from the analysis the variations of patients’ individual responses to each therapeutic stage of the programme. Another beneficial effect of prospective research is that it permits the optimisation of strategies for reducing statistical mortality (informing patients that they will be assessed after discharge, obtaining precise information for their subsequent location, etc.).

There are unavoidable ethical difficulties involved in the random assignment of addict subjects to the different treatment groups and the control or non-treatment group. The inclusion of subjects in the different treatment groups is made according to subjects’ characteristics and their addiction, always with the intention of obtaining maximum benefit and reducing the risk of abandonment and relapse. Thus, it is quite obvious that a patient cannot be assigned to a control group that will receive no effective treatment. Given this situation, it is difficult to establish the basic conditions that make possible a comparison between groups of patients, in which the differences between them would be attributable to the therapeutic operations to which they have been subjected, and not to the previous conditions required for forming part of them. This is especially the case given the impossibility of forming a pure experimental control group.

The solution offered for these problems is not a definitive one, but it may mitigate some of the negative effects on the validity of results. This solution has two conditions. On the one hand, the treatment groups should have explicit conditions (made explicit in an operative way) of admission and exit of the subjects of each intervention. In this way it would be possible to estimate the influence of subjects’ particular conditions on the therapeutic results. On the other hand, it is necessary to form a comparison group made up of subjects that, although identical to those of the treatment groups in terms of indicators related to the evolution of the addiction, have not received significant therapy, and whose situation, therefore, can be considered as following a "natural course" of addiction. These types of subject tend to be found among those that, having fulfilled the conditions for receiving treatment and admitted to a programme, abandon it after a short time.

The presence of a large number of subjects that are impossible to locate, or that refuse to participate in the treatment assessment, constitutes a risk of bias in the results that it is difficult to evaluate. In a work devoted specifically to this problem, Desmond et al. (1995) recommended the following measures: a) To collect as much information as possible on the subject during the treatment; b) To inform the subjects that they will be interviewed after the treatment; c) To give material incentives to subjects for participating in the research; d) To select suitable staff for carrying out the follow-up; e) To keep a detailed record of the activities carried out during the research work; f) To use information from other institutions; g) To carry out a brief interview; h) To carry out the interview as soon as possible after location of the subject; i) To make it easy for the subject to attend the interview; j) To allow plenty of time for carrying out the field work.

Some of these recommendations are intended to make subjects more motivated to participate, whilst in others the aim is to compensate for the lack of information resulting from the inevitable loss of part of the sample. The authors of this work (Desmond et al., 1995) maintain that the implementation of these measures has produced significant reductions in statistical mortality, keeping it within the recommended limits, according to the results of a field study.

In any case, if considerable losses of subjects occur, a useful procedure for palliating this problem is the comparison of subjects lost to the study with those that participate in it, in terms of a series of basic variables (pre-treatment consumption history, sociodemographic data, criminal record, etc.). The differences found in these types of variable may provide clues as to the direction of bias in the sample (Caspar, 1992).

There would appear to be a general consensus on the main variables to be studied and certain disagreement on methods and measurement instruments. Consumption behaviours, together with the data referring to work or criminal activity, are included in practically all follow-up and treatment evaluation studies, since they relate to a variety of objectives pursued by the majority of therapeutic programmes in this field. Nevertheless, it is highly recommendable to collect data on the patient’s evolution corresponding to the follow-up period, and not only on the point at which the assessment is made. In this way, more reliable conclusions can be drawn with regard to the effects of the programme assessed, and firmer relationships established between the variables studied (consumption, work activity, criminal behaviour, etc.) and other types of event subsequent to completion of the programme.

As far as instruments are concerned, disagreements are more evident. The use of tests providing results that can be compared from one assessment to another is a highly desirable goal within this field of research. It would not appear to be an easily achievable one, however, given that each study seems to require the adaptation of existing instruments to the peculiarities of the treatment under analysis. Graham (1994), taking this into account, developed a series of general recommendations for the selection of measures and instruments that should be adhered to by all follow-up studies of addiction treatments. Nevertheless, there are certain areas, frequently included in research, in which, wherever possible, the use of standardised instruments should be considered that take account of the validity and reliability of results, two essential measurement properties in the social sciences (Muñiz, 1998). In this context we might mention, for example (and among many others), the BDI (Beck Depression Inventory) for the assessment of depression, the GHQ (Goldberg Health Questionnaire) for the evaluation of state of health, or the AUDIT for obtaining information on alcohol-related problems.

Reliability studies that compare self-reports with the results of urine analyses show a high level of agreement –over 90% (Maisto, McKay and Connors, 1990). Nevertheless, the validity and reliability of the data remain open to question. Studies that have related subjects’ responses to questionnaire items with results obtained from other indicators (mainly consumption information validated through urine analysis) have been limited to short periods, and it is not clear that the validity and reliability of responses will be similar in studies in which subjects must report their behaviour (addictive, criminal, work, etc.) over a longer period of time (one year or more). In such cases, for example, subjects’ reports of their behaviour cannot be validated through the use of a urine analysis at the time of the interview. In view of this, it is necessary for subjects’ reports to be backed up by as much external validation as possible, using criteria that take account of the period to be studied. Two important sources of external information are institutional data (health service, judicial, employment, etc.) and data from the family context.

In recent years there has been a rapid proliferation of assistance programmes for drug-dependents in Spain. There is currently a consensus among the majority of professionals on the urgent need to develop studies for assessing the effectiveness of these programmes, with a view to making possible improved planning and decision-making. Nevertheless, studies for the assessment of the results of treatment carried out in Spain have been scarce. In the majority of cases they are follow-up studies, which, given their methodological characteristics, can only inform of the state of subjects at a particular moment, and from which we can make no reasonable inferences about the effects of the intervention.

Moreover, most of these studies are subject to methodological limitations. Some of these limitations affect studies for the assessment of programmes in any area of research, while others are peculiar to the field of drug-addiction, such as those related to the population characteristics and to the wide conceptual variety of treatment programmes.

In this context, there is a need for the planning and design of studies for programme assessment that are well controlled and that adhere to the methodological conditions that have demonstrated their usefulness for dealing with the different problems that may present themselves. Some of the recommendations for such conditions have been referred to in this work, though it does not constitute an exhaustive compendium of solutions to all the problems to which this type of study is subject. In any case, it can be considered that observation of these recommendations will lead to more valid and informative research on the effectiveness of treatment in the field of drug-dependency.

Eventually, this line of research should lead us to the design and preparation of assessment protocols for each one of the treatment modes currently in use (programmes with agonists, therapeutic communities, outpatient programmes with antagonists, etc.). This is a suggestion that coincides, moreover, with that which is proposed for other areas of social intervention (Fernández del Valle, 1992). Even so, a first essential step would be the consolidation and homogenisation of the content and objectives of each type of programme.

View this Table: [in this window] [in a new window]

Table 1. Most frequent methodological limitations in studies for the evaluation of drug-dependence treatments

View this Table: [in this window] [in a new window]

Table 2. Characteristics of studies for the evaluation of drug-dependency treatment programmes carried out in Spain