ARTICLE
Auteur(s) : Isabelle
Ray-Coquard1,2, Franck Chauvin3, Antoine
Lurkin2, Françoise Ducimetière2, Jean
Philippe Jacquin3, Cécile Agostini4, Isabelle
Pouchard5, Bernard Meyer6, Fadila
Farsi1, Patrick Castel1, Lionel
Perrier2, Thierry Philip1,2,3,4,5,6,7,8
1Centre Léon Bérard, 28, rue Laennec, 69008 Lyon,
France
2Gresac (LASS-CNRS FRE 2747), 28, rue Laennec 69008
Lyon, France
3Institut de Cancérologie de la Loire, 42000
Saint-Etienne, France
4Centre hospitalier, BP 1125, 73011 Chambéry, France
5Centre hospitalier, 28, rue Charlieu, 42300 Roanne,
France
6Clinique de Rillieux, 941 rue Capitaine-Julien, 69140
Rillieux, France
7Centre Val d’Aurelle, 34094 Montpellier Cedex,
France
8Fédération nationale des centres de lutte contre le
cancer, 101, rue Tolbiac, 75654 Paris, France
Understanding medical practices or the whys and wherefores of care
decision-making is among the major objectives of medical, economic
and sociological research in the current political environment. All
partners involved in patient management have now understood the
necessity of good medical practice and organization of care.
However, scientific studies and governmental reports published so
far have failed to provide information on the functioning and
performance of medical practice. In that particular domain, it
seems that the political structures and mental attitudes of a
country cannot be extrapolated to another. In consequence,
researchers cannot rely on results published by other international
groups. After identification of factors affecting medical practice
and notably validated recommendations for clinical practices, we
must therefore specifically assess the advantages and limitations
of clinical practice guidelines in France, as well as the impact of
network functioning on medical practice.
Factors affecting medical practice
Among the reasons most commonly cited to account for the
discrepancies between medical practices is the particular style and
professional behavior of the physician. Each physician may use
individualized medical instructions tailored to test and treatment
thresholds. As reported by Eisenberg and Williams [1], a study of
these specific thresholds can help understand a physician’s
practice behavior. The “test threshold” refers to the point at
which the likelihood of the disease is high enough for the
physician to order a diagnostic test. Diagnostic thresholds that
trigger medical prescription are governed by two factors: medical
knowledge, gained from either initial or continuing medical
education, and the physician’s own medical experience and
underlying motivations. The choice of treatment will thus naturally
vary with the knowledge of the practitioner and his experience in
the field.
Medical knowledge may be gained from several sources, although
it is impossible today to judge the respective merits of each of
these sources for medical decision. Several factors have been
identified and proved to be correlated with the medical decision.
The first factor is initial medical education, the quantity and
quality of information of which should be balanced with appropriate
methodological knowledge to help medical students understand and
analyze the information they receive [2]. The second factor is
continuing medical education which has become mandatory for French
physicians since 1996. Yet, a literature survey by P. Durieux
has failed to show that the distribution of educational material or
more formal continuing medical education activities influenced the
behavior of physicians [3]. No teaching method appears
substantially better that the others when efficacy is considered
[4]. Pharmaceutical industries are able to strongly modify medical
practices. Effectively, in a literature review on the subject,
Thomson found 13 studies where a local follow-up visit allowed
to improve medical practice; the impact of follow-up visits was
superior to no visit at all, with a 20 to 50% relative improvement
of prescribing practices [5]. This improvement rate is probably one
of the most important concerning impacts of strategies designed to
modify medical practice. Data from the scientific literature mostly
originate from articles published by medical journals or from
conference presentations. It is not known for sure how the
publication of a given treatment trial or a meta-analysis can
influence individual medical practice. Many consensus conferences
have been held and their conclusions have been published, but the
apparent impact on medical practice remains limited [6-8].
Modifications in the administrative organization of the medical
system can provide a valuable alternative to education programs.
However, their impact on medical practice is limited. No randomized
study has demonstrated a significant direct benefit from these
procedures, but several transversal studies have suggested that
they might be useful in efforts to modify medical practice [9]. The
low efficacy of these methods is mostly due to their being
perceived as repetitive and tedious. Several “spontaneous”
experiments have confirmed that financial incentives could
successfully influence medical practice. Eisenberg and Kabcenell
have suggested, for instance, that modifying the payment
methodology helped modify physicians’ behavior [10]. Yet, the
capacity to change behaviors through financial rewards or penalties
given “on purpose” remains to be demonstrated. The only randomized
trial investigating the impact of financial incentives on drug
prescription in the hospital has yielded negative results [11].
Overall, little is known of methods or factors capable of
influencing medical practice. They usually vary with local
conditions that are not easy to evaluate or modify. The studies
reported above seem to suggest that some approaches are effective
in certain situations. However, these local situations and their
impact on medical practice remain to be elucidated. Few studies
have investigated this aspect to date, and it seems particularly
difficult when one considers the number of factors possibly
involved. Whatever are the factors for medical practices, the
impact of these is essentially variations of medical practice not
always explained by the disease itself or the characteristics of
the patients.
Variations in medical practice
In the Western world, changes in medical practice have been
extensively described [12]. These variations, well documented by
English-speaking authors, have changed the behavior of caregivers
regarding the use of medical services and devices. Studies of
thousands of patients have shown that other parameters than disease
itself (race, socioeconomic status, or hospital resources, for
instance) could influence medical practice, and that the
utilization of medical services tended to vary substantially among
areas of a given country [13]. The first international reports on
geographic variations in medical practice date back to the late
sixties when several studies assessed the rate of recourse to
tonsillectomy or adenoidectomy, gallbladder removal, or different
types of gynecologic surgery. Results, obtained principally in the
United States, Great Britain, Canada and Sweden, indicated that the
utilization of medical care varied significantly, by a factor of 2
or more [14]. Since then, many further studies have confirmed
initial results and attempted to provide convincing explanations.
Variations in medical practice could not be attributed to increased
patient morbidity, but rather to variations in the organization and
payment of care and the greater availability of hospital resources,
particularly to the ratio of surgeons to the population. Yet, it is
not possible to determine for sure whether people within developed
countries receive “more medical care” or “better quality of care”.
Inter-country differences may be due to differences in political
orientations or differences in the concept of public health itself.
However, this hypothesis does not account for inter-area variations
within a given healthcare system. As previously reported by
Wennberg and Gittlesohn [15], the availability of surgical
treatment varies by a factor of 2 in some areas; this might be
consecutive to the presence of a given physician or medical team
who promote specific treatment strategies, whereas these strategies
come to an end when the physician or the medical facility leave the
area. Other studies investigating hospitalization causes across
different states suggested that variations depend on the existence
or not of a medical consensus. In the absence of a consensus,
physicians are uncertain as to what strategy to adopt, allowing for
a wider margin of appreciation. It has been shown that variations
in medical practice are much more pronounced for more recent
techniques. However, new hypotheses should be explored, such as
inter-region variations consecutive to the under-utilization of
validated hospital-based guidelines in some regions. Results
reported by Wennberg et al. in 1989 showed that, in the case of
low-variation medical conditions (such as the breaking of a leg,
gastro-intestinal bleeding or myocardial infarction), there is a
wide consensus about the need for hospitalization, whereas
diagnostic and treatment strategies are much more variable [16]. It
was concluded that medical practices vary with the level of
consensus or dissent between physicians. This is not surprising,
and the literature on the subject clearly shows that several
clinical situations generate no or little variations.
Variations of medical practice with geographical location have
been studied, in particular, in the domain of oncology [17], though
no clear causal connection could be evidenced. The cancer
registries of nine French districts were analyzed in search of
indications on the subject. Medical practices were found
heterogeneous regarding initial treatment for patients with breast
cancer, but the survey failed to identify explanatory factors
[18].
These variations of medical practice impact quality of care,
which is of concern not only to patients, but also to healthcare
managers and to the scientific community. Medical evaluation aims
at improving quality of care. It is thus crucial to analyze the
potential impact of practice variations. Measuring these variations
and identifying causative factors is a prerequisite for appropriate
medical evaluation. They will make it possible to focus scientific
knowledge and improve its circulation so that it becomes accessible
to physicians. Medical evaluation should provide a rational basis
for novel approaches in medical practice. The general aim of this
process is not to implement a uniform medical model that would
neglect the intrinsic diversity of patients and doctors, but to
develop better evidence-based information systems that would
guarantee, for all patients, an equal level of risk and equal
chance of cure.
Evaluation of medical practices
The World Health Organization (WHO) has defined evaluation of care
as “a process which makes it possible to guarantee each patient the
range of diagnostic and therapeutic acts whereby he can achieve the
best possible results in terms of health, in accordance with the
current state of medical science, at the most cost-effective price
for an equivalent result, with the least iatrogenic risk and with a
view to the greatest satisfaction in terms of procedures, outcome
and human contacts within the health system”. Beyond this ambitious
program, that sets goals but does not specify the means of
achieving compliance, evaluation of care is based on the
measurement of quality indicators in accordance with appropriate
criteria.
There are several reasons for evaluating quality of care:
variations of practice, especially for the most invasive diagnostic
or therapeutic techniques; continuous advances in medical
knowledge, which are difficult to keep pace with because of their
number and, above all, the uncertain scientific value of some
publications; rapidly progressing but expensive medical
technologies leading to increasingly complex diagnostic or
treatment procedures which can hardly be paid for due to the
development of cost-containment measures.
In agreement with the law of April 1991 on hospital reform,
French hospitals, either public or private, are under the
obligation to develop quality of care evaluation. In that context,
they are expected to promote evaluation standards for medical
practice, as well as specific procedures to guarantee the quality
and efficiency of the organization of care or of any activity
contributing to patient management. They must also implement
specific information systems adapted to the diseases and the
details of patient management; these tools will contribute both to
improve medical knowledge and to provide information on hospital
activity and costs in order to adjust the possibilities of care to
the needs of the patients.
Which step of the care process should be evaluated? In 1966,
Donabedian suggested that three different stages should be targeted
[19]: hospital structures, corresponding to resources available to
the physician under evaluation (skills and experience, facilities);
medical practice, corresponding to what is done to or for the
patient; health indicators, corresponding to the subsequent
evolution of patient health. High-quality care should translate
into good clinical outcome and improved health indicators
(mortality, morbidity). Yet, since patients are numerous and
diverse, most indicators can neither be interpreted correctly nor
totally relied on. The problems lie in identifying the most
important prognostic factors and in collecting data routinely so
that mortality results can be interpretable [20]. Once the
difference in mortality has been established, the variations of
medical practice, if any, likely to be responsible for this
situation remain to be identified. Measures of practice seem to be
more valuable indicators than outcome. Indeed, the process of care
can be measured more easily, with good reproducibility, and the
interpretation of data is easier. Besides, process measurements
identify specific shortcomings, thus permitting to settle the
problems.
Methodology in six points
It should be noted that, in English-speaking countries, the phrase
“medical audit” (or “medical evaluation”) encompasses all types of
evaluation in use by healthcare professionals: peer review, data
collection and analysis, auditing, guidelines and recommendations.
Clinical auditing is a comparison, based on standardized validated
criteria, of actual and reference care processes [21]. It measures
quality of care and outcomes in view of improving clinical
performance.
A review of the literature discriminated six different steps,
with different specificities, depending on the subject and the
domain of application.
Choice of the question and initiation of the study (ex: impact
of guidelines on medical practice)
Various care processes, from diagnosis to treatment or prevention,
can be evaluated. The question may focus on a particular clinical
situation, a given treatment protocol, any key element contributing
to quality of care (such as patient records), or a complete
clinical management process.
Selection of the criteria (those from the guidelines)
This step identifies the criteria judged to be representative of
quality of care. These criteria, further used as the reference
basis for comparing actual practice to optimal clinical service,
must provide valid, quantifiable and reliable data, with good
sensitivity and specificity. Several types of criteria will be
examined: validation (such as criteria governing inclusion in
treatment protocols), conformity, treatment complications, results,
patient satisfaction, as well as alignment with economic standards
and appropriate utilization of resources.
Selection of the type of study
The methodology is descriptive, based on prospective or
retrospective data collection. Advantages and drawbacks of each
method are reported in the table 1( Table
1 ).
Table 1 Advantages and drawback of each method
|
Retrospective/patient records
|
Prospective/observation
|
|
Advantages
|
- A posteriori evaluation
- Indirect collection of data
- Easy access to data
- Global overview
- Inexpensive
|
- Observation of facts, no interpretation
- Perception of details
- Immediate effect
|
|
Drawbacks
|
- Care noted but not given
- Missing data
- Missing records
|
- Long and expensive
- Difficult
- Observer/observed interaction bias
- “Unnatural” behavior of observed
- Observer is influenced
|
Collection of data
Data can be obtained from multiple sources. For retrospective
analyses, several sources are available: patient medical records,
nursing records, disease registries, activity reports of hospitals,
patient records at discharge, volume of drug prescriptions. The
number of records or observations generally considered to be
acceptable is 50 (descriptive methods do not involve complex
statistical analyses). Elaborating an accurate datasheet is a
prerequisite for guaranteeing the quality of a clinical audit, and
is critical to the credibility and acceptability of the results.
Periodical quality control of the datasheets, with assessment of
data integrity and documentation of missing data and exceptions, is
also required. It is essential that all healthcare professionals
and hospital staff involved in the study have a clear view of the
principal objective, namely to improve the quality of patient care.
Analysis of results
After the data collection is complete, the first analysis includes
the following points: respective ratio of complete, incomplete,
missing or invalid data; overall analysis of results, evaluation of
criterion conformity with predetermined standards. Then major
deviations are identified and analyzed. Deviations between
observations and pre-established standards may stem from three
different causes:
- – Professional causes: either heterogeneous,
incompatible or unsuitable care processes. Sometimes caregivers may
also not have received appropriate training to use them
correctly.
- – Organizational causes: these may result from
inappropriate coordination between the people involved, generally
due to inefficient communication (either in written or oral
form).
- – Structural causes: inadequate resources or
inappropriate use of resources.
Elaboration of recommendations
Recommendations should be graded according to two factors: ease of
use and quick implementation. This is an integral part of clinical
audits addressing quality of care. When evaluation has revealed
significant deviations from standards, a second audit is necessary.
Follow-up is thus an essential step of the “evaluation loop”. It
allows dynamic evolution of the system and promotes the improvement
of quality of care.
Success and limitations
The success of any clinical audit depends on a number of
inescapable conditions: Firstly, verify that the question raised
relates to an actually existing problem, with strong prevalence and
high costs. The primary objective must be quality. Secondly, make
sure to involve a group of committed professionals with training in
the evaluation of medical practice, and to work on accurate,
validated criteria established from appropriate reference sources
(scientific or medical literature, relevant references, reports
from experts). Finally, present the result in such a way that it
actually leads to improving medical practices.
One of the major limitations of this evaluation is the selection
of the themes explored. Most studies focus more on scientific and
technological aspects of care than on fair distribution of care or
psycho-social questions. The second limitation is methodology.
Probably due to cost reasons, most studies are retrospective, and
more often descriptive than analytical. The third major limitation
is the efficacy indicator chosen. Davis et al. have defined four
different efficacy levels: change perceived by physicians,
improvement of knowledge, modification of healthcare processes,
improvement of patient outcomes [22]. For example, Lomas in 1989,
showed us contrasted attitudes toward the use of cesarean section.
One third of the hospitals and obstetricians reported changing
their practice as a consequence of the guidelines, and
obstetricians reported rates of cesarean section falling from 72.2
to 61.1 % (P less than 0.01). The surveys also showed,
however, that knowledge of the content of the recommendations was
poor (67 % correct responses). Furthermore, data on actual
practice after the publication of the guidelines showed that the
rates of cesarean section were 15 to 49 % higher than the
rates reported by obstetricians [23].
We have seen that evaluation is “the process of determining the
value, the significance or the worth of something”. However, the
absence of clear objectives makes evaluation difficult or even
impossible. When it comes to medical auditing, objectives are
usually not clearly defined and sometimes rather controversial.
Improving the quality of patient care provided by physicians seems
simple enough, but becomes irrelevant when no common definition of
this quality is available. Most studies published to date have
focused on medical practice evaluation or quality insurance
programs, though with controversial validity and questionable
objectivity. Consequences, however, seem of little significance; in
the United-States, for instance, a country with a long history of
quality assurance services, only 20 % of physicians surveyed
are aware of changes in their practice that would be directly
attributable to medical auditing [24].
Evaluation of modifications of medical practice following
clinical audits
The impact of a clinical audit can be assessed using classical
methods of epidemiological studies measuring the potential impact
of medical interventions. A comparison is made between before and
after, or here and elsewhere data; studies are conducted on
chronological series or, more rarely, on randomized populations.
Before/after studies are frequent but, when not controlled, they
can produce erroneous indications on the impact of an intervention.
Many external factors (pharmaceutical firms, scientific literature,
changes in healthcare programs or patient management strategies,
seasonal variations, etc.) may influence the behavior of
caregivers. Variations observed may well be wrongly attributed to
the intervention whereas in reality they only indicate a more
general trend. Controlled before/after studies compare the behavior
of physicians before and after an intervention, and analyze
deviations from a control group (not randomized) not benefiting
from the intervention. Control and study populations must naturally
display identical characteristics, and the collection of data must
be performed simultaneously in both groups.
Chronological studies are based on several reference data (10 or
more) collected before and after the intervention, thus allowing to
determine whether observed changes are directly related to the
intervention or only mirror underlying general trends. Shermann,
for instance, has been able to demonstrate that the consensus
conference on prostate cancer treatment has had no real impact upon
medical practice [8].
Randomized trials may be more relevant for studying the impact
of an intervention than for evaluating the benefit of treatment
advances. However, there are major differences between the two.
Compared to treatment trials, the results of medical audits are
more dependent on the structural organization of the group under
study. They cannot be easily extrapolated to other groups with
different organizational structures. It may be worthwhile to
conduct double-blind treatment trials; yet, this cannot be the rule
when it comes to auditing medical practice since the physician (or
the hospital), unlike the patient, cannot be kept uninformed of the
intervention for which he is tested. Adverse events and treatment
complications are frequently recorded in the frame of treatment
trials. Conversely, medical audits rarely take into account the
side effects of the intervention (increased recourse to CT-scan
examinations when standard radiography tests are discouraged, for
instance). Finally, if patients participating in treatment trials
are selected at random, physicians can hardly be randomized to
follow a recommendation for treating some patients and not others.
In that case, and apart from very specific situations, the
randomization unit should not be the patient, but the physician,
the clinical unit or the hospital. This not only requires enrolling
a sufficient number of patients in the study, but also involving a
representative sample of physicians or hospitals. The experimental
design is identical in both cases: parallel-group randomized
trials, or complex randomized trials involving the use of control
groups to minimize the Hawthorne effect.
Auditing is thus of major interest to evaluate the impact of the
guidelines, to assess their clinical soundness, to judge the impact
of the strategy chosen for disseminating the recommendations, and
also to organize the dynamic loop of quality control leading to
elaborating new recommendations. In 1993, Grimshaw and Russell
identified methods that met criteria for scientific rigor; these
included randomized studies, but also more experimental methods
such as chronological series and studies with parallel control
groups [25].
Example of medical practice in oncology
In 2002, a search of Medline using the terms “medical audit” and
“neoplasm” yielded over 550 references. Medical audits have been an
integral part of the “cancer culture” ever since the disease has
become a public health challenge. Medical evaluation is useful to
assess the impact of recommendations on clinical practice; as such,
it has its place in many domains of oncology: epidemiology,
clinical research, implementation of new treatment advances,
management of care.
Epidemiology
Medical evaluation can help assess the maintenance of other
clinical tools (registries, medical education, functioning of
networks). Some authors, for instance, publish regular “activity
updates” [26]. Chronological series have also provided helpful
information on the training of surgery interns [27]. Moreover,
medical auditing can be used for identifying factors permitting to
improve these tools in the context of quality management.
Clinical research
Clinical trials proposed by major international collaborative
groups (EORTC, NCI, CALGB, etc.) are regularly audited, both to
preserve the integrity of clinical research and to prevent
scientific misconduct. Auditing guarantees optimal analysis
performance for data of patients enrolled in treatment trials.
Continuous auditing has also made it possible to elaborate
recommendations for judging scientific research based not on
medical outcomes but on methodological aspects. Shapiro and Charrow
have shown that scientific soundness should be questioned when 7%
major deviation from standard practice guidelines is reported [28].
Implementation of new treatment advances
As is often the case in medicine, new technological advances,
either for patient diagnosis or treatment, tend to be used in
addition to, rather than instead of, prior techniques. Several
audits investigating the place and role of these advances in cancer
treatment have been published. Whereas standard health indicators
are rarely used in medicine, they are often assessed in the
oncological setting. This is probably due to the specific nature of
the disease (major risk of death) and to obvious methodological
reasons (death is a relatively frequent event and can therefore be
documented). Several teams have investigated, for instance, the
relationship between patient mortality and medical practice
[29-32].
Healthcare management
In the domain of healthcare management, activity snapshots can
capture the activities of a clinical unit and check the adequacy of
observed results with allocated resources. In oncology, for
instance, medical audits can verify the quality of patient medical
records, or evaluate the clinical management of patients with lung
cancer as a function of facilities available locally, etc. They may
also provide evidence in support of a specific type of organization
or advocate the necessity to develop specific management strategies
(psycho-oncology unit, surgical quality control, network
organization, etc.) As an example, the hospital reform plan of 1996
has recommended the creation of healthcare networks. Physicians and
healthcare structures caring for cancer patients, who are
particularly aware that only this multidisciplinary approach can be
successful in treating this chronic and progressive disease, have
made efforts to develop alternatives to inpatient hospitalization.
Forty hospitals or structures of the Rhone-Alpes region have agreed
to collaborate in a regional cancer network (ONCORA) organized by
the comprehensive cancer center of Lyon, Leon Berard center,
serving as the reference hospital. All caregivers participating in
ONCORA have signed collaborative agreements with the center. Their
major objective is to improve the management of patients with
cancer. To this aim, they have agreed to elaborate common standards
for the diagnosis and treatment of cancer patients, to exchange
data both on patients and the activity of the different units or
structures involved, and to elaborate advanced consultations under
the guidance of the comprehensive cancer center. This collaboration
was confirmed in 1992 by the signing of a charter on priority
objectives such as the implementation of medical practice
guidelines [33]. This “thesaurus” was distributed to all physicians
of Leon Berard cancer center in 1994, and to oncologists
participating in the network in 1995. The whole process has been
subjected to an audit to evaluate the impact of the recommendations
on clinical practice (see next paragraph).
Impact of recommendations
The success of medical practice guidelines depends on many factors:
the clinical context [34], the methods used for the development
(in-house or outsource), distribution and implementation of these
guidelines, the support of opinion leaders [35], or the use of
computer facilities [36].
Probably because cancer is a very specific condition, with
uncertain prognosis and elevated costs, the medical community
regularly publishes, or simply “announces”, distributes and
implements, clinical recommendations. This can be illustrated by an
English experience published by Junor et al. in 1994, demonstrating
that multidisciplinary management improved the survival of patients
with ovarian cancer [29] . Recommendations were then published and
distributed to the medical community. Compliance was tested and
results were published in 1999 [37].
A literature review was performed to identify the changes in
medical practice (for both diagnosis and treatment) consecutive to
the implementation of these recommendations. Most of the clinical
studies analyzed were high-quality controlled randomized trials.
However, the efficacy of recommendation guidelines on patient
management is controversial, since only minor modifications have
been found (between 0 and 37 %) [25]. Our before/after study,
published in 1997, tested the implementation of the Thesaurus and
its use as of 1996 [38]. In 1993, only 32 % of treatment
decisions were consistent with the Thesaurus for the treatment of
both breast and colon cancer, versus 62 % in 1995. Deviations
between recommendations and practice principally concerned the
follow-up management of breast cancer patients, the treatment of
elderly patients and the choice of chemotherapy protocols. The
increasing correlation between medical decisions and
recommendations of the Thesaurus was tested in a multivariate
analysis [39], with the highest correlation found for patients with
disseminated disease (metastatic) and good general health
(performance status 0 or 1). This particular study also emphasized
the problem of elderly cancer patients who cannot fully benefit
from standard treatments. Other factors inherent to this particular
population (physiological age) and to its social environment (home
care…) must also be taken into account. Social environmental
factors were previously known to influence medical decision-making
and physicians’ compliance with recommendations. In this study, we
have shown that patients’ characteristics should also be considered
when judging the compliance of patient management to
recommendations and, even before that, when elaborating the
recommendations themselves. We have shown that the rate of medical
decisions in agreement with current scientific knowledge increased
from 71 to 81 % between the two periods of the study [39].
These results suggest that recommendations for clinical practice
and related educational programs (expected to stimulate the
implementation of guidelines) can efficiently change the behavior
of physicians and align medical decision with scientific knowledge.
Finally, another study from our group also permitted to show that
the implementation of guidelines has positive impact on health
expenses [40], and non-compliance with the guidelines leads to
additional costs. The potential impact of the strategy used for
developing and implementing the guidelines was tested among the
physicians involved (oncologists or organ specialists) in Oncora
cancer network [41]. Using the audit methodology, we evaluated : 1)
the behavioral changes since implementing the guidelines in the
network, 2) the variations between hospitals, and we proposed
explanatory hypotheses to these variations. A concurrent control
group was used to overcome the bias due to potential behavioral
modifications occurring in the absence of any intervention. We have
been able to show that distributing and implementing the Thesaurus
induced modifications of medical practice in the network (19 to 40%
conformity rate for breast cancer), whereas no significant
modification was noted in the control group during the same period
(6 to 7 % of conformity for breast cancer). Our study thus
confirmed the positive impact of medical practice guidelines on the
behavior of physicians participating in an oncology network.
However, it was well demonstrated that the stability of this level
of compliance could not be maintained over time, specially after
the intensive implementation strategy had stopped [42-45]. In a new
study, we evaluated the persistence of conformity to guidelines
through a new medical audit [46]. In 1999, the hospitals of the
previously compared experimental and control groups accepted to
re-assess the impact of CPG. In the experimental group, the
compliance of medical decisions with CPG was significantly higher
in 1999 than in 1996 for colon cancer: 73 versus 56 % (p =
0.003), and similar for the two periods for breast cancer : 36
versus 40 % (p = 0.24). In the control group, compliance was
identical for the two periods for breast cancer: 4 versus 7 %
(p = 0.19). The CPG program for cancer management seems to produce
persistent changes in medical practice in our cancer network in
terms of conformity with CPG.
Conclusion
Our purpose here is not to evaluate auditing tools, but to
determine the particular interventions and circumstances that would
efficiently help modulate medical practice. The oncology setting
and, more precisely, the experience of the ONCORA network, provide
the opportunity to elaborate clinical guidelines and promote their
use for modifying and rationalizing medical practice. Yet, if
reference guidelines can help modify practices, their elaboration
and dissemination must follow several inescapable rules. First, the
guidelines must be developed by an expert consensus panel, based on
evidence derived from a critical review of the literature. After
validation by physicians involved, they are disseminated widely
through publications as well as across patient records. Each
guideline must be elaborated and developed on a local basis.
However, several potential limitations must be taken into account
[47, 48] such as the lengthy development cycle of clinical
guidelines [49] and the possible development of new drugs [50] or
new economic policies. External review and feedback from
practitioners is also needed. However, the major drawback lies with
the tool itself, namely the validity of the guidelines. What is
meant by new available evidence? On what time basis should
guidelines be updated? What is their expected time to obsolescence,
knowing that half of them become outdated after 5.8 years? [51].
The exact role of networks for the development of medical
practice guidelines is not known. Only the Rhone-Alpes experience
has been published to date. But results from this study do not
discriminate the impact of network management from the impact of
guidelines. Regarding healthcare networks, whereas economists
promote competition, financial incentives and per-patient funding,
healthcare professionals favor the expansion of information and
self-evaluation systems. In this context, if reference guidelines
are based on unvalidated personal opinion rather than on cogent
scientific evidence, limiting medical evaluation to the appraisal
of deviations from medical guidelines would be hazardous.
The goal of medical research is no longer to propose innovative,
intellectually challenging projects based on disparate data models,
but to build functional systems for given types of populations
(elderly patients, chronic diseases like cancer), and to develop
regional healthcare programs with precise, accurate and sensitive
performance indicators. This can only be achieved by implementing
data collection systems using computer-based applications.
Acknowledgments
To Marie Dominique Reynaud for editing assistance, Josiane Popescu
for Medline search and Florence Germaine for assistance in
collecting the data. This work was partly supported by an PHRC
1997, ANAES 2001, Ligue contre le cancer (Rhône) 2002 grant from
the Ministry of Health (France).
Références
1 Eisenberg JM, Williams SV. Cost containment and
changing physicians’ practice behavior. Can the fox learn to guard
the chicken coop? JAMA 1981; 246: 2195-201.
2 Dolan JG, Bordley DR. Using the analytic hierarchy
process (AHP) to develop and disseminate guidelines. QRB Qual Rev
Bull 1992; 18: 440-7.
3 Durieux P, Ravaud P, Chaix C,
Durand-Zaleski I. Does continuing medical education improve
the way physicians conduct their practice? Presse Med 1999; 28:
468-72.
4 Davis DA, Thomson MA, Oxman AD, Haynes RB.
Changing physician performance. A systematic review of the effect
of continuing medical education strategies. JAMA 1995; 274:
700-5.
5 Thomson O’Brien MA, Oxman AD, Davis DA,
Haynes RB, Freemantle N, Harvey E. Educational
outreach visits: effects on professional practice and health care
outcomes (cochrane review). The Cochrane library 1999; 2: 1-16.
6 Lomas J. Words without action? The production,
dissemination, and impact of consensus recommendations.
AnnuRevPublic Health 1991; 12: 41-65; (41-65).
7 Grilli R, Alexanian AA, Apolone G,
Confalonieri C, Fossati R, Liati P, et al.
Trends in patterns of care for breast cancer in Italy (1979-1987).
Tumori 1990; 76: 184-9.
8 Sherman CR, Potosky AL, Weis KA,
Ferguson JH. The Consensus Development Program. Detecting
changes in medical practice following a consensus conference on the
treatment of prostate cancer. Int J Technol Assess Health Care
1992; 8: 683-93.
9 Gabbay J, McNicol MC, Spiby J, Davies SC,
Layton AJ. What did audit achieve? Lessons from preliminary
evaluation of a year’s medical audit. BMJ 1990; 301: 526-9.
10 Eisenberg JM, Kabcenell A. Organized practice and
the quality of medical care. Inq 1988; 25: 78-89.
11 Martin AR, Wolf MA, Thibodeau LA, Dzau V,
Braunwald E. A trial of two strategies to modify the
test-ordering behavior of medical residents. N Engl J Med 1980;
303: 1330-6.
12 Chassin MR, Brook RH, Park RE, Keesey J,
Fink A, Kosecoff J, et al. Variations in the use of
medical and surgical services by the Medicare population. N Engl J
Med 1986; 314: 285-90.
13 Gatsonis C, Normand SL, Liu C, Morris C.
Geographic variation of procedure utilization. A hierarchical model
approach. Med Care 1993; 31: YS54-YS59.
14 Vayda E. A comparison of surgical rates in Canada and in
England and Wales. N Engl J Med 1973; 289: 1224-9.
15 Wennberg J, Gittelsohn A. Variations in medical
care among small areas. Sci Am 1982; 246: 120-34.
16 Wennberg JE, Freeman JL, Shelton RM,
Bubolz TA. Hospital use and mortality among Medicare
beneficiaries in Boston and New Haven. N Engl J Med 1989; 321:
1168-73.
17 Grilli R, Scorpiglione N, Nicolucci A,
Mainini F, Penna A, Mari E, et al. Variation in
use of breast surgery and characteristics of hospitals’ surgical
staff. Int J Qual Health Care 1994; 6: 233-8.
18 Grosclaude P, Menegoz F, Chaplain G.
Evaluation du traitement du cancer en France. Enquête sur les
pratiques dans neuf départements français en 1990. Rev Epidem Sant
Publ 1995; 43(S): 7.
19 Donabedian A. Evaluating the quality of medical care.
Milbank Mem Fund Q 1966; 44(Suppl-206).
20 Davies HT, Crombie IK. Assessing the quality of
care. BMJ 1995; 311: 766.
21 Agence Nationale d’Accreditation et d’Evaluation en Santé.
In: L’audit clinique: bases méthodologiques de l’evaluation des
pratiques professionelles, vol. 1, 1 ed. Paris: ANAES, 1999:
7-30.
22 Davis DA, Thomson MA, Oxman AD,
Haynes RB. Evidence for the effectiveness of CME. A review of
50 randomized controlled trials. JAMA 1992; 268: 1111-7.
23 Lomas J, Anderson GM, Domnick-Pierre K,
Vayda E, Enkin MW, Hannah WJ. Do practice guidelines
guide practice? The effect of a consensus statement on the practice
of physicians. N Engl J Med 1989; 321: 1306-11.
24 Casanova JE. Status of quality assurance programs in
American hospitals. Med Care 1990; 28: 1104-9.
25 Grimshaw JM, Russell IT. Effect of clinical
guidelines on medical practice: a systematic review of rigorous
evaluations [see comments. Lancet 1993; 342: 1317-22.
26 Griffiths RW. Audit of histologically incompletely
excised basal cell carcinomas: recommendations for management by
re-excision. Br J Plast Surg 1999; 52: 24-8.
27 Aitken RJ, Thompson MR, Smith JA,
Radcliffe AG, Stamatakis JD, Steele RJ. Training in
large bowel cancer surgery: observations from three prospective
regional United Kingdom audits. BMJ 1999; 318: 702-3; [see
comments].
28 Shapiro MF, Charrow RP. The role of data audits in
detecting scientific misconduct. Results of the FDA program. JAMA
1989; 261: 2505-11.
29 Junor EJ, Hole DJ, Gillis CR. Management of
ovarian cancer: referral to a multidisciplinary team matters. Br J
Cancer 1994; 70: 363-70.
30 Basnett I, Gill M, Tobias JS. Variations in
breast cancer management between a teaching and a non- teaching
district. Eur J Cancer 1992; 28A: 1945-50.
31 Wolfe CD, Tilling K, Bourne HM, Raju KS.
Variations in the screening history and appropriateness of
management of cervical cancer in South East England. Eur J Cancer
1996; 32A: 1198-204.
32 Gillis CR, Hole DJ. Survival outcome of care by
specialist surgeons in breast cancer: a study of 3786 patients in
the west of Scotland. BMJ 1996; 312: 145-8; [see comments].
33 Centre Léon Bérard and Réseau ONCORA. Thesaurus ONCORA en
cancérologie. Paris: Arnette Blackwell, 1997; (1).
34 Grilli R, Lomas J. Evaluating the message: the
relationship between compliance rate and the subject of a practice
guideline. Med Care 1994; 32: 202-13.
35 Lomas J, Enkin M, Anderson GM, Hannah WJ,
Vayda E, Singer J. Opinion leaders vs audit and feedback
to implement practice guidelines. Delivery after previous cesarean
section. JAMA 1991; 265: 2202-7.
36 Davidoff F, Haynes B, Sackett D, Smith R.
Evidence based medicine. BMJ 1995; 310: 1085-6.
37 Sengupta PS, Jayson GC, Slade RJ,
Eardley A, Radford JA. An audit of primary surgical
treatment for women with ovarian cancer referred to a cancer
centre. Br J Cancer 1999; 80: 444-7.
38 Ray-Coquard I, Philip T, Lehmann M,
Fervers B, Farsi F, Chauvin F. Impact of a clinical
guidelines program for breast and colon cancer in a French cancer
center. JAMA 1997; 278: 1591-5.
39 Ray-Coquard I, Philip T, Négrier S,
Desseigne F, Rivoire M, Fervers B, et al. Is
colon cancer management in a comprehensive cancer centre
evidence-based? Colorectal Dis 1999; 1: 214-21.
40 Mille D, Roy T, Carrere MO, Ray I,
Ferdjaoui N, Spath HM, et al. Economic impact of
harmonizing medical practices: compliance with clinical practice
guidelines in the follow-up of breast cancer in a French
Comprehensive Cancer Center. J Clin Oncol 2000; 18: 1718-24.
41 Ray-Coquard I, Philip T, de Laroche G,
Froger X, Suchaud JP, Voloch A, et al. A
controlled "before-after" study: impact of a clinical guidelines
programme and regional cancer network organization on medical
practice. Br J Cancer 2002; 86: 313-21.
42 Murray MD, Harris LE, Overhage JM,
Zhou XH, Eckert GJ, Smith FE, et al. Failure of
computerized treatment suggestions to improve health outcomes of
outpatients with uncomplicated hypertension: results of a
randomized controlled trial. Pharmacotherapy 2004; 24: 324-37.
43 Tierney WM, Miller ME, McDonald CJ. The effect
on test ordering of informing physicians of the charges for
outpatient diagnostic tests. N Engl J Med 1990; 322: 1499-504.
44 Tierney WM. Improving clinical decisions and outcomes
with information: a review. Int J Med Inform 2001; 62: 1-9.
45 Dexter PR, Wolinsky FD, Gramelspacher GP,
Eckert GJ, Tierney WM. Opportunities for advance
directives to influence acute medical care. J Clin Ethics 2003; 14:
173-82.
46 Ray-Coquard I, Philip T, de Laroche G,
Froger X, Suchaud JP, Voloch A, et al.
Persistence of medical change at implementation of clinical
guidelines on medical practice: a controlled study in a cancer
network. J Clin Oncol 2005; 23: 4414-23.
47 Woolf SH, Grol R, Hutchinson A, Eccles M,
Grimshaw J. Clinical guidelines: potential benefits,
limitations, and harms of clinical guidelines. BMJ 1999; 318:
527-30.
48 Grilli R, Magrini N, Penna A, Mura G,
Liberati A. Practice guidelines developed by specialty
societies: the need for a critical appraisal. Lancet 2000; 355:
103-6.
49 Browman GP, Newman TE, Mohide EA,
Graham ID, Levine MN, Pritchard KI, et al.
Progress of clinical oncology guidelines development using the
Practice Guidelines Development Cycle: the role of practitioner
feedback. J Clin Oncol 1998; 16: 1226-31.
50 Pater JL, Browman GP, Brouwers MC,
Nefsky MF, Evans WK, Cowan DH. Funding new cancer
drugs in Ontario: closing the loop in the practice guidelines
development cycle. J Clin Oncol 2001; 19: 3392-6.
51 Shekelle PG, Woolf SH, Eccles M,
Grimshaw J. Clinical guidelines: developing guidelines. BMJ
1999; 318: 593-6.
|
Printable version
Version PDF