A Quality Indicator Can Be Biased by Intra-hospital Heterogeneity: the Case for Quality of Patient Record Keeping in France

Background: Since 2008, French health institutions providing medical, surgical and obstetrical care are assessed on the basis of a set of quality indicators. The French National Authority for Health developed a survey design in which 80 records are randomly selected from each institution. The main aim was to assess the effects of internal heterogeneity of a hospital that comprises several units. The survey method is based on the hypothesis of intra-institution homogeneity, which overlooks the fact that in wide hospitals homogeneity is related to departments and thus leads to overall intra-hospital heterogeneity.

Methods: Simulated databases were created to modelise the heterogeneity of our hospital and computed to assess the reliance of indicator measurement. We used real data from a large teaching hospital having internal heterogeneity related to each department.

Results: Variance under heterogeneity was greater than under homogeneity (3- to 18-fold) leading to an increased size of the confidence interval (CI) (at 95%) from 9 (given Haute Autorité de Santé sources) to 22 (for greatest internal heterogeneity).

Conclusions: The variations in a quality indicator can be explained by intra-institution heterogeneity and are not related to changes in the quality policy of the hospitals and may lead to errors in terms of pay for performance.