Quantifying Health Policy Uncertainty in China Using Newspapers: Text Mining Study

Background: From the severe acute respiratory syndrome (SARS) outbreak in 2003 to the COVID-19 pandemic in 2019, a series of health measures and policies have been introduced from the central to the local level in China. However, no study has constructed an uncertainty index that can reflect the volatility, risk, and policy characteristics of the health environment.

Objective: We used text mining analysis on mainstream newspapers to quantify the volume of reports about health policy and the total number of news articles and to construct a series of indexes that could reflect the uncertainty of health policy in China.

Methods: Using the Wisenews database, 11 of the most influential newspapers in mainland China were selected to obtain the sample articles. The health policy uncertainty (HPU) index for each month from 2003 to 2022 was constructed by searching articles containing the specified keywords and calculating their frequency. Robustness tests were conducted through correlation analysis. The HPU index was plotted using STATA (version 16.0), and a comparative analysis of the China and US HPU indexes was then performed.

Results: We retrieved 6482 sample articles from 7.49 million news articles in 11 newspapers. The China HPU index was constructed, and the robustness test showed a correlation coefficient greater than 0.74, which indicates good robustness. Key health events can cause index fluctuations. At the beginning of COVID-19 (May 2020), the HPU index climbed to 502.0. In December 2022, China's HPU index reached its highest value of 613.8 after the release of the "New Ten Rules" pandemic prevention and control policy. There were significant differences in HPU index fluctuations between China and the United States during SARS and COVID-19, as well as during the Affordable Care Act period.

Conclusions: National health policy is a guide for health development, and uncertainty in health policy can affect not only the implementation of policy by managers but also the health-seeking behavior of the people. Here, we conclude that changes in critical health policies, major national or international events, and infectious diseases with widespread impact can create significant uncertainty in China's health policies. The uncertainty of health policies in China and the United States is quite different due to different political systems and news environments. What is the same is that COVID-19 has brought great policy volatility to both countries. To the best of our knowledge, our work is the first systematic text mining study of HPU in China.