Considering News Background Can Improve GDP Projections in Periods of Instability
The accuracy of Russian GDP forecasts during periods of instability improves in 45% of cases when news reports are taken into account. However, during more stable periods, this advantage nearly disappears. News provides an up-to-date view of the economy and enables quicker responses to emerging challenges. This was revealed by an analysis of over 500,000 news reports conducted by Ivan Stankevich and Natalia Makeeva of the HSE Faculty of Economic Sciences (FES), and Nikita Lyubaykin. The study results have been published in Voprosy Ekonomiki.
Statistical agencies do not publish economic statistics in real time. For example, quarterly GDP data is published with a two-month delay. However, more up-to-date data is essential for understanding the current economic situation, especially considering the impact of the coronavirus pandemic and sanctions pressure. Therefore, experts attempt to forecast economic indicators using current data, such as daily bank card transactions or export and import volumes. However, these estimates may not always be very accurate. Assistant Professor at HSE FES Ivan Stankevich, lecturer Natalia Makeeva, and researcher Nikita Lyubaykin have examined how incorporating news reports can influence the accuracy of Russian GDP forecasts.
The authors analysed over 500,000 news reports from Telegram channels of major media and news outlets with audiences exceeding 100,000 people, spanning from 2014 to 2023. 'We used a diverse range of channels to avoid any bias from uneven coverage of events,' said the HSE FES researchers. Using deep learning methods, each news item was classified into one of 19 categories (such as 'economics and business,' 'society,' or 'culture') and assessed for tonality ('positive,' 'negative,' or 'neutral'). The data was then incorporated into existing forecasting models to determine whether incorporating news reports improved the accuracy of the forecasts. The authors also divided the observations into two periods: before and after the introduction of large-scale economic sanctions in 2022. The mean absolute error (MAE) of the forecasts was used as the primary measure of accuracy, representing the deviation of the model’s results from the actual indicators over the entire observation period. A lower MAE indicates a better model.
It was found that incorporating news reports during periods of instability improves forecast accuracy in 45% of cases and reduces the MAE for GDP by 0.64 percentage points (p.p.). This is a significant result. Before incorporating news reports, the average error was around 2 p.p., while in the updated model, this error decreased to less than 1.3 p.p. Considering that GDP fell by 2.4 p.p. in 2022 and increased by 3.6 p.p. in 2023, the improvement is quite substantial.
Ivan Stankevich
'Incorporating the emotional tone of the news is an effort to formalise and quantify how experts assess the current state of the economy. Often, economic problems are apparent to specialists before showing up in "traditional" statistics, such as low unemployment, stable stock markets, and a steady exchange rate. Experts write about this, causing the overall news sentiment to worsen, and models are signalling a risk of recession,' according to Ivan Stankevich, Assistant Professor at the HSE Faculty of Economic Sciences.
However, this approach is effective only during crises and periods of instability. In the pre-sanctions period, taking into account the news background has minimal impact on forecast accuracy, affecting only 30% of models, and decreasing MAE by only 0.26 p.p. The authors also note that models incorporating news of all tonalities, rather than just negative ones, perform best.
While news can help quickly assess the state of the economy during unstable times, it is advisable to explore additional methods to enhance the accuracy of short-term economic forecasts.