Hospitals must ensure preparedness and safety to maintain services during disasters. Results from the Hospital Safety Index (HSI) often indicate that hospitals are safe and capable; however, it is important to verify whether these assessments are consistent across different evaluators. This study compared hospital safety scores between internal evaluators (hospital staff) and external evaluators.

This cross-sectional study was conducted in 2022 using documented hospital safety assessment data from hospitals supervised by Kermanshah University of Medical Sciences (KUMS) in western Iran. Independent-samples t-tests, the intraclass correlation coefficient (ICC), and Bland–Altman plots were used to compare mean scores and assess agreement between evaluators.

For structural safety, the difference between internal and external evaluators was not significant (95% CI, -9.19 to 9.67). In contrast, the differences were pronounced for non-structural safety (95% CI, 7.00 to 20.63) and functional safety (95% CI, 10.56 to 28.84), with both confidence intervals excluding zero. Agreement between evaluators was poor across most hospital safety domains (ICC < 0.05).

Internal evaluators rated their hospitals significantly safer than external evaluators did. Given the pronounced differences in non-structural and functional safety, and the poor inter-rater agreement across most domains, relying solely on internal assessments may mislead hospital managers about disaster preparedness. Hospital managers should therefore regard external evaluation as an opportunity to reveal their hospital’s vulnerabilities in disaster safety.

The Hospital Safety Index (HSI) developed by the World Health Organization (WHO) was adopted by most countries to evaluate the safety of hospitals against disasters. This study aimed to assess the status of hospital safety from disasters between 2016 and 2022 in Kermanshah province in Iran.

This is a retrospective longitudinal study which investigated HSI data from 23 hospitals. Data were gathered by Farsi Hospital Safety Index (FHSI) and analyzed with a repeated measures analysis of variance (ANOVA).

The risk of hydro-meteorological (from 43.1 to 32.7) and biological hazards (51.3 to 35.5) significantly decreased. Although structural safety remained constant (from 67.8 to 70.1), nonstructural (from 51.5 to 71.2), and functional (from 47.1 to 71.2) safety scores increased significantly over study period.

The findings revealed hospitals safety in Kermanshah province gradually improved. However, the health-care stakeholders should pay the necessary attention to improving the structural safety of hospitals.