Coronavirus disease 2019 (COVID-19, caused by the virus SARS-CoV-2) first broke out in Wuhan, China in December 2019. Reference Wrigley Kelly, Murray and McCarthy1 The number of confirmed cases has continued to increase, causing damage to the society and the economy worldwide. Researchers around the globe are still working on the vaccine; meanwhile prevention is the only measure to control the spread of COVID-19. Although there are various COVID-19 vaccines (which were rolled out world-wide from December 2020), nearly all of them were developed, manufactured, and applied in a very short time. Reference Scott McLachlan and Dube2 On the other hand, the spread of rumors and false information is also accelerating. Reference Cuan-Baltazar, Muñoz-Perez and Robledo-Vega3,Reference Bastani and Bahrami4 For example, some people in remote rural areas in China were seduced to buy COVID-19 vaccines at the price of several 100s of RMB, while in fact, the Chinese government had provided it to the public for free. Some people doubt the effectiveness and safety of the vaccine, and therefore hesitate to get vaccinated, as they question whether these vaccines have passed large-scale clinical trials. Since many vaccines take 10 to 15 years to reach the public, the World Health Organization (WHO) said it did not believe a credible vaccine would be available in less than 18 months, Reference Grenfell and Drew5 but the timeline for COVID-19 vaccine was very different.
With the proliferation of mobile devices and the development of high-speed internet, online video platforms have gained increasing popularity, making videos an ideal tactic for disseminating COVID-19 related information. Reference He, Liu and He6 Although YouTube (http://www.youtube.com), which is a popular video website worldwide, is inaccessible in the mainland of China for some reasons, many similar domestic video websites are very popular. Reference Li, Yan and Yang7 Among these websites, iQiyi (http://www.iqiyi.com) has high visibility, with 350 million registered users and 3 million daily active users. 8 A prominent feature of iQiYi is that it has the function of whole network video searching, which allows users to access 10 popular online video websites at the same time (iQiYi, Tencent, Sohu, Youku, Tudou, acfun, bilibili, ifeng, CCTV, and 1905). It is worth noting that all these platforms have both websites and smart-phone based Apps. Similar to YouTube, the uploaded videos have different sources, lack peer-review process, and are likely to be of variable quality. Reference Keelan, Pavri-Garcia and Tomlinson9
The use of video websites as a source of information on the COVID-19 vaccine in mainland China has never been evaluated. Thus, this study aimed at understanding the characteristics of the viral iQiYi COVID-19 vaccine videos and assessing their contents.
Methods
Search strategy
Using the whole network search function, iQiYi was systematically searched on July 1, 2021 for videos containing relevant information about the COVID-19 vaccine. Reference Keelan, Pavri-Garcia and Tomlinson9 The keywords used included “新型冠状病毒疫苗,” which means COVID-19 vaccine, and “新冠疫苗,” which is the abbreviation of “新型冠状病毒疫苗” according to the Chinese language.
The inclusion criteria were: (1) in Chinese language; (2) available on July 1, 2021; (3) related to the COVID-19 vaccine in content. The exclusion criteria were: (1) duplicate videos, in part or as a whole; (2) videos that were only related to COVID-19 but not the COVID-19 vaccine; (3) popular science videos, not COVID-19 vaccine related, such as those related to military, economics and politics.
We used iQiYi’s sorting option (popularity), which is 1 of the 3 available sorting options (relevance, upload date, popularity). The first 5 pages (20 videos/page × 5 pages = 100 videos) of each search result were filtered in consideration that users do not exceed the first 5 pages of a search result. Reference Morahan-Martin10
All the videos that met the inclusion criteria were downloaded and saved, and characteristics such as titles, channels, topics, sources, formats, length, upload date, number of upload subscribers, total number of views, number of likes, number of forwards, and number of comments were extracted and saved for backup. Since the uploader of the video is not necessarily the producer of the video, we rigorously classified the source of the videos by content. If a video contained more than 1 topic, then each topic was separately listed. Each video was independently accessed by 2 reviewers, Zhang and Chang, and all disagreements were resolved by consensus.
Classification and scoring of videos
For variables like length, number of days since upload, number of uploader subscribers, number of views, number of likes, number of forwards, and number of comments, videos were sorted and calculated, while for variables like channels, sources, topics and formats, videos were classified and counted.
The overall quality of the videos was evaluated with the 5-grade Global Quality Scale (GQS) standard. Reference Bernard, Langille and Hughes11 The GQS is a tool for evaluating internet resources, and according to the total score, a video graded as excellent and of good quality is considered to be of high quality, a video graded as being of moderate quality is considered of intermediate quality, and generally a video graded as having poor quality is regarded as being of low quality. Reference Rausch Osthoff, Niedermann and Braun12
Reliability and credibility of video content was assessed by modified Health on the Net Foundation Code of Conduct (HONCode). Reference Goobie, Guler and Johannson13 The HONCode is the code of conduct of the non-profit Health On the Net (HON) Foundation and is applicable to voluntary health/medical websites. It was launched in 1995 and has developed into the most common and trustworthy code for health/medical websites. Reference Boyer, Selby and Scherrer14 Evaluation involves assessing for authoritative, complementary, privacy, attribution, justifiability, transparency, financial disclosure, and advertising policy.
The quality and reliability of video content (i.e. integrity, comprehensibility, relevance, depth and accuracy of information provided) was evaluated according to DISCERN criteria, which were previously used to evaluate the quality of health information on YouTube. Reference Singh, Singh and Singh15 DISCERN assesses quality and reliability by grading 8 items (concerning aims, bias, relevance, etc.), with each item being scored on a scale of 1 to 5, where 1 is “poor” and 5 is “excellent” in terms of quality. The higher the score, the better the information. Reference Charnock, Shepperd and Needham16
Statistical analysis
Statistical analysis was performed using SPSS 22.0 (IBM Corporation, Armonk, NY). A descriptive analysis was then used to describe the basic characteristics of the COVID-19 vaccine information on iQiYi. Categorical variables were stated as number of videos and percentage (%), while numerical variables were reported as median with minimum and maximum values.
Results
Collected data
The search for the 2 terms yielded 1383000 videos in total (“新型冠状病毒疫苗”- 715000 videos, “新冠疫苗”- 668000 videos). The first 100 eligible videos in each category were recorded. Videos that were duplicated were also recorded once. In total, 138 videos met our inclusion criteria. After excluding 48 non-related videos, a total of 90 videos met the inclusion criteria and were analyzed. Figure 1 represents the search process.
Search process flow diagram.
Summary and characteristics of videos
A summary of the videos, including the length, number of days since the video was uploaded, number of uploader subscribers, video popularity, and engagement (number of views, number of likes, number of forwards, number of comments) were recorded (Table 1). Missing information about the videos due to privacy policies or other reasons was not included in the study. The 90 reviewed videos were uploaded between December 19, 2020 and June 25, 2021. The minimum length of the videos was 0:24 seconds and the maximum length of the videos was 15:01 minutes. The total number of uploader subscribers, views, likes, forwards and comments of all the videos was 7173246, 1165596, 14498, 1450, and 95 respectively.
Summary of included videos
The categories of channels were provided by iQiYi. Nearly 50% of the videos were from the health channel (44 videos, 48.89%), followed by the news channel (35 videos, 38.89%). There were 2 separate videos (2.22%) from the maternal and child channel and the military channel. Subsequently, the channels on Economics, origin, life, entertainment, and public welfare had a total of 5 videos, with only 1 on each channel. The channels with 2 videos were not mentioned (2.22%).
Among the 90 videos, 40 (44.44%) were from media organizations (the logo or QR code of the media organization was usually displayed at the end of the videos). Videos of medical professional individuals ranked second with 23 videos (25.56%), and 12 videos (13.33%) came from the television (the logo of the TV station was always displayed in 1 corner of the video). Similarly, private individuals/laypersons posted 12 videos (13.33%), while the source of the other videos could not be inferred from the content.
The topics covered by the videos were mainly precautions, adverse drug reactions, mechanisms, vaccination procedures, national vaccination policies, and research and development. While most videos only contained 1 or 2 topic categories, in all the categories, precautions ranked top (69 videos, 66.35%), which were notifications about the interaction and contraindication of the vaccine. Adverse drug reactions were another major concern, as potential side effects of the COVID-19 vaccine including lethargy, dizziness, fatigue, nausea, rash, and fever were introduced in 10 videos (9.62%). A total of 9 videos (8.65%) addressed vaccine science and mechanisms, as well as the different varieties of vaccines in application. Following ranked vaccination procedure (7 videos, 6.73%) and national vaccination policy (6 videos, 5.77%), these 2 kinds of videos told people where and how to get vaccinated. For the research and development issue, there were 3 videos (2.88%) which talked about the manufacturing process and clinical trial of the COVID-19 vaccine.
The videos were divided into 7 types based on format parameters, as follows: (1) presentation (34 videos, 37.79%); (2) news report (22 videos, 24.44%); (3) interview (12 videos, 13.33%); (4) video clip (10 videos, 11.11%); (5) animation with voice explanation (8 videos, 8.89%); (6) short play (actors’ performance) (2 videos, 2.22%); (7) animation with text explanation (2 videos, 2.22%). The results are shown in Figure 2.
Characteristics of included videos. (Top left: Channels; Top right: Sources; Bottom left: Topics; Bottom right: Formats).
Quality, reliability and credibility
For the GQS score, more than 4/5 of all the videos (75 videos, 83.34%) were of high quality, 14 videos (15.56%) were of intermediate quality, and only 1 video (1.11%) was classified as low quality (Table 2).
GQS score of included videos
The percentages of videos adhering to each HONCode principle are shown in Table 3. In general, 77 (85.56%) videos clearly stated whether the information came from a qualified medical professional or not. The information provided in 85 (94.44%) videos were meant to support the patient’s self-management; additionally, 54 (60.00%) videos satisfied “attribution” criteria. Some videos were scored poorly regarding justifiability (7 videos, 7.77%), while 55 (61.11%) videos provided the viewers with contact information, and 13 (14.44%) videos mentioned financial disclosure. In addition to all of this, 6 videos (6.67%) included advertisements which were clearly differentiable to the viewers. The screened COVID-19 vaccine videos on iQiYi took no account of privacy.
HONCode adherence of included videos
Videos attained an overall median DISCERN score of 32 with a range from 16 to 36. In applying the DISCERN criteria to the video assessment, taking into consideration “clear aims,” “achieve aims,” “relevant,” and “unbiased,” there were (separately) 86 videos, 84 videos, 85 videos, and 85 videos all rated as 5, meaning that the videos stated what they were meant to cover and they met the purpose. A bit more than 50% of videos had sources of information that could be identified (49 videos were rated as 5), and when the information reported in the publication was generated (46 videos were rated as 5). About 50% of the videos (46 videos were rated as 5) acknowledged areas of uncertainty, e.g., what measures should be taken if vaccinated women found themselves to be pregnant shortly afterwards, or the interaction between COVID-19 vaccine and other vaccines such as Human Papillomavirus (HPV) vaccine. No videos provided additional sources of support and information. The results are shown in Figure 3.
DISCERN score of included videos (score: 
1 
2 
3 
4 
5).
Discussion
In this era of globalization, it may be difficult to prevent the spread of COVID-19, but the most effective way to prevent panic among the people is the provision of correct and timely information to meet public needs from a scientific point of view. 17 Ensuring the quality, reliability and credibility of health information, protecting the rights of the individual to freedom of expression and opinion, and avoiding false information, in particular in public health emergency are very important. The COVID-19 vaccine was approved to be put into use as early as the second part of 2020. Reference Staff18 When it was first made available to the public, people had many questions about it, they doubted how the COVID-19 vaccine could be developed and manufactured in such a short time, how effective, safe or even how expensive the vaccine was. People wanted to know where and how to get vaccinated, as well as who was eligible to be vaccinated. Chinese video website iQiYi provided a different medium to disseminate information about the COVID-19 vaccine to the public. This video-based information source helped the public get better understanding, and like other vaccination campaigns, it also focused on raising the awareness of the public about the efficacy and safety of vaccines. Reference Bonnevie, Rosenberg and Kummeth19,Reference Patel, Jeve and Sherman20
Several authors have evaluated the characteristics of YouTube videos providing information about COVID-19 vaccines. Reference Ahmad, Murad and Baig21,Reference Marwah, Carlson and Rosseau22 As far as we know, this is the first study to assess the content and quality of COVID-19 vaccine related videos on iQiYi. The results revealed that most screened videos were of good quality, and these videos partly conformed to HONCode and DISCERN principle.
The length of the videos and audience engagement measures like the number of likes were important factors for iQiYi videos to be popular. The duration of the videos were not long as ost of them only lasted for several minutes - it was the consideration that people don’t have the time and patience to see a long sermon. The videos examined gained a total of nearly 1.2 million views. The data illustrated that videos about COVID-19 vaccine on iQiYi were extremely hot as the likes on the videos will orally spread the propaganda which ultimately leads to more views. The engagement of the public was not too much, although the videos were viewed tens of thousands of times, the number of likes, forwards and comments were relatively low. Not too many people shared the videos on their social media account, let alone leave some words under the videos. This may have been due to the limited function of the video websites, as iQiYi didn’t rate the videos and the audiences’ engagement was non-compulsory. The only incentive for the viewers to engage in interaction with the videos were the videos themselves. For the uploader, they seldom responded to the viewers’ comments actively and promptly.
Most of the included videos were classified into health channels by iQiYi. Media organizations rely on their subscribers to keep the business running, so they put in great effort, money and time to the make attractive original videos. Video websites were used as 1 of their major platforms to transmit information. This study showed that media organization played a big part in video spread of COVID-19 vaccine related information.
It seems the video website is a powerful and useful tool for medical professionals like physicians, pharmacists, nurses, and even dietitian, as more and more of them realized that it was 1 of their responsibilities to convey the correct information about COVID-19 to the public. They set up personal accounts on video websites to promote medical knowledge and seize the opportunity to improve their popularity. Given the fact that Chinese doctors often work overtime and experience energy deficiencies, they are enthusiastic about science popularization. Reference Wu, Liu and Wang23 Some of them already became online celebrity doctors and had a large population of fans. For example, Doctor Wenhong Zhang, who was very famous in China for his rich knowledge, humorous language, and wise judgment, was always invited to give speeches on various occasions and was awarded many honors and awards.
Since it is not necessary to have professional equipment, software, or skills to produce videos, warm-hearted non-professional people can easily share their personal experience on the video platform. Some of them introduced or reemphasized the national or local vaccination policy, others recorded their own vaccination process to show the public the procedure and how it felt. This indicated that individuals do not only want to be passively educated and informed about COVID-19 vaccine information, but they actively want to take part in sharing their knowledge and experience to others. This result could indicate that videos published by medical professionals commonly serve a higher educational goal, while videos published by laypersons mainly serve a higher social purpose.
There’s quite a lot of useful information about the COVID-19 vaccine available on iQiYi, and precautions and interactions were the most concerned topics about the COVID-19 vaccine. At that time, the Chinese government called on the public to be vaccinated. This was obligatory and was promoted as a political duty in some local areas. Since getting vaccination means making one’s contribution to the country, people are actively encouraged to get vaccinated for their country. The special population, for example, people with comorbidities like diabetes, hypertension, malignant tumors, or pregnant and lactating women need to know if it is appropriate for them to get vaccinated. The general population was also eager to get the knowledge of the interaction and contraindication between the COVID-19 vaccine and food, medications, and other vaccines. The above-mentioned concerns were discussed in the screened videos, and among the video topics, research and development were the least cared about, maybe because vaccine development process, although expressed in a clear and approachable manner, was still complicated and obscure to non-professionals. Since the COVID-19 vaccine was already in the clinical application stage worldwide, the research and development process were not the focus of attention anymore.
The forms of videos were versatile. The carefully designed and produced news reports, interviews, animations, short plays, etc., were vivid, interesting and easy to understand. Presentation or personal speech is the most used format, just because it is simple and does not need a lot of resources to produce. These kinds of personal speech were portrayed as a person’s chest and head, or a big face occupying part of the screen. This was intuitive and educational but lacked a bit of flexibility, after all, only high-quality eye-catching videos would appeal to the viewers.
iQiyi is not a medical professional website, rather it is all inclusive. Though censorship for video website exists, the video content is subject to review for gambling, pornography, violence or other illegal content, and there is no standard for the professional review of the videos. Health information provided on iQiYi is mainly targeted to non-expert people, which further increases the need to ensure that viewers are provided with accurate information. Presently, the tools used for evaluating videos are limited in scope. Reference Nagpal, Karimianpour and Mukhija24 While the current available tool, COVID-19 Specific Score (CSS) evaluates COVID-19 information, it addresses topics regarding the pandemic itself such as the coronavirus’ epidemiology and transmission, thus it cannot be applied to information on COVID-19 vaccines. Reference Tan, Pua and Wong25,Reference Li, Bailey and Huynh26 There is currently no quality assessment tool for COVID-19 vaccines videos.
The GQS score of the videos were relatively high, maybe due to the reason that the included videos ranked top on the list based on popularity. If the videos were not well made, they would not be welcomed by the viewers.
Although a majority of the videos were rated as being of good quality based on their GQS score, most videos only achieved part adherence to HONCode and DISCERN principles. After all, HONCode and DISCERN principles are evaluation tools for health websites, not for video websites specifically. Although these tools are currently the best available tools, they may not be able to capture the complexities of iQiYi videos but are better suited for text media. A frequently missing quality indicator in iQiYi videos is the referencing of information provided. Thus, these videos were unable to fulfil both HONCode principles (e.g., attribution and transparency) and DISCERN reliability indicators (e.g., clear sources of information), and were unable to attain high scores. This may be due to the lack of a standardized method for referencing data sources in videos, and frequent sharing of opinions or experience in these videos rather than sharing of evidence-based information. This is a major constraint in all videos, and producers of future videos should have regularly verifiable sources for the presentation of evidence-based information. Advertisements were also included in several TV videos, and while it is clear that advertisements supported the funding of the TV station to make these programs which were usually health education programs aimed at teaching people how to live long. All the videos were not concerned with the privacy of the patients, probably because these videos were not for treatment or education purposes. Also, there were no patients in these videos, hence, the privacy principle was not applicable here. The screened videos ranked top on the popularity list, it seemed that they were well received by the public, however, the interaction between uploaders and viewers were limited, e.g., the comments of the videos were scarce. The evaluation tools used in this study were not involved in rating how well the videos were received and interpreted by the general public. Therefore, the evaluation of understandability and action-ability of the videos are important aspects needed to be considered.
Since available tools cannot fully reflect the characteristics and completeness of versatile videos, or if they contain surplus or items that are not applicable, it is urgent to develop appropriate tools or set new standards for video website platforms to evaluate health information in order to better capture the way information is disseminated and public interaction with them. People around the world are increasingly using the internet to collect information about the COVID-19 vaccine. Since anything relating to COVID-19 on the video platform would be hot, multilateral efforts are required among the public, government and websites, to maximize the potential of video-based information and to minimize the amount of misleading or useless information.
This study has several limitations. First, effective and appropriate tools or standards should be developed to help people understand the modern video-based health information better. Secondly, the videos were sorted by popularity. The sorting criteria can influence the search results. Thirdly, iQiYi search results are dynamic, and will change as more new videos are uploaded and old videos are withdrawn. Therefore, this cross-sectional study comprises of a snapshot of information on the COVID-19 vaccine at a certain time, also, we do not know how different the results would be if more video samples were taken. Furthermore, short video platforms are of high popularity in recent years like TikTok and were not in the whole network search scope of iQiYi, hence, they were not included in this study.
Conclusion
In conclusion, our study played a key role in understanding how iQiYi described the COVID-19 vaccine. The COVID-19 vaccine videos from iQiYi show significant differences in sources and content. It is urgent to explore practical tools for video websites assessment, and also, there is the need for joint efforts to transfer correct and effective health information to people.
Funding statement
The authors received non-financial support from the Science and Technology Development Center of Chinese Pharmaceutical Association (CMEI2021KPYJ00306) for the research, authorship, and/or publication of this article.
Conflict of interests
The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical standards
This study did not require approval by the institutional review board because it involved the use of public access data only.
