@Article{信息:doi 10.2196 / / jmir.4.1。e2，作者=“Craigie, Mark and Loader, Brian and Burrows, Roger and Muncer, Steven”，标题=“Internet上健康信息的可靠性:专家评级的检验”，期刊=“J Med Internet Res”，年份=“2002”，月份=“Jan”，日期=“17”，卷=“4”，数字=“1”，页面=“e2”，关键词=“新闻组”;互联网;评级信息;可靠性;结果的可重复性;统计数据;背景:近年来，利用医学专家对互联网上与健康相关的网站内容进行评级的做法非常盛行。在这项研究中，通常的做法是让一名医学专家对网站的内容进行评级。在许多情况下，专家认为互联网上的健康信息很差，甚至有潜在的危险。 However, one problem with this approach is that there is no guarantee that other medical experts will rate the sites in a similar manner. Objectives: The aim was to assess the reliability of medical experts' judgments of threads in an Internet newsgroup related to a common disease. A secondary aim was to show the limitations of commonly-used statistics for measuring reliability (eg, kappa). Method: The participants in this study were 5 medical doctors, who worked in a specialist unit dedicated to the treatment of the disease. They each rated the information contained in newsgroup threads using a 6-point scale designed by the experts themselves. Their ratings were analyzed for reliability using a number of statistics: Cohen's kappa, gamma, Kendall's W, and Cronbach's alpha. Results: Reliability was absent for ratings of questions, and low for ratings of responses. The various measures of reliability used gave conflicting results. No measure produced high reliability. Conclusions: The medical experts showed a low agreement when rating the postings from the newsgroup. Hence, it is important to test inter-rater reliability in research assessing the accuracy and quality of health-related information on the Internet. A discussion of the different measures of agreement that could be used reveals that the choice of statistic can be problematic. It is therefore important to consider the assumptions underlying a measure of reliability before using it. Often, more than one measure will be needed for ``triangulation'' purposes. ", issn="1438-8871", doi="10.2196/jmir.4.1.e2", url="//www.mybigtv.com/2002/1/e2/", url="https://doi.org/10.2196/jmir.4.1.e2", url="http://www.ncbi.nlm.nih.gov/pubmed/11956034" }