TY - JOUR AU - Boyer, Célia AU - Dolamic, Ljiljana PY - 2015 DA - 2015/06/02 TI - HONcode网站符合性自动检测与人工检测的比较:评估JO - J Med Internet Res SP - e135 VL - 17 IS - 6kw - HONcode KW -分类KW -人工智能KW -自然语言处理KW -质量标准AB -背景:要获得HONcode认证，一个网站必须符合HONcode行为准则的8个原则。在目前的手动认证过程中，HONcode专家对候选网站使用每个原则的精确指南进行评估。在欧洲KHRESMOI项目范围内，网络健康(HON)基金会开发了一个自动化系统，以协助检测网站的HONcode符合性。实施HONcode审查的自动化协助可以加快当前耗时的HONcode认证和持续监视任务。此外，作为通用搜索引擎插件使用的自动化工具可能有助于检测遵守HONcode原则但尚未获得认证的健康网站。目的:本研究的目的是确定自动化系统是否能够像人类专家一样完成识别健康网站HONcode原则的任务。方法:以HONcode高级专家的人工评价为基线，比较HONcode自动检测系统与HONcode高级专家的检测能力。一组27个与健康相关的网站由HONcode的资深专家手工评估，以确定它们是否符合HONcode的8项原则。基于监督机器学习的HONcode合规性检测自动化系统对同一组网站进行处理。并对两种方法的结果进行了比较。 Results: For the privacy criterion, the automated system obtained the same results as the human expert for 17 of 27 sites (14 true positives and 3 true negatives) without noise (0 false positives). The remaining 10 false negative instances for the privacy criterion represented tolerable behavior because it is important that all automatically detected principle conformities are accurate (ie, specificity [100%] is preferred over sensitivity [58%] for the privacy criterion). In addition, the automated system had precision of at least 75%, with a recall of more than 50% for contact details (100% precision, 69% recall), authority (85% precision, 52% recall), and reference (75% precision, 56% recall). The results also revealed issues for some criteria such as date. Changing the “document” definition (ie, using the sentence instead of whole document as a unit of classification) within the automated system resolved some but not all of them. Conclusions: Study results indicate concordance between automated and expert manual compliance detection for authority, privacy, reference, and contact details. Results also indicate that using the same general parameters for automated detection of each criterion produces suboptimal results. Future work to configure optimal system parameters for each HONcode principle would improve results. The potential utility of integrating automated detection of HONcode conformity into future search engines is also discussed. SN - 1438-8871 UR - //www.mybigtv.com/2015/6/e135/ UR - https://doi.org/10.2196/jmir.3831 UR - http://www.ncbi.nlm.nih.gov/pubmed/26036669 DO - 10.2196/jmir.3831 ID - info:doi/10.2196/jmir.3831 ER -