About: Evaluating human versus machine learning performance in classifying research abstracts

Facets (new session)
Description
Metadata
Settings
- owl:sameAs
- Inference Rule:

About: Evaluating human versus machine learning performance in classifying research abstracts Goto Sponge NotDistinct Permalink

An Entity of Type : schema:ScholarlyArticle, within Data Space : covidontheweb.inria.fr associated with source document(s)

Attributes	Values
type	Academic Article research paper schema:ScholarlyArticle
isDefinedBy	Covid-on-the-Web dataset
has title	Evaluating human versus machine learning performance in classifying research abstracts
Creator	Cai, Qing Goh, Yeow Khiam, · Khor, Aik Ko, · Theseira, · Xin, ·
Source	Medline; PMC
abstract	We study whether humans or machine learning (ML) classification models are better at classifying scientific research abstracts according to a fixed set of discipline groups. We recruit both undergraduate and postgraduate assistants for this task in separate stages, and compare their performance against the support vectors machine ML algorithm at classifying European Research Council Starting Grant project abstracts to their actual evaluation panels, which are organised by discipline groups. On average, ML is more accurate than human classifiers, across a variety of training and test datasets, and across evaluation panels. ML classifiers trained on different training sets are also more reliable than human classifiers, meaning that different ML classifiers are more consistent in assigning the same classifications to any given abstract, compared to different human classifiers. While the top five percentile of human classifiers can outperform ML in limited cases, selection and training of such classifiers is likely costly and difficult compared to training ML models. Our results suggest ML models are a cost effective and highly accurate method for addressing problems in comparative bibliometric analysis, such as harmonising the discipline classifications of research from different funding agencies or countries.
has issue date	2020-07-18(xsd:dateTime)
bibo:doi	10.1007/s11192-020-03614-2
bibo:pmid	32836529
has license	cc-by
sha1sum (hex)	44557db9dce19a72a599007f5a5c00ac8e031a79
schema:url	https://doi.org/10.1007/s11192-020-03614-2
resource representing a document's title	Evaluating human versus machine learning performance in classifying research abstracts
has PubMed Central identifier	PMC7367789
has PubMed identifier	32836529
schema:publication	Scientometrics
resource representing a document's body	covid:44557db9dce19a72a599007f5a5c00ac8e031a79#body_text
is schema:about of	named entity 'panels' named entity 'human' named entity 'training' named entity 'study' named entity 'addressing' named entity 'training' named entity 'meaning' named entity 'classifying' covid:arg/44557db9dce19a72a599007f5a5c00ac8e031a79 named entity 'classifiers' named entity 'agencies' named entity 'groups' named entity 'abstracts' named entity 'highly' named entity 'accurate' named entity 'humans' named entity 'training' named entity 'human' named entity 'harmonising' named entity 'human' named entity 'abstracts' named entity 'classification models' named entity 'research abstracts' named entity 'compared' named entity 'word clustering' named entity 'ground truth' named entity 'training set' named entity 'test set' named entity 'ground truth' named entity 'NSF' named entity 'European Research Council' named entity 'software engineering' named entity 'email' named entity 'F1 score' named entity 'prior knowledge' named entity 'European Research Council' named entity 'Almost Perfect' named entity 'ground truth' named entity 'randomly assigned' named entity 'email' named entity 'unsupervised classification' named entity 'email' named entity 'research abstracts' named entity 'database' named entity 'research policy' named entity '95%' named entity 'SVM' named entity 'co-citation' named entity 'test set' named entity 'training set' named entity 'test set' named entity 'statistical analysis' named entity 'flat rate' named entity 'supervised classification' named entity 'postdoctoral researchers' named entity 'Nanyang Technological University' named entity 'Nanyang Technological University' named entity 'training set' named entity 'selection effects' named entity 'scientometric' named entity 'confusion matrix' named entity 'probability' named entity 'algorithm' named entity 'supervised classification' named entity 'research teams' named entity 'unsupervised classification' named entity 'training set' named entity 'test set' named entity 'Ph.D.' named entity 'natural sciences' named entity 'supervised classification' named entity 'machine learning' named entity 'ground truth' named entity 'standard errors' named entity 'Moed'

Faceted Search & Find service v1.13.91 as of Mar 24 2020

Alternative Linked Data Documents: Sponger | ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3229 as of Jul 10 2020, on Linux (x86_64-pc-linux-gnu), Single-Server Edition (94 GB total memory)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2024 OpenLink Software