ClueWeb 09 Labeled Near-Duplicate News Articles

Language:
English
This data release is a companion to the paper Duplicate News Story Detection Revisited by Omar Alonso, Dennis Fetterly, and Mark Manasse published at The Ninth Asia Information Retrieval Societies Conference (AIRS 2013) in December 2013. Last published: August 28, 2013.
  • Version:

    1.0

    File Name:

    news-dedupe-data.zip

    Date Published:

    5/11/2016

    File Size:

    14.1 MB

      This data release is a companion to the paper Duplicate News Story Detection Revisited by Omar Alonso, Dennis Fetterly, and Mark Manasse published at The Ninth Asia Information Retrieval Societies Conference (AIRS 2013) in December 2013. The package provides approximately 5.5 million document identifiers of a subset of the ClueWeb’09 “Category A English” documents that are likely to be from news sources. The package also contains two sets of human generated labels. The first set of labels is the assessment of 456 pairs of documents by the authors as near-duplicate, non-duplicate, containment, near-duplicate irrelevant, or non-duplicate irrelevant. The second set of labels is 710 labels obtained via a crowdsourcing system where the pairs of articles are labeled as near-duplicate or non-duplicate articles. Finally, the data release contains the experimental design templates used for the crowdsourced assessments.
  • Supported Operating System

    Windows 10 , Windows 7, Windows 8

      • Windows 7, Windows 8, or Windows 10
      • Click Download and follow the instructions.
Site feedback
Microsoft

What category would you like to give web site feedback on?



Rate your level of satisfaction with this web page today:

Comments:

Submit