docgen.templates.trec02-ar.template Maven / Gradle / Ivy

Go to download
# Anserini Regressions: TREC 2002 Monolingual Arabic

This page documents BM25 regression experiments for monolingual Arabic document retrieval as part of the [TREC 2002 CLIR Track](https://trec.nist.gov/pubs/trec11/t11_proceedings.html).
The description of the document collection can be found on the [TREC data page](https://trec.nist.gov/data/docs_noneng.html).

The exact configurations for these regressions are stored in [this YAML file](${yaml}).
Note that this page is automatically generated from [this template](${template}) as part of Anserini's regression pipeline, so do not modify this page directly; modify the template instead.

From one of our Waterloo servers (e.g., `orca`), the following command will perform the complete regression, end to end:

```
python src/main/python/run_regression.py --index --verify --search --regression ${test_name}
```

## Indexing

Typical indexing command:

```
${index_cmds}
```

The collection comprises Agence France Presse (AFP) Arabic newswire, from [LDC2001T55 (Arabic Newswire Part 1)](https://catalog.ldc.upenn.edu/LDC2001T55).
Inside the LDC2007T38 distribution, there should be a directory named `transcripts`, which contains 2,337 gzipped files in 7 directories, `1994` ... `2000`.
The path above `/path/to/trec02-ar/` should point to this `transcripts/` directory.
The collection contains 383,872 documents.

For additional details, see explanation of [common indexing options](common-indexing-options.md).

## Retrieval

Topics and qrels are stored in [`src/main/resources/topics-and-qrels/`](../src/main/resources/topics-and-qrels/), downloaded from NIST's page for [non-English topics](https://trec.nist.gov/data/topics_noneng/index.html) and [non-English relevance judgments](https://trec.nist.gov/data/qrels_noneng/index.html):

+ [`topics.trec02ar-ar.txt`](../src/main/resources/topics-and-qrels/topics.trec02ar-ar.txt): TREC 2002 cross language topics in Arabic
+ [`qrels.trec02ar.txt`](../src/main/resources/topics-and-qrels/qrels.trec02ar.txt): TREC 2002 cross language relevance judgements

After indexing has completed, you should be able to perform retrieval as follows:

```
${ranking_cmds}
```

Evaluation can be performed using `trec_eval`:

```
${eval_cmds}
```

## Effectiveness

With the above commands, you should be able to reproduce the following results:

${effectiveness}