PyTerrier - v1.0

🔍 Retrieve. 🧠 Rerank. 💬 Answer. ⚙️ Experiment.

Build (sparse|learned sparse|dense) indexing and retrieval pipelines for search and RAG use-cases, and conduct experiments on standard datasets.

For example, build a re-ranking pipeline combining a Terrier BM25 retriever and the MonoT5 neural reranker (each of these are PyTerrier Transformer classes):

import pyterrier as pt
import pyterrier_t5
bm25 = pt.terrier.TerrierIndex.from_hf("pyterrier/vaswani.terrier").bm25() % 100
monot5 = bm25 >> pt.get.get_text(pt.get_dataset('irds:vaswani')) >> pyterrier_t5.MonoT5ReRanker()
monot5.search("What are chemical reactions?")

In notebook environments, PyTerrier transformers and pipelines can be visualised.

You can easily build pipeline for query expansion, learning-to-rank, dense retrieval and even RAG.

Once you have working pipelines, you can formulate an experiment to compare their effectiveness using the pt.Experiment function:

from pyterrier.measures import *
pt.Experiment(
  [bm25, monot5]
  pt.get_dataset('vaswani').get_topics(),
  pt.get_dataset('vaswani').get_qrels(),
  [nDCG@10, AP@100]
)

You can easily perform retrieval experiments using many standard datasets, including all from the ir_datasets package. E.g., use pt.datasets.get_dataset("irds:medline/2004/trec-genomics-2004") to get the TREC Genomics 2004 dataset. A full catalogue of ir_datasets is available here.

The easiest way to get started with PyTerrier is to use one of our Colab notebooks - look for the badges below.

1. pip install 'pyterrier[all]'
2. You may need to set JAVA_HOME environment variable if Pyjnius cannot find your Java installation.

PyTerrier has additional plugins for everything from dense retrieval to RAG:

• Pyterrier_DR: [Github] - single-representation dense retrieval
• Pyterrier_RAG: [Github] - retrieval augmented generation and LLM access
• PyTerrier_ColBERT: [Github] - mulitple-representation dense retrieval and/or neural reranking
• PyTerrier_PISA: [Github] - fast in-memory indexing and retrieval using PISA
• PyTerrier_T5: [Github] - neural reranking: monoT5, duoT5
• PyTerrier_GenRank [Github] - generative listwise reranking: RankVicuna, RankZephyr
• PyTerrier_doc2query: [Github] - neural augmented indexing
• PyTerrier_SPLADE: [Github] - neural augmented indexing

You can see examples of how to use these, including notebooks that run on Google Colab, in the contents of our Search Solutions 2022 tutorial.

PyTerrier is subject to the terms detailed in the Mozilla Public License Version 2.0. The Mozilla Public License can be found in the file LICENSE.txt. By using this software, you have agreed to the licence.

The source and binary forms of PyTerrier are subject to the following citation license:

By downloading and using PyTerrier, you agree to cite at the undernoted paper describing PyTerrier in any kind of material you produce where PyTerrier was used to conduct search or experimentation, whether be it a research paper, dissertation, article, poster, presentation, or documentation. By using this software, you have agreed to the citation licence.

Declarative Experimentation in Information Retrieval using PyTerrier. Craig Macdonald and Nicola Tonellotto. In Proceedings of ICTIR 2020.

@inproceedings{pyterrier2020ictir,
    author = {Craig Macdonald and Nicola Tonellotto},
    title = {Declarative Experimentation inInformation Retrieval using PyTerrier},
    booktitle = {Proceedings of ICTIR 2020},
    year = {2020}
}

• Craig Macdonald, University of Glasgow
• Sean MacAvaney, University of Glasgow
• Nicola Tonellotto, University of Pisa
• Alex Tsolov, University of Glasgow
• Arthur Câmara, TU Delft
• Alberto Ueda, Federal University of Minas Gerais
• Sean MacAvaney, University of Glasgow
• Chentao Xu, University of Glasgow
• Sarawoot Kongyoung, University of Glasgow
• Zhan Su, Copenhagen University
• Marcus Schutte, TU Delft
• Lukas Zeit-Altpeter, Friedrich Schiller University Jena