Experiment

class coba.experiments.Experiment

Experiment for environments, learners and evaluators.

Constructors

__init__(environments: Environment | Sequence[Environment], learners: Learner | Sequence[Learner], evaluator: Evaluator | Sequence[Evaluator] = SequentialCB(), description: str = None) → None

__init__(eval_tuples: Sequence[Tuple[Learner, Environment] | Tuple[Learner, Environment, Evaluator]], description: str = None) → None

Instantiate an Experiment.

Parameters:

environments – The collection of environments to use in the experiment.
learners – The collection of learners to use in the experiment.
evaluator – The evaluation task we wish to perform on learners and environments.
eval_tuples – The learner-environment-evaluator triples we wish to evaluate.
description – A description of the experiment for documentaiton purposes.

Methods

config(processes: int | None = None, maxchunksperchild: int | None = None, maxtasksperchunk: int | None = None) → Experiment

Configure how the experiment will be executed.

Parameters:

processes – The number of processes to create for evaluating the experiment.
maxchunksperchild – The number of chunks each process evaluate before being restarted. A value of 0 means that all processes will survive until the end of the experiment.
maxtasksperchunk – The maximum number of tasks a chunk can have. If a chunk has too many tasks it will be split into smaller chunks. A value of 0 means that chunks are never broken down into smaller chunks.

Returns:

The configured Experiment.

Run the experiment and return the results.

Parameters:

result_file – The file for writing and restoring results.
quiet – Indicates that logged output should be turned off.
processes – The number of processes to create for evaluating the experiment.
maxchunksperchild – The number of chunks each process evaluates before being restarted. A value of 0 means that processes will survive until the end of the experiment. Unless otherwise specified chunks are individual environment-learner-evaluator triples.
maxtasksperchunk – The maximum number of tasks a chunk can have. If a chunk has too many tasks it will be split into smaller chunks. A value of 0 means that chunks are never broken down into smaller chunks.
seed – The seed that will determine all randomness within the experiment.

Returns:

Result of the experiment.

Attributes

maxchunksperchild: The number of tasks chunks to perform per process before restarting an evaluation process.

maxtasksperchunk: The maximum number of tasks allowed in a chunk before breaking a chunk into smaller chunks.