Evaluate a challenger

evaluate_challenger

evaluate_challenger(
    challenger_python_code_uri_or_local_path=None,
    output_bucket=None,
    output_prefix=None,
    runtime_configuration=None,
    region=None,
)

Compute all the benchmark scores for the given challenger dataset, by calling all functions of the metrics module. It generates and executes a notebook based on the python code that opens the challenger dataset as challenger_dataset: xarray.Dataset.

This function is used for official evaluation.

The output notebook file name is automatically derived from the challenger file name: glonet.py becomes glonet.global.report.ipynb.

Parameters

Name	Type	Description	Default
challenger_python_code_uri_or_local_path	str	The python content that opens the challenger dataset. Required. Can be a remote file (URL), a DataURI or the path to a local file. Can also be configured with environment variable OCEANBENCH_CHALLENGER_PYTHON_CODE_URI_OR_LOCAL_PATH.	None
output_bucket	str	The destination S3 bucket of the executed notebook. If not provided, the notebook is written on the local filesystem. If provided, uses AWS S3 environment variables. Can also be configured with environment variable OCEANBENCH_OUTPUT_BUCKET.	None
output_prefix	str	The destination S3 prefix of the executed notebook. If output_bucket is not provided, this option is ignored. If provided, uses AWS S3 environment variables. Can also be configured with environment variable OCEANBENCH_OUTPUT_PREFIX.	None
runtime_configuration	RuntimeConfiguration	Runtime settings applied inside the generated notebook execution, including staging and remote retry behavior.	None
region	str, RegionSpec, or None	The OceanBench region to evaluate on. This can be an official region id such as global or ibi, or a custom oceanbench.regions.RegionSpec value.	None