gpt_engineer.benchmark.__main__.main

gpt_engineer.benchmark.__main__.main(path_to_agent: str, bench_config: str = '/home/docs/checkouts/readthedocs.org/user_builds/gpt-engineer/checkouts/latest/gpt_engineer/benchmark/default_bench_config.toml', yaml_output: str | None = None, verbose: bool | None = False, use_cache: bool | None = True)[source]

The main function that runs the specified benchmarks with the given agent and outputs the results to the console.

Parameters:

  • path_to_agent (str) – The file path to the Python module that contains a function called ‘default_config_agent’.

  • bench_config (str, default=default_bench_config.toml) – Configuration file for choosing which benchmark problems to run. See default config for more details.

  • yaml_output (Optional[str], default=None) – Pass a path to a yaml file to have results written to file.

  • verbose (Optional[bool], default=False) – A flag to indicate whether to print results for each task.

  • use_cache (Optional[bool], default=True) – Speeds up computations and saves tokens when running the same prompt multiple times by caching the LLM response.

Return type:

None