hardware_rc.dqn_rc

A class that implements DQN for a MEMS-based reservoir computer for discrete reinforcement learning tasks.

MEMS dynamics simulation is done with a rk4 DDE solver implemented with JAX for speed.

Objects are created with, optionally, a list of hyperparameters, environment from Farama Gymnasium, previously trained models, and singular or specific hyperparameter values.

Classes

DQNConfig(env_name, rc_seed, mask_seed, ...)

DQN_RC([env, reward_function, env_name, ...])

class hardware_rc.dqn_rc.DQN_RC(env=None, *, reward_function=None, env_name=None, watch=True, config: DQNConfig | Mapping[str, Any] | None = None, model: str | None = None, **overrides: Any)[source]

Bases: object

train(*, env=None, meta={}, trials=None, folder_path=None, wandb_on=True, save_model=True, ray_on=False, ray_metric='avg_reward', ray_mode='max', ray_report_freq=1, full_validate=False, val_size=5, val_size_min=1, threed_it=False, resume=False, wandb_id=None)[source]

Runs a reinforcement learning training on the given environment for the initialized DQN_RC model.

Parameters:

  • env (gym.Env) – The environment to train on (likely already defined during agent creation).

  • meta (dict) – A dictionary containing metadata for the training session.

  • trials (int) – The number of trials to train on.

  • folder_path (str) – The path to save the model to.

  • wandb_on (bool) – Whether to use wandb for logging.

  • save_model (bool) – Whether to save the model to disk.

  • full_validate (bool) – Whether to fully validate every trial or when an agent gets close to current best reward.

  • val_size (int) – The number of environment seeds in the validation set.

  • val_size_min (int) – The minimum number of samples in the validation set, if full_validate is False.

  • threed_it (bool) – Used to create policy map (to be implemented more easily)

  • resume (bool) – Whether to resume training from a checkpoint.

  • wandb_id (str) – The wandb run id to resume training from.

Returns:

The path to the best model found during training.

Return type:

best_model_path (str)