Basic Usage

This package is designed to train and analyze.

Importing the package:

# Import the package
import hardware_rc as hrc

# Or import specific classes
from hardware_rc import DQN_RC, PPO_RC

Running a DQN RC training:

---

Run a DQN RC training for CartPole-v1 with default hyperparameters:

from hardware_rc import DQN_RC

# Create a DQN_RC agent with default hyperparameters and CartPole-v1 environment
dqn_agent = DQN_RC(env_name="CartPole-v1", watch=True)

# If no environment object is provided, one is created using env_name

# watch=True sets render_mode='human' in the created environment
# (only happens if no environment is provided)

# Train the agent for 100 episodes
agent_path = dqn_agent.train(trials=100)

STANDARD USAGE:

CartPole-v1 training while:

• specifying hyperparameters

• initializing environment

• enabling weights and biases

• analyzing best model

import gymnasium as gym
from hardware_rc import DQN_RC, AnalyzeRun

env_name = 'CartPole-v1'

# Specify hyperparameters in dictionary
hp = {
    'N': 300,
    'theta': 1,
    'amplification': 360,
    'trials': 300,
    # full list of possible hyperparamters found in `hyperparameter overview`
}

# Specify metadata for weights and biases logging
meta = {
    "project": "MEMS-RC-RL",
    "group": f"tutorial",
    'job_type': 'single',
    'env': env_name,
    "algo": "DQN_RC",
    "run_name": f"{env_name}-DQN"
}

# Initialize environment
# No render_mode specified so everything will happen in background (faster)
env = gym.make(env_name)

# Initialize agent with env and hyperparameters
dqn_agent = DQN_RC(env, config=hp)

# Run training
# Defaults to number of trials in hyperparameters
agent_path = dqn_agent.train(meta=meta, wandb_on=True)

# Init analyze object
analyze = AnalyzeRun(agent_path)

# test model on 25 seeds and create gif of best run
analyze.run_and_gif(test_range[0,25],
                    win_thresh=499, metric='max'
                    gif_title='CartPole-v1 - Example')

Running a PPO RC training:

---

Run a DQN RC training for CartPole-v1 with default hyperparameters:

from hardware_rc import PPO_RC

# Create a PPO_RC agent with default hyperparameters and CartPole-v1 environment
ppo_agent = PPO_RC(env_name="MountainCarContinuous-v0", watch=True)

# If no environment object is provided, one is created using env_name

# watch=True sets render_mode='human' in the created environment
# (only happens if no environment is provided)

# Train the agent for 100 episodes
agent_path = ppo_agent.train(trials=100)

STANDARD USAGE:

MountainCarContinuous-v0 training while:

• specifying hyperparameters

• initializing environment

• enabling weights and biases

• analyzing best model

import gymnasium as gym
from hardware_rc import PPO_RC, AnalyzeRun

env_name = 'MountainCarContinuous-v0'

# Specify hyperparameters in dictionary
hp = {
    'N_envs': 10,
    'T_horizon': 2048,
    'N': 300,
    'theta': 1,
    'trials': 300,
    'learning_rate': 3e-4,
    # full list of possible hyperparamters found in `hyperparameter overview`
}

# Specify metadata for weights and biases logging
meta = {
    "project": "MEMS-RC-RL",
    "group": f"tutorial",
    'job_type': 'single',
    'env': env_name,
    "algo": "PPO_RC",
    "run_name": f"{env_name}-PPO"
}

# Initialize environment - need vectorized env for PPO
# No render_mode specified so everything will happen in background (faster)
venv = gym.make_vec(env_name, num_envs=hparams['N_envs'], vectorization_mode='async')

# Initialize agent with env and hyperparameters
ppo_agent = PPO_RC(venv, config=hp)

# Run training with weights and biases logging
# Defaults to number of trials in hyperparameters
agent_path = ppo_agent.train(meta=meta, wandb_on=True)

# Init analyze object
analyze = AnalyzeRun(agent_path)

# test model on 25 seeds and create gif of best run
analyze.run_and_gif(test_range[0,25],
                    win_thresh=85, metric='max'
                    gif_title='MountainCarContinuous - Example')