Basic Usage
####

This package is designed to `train` and `analyze`.


Importing the package:
----

.. code-block:: python

    # Import the package
    import hardware_rc as hrc

.. code-block:: python

    # Or import specific classes
    from hardware_rc import DQN_RC, PPO_RC




Running a DQN RC training:
----
----

**Run a DQN RC training for CartPole-v1 with default hyperparameters:**

.. code-block:: python

    from hardware_rc import DQN_RC

    # Create a DQN_RC agent with default hyperparameters and CartPole-v1 environment
    dqn_agent = DQN_RC(env_name="CartPole-v1", watch=True)

    # If no environment object is provided, one is created using env_name

    # watch=True sets render_mode='human' in the created environment 
    # (only happens if no environment is provided)
    
    # Train the agent for 100 episodes
    agent_path = dqn_agent.train(trials=100)

**STANDARD USAGE:**

CartPole-v1 training while: 
    - specifying hyperparameters 
    - initializing environment 
    - enabling weights and biases
    - analyzing best model

.. code-block:: python

    import gymnasium as gym
    from hardware_rc import DQN_RC, AnalyzeRun

    env_name = 'CartPole-v1'

    # Specify hyperparameters in dictionary
    hp = {
        'N': 300,
        'theta': 1,
        'amplification': 360,
        'trials': 300,
        # full list of possible hyperparamters found in `hyperparameter overview`
    }

    # Specify metadata for weights and biases logging
    meta = {
        "project": "MEMS-RC-RL",
        "group": f"tutorial",
        'job_type': 'single',
        'env': env_name,
        "algo": "DQN_RC",
        "run_name": f"{env_name}-DQN"
    }

    # Initialize environment 
    # No render_mode specified so everything will happen in background (faster)
    env = gym.make(env_name)

    # Initialize agent with env and hyperparameters
    dqn_agent = DQN_RC(env, config=hp)
    
    # Run training
    # Defaults to number of trials in hyperparameters
    agent_path = dqn_agent.train(meta=meta, wandb_on=True)

    # Init analyze object
    analyze = AnalyzeRun(agent_path)

    # test model on 25 seeds and create gif of best run
    analyze.run_and_gif(test_range[0,25],
                        win_thresh=499, metric='max'
                        gif_title='CartPole-v1 - Example')
    

Running a PPO RC training:
----
----

**Run a DQN RC training for CartPole-v1 with default hyperparameters:**

.. code-block:: python

    from hardware_rc import PPO_RC

    # Create a PPO_RC agent with default hyperparameters and CartPole-v1 environment
    ppo_agent = PPO_RC(env_name="MountainCarContinuous-v0", watch=True)

    # If no environment object is provided, one is created using env_name

    # watch=True sets render_mode='human' in the created environment 
    # (only happens if no environment is provided)
    
    # Train the agent for 100 episodes
    agent_path = ppo_agent.train(trials=100)

**STANDARD USAGE:**

MountainCarContinuous-v0 training while: 
    - specifying hyperparameters 
    - initializing environment 
    - enabling weights and biases
    - analyzing best model

.. code-block:: python

    import gymnasium as gym
    from hardware_rc import PPO_RC, AnalyzeRun

    env_name = 'MountainCarContinuous-v0'

    # Specify hyperparameters in dictionary
    hp = {
        'N_envs': 10,
        'T_horizon': 2048,
        'N': 300,
        'theta': 1,
        'trials': 300,
        'learning_rate': 3e-4,
        # full list of possible hyperparamters found in `hyperparameter overview`
    }

    # Specify metadata for weights and biases logging
    meta = {
        "project": "MEMS-RC-RL",
        "group": f"tutorial",
        'job_type': 'single',
        'env': env_name,
        "algo": "PPO_RC",
        "run_name": f"{env_name}-PPO"
    }

    # Initialize environment - need vectorized env for PPO
    # No render_mode specified so everything will happen in background (faster)
    venv = gym.make_vec(env_name, num_envs=hparams['N_envs'], vectorization_mode='async')

    # Initialize agent with env and hyperparameters
    ppo_agent = PPO_RC(venv, config=hp)
    
    # Run training with weights and biases logging
    # Defaults to number of trials in hyperparameters
    agent_path = ppo_agent.train(meta=meta, wandb_on=True)

    # Init analyze object
    analyze = AnalyzeRun(agent_path)

    # test model on 25 seeds and create gif of best run
    analyze.run_and_gif(test_range[0,25],
                        win_thresh=85, metric='max'
                        gif_title='MountainCarContinuous - Example')