gym_acnportal.algorithms

Algorithms used for deploying trained RL models.

Package Contents

Classes

GymBaseAlgorithm

Abstract algorithm class for Simulations using a reinforcement

GymTrainedAlgorithm

Algorithm class for Simulations using a reinforcement learning

SimRLModelWrapper

Abstract wrapper class that wraps a reinforcement learning
class gym_acnportal.algorithms.GymBaseAlgorithm(max_recompute: int = 1)

Bases: acnportal.algorithms.BaseAlgorithm

Abstract algorithm class for Simulations using a reinforcement learning agent that operates in an Open AI Gym environment.

Implements abstract class BaseAlgorithm.

Simulations that run GymAlgorithm-style schedulers have two entry points for simulation control. First, a reinforcement learning agent may call model.learn(vec_env), which will step through the simulation. In this case, schedule() will never be called as all simulation progression would be handled by the agent stepping the environment. As such, the GymTrainingAlgorithm class does not implement a schedule function. See GymTrainingAlgorithm for this case.

Alternatively, the Simulator may call scheduler.run(), causing this GymAlgorithm to run model.predict() on a trained model. See GymTrainedAlgorithm for this case.

Algorithm class for Simulations using a reinforcement learning agent that operates in an Open AI Gym environment.

Implements abstract class GymBaseAlgorithm.

Simulations that run GymAlgorithm-style schedulers have two entry points for simulation control. First, the Simulator may call scheduler.run(), causing this GymAlgorithm to run model.predict(). Alternatively, one may call model.learn(vec_env), which instead will step through the simulation. See GymTrainingAlgorithm for this case.

Args:

max_recompute (int): See BaseAlgorithm.

_env :Optional[BaseSimEnv]
max_recompute :Optional[int]
__deepcopy__(self, memodict: Optional[Dict] = None) → gym_acnportal.algorithms.gym_algorithm.GymBaseAlgorithm
register_interface(self, interface: acnportal.acnsim.Interface) → None

NOTE: Registering an interface sets the environment’s interface to GymTrainedInterface.

property env(self) → gym_acnportal.gym_acnsim.envs.BaseSimEnv

Return the algorithm’s gym environment.

Returns:

BaseSimEnv: A gym environment that wraps a simulation.

Raises:
ValueError: Exception raised if vec_env is accessed prior to

an vec_env being registered.

register_env(self, env: gym_acnportal.gym_acnsim.envs.BaseSimEnv) → None

Register a model that outputs schedules for the simulation.

Args:

env (BaseSimEnv): An vec_env wrapping a simulation.

Returns:

None

abstract schedule(self, active_evs) → Dict[str, List[float]]

NOT IMPLEMENTED IN GymBaseAlgorithm.

class gym_acnportal.algorithms.GymTrainedAlgorithm(max_recompute: int = 1)

Bases: gym_acnportal.algorithms.gym_algorithm.GymBaseAlgorithm

Algorithm class for Simulations using a reinforcement learning agent that operates in an Open AI Gym environment.

Implements abstract class GymBaseAlgorithm.

Simulations that run GymAlgorithm-style schedulers have two entry points for simulation control. First, the Simulator may call scheduler.run(), causing this GymAlgorithm to run model.predict(). Alternatively, one may call model.learn(vec_env), which instead will step through the simulation. See GymTrainingAlgorithm for this case.

_env :BaseSimEnv
_model :Optional[SimRLModelWrapper]
property model(self) → gym_acnportal.algorithms.gym_algorithm.SimRLModelWrapper

Return the algorithm’s predictive model.

Returns:
SimRLModelWrapper: A predictive model that returns an array

of actions given an environment wrapping a simulation.

Raises:
ValueError: Exception raised if model is accessed prior to

a model being registered.

register_model(self, model: gym_acnportal.algorithms.gym_algorithm.SimRLModelWrapper) → None

Register a model that outputs schedules for the simulation.

Args:
model (SimRLModelWrapper): A model that can be used for

predictions in ACN-Sim.

Returns:

None

schedule(self, active_evs: List[EV]) → Dict[str, List[float]]

Creates a schedule of charging rates for each EVSE in the network. This only works if a model and environment have been registered.

Implements BaseAlgorithm.schedule().

class gym_acnportal.algorithms.SimRLModelWrapper(model: object = None)

Abstract wrapper class that wraps a reinforcement learning agent for general use with ACN-Sim. Users should define a new class that implements the predict method.

model :object
abstract predict(self, observation: object, reward: float, done: bool, info: Dict[Any, Any] = None) → numpy.ndarray

Given an observation, reward, done, and info from an environment, return a prediction for the next optimal action as a numpy array.

Args:

observation: An observation of the environment. reward: The last reward returned by the environment. done: If true, this environment’s simulation is done

running.

info: Info for debugging and testing from this environment.

In a deployment model, this information is restricted.

Returns: