Utilities

Logger

class genrl.utils.logger.CSVLogger(logdir: str)

Bases: object

CSV Logging class

Parameters:logdir (string) – Directory to save log at

close() → None

Close the logger

write(kvs: Dict[str, Any], log_key) → None

Add entry to logger

Parameters:kvs (dict) – Entries to be logged

class genrl.utils.logger.HumanOutputFormat(logdir: str)

Bases: object

Output from a log file in a human readable format

Parameters:logdir (string) – Directory at which log is present

close() → None

max_key_len(kvs: Dict[str, Any]) → None

Finds max key length

Parameters:kvs (dict) – Entries to be logged

round(num: float) → float

Returns a rounded float value depending on self.maxlen

Parameters:num (float) – Value to round

write(kvs: Dict[str, Any], log_key) → None

Log the entry out in human readable format

Parameters:kvs (dict) – Entries to be logged

write_to_file(kvs: Dict[str, Any], file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>) → None

Log the entry out in human readable format

Parameters:

• kvs (dict) – Entries to be logged
• file (io.TextIOWrapper) – Name of file to write logs to

class genrl.utils.logger.Logger(logdir: str = None, formats: List[str] = ['csv'])

Bases: object

Logger class to log important information

Parameters:

• logdir (string) – Directory to save log at
• formats (list) – Formatting of each log [‘csv’, ‘stdout’, ‘tensorboard’]

close() → None

Close the logger

formats

Return save format(s)

logdir

Return log directory

write(kvs: Dict[str, Any], log_key: str = 'timestep') → None

Add entry to logger

Parameters:

• kvs (dict) – Entry to be logged
• log_key (str) – Key plotted on log_key

class genrl.utils.logger.TensorboardLogger(logdir: str)

Bases: object

Tensorboard Logging class

Parameters:logdir (string) – Directory to save log at

close() → None

Close the logger

write(kvs: Dict[str, Any], log_key: str = 'timestep') → None

Add entry to logger

Parameters:

• kvs (dict) – Entries to be logged
• log_key (str) – Key plotted on x_axis

genrl.utils.logger.get_logger_by_name(name: str)

Gets the logger given the type of logger

Parameters:name (string) – Name of the value function needed Returns:Logger

Utilities

genrl.utils.utils.cnn(channels: Tuple = (4, 16, 32), kernel_sizes: Tuple = (8, 4), strides: Tuple = (4, 2), **kwargs) → Tuple

(Generates a CNN model given input dimensions, channels, kernel_sizes and

strides)

param channels:Input output channels before and after each convolution param kernel_sizes:  Kernel sizes for each convolution param strides:Strides for each convolution param in_size:Input dimensions (assuming square input) type channels:tuple type kernel_sizes:  tuple type strides:tuple type in_size:int returns:(Convolutional Neural Network with convolutional layers and

activation layers)

genrl.utils.utils.get_env_properties(env: Union[gym.core.Env, genrl.environments.vec_env.vector_envs.VecEnv], network: Union[str, Any] = 'mlp') → Tuple[int]

Finds important properties of environment

param env:Environment that the agent is interacting with type env:Gym Environment param network:Type of network architecture, eg. “mlp”, “cnn” type network:str returns:(State space dimensions, Action space dimensions,

discreteness of action space and action limit (highest action value)

rtype:int, float, …; int, float, …; bool; int, float, …

genrl.utils.utils.get_model(type_: str, name_: str) → Union

Utility to get the class of required function

param type_:“ac” for Actor Critic, “v” for Value, “p” for Policy param name_:Name of the specific structure of model. (

Eg. “mlp” or “cnn”)

type type_:string returns:Required class. Eg. MlpActorCritic

genrl.utils.utils.mlp(sizes: Tuple, activation: str = 'relu', sac: bool = False)

Generates an MLP model given sizes of each layer

param sizes:Sizes of hidden layers param sac:True if Soft Actor Critic is being used, else False type sizes:tuple or list type sac:bool returns:(Neural Network with fully-connected linear layers and

activation layers)

genrl.utils.utils.noisy_mlp(fc_layers: List[int], noisy_layers: List[int], activation='relu')

Noisy MLP generating helper function

Parameters:

• fc_layers (list of int) – List of fully connected layers
• noisy_layers (list of int) – :ist of noisy layers
• activation (str) – Activation function to be used. [“tanh”, “relu”]

Returns:

Noisy MLP model

genrl.utils.utils.safe_mean(log: Union[torch.Tensor, List[int]])

Returns 0 if there are no elements in logs

genrl.utils.utils.set_seeds(seed: int, env: Union[gym.core.Env, genrl.environments.vec_env.vector_envs.VecEnv] = None) → None

Sets seeds for reproducibility

Parameters:

• seed (int) – Seed Value
• env (Gym Environment) – Optionally pass gym environment to set its seed

Models

class genrl.utils.models.TabularModel(s_dim: int, a_dim: int)

Bases: object

Sample-based tabular model class for deterministic, discrete environments

Parameters:

• s_dim (int) – environment state dimension
• a_dim (int) – environment action dimension

add(state: numpy.ndarray, action: numpy.ndarray, reward: float, next_state: numpy.ndarray) → None

add transition to model :param state: state :param action: action :param reward: reward :param next_state: next state :type state: float array :type action: int :type reward: int :type next_state: float array

is_empty() → bool

Check if the model has been updated or not

Returns:True if model not updated yet Return type:bool

sample() → Tuple

sample state action pair from model

Returns:state and action Return type:int, float, .. ; int, float, ..

step(state: numpy.ndarray, action: numpy.ndarray) → Tuple

return consequence of action at state

Returns:reward and next state Return type:int; int, float, ..

genrl.utils.models.get_model_from_name(name_: str)

get model object from name

Parameters:name (str) – name of the model [‘tabular’] Returns:the model