Rules of the Game

This page is organized as follow:

Objectives

The Rules module define what is “Legal” and what is not. For example, it can be usefull, at the beginning of the training of an grid2op.Agent.BaseAgent to loosen the rules in order to ease the learning process, and have the agent focusing more on the physics. When the agent is performing well enough, it is then possible to make the rules more and more complex up to the target complexity.

Rules includes:

  • checking the number of powerline that can be connected / disconnected at a given time step

  • checking the number of substations for which the topology can be reconfigured at a given timestep

If an action “break the rules” it is replaced by a do nothing. Note that the Rules of the game is different from the concept of Ambiguous Action.

Behaviour

In this section, we detail the behaviour of the “cooldown” on the line and substation to make sure it’s clear in the documentation.

The general rules is pretty simple:

  1. if obs.time_before_cooldown_line[l_id] > 0 for a given line id l_id, any attempt to modify the status of the powerline l_id will be illegal.

  2. if obs.time_before_cooldown_line[l_id] == 0 for a given line id l_id, it is possible to modify the status of powerline l_id without any issue. And at the next step, you will need to wait for env.parameters.NB_TIMESTEP_COOLDOWN_LINE steps before being able to modify the status of the powerline l_id

  3. if obs.time_before_cooldown_sub[s_id] > 0 for a given substation id s_id, any attempt to modify the topology of the substation s_id will be illegal.

  4. if obs.time_before_cooldown_sub[s_id] == 0 for a given substation id s_id, it is possible to modify the topology of substation s_id without any issue. And at the next step, you will need to wait for env.parameters.NB_TIMESTEP_COOLDOWN_SUB steps before being able to modify the topology of the substation s_id

To illustrate this, let’s create an environment that we’ll use for an example and force the cooldowns:

import grid2op
env_name = "l2rpn_case14_sandbox"
env = grid2op.make(env_name)
env.set_id(0)
env.seed(0)

# for this example, we enforce that we need to wait 3 steps
# before being able to change again a line or a substation
param = env.parameters
param.NB_TIMESTEP_COOLDOWN_SUB = 3
param.NB_TIMESTEP_COOLDOWN_LINE = 3
env.change_parameters(param)

obs = env.reset()

# in summary (see descriptions bellow for more information) :
act_line_1 = env.action_space({"set_line_status": [(1, -1)]})
obs, reward, done, info = env.step(act_line_1)  # legal, obs.time_before_cooldown_line[1] = 3
obs, reward, done, info = env.step(act_line_1)  # illegal, obs.time_before_cooldown_line[1] = 2
obs, reward, done, info = env.step(act_line_1)  # illegal, obs.time_before_cooldown_line[1] = 1
obs, reward, done, info = env.step(act_line_1)  # illegal, obs.time_before_cooldown_line[1] = 0
obs, reward, done, info = env.step(act_line_1)  # legal, obs.time_before_cooldown_line[1] = 3

Cooldown on substation

The general rule is:

  1. if obs.time_before_cooldown_sub[s_id] > 0 for a given substation id s_id, any attempt to modify the topology of the substation s_id will be illegal.

  2. if obs.time_before_cooldown_sub[s_id] == 0 for a given substation id s_id, it is possible to modify the topology of substation s_id without any issue. And at the next step, you will need to wait for env.parameters.NB_TIMESTEP_COOLDOWN_SUB steps before being able to modify the topology of the substation s_id

Now let’s illustrate this with an action on substation 1:

obs = env.reset()

# let's do an action
act_sub1 = env.action_space({"set_bus": {"substations_id": [(1, [1, 2, 2, 1, 1, 1])]}})
# and see what happens
obs, reward, done, info = env.step(act_sub1)

Now, you can see that:

obs.time_before_cooldown_sub
# >>> array([0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)

This is because in the parameters we specified param.NB_TIMESTEP_COOLDOWN_SUB = 3 and that we acted on substaiton 1. In this case, the substation 1 obs.time_before_cooldown_sub[1] is 3.

And as long as obs.time_before_cooldown_sub[1] > 0 you will not be able to act on this substation again, whether you do the same action or another.

# let's do another action
act_sub1_ = env.action_space({"set_bus": {"substations_id": [(1, [1, 2, 2, 1, 2, 1])]}})
# and see what happens
obs, reward, done, info = env.step(act_sub1_)
info['is_illegal']
# >>> True  => the action is illegal
info["exception"]
# >>> [Grid2OpException IllegalAction IllegalAction('Substation with ids [1] have been modified illegally (cooldown)')]
# the exception raised explains why

# at this stage we have to wait still 2 steps before doing an action
obs.time_before_cooldown_sub
# >>> array([0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)

You can do an action on another substation without any issue:

# we can do an action on another substation if we want to
act_sub2 = env.action_space({"set_bus": {"substations_id": [(8, [2, 2, 1, 1, 1])]}})
obs, reward, done, info = env.step(act_sub2)
info['is_illegal']
# >>> False  => the action is perfectly legal
info["exception"]
# >>> []
obs.time_before_cooldown_sub
# >>> array([0, 1, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0], dtype=int32)

At this stage you still cannot do an action on substation 1:

# we can do an action on another substation if we want to
obs, reward, done, info = env.step(act_sub1)
info['is_illegal']
# >>> True  => the action is illegal
info["exception"]
# >>> [Grid2OpException IllegalAction IllegalAction('Substation with ids [1] have been modified illegally (cooldown)')]
# the exception raised explains why

obs.time_before_cooldown_sub
# >>> array([0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0], dtype=int32)

And now you can (after 3 steps, and “because” obs.time_before_cooldown_sub[1] == 0) you can do an action on the substation 1, as shown bellow:

# we can do an action on another substation if we want to
obs, reward, done, info = env.step(act_sub1_)
info['is_illegal']
# >>> False  => the action is perfectly legal, you have waited 3 steps before being able to do an action on sub 1 again
info["exception"]
# >>> []

obs.time_before_cooldown_sub
# >>> array([0, 3, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0], dtype=int32)

Cooldown on powerlines

Actions on powerline status behaves exactly the same way as actions on substation:

  1. if obs.time_before_cooldown_line[l_id] > 0 for a given line id l_id, any attempt to modify the status of the powerline l_id will be illegal.

  2. if obs.time_before_cooldown_line[l_id] == 0 for a given line id l_id, it is possible to modify the status of powerline l_id without any issue. And at the next step, you will need to wait for env.parameters.NB_TIMESTEP_COOLDOWN_LINE steps before being able to modify the status of the powerline l_id

We invite you to read the paragraph on the substation Cooldown on substation for more information.

Corner cases

There are some “corner cases”:

  1. you can change, at any step a powerline and a line status (for the general rules)

  2. You can modify the status of powerline by modifying the topology, see Note on powerline status for such examples. The cooldown are applied with respect to the type of the action decided by grid2op, and not the way the action are given.

Substation and powerline status

With the default rules, you can change both a powerline status and a substation at the same step. For example, you can change topology of substation 1 AND disconnect powerline 10, as shown bellow:

obs = env.reset()

# let's do an action
act = env.action_space({"set_bus": {"substations_id": [(1, [1, 2, 2, 1, 1, 1])]},
                        "set_line_status": [(10, -1)]}
                      )
# and see what happens
obs, reward, done, info = env.step(act)

info['is_illegal']
# >>> False  => the action is perfectly legal, you have waited 3 steps before being able
#               to do an action on sub 1 or line 9 again
info["exception"]
# >>> []

obs.time_before_cooldown_sub
# >>> array([0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)
obs.time_before_cooldown_line
# >>> array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)

Actions can have “side effect”

We explained in section Note on powerline status of the documentation that you can perform action on powerlines by specifying topology, for example:

obs = env.reset()

# let's do an action
act = env.action_space({"set_bus": {"substations_id": [(1, [1, 2, 2, -1, 1, 1])]}}
                      )
# and see what happens
obs, reward, done, info = env.step(act)

info['is_illegal']
# >>> False  => the action is perfectly legal, you have waited 3 steps before being able
#               to do an action on sub 1 or line 4 again
info["exception"]
# >>> []

obs.time_before_cooldown_sub
# >>> array([0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)
obs.time_before_cooldown_line
# >>> array([0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)

This action disconnects powerline 4 AND change the action of substation 1 at the same time (this is because one of the side of line 4 is connected to substation 1, and it’s the -1 provided in the example). Now let’s try to do an action on substation 4 (the other side of line 4 is connected at this substation):

# let's do an action
act_sub4 = env.action_space({"set_bus": {"substations_id": [(4, [2, 2, 2, 1, 1] )]}})

# and see what happens
obs, reward, done, info = env.step(act_sub4)

info['is_illegal']
# >>> True
info["exception"]
# >>> [Grid2OpException IllegalAction IllegalAction('Powerline with ids [4] have been modified illegally (cooldown)')]

obs.time_before_cooldown_sub
# >>> array([0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)
obs.time_before_cooldown_line
# >>> array([0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)

This action is illegal. In fact, if it were made on the grid, powerline 4 would be connected, but the rules prevent its reconnection: there is a cooldown on powerline 4 (you still have to wait 3 steps before reconnecting it).

There are multiple ways to overcome this issue:

  • modify the action “a posteriori” to “leave powerline 4 disconnected” (post processing)

  • know which elements not to modify, and leave it “alone” (pre processing)

For example, we show how to modify the code to assign 0 (meaning “I don’t change” in grid2op framework) to the lines that are already disconnected (so the lines such that obs.line_status is False)

import numpy as np
obs = env.reset()
act = env.action_space({"set_bus": {"substations_id": [(1, [1, 2, 2, -1, 1, 1])]}})
obs, reward, done, info = env.step(act)

act_sub4_clean = env.action_space({"set_bus": {"substations_id": [(4, [2, 2, 2, 1, 1])]}})
## post processing
act_sub4_clean.remove_line_status_from_topo()  # grid2op >= 1.8.0
# act_sub4_clean.line_or_set_bus = [(l_id, 0) for l_id in np.arange(obs.n_line) if not obs.line_status[l_id]]
# act_sub4_clean.line_ex_set_bus = [(l_id, 0) for l_id in np.arange(obs.n_line) if not obs.line_status[l_id]]
## continue as usual
obs, reward, done, info = env.step(act_sub4_clean)

info['is_illegal']
# >>> False
info["exception"]
# >>> []

obs.time_before_cooldown_sub
# >>> array([0, 2, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)
obs.time_before_cooldown_line
# >>> array([0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)

And as you can see, the action could be effectively made on the grid. Powerline 4 is not reconnected, the cooldown decreased for powerline 4 and substation 1,

Alternatively, you could have specified your action like: act_sub4_clean = env.action_space({“set_bus”: {“substations_id”: [(4, [2, 0, 2, 1, 1])]}}) (which is basically what the code above did) but it requires some manipulation to know that the end side of powerline 4 is on substation 4 (obs.line_ex_to_subid[4] == 4), and then that this element is the second component of this substation (obs.line_ex_to_sub_pos[4] == 1). We do not recommend using this method.

Note

As for grid2op 1.8.0 we added the function act.remove_line_status_from_topo(obs) that does this job

Behaviour of the obs.simulate function

TODO

Detailed Documentation by class

Classes:

AlwaysLegal()

This subclass doesn't implement any rules regarding the legality of the actions.

BaseRules()

This class is a base class that determines whether or not an action is legal in certain environment.

DefaultRules()

This subclass combine both LookParam and PreventReconnection.

LookParam()

This subclass only check that the number of powerlines reconnected / disconnected by the agent.

PreventDiscoStorageModif()

This subclass only check that the action do not modify the storage power (charge / discharge) of a disconnected storage unit.

PreventReconnection()

A subclass is used to check that an action will not attempt to reconnect a powerlines disconnected because of an overflow, or to check that 2 actions acting on the same powerline are distant from the right number of timesteps (see grid2op.Parameters.Parameters.NB_TIMESTEP_LINE_STATUS_REMODIF) or if two topological modification of the same substation are too close in time (see grid2op.Parameters.Parameters.NB_TIMESTEP_TOPOLOGY_REMODIF)

RulesByArea(areas_list)

This subclass combine PreventReconnection, :class: PreventDiscoStorageModif to be applied on the whole grid at once, while a specifique method look for the legality of simultaneous actions taken on defined areas of a grid.

RulesChecker([legalActClass])

Class that define the rules of the game.

class grid2op.Rules.AlwaysLegal[source]

This subclass doesn’t implement any rules regarding the legality of the actions. All actions are legal.

Methods:

__call__(action, env)

All actions being legal, this returns always true.

__call__(action, env)[source]

All actions being legal, this returns always true. See BaseRules.__call__() for a definition of the parameters of this function.

class grid2op.Rules.BaseRules[source]

This class is a base class that determines whether or not an action is legal in certain environment. See the definition of BaseRules.__call__() for more information.

Basically, this is an empty class with an overload of the __call__ operator that should return True or False depending on the legality of the action.

In grid2op.Environment, only action of the users are checked for legality.

Methods:

__call__(action, env)

As opposed to "ambiguous action", "illegal action" are not illegal per se.

can_use_simulate(nb_simulate_call_step, ...)

This function can be overriden.

initialize(env)

This function is used to inform the class instance about the environment specification.

Attributes:

__weakref__

list of weak references to the object (if defined)

abstractmethod __call__(action, env)[source]

As opposed to “ambiguous action”, “illegal action” are not illegal per se. They are legal or not on a certain environment. For example, disconnecting a powerline that has been cut off for maintenance is illegal. Saying to action to both disconnect a powerline and assign it to bus 2 on it’s origin side is ambiguous, and not tolerated in Grid2Op.

Parameters:
  • action (grid2op.Action.Action) – The action of which the legality is tested.

  • env (grid2op.Environment.Environment) – The environment on which the action is performed.

Returns:

  • is_legal (bool) – Whether the action is legal or not

  • reason – The cause of the illegal part of the action (should be a grid2op exception)

__weakref__

list of weak references to the object (if defined)

can_use_simulate(nb_simulate_call_step, nb_simulate_call_episode, param)[source]

This function can be overriden.

It is expected to return either SimulateUsedTooMuchThisStep or SimulateUsedTooMuchThisEpisode if the number of calls to obs.simulate is too high in total or for the given step

initialize(env)[source]

This function is used to inform the class instance about the environment specification. It can be the place to assert the defined rules are suited for the environement. :param env: The environment on which the action is performed. The environement instance is not fully initialized itself. :type env: grid2op.Environment.Environment

class grid2op.Rules.DefaultRules[source]

This subclass combine both LookParam and PreventReconnection. An action is declared legal if and only if:

  • It doesn’t disconnect / reconnect more power lines than what stated in the actual game _parameters grid2op.Parameters

  • It doesn’t attempt to act on more substations that what is stated in the actual game _parameters grid2op.Parameters

  • It doesn’t attempt to modify the power produce by a turned off storage unit

Methods:

__call__(action, env)

See BaseRules.__call__() for a definition of the _parameters of this function.

can_use_simulate(nb_simulate_call_step, ...)

This function can be overriden.

__call__(action, env)[source]

See BaseRules.__call__() for a definition of the _parameters of this function.

..versionchanged:: 1.10.2

In grid2op 1.10.2 this function is not called when the environment is reset: The “action” made by the environment to set the environment in the desired state is always legal

can_use_simulate(nb_simulate_call_step, nb_simulate_call_episode, param)[source]

This function can be overriden.

It is expected to return either SimulateUsedTooMuchThisStep or SimulateUsedTooMuchThisEpisode if the number of calls to obs.simulate is too high in total or for the given step

class grid2op.Rules.LookParam[source]

This subclass only check that the number of powerlines reconnected / disconnected by the agent.

This class doesn’t require any environment information. The “env” argument is only used to look for the game rules implemented in grid2op.Parameters.

See BaseRules.__call__() for a definition of the parameters of this function.

Methods:

__call__(action, env)

See BaseRules.__call__() for a definition of the parameters of this function.

can_use_simulate(nb_simulate_call_step, ...)

This function can be overriden.

__call__(action, env)[source]

See BaseRules.__call__() for a definition of the parameters of this function.

..versionchanged:: 1.10.2

In grid2op 1.10.2 this function is not called when the environment is reset: The “action” made by the environment to set the environment in the desired state is always legal

can_use_simulate(nb_simulate_call_step, nb_simulate_call_episode, param)[source]

This function can be overriden.

It is expected to return either SimulateUsedTooMuchThisStep or SimulateUsedTooMuchThisEpisode if the number of calls to obs.simulate is too high in total or for the given step

class grid2op.Rules.PreventDiscoStorageModif[source]

This subclass only check that the action do not modify the storage power (charge / discharge) of a disconnected storage unit.

See BaseRules.__call__() for a definition of the parameters of this function.

Methods:

__call__(action, env)

See BaseRules.__call__() for a definition of the parameters of this function.

__call__(action, env)[source]

See BaseRules.__call__() for a definition of the parameters of this function.

..versionchanged:: 1.10.2

In grid2op 1.10.2 this function is not called when the environment is reset: The “action” made by the environment to set the environment in the desired state is always legal

class grid2op.Rules.PreventReconnection[source]

A subclass is used to check that an action will not attempt to reconnect a powerlines disconnected because of an overflow, or to check that 2 actions acting on the same powerline are distant from the right number of timesteps (see grid2op.Parameters.Parameters.NB_TIMESTEP_LINE_STATUS_REMODIF) or if two topological modification of the same substation are too close in time (see grid2op.Parameters.Parameters.NB_TIMESTEP_TOPOLOGY_REMODIF)

Methods:

__call__(action, env)

This function check only that the action doesn't attempt to reconnect a powerline that has been disconnected due to an overflow.

__call__(action, env)[source]

This function check only that the action doesn’t attempt to reconnect a powerline that has been disconnected due to an overflow.

See BaseRules.__call__() for a definition of the parameters of this function.

..versionchanged:: 1.10.2

In grid2op 1.10.2 this function is not called when the environment is reset: The “action” made by the environment to set the environment in the desired state is always legal

class grid2op.Rules.RulesByArea(areas_list)[source]

This subclass combine PreventReconnection, :class: PreventDiscoStorageModif to be applied on the whole grid at once, while a specifique method look for the legality of simultaneous actions taken on defined areas of a grid. An action is declared legal if and only if:

  • It doesn’t reconnect more power lines than what is stated in the actual game _parameters grid2op.Parameters

  • It doesn’t attempt to act on more substations and lines within each area that what is stated in the actual game _parameters grid2op.Parameters

  • It doesn’t attempt to modify the power produce by a turned off storage unit

Example

If you want the environment to take into account the rules by area, you can achieve it with:

import grid2op
from grid2op.Rules.rulesByArea import RulesByArea

# First you set up the areas within the RulesByArea class
my_gamerules_byarea = RulesByArea([[0,1,2,3,4,5,6,7],[8,9,10,11,12,13,14]])
# Then you create your environment with it:
NAME_OF_THE_ENVIRONMENT = "l2rpn_case14_sandbox"
env = grid2op.make(NAME_OF_THE_ENVIRONMENT,gamerules_class=my_gamerules_byarea)

Methods:

__call__(action, env)

See BaseRules.__call__() for a definition of the _parameters of this function.

__init__(areas_list)

The initialization of the rule with a list of list of ids of substations composing the aimed areas.

_lookparam_byarea(action, env)

See BaseRules.__call__() for a definition of the parameters of this function.

can_use_simulate(nb_simulate_call_step, ...)

This function can be overriden.

initialize(env)

This function is used to inform the class instance about the environment specification and check no substation of the grid are left ouside an area.

__call__(action, env)[source]

See BaseRules.__call__() for a definition of the _parameters of this function.

..versionchanged:: 1.10.2

In grid2op 1.10.2 this function is not called when the environment is reset: The “action” made by the environment to set the environment in the desired state is always legal

__init__(areas_list)[source]

The initialization of the rule with a list of list of ids of substations composing the aimed areas. :param areas_list: :type areas_list: list of areas, each placeholder containing the ids of substations of each defined area

_lookparam_byarea(action, env)[source]

See BaseRules.__call__() for a definition of the parameters of this function.

can_use_simulate(nb_simulate_call_step, nb_simulate_call_episode, param)[source]

This function can be overriden.

It is expected to return either SimulateUsedTooMuchThisStep or SimulateUsedTooMuchThisEpisode if the number of calls to obs.simulate is too high in total or for the given step

initialize(env)[source]

This function is used to inform the class instance about the environment specification and check no substation of the grid are left ouside an area. :param env: An environment instance properly initialized. :type env: grid2op.Environment.Environment

class grid2op.Rules.RulesChecker(legalActClass=<class 'grid2op.Rules.AlwaysLegal.AlwaysLegal'>)[source]

Class that define the rules of the game.

Methods:

__call__(action, env)

Says if an action is legal or not.

__init__([legalActClass])

param legalActClass:

The class that will be used to tell if the actions are legal or not. The class must be given, and not

initialize(env)

This function is used to inform the class instance about the environment specification.

Attributes:

__weakref__

list of weak references to the object (if defined)

__call__(action, env)[source]

Says if an action is legal or not.

Parameters:
Returns:

  • is_legal (bool) – Assess if the given action is legal or not. True: the action is legal, False otherwise

  • reason – A grid2op IllegalException given the reason for which the action is illegal

__init__(legalActClass=<class 'grid2op.Rules.AlwaysLegal.AlwaysLegal'>)[source]
Parameters:

legalActClass (type) – The class that will be used to tell if the actions are legal or not. The class must be given, and not an object of this class. It should derived from BaseRules.

__weakref__

list of weak references to the object (if defined)

initialize(env)[source]

This function is used to inform the class instance about the environment specification. It can be the place to assert the defined rules are suited for the environement. :param env: The environment on which the action is performed. :type env: grid2op.Environment.Environment

If you still can’t find what you’re looking for, try in one of the following pages:

Still trouble finding the information ? Do not hesitate to send a github issue about the documentation at this link: Documentation issue template