Opponent Modeling ¶

This page is organized as follow:

Objectives ¶

Power systems are a really important tool today, that can be as resilient as possible to avoid possibly dramatic consequences.

In grid2op, we chose to enforce this property by implementing an “Opponent” (modeled thanks to the BaseOpponent that can take malicious actions to deteriorate the state of the powergrid and make tha Agent (grid2op.Agent) fail. To make the agent “game over” is really easy (for example it could isolate a load by forcing the disconnection of all the powerline that powers it). This would not be fair, and that is why the Opponent has some dedicated budget (modeled with the BaseActionBudget).

The class OpponentSpace has the delicate role to: - send the necessary information for the Opponent to attack properly. - make sure the attack performed by the opponent is legal - compute the cost of such attack - make sure this cost is not too high for the opponent budget.

How to create an opponent in any environment ¶

This section is a work in progress, it will only cover how to set up one type of opponent, and supposes that you already know which lines you want to attack, at which frequency etc.

More detailed information about the opponent will be provide in the future.

The set up for the opponent in the “l2rpn_neurips_track1” has the following configuration.

lines_attacked = ["62_58_180", "62_63_160", "48_50_136", "48_53_141", "41_48_131", "39_41_121",
              "43_44_125", "44_45_126", "34_35_110", "54_58_154"]
rho_normalization = [0.45, 0.45, 0.6, 0.35, 0.3, 0.2,
                     0.55, 0.3, 0.45, 0.55]
opponent_attack_cooldown = 12*24  # 24 hours, 1 hour being 12 time steps
opponent_attack_duration = 12*4  # 4 hours
opponent_budget_per_ts = 0.16667  # opponent_attack_duration / opponent_attack_cooldown + epsilon
opponent_init_budget = 144.  # no need to attack straightfully, it can attack starting at midday the first day
config = {
    "opponent_attack_cooldown": opponent_attack_cooldown,
    "opponent_attack_duration": opponent_attack_duration,
    "opponent_budget_per_ts": opponent_budget_per_ts,
    "opponent_init_budget": opponent_init_budget,
    "opponent_action_class": PowerlineSetAction,
    "opponent_class": WeightedRandomOpponent,
    "opponent_budget_class": BaseActionBudget,
    'kwargs_opponent': {"lines_attacked": lines_attacked,
                        "rho_normalization": rho_normalization,
                        "attack_period": opponent_attack_cooldown}
}

To create the same type of opponent on the case14 grid you can do:

import grid2op
from grid2op.Action import PowerlineSetAction
from grid2op.Opponent import RandomLineOpponent, BaseActionBudget
env_name = "l2rpn_case14_sandbox"

env_with_opponent = grid2op.make(env_name,
                                 opponent_attack_cooldown=12*24,
                                 opponent_attack_duration=12*4,
                                 opponent_budget_per_ts=0.5,
                                 opponent_init_budget=0.,
                                 opponent_action_class=PowerlineSetAction,
                                 opponent_class=RandomLineOpponent,
                                 opponent_budget_class=BaseActionBudget,
                                 kwargs_opponent={"lines_attacked":
                                      ["1_3_3", "1_4_4", "3_6_15", "9_10_12", "11_12_13", "12_13_14"]}
                                 )
# and now you have an opponent on the l2rpn_case14_sandbox
# you can for example
obs = env_with_opponent.reset()

act = ...  # chose an action here
obs, reward, done, info = env_with_opponent.step(act)

And for the track2 of neurips, if you want to make it even more complicated, you can add an opponent in the same fashion:

import grid2op
from grid2op.Action import PowerlineSetAction
from grid2op.Opponent import RandomLineOpponent, BaseActionBudget
env_name = "l2rpn_neurips_2020_track2_small"

env_with_opponent = grid2op.make(env_name,
                                 opponent_attack_cooldown=12*24,
                                 opponent_attack_duration=12*4,
                                 opponent_budget_per_ts=0.5,
                                 opponent_init_budget=0.,
                                 opponent_action_class=PowerlineSetAction,
                                 opponent_class=RandomLineOpponent,
                                 opponent_budget_class=BaseActionBudget,
                                 kwargs_opponent={"lines_attacked":
                                                     ["26_31_106",
                                                      "21_22_93",
                                                      "17_18_88",
                                                      "4_10_162",
                                                      "12_14_68",
                                                      "14_32_108",
                                                      "62_58_180",
                                                      "62_63_160",
                                                      "48_50_136",
                                                      "48_53_141",
                                                      "41_48_131",
                                                      "39_41_121",
                                                      "43_44_125",
                                                      "44_45_126",
                                                      "34_35_110",
                                                      "54_58_154",
                                                      "74_117_81",
                                                      "80_79_175",
                                                      "93_95_43",
                                                      "88_91_33",
                                                      "91_92_37",
                                                      "99_105_62",
                                                      "102_104_61"]}
                                 )
# and now you have an opponent on the l2rpn_case14_sandbox
# you can for example
obs = env_with_opponent.reset()

act = ...  # chose an action here
obs, reward, done, info = env_with_opponent.step(act)

To summarize what is going on here:

opponent_attack_cooldown: give the minimum number of time between two attacks (here 1 attack per day)
opponent_attack_duration: duration for each attack (when a line is attacked, it will not be possible to reconnect it for that many steps). In the example it’s 4h (so 48 steps)
opponent_action_class: type of the action the opponent will perform (in this case PowerlineSetAction)
opponent_class: type of the opponent. Change it at your own risk.
opponent_budget_class: Each attack will cost some budget to the opponent. If no budget, the opponent cannot attack. This specifies how the budget are computed. Do not change it.
opponent_budget_per_ts: increase of the budget of the opponent per step. The higher this number, the faster the the opponent will regenerate its budget.
opponent_init_budget: initial opponent budget. It is set to 0 to “give” the agent a bit of time before the opponent is triggered.
kwargs_opponent: additional information for the opponent. In this case we provide for each grid the powerline it can attack.

Note

This is only valid for the RandomLineOpponent that disconnect powerlines randomly (but not uniformly!). For other type of Opponent, we don’t provide any information in the documentation at this stage. Feel free to submit a github issue if this is an issue for you.

How to deactivate an opponent in an environment ¶

If you come accross an environment with an “opponent” already present but for some reasons you want to deactivate it, you can do this by customization the call to “grid2op.make” like this:

import grid2op
from grid2op.Action import DontAct
from grid2op.Opponent import BaseOpponent, NeverAttackBudget
env_name = ...


# if you want to disable the opponent you can do (grid2op >= 1.9.4)
kwargs_no_opp = grid2op.Opponent.get_kwargs_no_opponent()
env_no_opp = grid2op.make(env_name, **kwargs_no_opp)
# and there the opponent is disabled

# or, in a more complex fashion (or for older grid2op version <= 1.9.3)
env_without_opponent = grid2op.make(env_name,
                                    opponent_attack_cooldown=999999,
                                    opponent_attack_duration=0,
                                    opponent_budget_per_ts=0,
                                    opponent_init_budget=0,
                                    opponent_action_class=DontAct,
                                    opponent_class=BaseOpponent,
                                    opponent_budget_class=NeverAttackBudget,
                                    ...  # other arguments pass to the "make" function
                                    )

Note

Currently it’s not possible to deactivate an opponent once the environment is created.

If you want this feature, you can comment the issue https://github.com/rte-france/Grid2Op/issues/426

Detailed Documentation by class ¶

Classes:

`BaseActionBudget`(action_space)	INTERNAL
`BaseOpponent`(action_space)
`FromEpisodeDataOpponent`(action_space)
`GeometricOpponent`(action_space)	This opponent will disconnect lines randomly among the attackable lines lines_attacked.
`GeometricOpponentMultiArea`(action_space)	This opponent is a combination of several similar opponents (of Kind Geometric Opponent at this stage) attacking on different areas.
`NeverAttackBudget`(action_space)	This class define an unlimited budget for the opponent.
`OpponentSpace`(compute_budget, init_budget, ...)	Is similar to the action space, but for the opponent.
`RandomLineOpponent`(action_space)	An opponent that disconnect at random any powerlines among a specified list given at the initialization.
`UnlimitedBudget`(action_space)	This class define an unlimited budget for the opponent.
`WeightedRandomOpponent`(action_space)	This opponent will disconnect lines randomly among the attackable lines lines_attacked.

class grid2op.Opponent.BaseActionBudget(action_space)[source]¶

INTERNAL

Warning

/!\ Internal, do not use unless you know what you are doing /!\

This is the base class representing the action budget. It makes sure the opponent uses the correct type of “action”, and compute the bugdet associated to it.

class grid2op.Opponent.BaseOpponent(action_space)[source]¶

Methods:

`attack`(observation, agent_action, ...)	This method is the equivalent of "act" for a regular agent.
`get_state`()	This function should return the internal state of the Opponent.
`init`(partial_env, **kwargs)	Generic function used to initialize the derived classes.
`reset`(initial_budget)	This function is called at the end of an episode, when the episode is over.
`set_state`(my_state)	This function is used to set the internal state of the Opponent.
`tell_attack_continues`(observation, ...)	The purpose of this method is to tell the agent that his attack is being continued and to indicate the current state of the grid.

attack(observation, agent_action, env_action, budget, previous_fails)[source]¶

This method is the equivalent of “act” for a regular agent.

Opponent, in this framework can have more information than a regular agent (in particular it can view time step t+1), it has access to its current budget etc.

Parameters:

observation (grid2op.Observation.Observation) – The last observation (at time t)
opp_reward (float) – THe opponent “reward” (equivalent to the agent reward, but for the opponent) TODO do i add it back ???
done (bool) – Whether the game ended or not TODO do i add it back ???
agent_action (grid2op.Action.Action) – The action that the agent took
env_action (grid2op.Action.Action) – The modification that the environment will take.
budget (float) – The current remaining budget (if an action is above this budget, it will be replaced by a do nothing.
previous_fails (bool) – Wheter the previous attack failed (due to budget or ambiguous action)

Returns:

attack (grid2op.Action.Action) – The attack performed by the opponent. In this case, a do nothing, all the time.
duration (int) – The duration of the attack

get_state()[source]¶

This function should return the internal state of the Opponent.

This means that after a call to opponent.set_state(opponent.get_state()) the opponent should do the exact same things than without these calls.

init(partial_env, **kwargs)[source]¶: Generic function used to initialize the derived classes. For example, if an opponent reads from a file, the path where is the file is located should be pass with this method.

reset(initial_budget)[source]¶

This function is called at the end of an episode, when the episode is over. It aims at resetting the self and prepare it for a new episode.

Parameters:: initial_budget (float) – The initial budget the opponent has

set_state(my_state)[source]¶

This function is used to set the internal state of the Opponent.

Parameters:: my_state –

tell_attack_continues(observation, agent_action, env_action, budget)[source]¶

The purpose of this method is to tell the agent that his attack is being continued and to indicate the current state of the grid.

At every time step, either “attack” or “tell_acttack_continues” is called exactly once.

Parameters:

observation (grid2op.Observation.Observation) – The last observation (at time t)
agent_action (grid2op.Action.Action) – The action that the agent took
env_action (grid2op.Action.Action) – The modification that the environment will take.
budget (float) – The current remaining budget (if an action is above this budget, it will be replaced by a do nothing.

class grid2op.Opponent.FromEpisodeDataOpponent(action_space)[source]¶

Warning

This can only be used if your environment uses grid2op.Chronics.FromOneEpisodeData or XXX (from a list of episode data or directory) class otherwise it will NOT work.

New in version 1.9.4.

See also

grid2op.Chronics.FromOneEpisodeData

Examples

Provided that you stored some data in path_agent using a grid2op.Runner.Runner for example you can use this class with:

import grid2op
from grid2op.Chronics import FromOneEpisodeData
from grid2op.Opponent import FromEpisodeDataOpponent
from grid2op.Episode import EpisodeData

path_agent = ....  # same as above
env_name = .... # same as above

# path_agent is the path where data coming from a grid2op runner are stored
# NB it should come from a do nothing agent, or at least
# an agent that does not modify the injections (no redispatching, curtailment, storage)
li_episode = EpisodeData.list_episode(path_agent)
ep_data = li_episode[0]

env = grid2op.make(env_name,
                   chronics_class=FromOneEpisodeData,  # super important
                   data_feeding_kwargs={"ep_data": ep_data},  # super important
                   opponent_class=FromEpisodeDataOpponent,  # important
                   opponent_attack_cooldown=1,  # super important
              )
# ep_data can be either a tuple of 2 elements (like above)
# or a full path to a saved episode
# or directly an object of type EpisodeData

obs = env.reset()

# and now you can use "env" as any grid2op environment.

Parameters:: BaseOpponent (_type_) – _description_

Methods:

`attack`(observation, agent_action, ...)	This method is the equivalent of "act" for a regular agent.
`get_state`()	This function should return the internal state of the Opponent.
`init`(partial_env, **kwargs)	Generic function used to initialize the derived classes.
`reset`(initial_budget)	This function is called at the end of an episode, when the episode is over.
`set_state`(state)	This function is used to set the internal state of the Opponent.
`tell_attack_continues`(observation, ...)	The purpose of this method is to tell the agent that his attack is being continued and to indicate the current state of the grid.

attack(observation, agent_action, env_action, budget, previous_fails)[source]¶

This method is the equivalent of “act” for a regular agent.

Opponent, in this framework can have more information than a regular agent (in particular it can view time step t+1), it has access to its current budget etc.

Parameters:

observation (grid2op.Observation.Observation) – The last observation (at time t)
opp_reward (float) – THe opponent “reward” (equivalent to the agent reward, but for the opponent) TODO do i add it back ???
done (bool) – Whether the game ended or not TODO do i add it back ???
agent_action (grid2op.Action.Action) – The action that the agent took
env_action (grid2op.Action.Action) – The modification that the environment will take.
budget (float) – The current remaining budget (if an action is above this budget, it will be replaced by a do nothing.
previous_fails (bool) – Wheter the previous attack failed (due to budget or ambiguous action)

Returns:

attack (grid2op.Action.Action) – The attack performed by the opponent. In this case, a do nothing, all the time.
duration (int) – The duration of the attack

get_state()[source]¶

This function should return the internal state of the Opponent.

This means that after a call to opponent.set_state(opponent.get_state()) the opponent should do the exact same things than without these calls.

init(partial_env, **kwargs)[source]¶: Generic function used to initialize the derived classes. For example, if an opponent reads from a file, the path where is the file is located should be pass with this method.

reset(initial_budget)[source]¶

This function is called at the end of an episode, when the episode is over. It aims at resetting the self and prepare it for a new episode.

Parameters:: initial_budget (float) – The initial budget the opponent has

set_state(state)[source]¶

This function is used to set the internal state of the Opponent.

Parameters:: my_state –

tell_attack_continues(observation, agent_action, env_action, budget)[source]¶

The purpose of this method is to tell the agent that his attack is being continued and to indicate the current state of the grid.

At every time step, either “attack” or “tell_acttack_continues” is called exactly once.

Parameters:

observation (grid2op.Observation.Observation) – The last observation (at time t)
agent_action (grid2op.Action.Action) – The action that the agent took
env_action (grid2op.Action.Action) – The modification that the environment will take.
budget (float) – The current remaining budget (if an action is above this budget, it will be replaced by a do nothing.

class grid2op.Opponent.GeometricOpponent(action_space)[source]¶

This opponent will disconnect lines randomly among the attackable lines lines_attacked. The sampling is done according to the lines load factor (ratio <current going through the line> to <thermal limit of the line>) (see init for more details).

The time of the attack is sampled according to a geometric distribution

Methods:

`attack`(observation, agent_action, ...)	This method is the equivalent of "attack" for a regular agent.
`get_state`()	This function should return the internal state of the Opponent.
`init`(partial_env[, lines_attacked, ...])	Generic function used to initialize the derived classes.
`reset`(initial_budget)	This function is called at the end of an episode, when the episode is over.
`set_state`(my_state)	This function is used to set the internal state of the Opponent.
`tell_attack_continues`(observation, ...)	The purpose of this method is to tell the agent that his attack is being continued and to indicate the current state of the grid.

attack(observation, agent_action, env_action, budget, previous_fails)[source]¶

This method is the equivalent of “attack” for a regular agent. Opponent, in this framework can have more information than a regular agent (in particular it can view time step t+1), it has access to its current budget etc. :param observation: The last observation (at time t) :type observation: grid2op.Observation.Observation :param opp_reward: THe opponent “reward” (equivalent to the agent reward, but for the opponent) TODO do i add it back ??? :type opp_reward: float :param done: Whether the game ended or not TODO do i add it back ??? :type done: bool :param agent_action: The action that the agent took :type agent_action: grid2op.Action.Action :param env_action: The modification that the environment will take. :type env_action: grid2op.Action.Action :param budget: The current remaining budget (if an action is above this budget, it will be replaced by a do nothing. :type budget: float :param previous_fails: Wheter the previous attack failed (due to budget or ambiguous action) :type previous_fails: bool

Returns:

attack (grid2op.Action.Action) – The attack performed by the opponent. In this case, a do nothing, all the time.
duration (int) – The duration of the attack (if None then the attack will be made for the longest allowed time)

get_state()[source]¶

This function should return the internal state of the Opponent.

This means that after a call to opponent.set_state(opponent.get_state()) the opponent should do the exact same things than without these calls.

init(partial_env, lines_attacked=(), attack_every_xxx_hour=24, average_attack_duration_hour=4, minimum_attack_duration_hour=2, pmax_pmin_ratio=4, **kwargs)[source]¶

Generic function used to initialize the derived classes. For example, if an opponent reads from a file, the path where is the file is located should be pass with this method.

Parameters:

partial_env (grid2op Environment) – A pointer to the environment that initializes the opponent
lines_attacked (list) – The list of lines that the XPOpponent should be able to disconnect
attack_every_xxx_hour (float) –
Provide the average duration between two attacks. Note that this should be greater than average_attack_duration_hour as, for now, an agent can only do one consecutive attack. You should provide it in “number of hours” and not in “number of steps”

It is used to compute the attack_hazard_rate. Attacks time are sampled with a duration distribution. For this opponent, we use the simplest of these distributions : The geometric disribution https://en.wikipedia.org/wiki/Geometric_distribution (the discrete time counterpart of the exponential distribution). The attack_hazard_rate is the main parameter of this distribution. It can be seen as the (constant) probability of having an attack in the next step. It is also the inverse of the expectation of the time to an attack.
average_attack_duration_hour (float) –
Give, in number of hours, the average attack duration. This should be greater than recovery_minimum_duration_hour

Used to compute the recovery_rate: Recovery times are random or at least should have a random part. In our case, we will say that the recovery time is equal to a fixed time (safety procedure time) plus a random time (investigations and repair operations) sampled according to a geometric distribution
minimum_attack_duration_hour (int) – Minimum duration of an attack (give it in hour)
pmax_pmin_ratio (float) – Ratio between the probability of the most likely line to be disconnected and the least likely one.

reset(initial_budget)[source]¶

This function is called at the end of an episode, when the episode is over. It aims at resetting the self and prepare it for a new episode.

Parameters:: initial_budget (float) – The initial budget the opponent has

set_state(my_state)[source]¶

This function is used to set the internal state of the Opponent.

Parameters:: my_state –

tell_attack_continues(observation, agent_action, env_action, budget)[source]¶

The purpose of this method is to tell the agent that his attack is being continued and to indicate the current state of the grid.

At every time step, either “attack” or “tell_acttack_continues” is called exactly once.

Parameters:

observation (grid2op.Observation.Observation) – The last observation (at time t)
agent_action (grid2op.Action.Action) – The action that the agent took
env_action (grid2op.Action.Action) – The modification that the environment will take.
budget (float) – The current remaining budget (if an action is above this budget, it will be replaced by a do nothing.

class grid2op.Opponent.GeometricOpponentMultiArea(action_space)[source]¶

This opponent is a combination of several similar opponents (of Kind Geometric Opponent at this stage) attacking on different areas. The difference between unitary opponents is mainly the attackable lines (which belongs to different pre-identified areas

Methods:

`attack`(observation, agent_action, ...)	This method is the equivalent of "attack" for a regular agent.
`get_state`()	This function should return the internal state of the Opponent.
`init`(partial_env[, lines_attacked, ...])	Generic function used to initialize the derived classes.
`reset`(initial_budget)	This function is called at the end of an episode, when the episode is over.
`seed`(seed)	INTERNAL
`set_state`(my_state)	This function is used to set the internal state of the Opponent.
`tell_attack_continues`(observation, ...)	The purpose of this method is to tell the agent that his attack is being continued and to indicate the current state of the grid.

attack(observation, agent_action, env_action, budget, previous_fails)[source]¶

This method is the equivalent of “attack” for a regular agent. Opponent, in this framework can have more information than a regular agent (in particular it can view time step t+1), it has access to its current budget etc. Here we take the combination of unitary opponent attacks if they happen at the same time. We choose the attack duration as the minimum duration of several simultaneous attacks if that happen.

Parameters:

observation (grid2op.Observation.Observation) – see the GeometricOpponent::attack documentation
opp_reward (float) – see the GeometricOpponent::attack documentation
done (bool) – see the GeometricOpponent::attack documentation
agent_action (grid2op.Action.Action) – see the GeometricOpponent::attack documentation
env_action (grid2op.Action.Action) – see the GeometricOpponent::attack documentation
budget (float) – see the GeometricOpponent::attack documentation
previous_fails (bool) – see the GeometricOpponent::attack documentation

Returns:

attack (grid2op.Action.Action) – see the GeometricOpponent::attack documentation
duration (int) – see the GeometricOpponent::attack documentation

get_state()[source]¶

This function should return the internal state of the Opponent.

This means that after a call to opponent.set_state(opponent.get_state()) the opponent should do the exact same things than without these calls.

init(partial_env, lines_attacked=None, attack_every_xxx_hour=24, average_attack_duration_hour=4, minimum_attack_duration_hour=2, pmax_pmin_ratio=4, **kwargs)[source]¶

Generic function used to initialize the derived classes. For example, if an opponent reads from a file, the path where is the file is located should be pass with this method. This is based on init from GeometricOpponent, only parameter lines_attacked becomes a list of list

Parameters:

partial_env (grid2op Environment) – see the GeometricOpponent::init documentation
lines_attacked (list(list)) – The lists of lines attacked by each unitary opponent (this is a list of list: the size of the outer list is the number of underlying opponent / number of areas and for each inner list it gives the name of the lines to attack.)
attack_every_xxx_hour (float) – see the GeometricOpponent::init documentation
average_attack_duration_hour (float) – see the GeometricOpponent::init documentation
minimum_attack_duration_hour (int) – see the GeometricOpponent::init documentation
pmax_pmin_ratio (float) – see the GeometricOpponent::init documentation

reset(initial_budget)[source]¶

This function is called at the end of an episode, when the episode is over. It aims at resetting the self and prepare it for a new episode.

Parameters:: initial_budget (float) – The initial budget the opponent has

seed(seed)[source]¶

INTERNAL

Warning

/!\ Internal, do not use unless you know what you are doing /!\ We do not recommend to use this function outside of the two examples given in the description of this class.

Set the seeds of the source of pseudo random number used for these several unitary opponents.

Parameters:: seed (int) – The root seed to be set for the random number generator.
Returns:: seeds – The associated list of seeds used.
Return type:: list

set_state(my_state)[source]¶

This function is used to set the internal state of the Opponent.

Parameters:: my_state –

tell_attack_continues(observation, agent_action, env_action, budget)[source]¶

The purpose of this method is to tell the agent that his attack is being continued and to indicate the current state of the grid.

At every time step, either “attack” or “tell_acttack_continues” is called exactly once.

Parameters:

observation (grid2op.Observation.Observation) – The last observation (at time t)
agent_action (grid2op.Action.Action) – The action that the agent took
env_action (grid2op.Action.Action) – The modification that the environment will take.
budget (float) – The current remaining budget (if an action is above this budget, it will be replaced by a do nothing.

class grid2op.Opponent.NeverAttackBudget(action_space)[source]¶

This class define an unlimited budget for the opponent.

It SHOULD NOT be used if the opponent is allowed to take any actions!

class grid2op.Opponent.OpponentSpace(compute_budget, init_budget, opponent, attack_duration, attack_cooldown, budget_per_timestep=0.0, action_space=None)[source]¶

Is similar to the action space, but for the opponent.

This class is used to express some “constraints” on the opponent attack. The opponent is free to attack whatever it wants, for how long it wants and when it wants. This class ensures that the opponent does not break any rules.

action_space¶

The action space defining which action the Opponent are allowed to take

Type:: grid2op.Action.ActionSpace

init_budget¶

The initial budget of the opponent

Type:: float

compute_budget¶

The tool used to compute the budget

Type:: grid2op.Opponent.ActionBudget

opponent¶

The agent that will take malicious actions.

Type:: grid2op.Opponent.BaseOpponent

previous_fails¶

Whether the last attack of the opponent failed or not

Type:: bool

budget_per_timestep¶

The increase of the opponent budget per time step (if any)

Type:: float

Methods:

`attack`(observation, agent_action, env_action)	This function calls the attack from the opponent.
`close`()	if this has a reference to a backend, you need to close it for grid2op to work properly.
`has_failed`()	This signal is sent by the environment and indicated the opponent attack could not be implmented on the powergrid, most likely due to the attack to be ambiguous.
`init_opponent`(partial_env, **kwargs)	Generic function used to initialize the opponent.
`reset`()	Reset the state of the Opponent to its original state, in particular re assign the proper budget to it.

attack(observation, agent_action, env_action)[source]¶

This function calls the attack from the opponent.

It check whether the budget is consistent with the attack (budget should be more that the cosst associated with the attack). If the attack cost too much, then it is replaced by a “do nothing” action. Otherwise, the attack will be implemented by the environment.

Note that if the attack is “ambiguous” it will fails (the environment will replace it by a “do nothing” action), but the budget will still be consumed.

NB it is expected that this function update the OpponentSpace.last_attack attribute with None if the opponent choose not to attack, or with the attack of the opponent otherwise.

Parameters:

observation (grid2op.Observation.Observation) – The last observation (at time t)
agent_action (grid2op.Action.Action) – The action that the agent took
env_action (grid2op.Action.Action) – The modification that the environment will take.

Returns:

res – (or “do nothing” if the attack was too costly) or class:NoneType : Returns None if no action is taken

Return type:

grid2op.Action.Action : The attack the opponent wants to perform

close()[source]¶: if this has a reference to a backend, you need to close it for grid2op to work properly. Do not forget to do it.

has_failed()[source]¶: This signal is sent by the environment and indicated the opponent attack could not be implmented on the powergrid, most likely due to the attack to be ambiguous.

init_opponent(partial_env, **kwargs)[source]¶: Generic function used to initialize the opponent. For example, if an opponent reads from a file, the path where is the file is located should be pass with this method.

reset()[source]¶: Reset the state of the Opponent to its original state, in particular re assign the proper budget to it.

class grid2op.Opponent.RandomLineOpponent(action_space)[source]¶

An opponent that disconnect at random any powerlines among a specified list given at the initialization.

Methods:

`attack`(observation, agent_action, ...)	This method is the equivalent of "attack" for a regular agent.
`init`(partial_env[, lines_attacked])	INTERNAL

attack(observation, agent_action, env_action, budget, previous_fails)[source]¶

This method is the equivalent of “attack” for a regular agent.

Opponent, in this framework can have more information than a regular agent (in particular it can view time step t+1), it has access to its current budget etc.

Parameters:

observation (grid2op.Observation.Observation) – The last observation (at time t)
opp_reward (float) – THe opponent “reward” (equivalent to the agent reward, but for the opponent) TODO do i add it back ???
done (bool) – Whether the game ended or not TODO do i add it back ???
agent_action (grid2op.Action.Action) – The action that the agent took
env_action (grid2op.Action.Action) – The modification that the environment will take.
budget (float) – The current remaining budget (if an action is above this budget, it will be replaced by a do nothing.
previous_fails (bool) – Wheter the previous attack failed (due to budget or ambiguous action)

Returns:

attack (grid2op.Action.Action) – The attack performed by the opponent. In this case, a do nothing, all the time.
duration (int) – The duration of the attack (if None then the attack will be made for the longest allowed time)

init(partial_env, lines_attacked=[], **kwargs)[source]¶

INTERNAL

Warning

/!\ Internal, do not use unless you know what you are doing /!\ Used when the opponent is created.

Parameters:

partial_env –
lines_attacked –
kwargs –

class grid2op.Opponent.UnlimitedBudget(action_space)[source]¶

This class define an unlimited budget for the opponent.

It SHOULD NOT be used if the opponent is allowed to take any actions!

class grid2op.Opponent.WeightedRandomOpponent(action_space)[source]¶

This opponent will disconnect lines randomly among the attackable lines lines_attacked. The sampling is weighted by the lines current usage rate divided by some factor rho_normalization (see init for more details).

When an attack becomes possible, the time of the attack will be sampled uniformly in the next attack_period steps (see init).

Methods:

`attack`(observation, agent_action, ...)	This method is the equivalent of "attack" for a regular agent.
`init`(partial_env[, lines_attacked, ...])	Generic function used to initialize the derived classes.
`reset`(initial_budget)	This function is called at the end of an episode, when the episode is over.
`tell_attack_continues`(observation, ...)	The purpose of this method is to tell the agent that his attack is being continued and to indicate the current state of the grid.

attack(observation, agent_action, env_action, budget, previous_fails)[source]¶

This method is the equivalent of “attack” for a regular agent.

Opponent, in this framework can have more information than a regular agent (in particular it can view time step t+1), it has access to its current budget etc.

Parameters:

observation (grid2op.Observation.Observation) – The last observation (at time t)
opp_reward (float) – THe opponent “reward” (equivalent to the agent reward, but for the opponent) TODO do i add it back ???
done (bool) – Whether the game ended or not TODO do i add it back ???
agent_action (grid2op.Action.Action) – The action that the agent took
env_action (grid2op.Action.Action) – The modification that the environment will take.
budget (float) – The current remaining budget (if an action is above this budget, it will be replaced by a do nothing.
previous_fails (bool) – Wheter the previous attack failed (due to budget or ambiguous action)

Returns:

attack (grid2op.Action.Action) – The attack performed by the opponent. In this case, a do nothing, all the time.
duration (int) – The duration of the attack

init(partial_env, lines_attacked=[], rho_normalization=[], attack_period=288, **kwargs)[source]¶

Generic function used to initialize the derived classes. For example, if an opponent reads from a file, the path where is the file is located should be pass with this method.

Parameters:

lines_attacked (list) – The list of lines that the WeightedRandomOpponent should be able to disconnect
rho_normalization (list) – The list of mean usage rates for the attackable lines. Should have the same length as lines_attacked. If no value is given, no normalization will be performed. The weights for sampling the attacked line are rho / rho_normalization.
attack_period (int) – The number of steps among which the attack may happen. If attack_period=10, then whenever an attack can be made, it will happen in the 10 next steps.

reset(initial_budget)[source]¶

This function is called at the end of an episode, when the episode is over. It aims at resetting the self and prepare it for a new episode.

Parameters:: initial_budget (float) – The initial budget the opponent has

tell_attack_continues(observation, agent_action, env_action, budget)[source]¶

The purpose of this method is to tell the agent that his attack is being continued and to indicate the current state of the grid.

At every time step, either “attack” or “tell_acttack_continues” is called exactly once.

Parameters:

observation (grid2op.Observation.Observation) – The last observation (at time t)
agent_action (grid2op.Action.Action) – The action that the agent took
env_action (grid2op.Action.Action) – The modification that the environment will take.
budget (float) – The current remaining budget (if an action is above this budget, it will be replaced by a do nothing.

If you still can’t find what you’re looking for, try in one of the following pages:

Still trouble finding the information ? Do not hesitate to send a github issue about the documentation at this link: Documentation issue template

Opponent Modeling¶

Objectives¶

How to create an opponent in any environment¶

How to deactivate an opponent in an environment¶

Detailed Documentation by class¶

Opponent Modeling ¶

Objectives ¶

How to create an opponent in any environment ¶

How to deactivate an opponent in an environment ¶

Detailed Documentation by class ¶