Known issues and workarounds

In this section we will detail what are the common questions we have regarding grid2op and how to best solve them (if we are aware of such a way…)

Pickle issues

The most common (and oldest) issue regarding grid2op is its interaction with the pickle module in python.

This module is used internally by the multiprocessing module and many others.

By default (and “by design”) grid2op will create the classes when an environment is loaded. You can notice it like this:

import grid2op

env_name = "l2rpn_case14_sandbox"
env = grid2op.make(env_name)

print(type(env))

This will show something like Environment_l2rpn_case14_sandbox. This means that, not only the object env is created when you call grid2op.make but also the class that env belongs too (in this case Environment_l2rpn_case14_sandbox).

Note

We decided to adopt this design so that the powergrid reprensentation in grid2op is not copied and can be access pretty easily from pretty much every objects.

For example you can call env.n_gen, type(env).n_gen, env.backend.n_gen, type(env.backend).n_gen, obs.n_gen, type(obs).n_gen, act.n_gen, type(act).n_gen, env.observation_space.n_gen, type(env.observation_space).n_gen well… you get the idea

But allowing so makes it “hard” for python to understand how to transfer objects from one “process” to another or to save / restore it (indeed, python does not save the entire class definition it only saves the class names.)

This type of issue takes the form of an error with:

• XXX_env_name (eg CompleteObservation_l2rpn_wcci_2022) is not serializable.

• _pickle.PicklingError: Can’t pickle <class ‘abc._ObsEnv_l2rpn_case14_sandbox’>: attribute lookup _ObsEnv_l2rpn_case14_sandbox on abc failed

Automatic ‘class_in_file’

To solve this issue, we are starting from grid2op 1.10 to introduce some ways to get around this automatically. It will be integrated incrementally to make sure not to break any previous code.

The main idea is that grid2op will define the class as it used to (no change there) but instead of keeping them “in memory” it will write it on the hard drive (in a folder within the environment data) each time an environment is created.

This way, when pickle or multiprocessing will attempt to load the environment class, they will be able to because the files are stored on the hard drive.

There are some drawbacks of course. The main one being that creating an environment can take a bit more time (especially if you have slow I/O). It will also use a bit of disk space (a few kB so nothing to worry about).

For now we tested it on multi processing and it gives promising results.

TL;DR: Enable this feature by calling grid2op.make(env_name, class_in_file=True) and you’re good to go.

To enable this, you can:

• define a default behaviour by editing the ~/.grid2opconfig.json global parameters

• define the environment variable grid2op_class_in_file BEFORE importing grid2op

• use the kwargs class_in_file when calling the grid2op.make function

Note

In case of “conflicting” instruction grid2op will do the following:

• if class_in_file is provided in the call to grid2op.make(…) it will use this and ignore everything else

• (else) if the environment variable grid2op_class_in_file is defined, grid2op will use it

• (else) if the configuration file is present and the key class_in_file is there, grid2op will use it

• (else) it will use its default behaviour (as of writing, grid2op 1.10.3) it is to DEACTIVATE this feature (in the near future the default will change and it will be activated by default)

For example:

The file ~/.grid2opconfig.json can look like:

{
    "class_in_file" : false
}

or

{
    "class_in_file" : true
}

If you prefer to work with environment variables, we recommend you do something like :

import os

os.environ["grid2op_class_in_file"] = "true"  # or "false" if you want to disable it

import grid2op

And if you prefer to use it directly in grid2op.make(…) funciton, you can do it with:

import grid2op
env_name = "l2rpn_case14_sandbox"
env = grid2op.make(env_name, class_in_file=True) # or `class_in_file=False`

If you want to know if you environment has used this new feature, you can check with:

import grid2op
env = grid2op.make(...)
print(env.classes_are_in_files())

Danger

If you use this, make sure (for now) that the original grid2op environment that you have created is not deleted. If that is the case then the folder containing the classes definition will be removed and you might not be able to work with grid2op correctly.

Experimental read_from_local_dir

Before grid2op 1.10.3 the only way to get around pickle / multiprocessing issue was a “two stage” process: you had first to tell grid2op to generate the classes and then to tell it to use it in all future environment.

This had the drawbacks that if you changed the backend classes, or the observation classes or the action classes, you needed to start the whole process again. ANd it as manual so you might have ended up doing some un intended actions which could create some “silent bugs” (the worst kind, like for example not using the right class…)

To do it you first needed to call, once (as long as you did not change backend class or observation or action etc.) in a SEPARATE python script:

import grid2op
env_name = "l2rpn_case14_sandbox"  # or any other name

env = grid2op.make(env_name, ...)  # again: redo this step each time you customize "..."
# for example if you change the `action_class` or the `backend` etc.

env.generate_classes()

And then, in another script, the main one you want to use:

import grid2op
env_name = SAME NAME AS ABOVE
env = grid2op.make(env_name,
                    experimental_read_from_local_dir=True,
                    SAME ENV CUSTOMIZATION AS ABOVE)

As of grid2op 1.10.3 this process can be made automatically (not without some drawbacks, see above). It might interact in a weird (and unpredictable) way with the class_in_file so we would recommend to use one OR (exclusive OR, XOR for the mathematicians) the other but avoid mixing the two:

• either use grid2op.make(…, class_in_file=True)

• or use grid2op.make(…, experimental_read_from_local_dir=True)

Thus we DO NOT recommend to use something like grid2op.make(…, experimental_read_from_local_dir=True, class_in_file=True)

If you still can’t find what you’re looking for, try in one of the following pages:

• Index

• Search Page

Still trouble finding the information ? Do not hesitate to send a github issue about the documentation at this link: Documentation issue template