Skip to main content

Environments

Gymnasium environments in the ADK are part of the RL path when you use RLExecutor. They translate actions into simulator steps, define reward semantics, and shape observations — the usual reinforcement-learning environment role. The RL agent / environment split in genie setup templates follows from that executor choice, not from the core ADK itself.

If you implement a custom BaseExecutor subclass instead, you typically call OptimizationContext.step_world directly and do not need a Gymnasium environment at all.

Registering built-in env IDs​

Gymnasium IDs are registered when you import adk in the agent process (genie run / python src/main.py). The genie CLI sets GENIE=1 and skips registration, so always import adk (or gymnasium after import adk) from agent code, not from CLI-only scripts.

Every built-in env requires env_data from RLExecutor:

import adk  # registers env IDs
import gymnasium as gym
from adk.executors.rl import RLAgentEnv

class MyAgent(RLAgentEnv):
    def __init__(self, env_data, agent_data):
        super().__init__(env_data, agent_data)
        self.env = gym.make("AI4EE-Direct-Action-Env", env_data=env_data)

RLExecutor wiring​

On each run, RLExecutor constructs EnvData from the optimization context and passes it to your RLAgentEnv constructor. Your agent must still expose the Gymnasium interface (reset, step, observation_space, action_space, …) because RLExecutor calls those methods on the agent instance directly — even when stepping logic lives in a separate self.env object.

See RL Agents and RL Run Data for how your agent receives and uses this data.

Built-in environments​

The ADK registers these Gymnasium environment IDs (note the AI4EE prefix):

IDClassDocumentation
"AI4EE-Direct-Action-Env"DirectActionEnvAPI
"AI4EE-Episodic-Direct-Action-Env"EpisodicDirectActionEnvSame family as DirectActionEnv; multi-step episodes
"AI4EE-Additive-Action-Env"AdditiveActionEnvAPI
"AI4EE-Clipped-Additive-Action-Env"ClippedAdditiveActionEnvAPI
"AI4EE-Direct-Action-Graph-Env"DirectActionEnv (graph)Graph-instrumented direct action; see source
"AI4EE-Dynamic-Env"DynamicEnvAdvanced; see adk.executors.rl.envs.dynamic_env
"AI4EE-Dynamic-Env-V1"DynamicEnvV1Advanced variant
"AI4EE-Dynamic-Env-V2"DynamicEnvV2Advanced variant
"AI4EE-Single-Optimum-Env"SingleOptimumEnvAdvanced; single-optimum reward shaping
"AI4EE-PGRL-Env-V1"PGRLEnvAdvanced; PGRL-specific
"AI4EE-Raw-Simulation-Env"RawSimulationEnvAdvanced; minimal reward wrapper

IDs without dedicated API pages are advanced built-ins maintained for specific agent projects. Inspect adk/__init__.py registrations and the matching module under adk/executors/rl/envs/ before adopting them.

Custom environments​

Subclass OptimizationEnv to implement reward logic, action handling, and stepping against EnvData.step_world. This base class backs the built-in environments and is the recommended starting point for custom envs on the RL path.

Pass env_data into gymnasium.make for built-in envs, or construct your OptimizationEnv subclass directly in the RL agent constructor.