The Guppy Gym

The guppy gym is an OpenAI gym environment for learning and evaluating controlled robotic fish that interact with (simulated) live fish.

The guppy gym is based on the kilobots gym.

For reinforcement learning using the guppy gym, see also the rl repo.

Documentation

Documentation can be found here.

Installation

Linux and python3.8

python3 -m pip install wheel
python3 -m pip config set global.extra-index-url https://git.imp.fu-berlin.de/api/v4/projects/6392/packages/pypi/simple
python3 -m pip install robofish-gym-guppy

python3.9 and above

The dependency Box2d needs swig to be installed if python 3.9 or above is used. swig is a system package and cannot be installed with pip.

sudo apt install swig
python3 -m pip config set global.extra-index-url https://git.imp.fu-berlin.de/api/v4/projects/6392/packages/pypi/simple
python3 -m pip install robofish-gym-guppy

macOS

brew install swig
python3 -m pip config set global.extra-index-url https://git.imp.fu-berlin.de/api/v4/projects/6392/packages/pypi/simple
python3 -m pip install robofish-gym-guppy
python3 -m pip install box2d-kengz # needed only on macOS

Getting started

The repository consists of environment classes based on OpenAI gym (gym.Env) and simluations of robots. The fish_models repository contains models of fish behavior and can be used with the guppy gym.

Example:

import numpy as np
import robofish.gym_guppy
import fish_models

NUM_ROBOTS = 2
NUM_GUPPIES = 3

rng = np.random.default_rng(0)

# Initialize the environment.
env = robofish.gym_guppy.envs.ConfigurableGuppyEnv(
    guppy_type=fish_models.ModelGuppy,
    guppy_args={
        "vault": fish_models.vault.Vault(),
        "model": fish_models.models.LiveFemaleFemaleConstantSpeedCouzinModel(),
        "frequency": 25.0,
        "world_conversion_factor": 100.0,
    },
    num_guppies=NUM_GUPPIES,
    robot_type="TurnRobot", # types may be passed as strings
    num_robots=NUM_ROBOTS,

)

# Reset the environment. This will cause the agents to be initialized.
initial_observation = env.reset()

# Run the simulation for 10 steps.
for _ in range(10):
    action = rng.normal(size=(NUM_ROBOTS, 1))  # Sample a random actions.
    observation, reward, done, info = env.step(action=action)

# Get the robot and guppy poses. The info dictionary also contains other
# information, such as actions taken, etc.
robot_poses = [np.stack(info["robot_poses"][i] for i in info["robot_poses"])]
guppy_poses = [np.stack(info["guppy_poses"][i] for i in info["guppy_poses"])]

Guppies

You should not use the guppies that are built into the guppy gym. Instead, you should use fish_models.

(The guppies that are built into the guppy gym are either deprecated, are only for running tests of the guppy gym intself, or are base classes that are used by fish_models.)

Robots

The choice of robot determines the action space. For example the very simple TurnSpeedRobot expects an action that consists of a turn and a speed value. This robot model may be used for debugging due to it's simplicity, or for learning policies that do not have to be able to control a physical robot. If you want to train a policy that should later control a physical robot, you need to use PolarCoordinateTargetRobot (the default robot class in ConfigurableGuppyEnv). PolarCoordinateTargetRobot includes a simulation of the physical robot, including PID controller.

Rendering

To visualize trajectories use robofish-trackviewer. The use of env.render() is deprecated.

""".. include:: ../../../README.md"""  # noqa: D415
# isort:skip_file
# TODO: Sorting the imports causes a circular import error. Fix this.
from .guppies import (
    Agent,
    TurnBoostAgent,
    ConstantVelocityAgent,
    TurnSpeedAgent,
    VelocityControlledAgent,
    Guppy,
    DummyGuppy,
    TurnSpeedGuppy,
    TurnBoostGuppy,
    BaseCouzinGuppy,
    ClassicCouzinGuppy,
    BoostCouzinGuppy,
    AdaptiveCouzinGuppy,
    BiasedAdaptiveCouzinGuppy,
    PerturbedAdaptiveCouzinGuppy,
    Robot,
    TurnSpeedRobot,
    TurnBoostRobot,
    GlobalTargetRobot,
    PolarCoordinateTargetRobot,
    TurnRobot,
    GoToRobot,
    TorqueGuppy,
    TorqueThrustAgent,
    AdaptiveAgent,
    BoostGuppy,
    RandomTurnBoostGuppy,
)

from .envs import GuppyEnv, GoalGuppyEnv, VariableStepGuppyEnv, VariableStepGoalGuppyEnv
from .wrappers import (
    RayCastingWrapper,
    FrameStack,
    DiscreteActionWrapper,
    IgnorePastWallsWrapper,
)
from .reward import (
    clipped_reward,
    exp,
    negative_distance_to_swarm,
    follow_reward,
    approach_swarm_reward,
    proximity_to_center_reward,
    wall_avoidance_gate,
    in_zor_gate,
    in_zoi_gate,
    in_zoi_reward,
    negative_distance_to_center,
    goal_reward,
    relative_goal_reward,
)


__all__ = [
    "Agent",
    "TurnBoostAgent",
    "ConstantVelocityAgent",
    "TurnSpeedAgent",
    "VelocityControlledAgent",
    "Guppy",
    "DummyGuppy",
    "TurnSpeedGuppy",
    "TurnBoostGuppy",
    "BaseCouzinGuppy",
    "ClassicCouzinGuppy",
    "BoostCouzinGuppy",
    "AdaptiveCouzinGuppy",
    "BiasedAdaptiveCouzinGuppy",
    "PerturbedAdaptiveCouzinGuppy",
    "Robot",
    "TurnSpeedRobot",
    "TurnBoostRobot",
    "GlobalTargetRobot",
    "PolarCoordinateTargetRobot",
    "TurnRobot",
    "GoToRobot",
    "TorqueGuppy",
    "TorqueThrustAgent",
    "AdaptiveAgent",
    "BoostGuppy",
    "RandomTurnBoostGuppy",
    "GuppyEnv",
    "GoalGuppyEnv",
    "VariableStepGuppyEnv",
    "VariableStepGoalGuppyEnv",
    "RayCastingWrapper",
    "FrameStack",
    "DiscreteActionWrapper",
    "IgnorePastWallsWrapper",
    "clipped_reward",
    "exp",
    "negative_distance_to_swarm",
    "follow_reward",
    "approach_swarm_reward",
    "proximity_to_center_reward",
    "wall_avoidance_gate",
    "in_zor_gate",
    "in_zoi_gate",
    "in_zoi_reward",
    "negative_distance_to_center",
    "goal_reward",
    "relative_goal_reward",
]

__pdoc__ = {x: False for x in __all__}

Sub-modules

robofish.gym_guppy.bodies

Base classes for fish and robots that are related to physics simulation …

robofish.gym_guppy.envs

Environment classes for reinforcement learning and other uses …

robofish.gym_guppy.guppies

Guppies and robots.

robofish.gym_guppy.reward

Reward functions.

robofish.gym_guppy.tools

Helper functions and utilities.

robofish.gym_guppy.wrappers

Wrappers for the Robofish environment.