botblocks
projectsgalleryworkspacedocsprofile
introdemospythondashboardcli

python api

worlds and lifecycle

every environment owns a list of things:

from botblocks import Box, MujocoEnv, Plane, Robot, Sphere

robot = Robot().load('public/so101')
env = MujocoEnv([Plane(), Box(size=0.03), Sphere(size=0.03), robot]).start()

Thing accepts pos=[x, y, z], rot=[roll, pitch, yaw], and an optional name. distances are in meters and angles in radians. Box.size is its side length; Sphere.size is its diameter. their initial pos places the bottom at that height, so pos=[0, 0, 0] rests on the ground. live poses and later set_pos calls use the body's center. Plane supplies the ground at z=0.

Asset loads a project glb and its generated collision mesh:

from botblocks import Asset

cup = Asset('libero/cup.glb')
basket = Asset('libero/basket.glb', fixed=True)
@robot.ready
def setup(bot, env):
    pass                    # once at start and again after reset

@robot.loop
def tick(bot, env):
    pass                    # repeatedly, synchronized to frames

useful world methods:

env.time()                  # wall-clock seconds since start or reset
env.next_frame()            # wait for the next frame
env.reset(wait=True)        # reset things, subsystems, and physics
env.metric(reward=0.8)      # stream numeric values to the workspace metrics pane

thing.pose                  # live Pose(pos, rot)
thing.velocity              # live world-frame linear velocity [x, y, z]
thing.set_pos([x, y, z])    # move a thing immediately
thing.set_pose([x, y, z], [roll, pitch, yaw])

start(wait=True, view=True, loop=True) normally blocks and runs lifecycle callbacks. wait=False returns with callbacks running in background threads. keep the script alive while those threads run. view=False omits the environment from the initial workspace selection; it does not supply a camera renderer. loop=False disables the background lifecycle loop for callers that drive steps themselves.

environments

  • MujocoEnv: physics, contacts, free bodies, sensors, and simulated actuators
  • ViewerEnv: physics-free kinematics; servo targets snap to their poses and motors integrate as angular velocity
  • LocalEnv: providers read and command hardware on the local machine

MujocoEnv uses a 0.005-second physics timestep by default. drive(True) stops automatic stepping so your script can drive physics; drive(False) resumes realtime simulation. step_sim(0) refreshes state without advancing time.

env = MujocoEnv([Plane(), Box()], timestep=0.005).start(wait=False, loop=False)
env.drive(True)
env.step_sim(4)              # advance 0.02 simulated seconds
env.drive(False)

step_sim(n, ctrl=[[f'{robot.id}/{servo.name}', target]]) applies targets together before stepping. use MujocoEnv(things, nonconvex=True) for nonconvex asset collisions; an individual Asset(..., nonconvex=False) can opt out.

robots

robot = Robot().load('public/so101')
robot = Robot().load('menagerie/unitree_go2')
robot = Robot().load('my-project/my-robot')
robot = Robot('public/so101') # constructor shorthand

robot['shoulder']            # named subsystem
robot['goal'] = Command(n=3) # add an in-script observation subsystem

use the gallery or project page to inspect a robot's configured subsystem names. code should not assume that every robot has an arm, drive, camera, or imu.

actuators and sensors

hinges use radians and rad/s; sliding joints use meters and m/s.

servo.target(position)      # default speed=None: no target slew limit
servo.target(position, speed=1.0) # slew target at 1 rad/s (hinges) in mujoco
servo.target(0.1, relative=True)  # offset from the current position
servo.read()
servo.velocity()

motor.power(value) # normally -1..1
motor.read()
motor.velocity()

encoder.read()
encoder.velocity()

drive.power(fwd, turn=0)     # DiffDrive: left=fwd+turn, right=fwd-turn
imu.gravity()                # body-frame gravity direction
imu.pos()                    # world position
imu.vel()                    # body-frame linear velocity
imu.vel_world()              # world-frame linear velocity
imu.ang_vel()                # body-frame angular velocity
imu.ang()                    # roll, pitch, yaw
imu.accel()                 # world-frame linear acceleration
imu.quat()                  # [w, x, y, z]
camera.snap()                # raw rgba BytesId, also appears in snaps
camera.snap_img()            # rgb PIL.Image
camera.w = camera.h = 64      # set before starting the env for smaller rendered images
subsystem.pose               # world pose of its mounted body

a simulated camera needs an attached viewer or the host renderer before it can answer a snapshot. use BOTBLOCKS_RENDER=1 botblocks run script.py --headless for unattended camera scripts. snap() caches frames according to camera.fps (default 30); set it to float('inf') when every manual simulation step needs a fresh image.

arm ik

an Arm is a composite over a configured servo chain:

arm.move([dx, dy, dz])
arm.move([dx, dy, dz], aim=[ax, ay, az], speed=0.1) # aim: the direction the gripper points
arm.goto([x, y, z])
arm.goto([x, y, z], aim=[0, 0, -1], speed=0.1)      # absolute, gripper pointing down
arm.goto([x, y, z], rpy=[roll, pitch, yaw], speed=0.1)
arm.lineto([x, y, z], duration=2) # smooth cartesian line; blocks until the arm arrives or stops
arm.lineto([x, y, z], duration=2, aim=[0, 0, -1])
arm.fk()                     # current 4x4 tool transform
arm.describe()               # joint angles and tool pose
arm.target_all(angles)       # servo-chain order; waits for joint targets

move is relative; goto and lineto use absolute world coordinates. all three take either aim or rpy; their speed defaults to 1.0. goto and move set joint targets and return immediately. lineto follows a cartesian line and waits for arrival or settling; it returns stopped ...cm short if the tool stops short. inspect describe() and a camera snapshot before assuming a grasp or placement succeeded.

unreachable positions, orientations, and self-colliding solutions return refused: ... and emit a warning. None means no refusal was reported; it does not prove an object was picked up.

prefer aim over rpy. aim constrains two of the three orientation degrees of freedom, which is what most arms have to spare: a 5-dof arm like so101 spends 3 dof on position and has exactly 2 left, so a full rpy is over-constrained and will be refused for all but a lucky few poses. so101 can only aim within the vertical plane through its shoulder — straight down works anywhere it can reach, sideways generally does not.

keyboard input

Keyboard exposes the currently held keys; keys.new_down() returns new key presses since its previous call. click outside the editor before driving. KeyboardCounter integrates key pairs into numeric axes: the first key increases an axis, the second decreases it.

from botblocks import KeyboardCounter, ViewerEnv

keys = KeyboardCounter(pairs=['qa', 'ws'], init=[0, -0.5], rate=0.8)

@robot.loop
def drive(bot, env):
    shoulder, elbow = keys.get_axes()
    bot['shoulder'].target(shoulder)
    bot['elbow'].target(elbow)

ViewerEnv([Plane(), robot, keys]).start()

reinforcement learning

install the [rl] extra locally. GymEnv turns selected subsystems into gymnasium observation and action spaces: subsystems with observe() and obs_dim supply observations; those with act() and act_range() supply actions. subsys=[robot] expands to all of that robot's subsystems. raw cameras need a custom gym environment, as in the navigate demo.

the input is [sensor observations, previous action], with actions in subsystem order. previous actions start at zero on reset, including automatic resets during sharded training. Policy uses the same input layout; call policy.reset() when starting a new episode. checkpoints trained without previous actions need retraining.

from botblocks import GymEnv, Plane, Robot
import math

furuta = Robot().load('public/furuta')

@furuta.loop
def reward(bot, env):
    angle = bot['encoder'].read()
    env.reward(-math.cos(angle))

GymEnv([Plane(), furuta], subsys=[furuta['encoder'], furuta['servo']]).train(
    n=32, shards=4, steps=1_000_000, save='/tmp/furuta')

reward terms can be a value or [value, weight]; the reward is their weighted sum and the unweighted terms appear in metrics. env.reward(..., done=True) marks termination.

@robot.loop
def reward(bot, env):
    velocity = -sum(bot['imu'].vel_world() ** 2)
    height = -(bot['imu'].pos()[2] - 0.20) ** 2
    env.reward(velocity=[velocity, 0.1], height=height)

training uses stable-baselines3 ppo. n is the total robot count and should be divisible by shards; each shard has its own simulation. sharded training copies the first robot onto a plane, so use n=1, shards=1 to preserve a custom scene. a shard resets when all its robots terminate or the episode horizon expires. the defaults are dt=0.02, seconds=7.68, and phys_dt=0.005.

train saves a ppo .zip and observation-normalization .pkl. metrics stream to the workspace. medium and gpu containers provide more compute:

botblocks deploy train.py --size medium

play a saved policy with:

from botblocks import MujocoEnv, Plane, Policy, Robot

furuta = Robot('public/furuta')
policy = Policy('/tmp/furuta', subsys=[furuta['encoder'], furuta['servo']], vecnorm='/tmp/furuta.pkl')

@furuta.ready
def reset_policy(bot, env):
    policy.reset()

@furuta.loop
def act(bot, env):
    policy.step()

MujocoEnv([Plane(), furuta]).start()

keep the same subsystem order as training and pass the saved normalization file. for exact control timing, use manual simulation steps at the training dt.

onnx policies

OnnxInference loads a local file or project asset. its predict() expects a model input named obs; set inp_len to that model's input width and pred_len to its action count. the walk demo shows observation construction, action scaling, and reset handling for a real exported policy.

hardware providers

LocalEnv(things, providers=[ProviderClass]) instantiates provider classes for matching subsystems. subclass Provider and set provides = Servo (or another subsystem type). a servo provider implements set_ctrl, set_pos, read, and velocity; a camera provider implements snap. the environment calls bind(client, on, name, state, joint) on each provider.

pass classes in providers, not instances assigned to subsystem.provider: startup resolves and binds providers from the environment list. the bundled ServoSTS and CameraUSB adapters are experimental: ServoSTS still needs its binding signature updated for this interface, and CameraUSB supplies png bytes while Camera.snap_img() expects raw rgba. they also require scservo_sdk and opencv respectively, which the base install does not include.