Dormant pruning demo¶
A Bumblebee-like robot pruning a row of a generated winter vineyard: a skid-steer UGV drives the alley, stops at each vine, and an arm on a linear slide cuts every cane back to two buds with a bypass shear. The cuts are planned from the scene's own ground truth -- which cane, where each bud is -- rather than from a sensor. Last season's tall weeds still stand under the vines and in the alley, their long stems bending as the robot drives and works through them. The trellis wires are hidden: they have no collider yet, so the canes and the shear would pass through them.
Contents: Requirements · Running the demo · Teleoperation · Seeing the colliders · Recording the video · What it shows · How the shear cuts · The robot · Layout
Requirements¶
- uv installed
- Isaac Sim 6.2 compatible hardware
- Rust installed (this demo builds the scene generator from source)
Running the demo¶
uv run main.py
The first run downloads the Warthog, generates the vineyard and converts the
robot, and caches all three; later runs start straight up. --help lists the launcher's options --
--row picks the row pruned, --physics the backend and --viz the viewer.
Only the default backend, Newton coupled with VBD, bends a cane or a weed, so only it can cut a cane. Under any other the canes and weeds spawn static and the robot drives the row with nothing to do.
Teleoperation¶
uv run main.py --teleop
The planner stands down and the keyboard has the robot, with the Kit window focused: the left hand drives, the right hand cuts. The viewport lists the keys in its bottom right corner.
| key | what it does |
|---|---|
W / S |
drive forward / back |
A / D |
turn left / right |
↑ / ↓ |
jog the mouth forward / back along the robot |
← / → |
jog it toward the row / away from it |
Page Up / Page Down |
jog it up / down |
Enter |
close the shear, cutting every cane the blade closes on, and open it again; the robot holds still while it shuts |
The mouth is a gantry head rather than a wrist: it stays squared up to the row -- blades out over the rail, pivot upright, so an upright cane lies across it -- and its position is held in the robot's own frame, so it rides along when the base drives. It starts out over the rail at spur height, a stand-off short of the row, and a jog the arm cannot reach is refused.
Seeing the colliders¶
uv run main.py --colliders
draws every collider the physics steps, translucent over the scene: blue
for the ground and the vines' wood, green for the canes, magenta for the
robot. Kit draws a PhysX scene's colliders itself, but this one runs on
Newton, which Kit knows nothing about, so the drawing is the framework's
ColliderView. It follows what changes while the demo runs: a cut cane's
two segments, one shortened and one stretched, and the shear's slices,
which stand at a cane while the drawn blade closes on through it.
Recording the video¶
./record.sh
films a vine of the default scene pruned, in Isaac Sim's RTX renderer and
headless, and writes target/video/pruning.mp4 at the repo root, captioned
and small enough to upload, and a contact sheet beside it. Its storyboard --
the cases filmed, one per cane, the camera, the pacing -- is the top of
video.py; the skill covers
re-recording it and changing what it shows.
What it shows¶
The layout is Bumblebee's (Silwal et al., 2021): a Clearpath Warthog in the alley with an aluminium frame on its back, an arm on a 1.35 m slide along the frame -- a UR10 here, where the paper has a UR5 -- and a bypass shear in its hand. So is the pipeline, minus the cameras:
- The rule. Keep two buds on every cane and cut midway between the second
and the third. The scene is generated dormant, so every cane is a rod a
solver bends, with a
Bud_NNprim at every node; the planner reads the buds off the stage and their live positions off the rod bodies carrying them. - The approach. Each cut is a pose for the shear's mouth, squared up to the cane: the pivot along it, the blades along the approach, aimed afresh at the buds when the cut's turn comes. The arm plans to a point 15 cm out, where the cut is aimed once more -- the swing out brushes through the canes, the one to cut among them, with the shear shut, since the claw of an open hook would gather one into the mouth and carry it on to every cut after -- then opens the shear, closes in on a straight line to the cut and settles there. The shear's head and blades collide with the canes, so a cane the mouth comes in on is funnelled between the edges or pushed aside, bending as it goes.
- The order. The cuts on a vine are taken nearest neighbour first. A cut whose bud to take is gone by its turn -- caught by the blade on a neighbouring cane, or by a falling piece -- is left out. Once the last is made the arm folds home, at the same top speed it swings at, and the robot drives on.
- The cut. Once the arm has settled, the shear closes, and the moving blade carries the cane across the mouth onto the fixed one and cuts it there, as the next section tells. Every cane the blade closes on is cut, each as the blade reaches it. A cane the blade never reaches, pushed out of the mouth on the way in or never in it, is a miss.
- The fall. The piece meets its own vine -- the stub, the other canes, the cordon -- the ground and the robot, and passes through every other vine: it catches on the stub for a moment, tips, and slides down beside it. One that falls into the open mouth or onto the blades would ride along to the next cut, where the blade does not cut it, since it is loose; so the arm, backed out, tips the blades down until it slides off.
The score is printed after every vine, and against the paper's at the end: cuts reachable, cuts made, cuts made at the right place -- the kept bud still on the vine and the next one gone -- and seconds per vine. Bumblebee reports 87% of cuts made at the right place, 213 s per vine and 68% of canes reachable from one side. On the default scene the demo reaches all 72 cuts of its row and makes every one at the right place, at 20 s of simulated time per vine. Because the simulation knows the true plant, the score keeps "chose the right cut" apart from "made the cut", which a field trial cannot.
How the shear cuts¶
The shear's blades are a bypass pruner's. The moving one is the cutting
blade, its edge convex and ground on its outer face; it closes past the
fixed one, the hook, whose concave edge is the crook a cane is held in and
whose claw reaches past the cutting blade's tip. The two edges are one
curve when shut, so the open mouth spans the same angle at every distance
out from the pivot. A cane coming in outside the hook's tip slides off its
back, so the mouth's point a cane is brought to keeps clear of the tip: the
centre of the widest circle between the straight lines from the pivot to
the two tips. Each blade is drawn as a mesh shear.py writes beside the
URDF, and collides as convex pieces along it, the moving blade's being the
slices below.
A real bypass shear closes its moving blade past a fixed one, and the wood between them gives. A simulated cane does not give: a blade closing on it would squeeze it against the fixed blade until the solver shoved it out sideways. So the cut is made by rule, a control tick at a time, and the moving blade's collider stops at the cane while the drawn blade closes on through it, the way a real edge goes into the wood.
The collider does not stop whole. The moving blade collides as eight slices end to end along it, a centimetre each, and each slice stops on its own:
pivot tip
[ 0 ][ 1 ][ 2 ][ 3 ][ 4 ][ 5 ][ 6 ][ 7 ] one convex piece each
( cane ) 5 and 6 stand at it,
the rest close with the blade
Every tick while the shear closes, shear.Stroke:
- Carries. A slice rides the blade, and pushes a cane in the mouth across, onto the fixed blade.
- Stands. A slice about to reach a cane that lies against the fixed blade stops a millimetre off its bark, at the angle where the edge touches it, and stays there. Only the slices along which the edge meets the cane stop, those within a cane's radius of it, so the tip and the root close on with the blade and collide where it is drawn.
- Cuts. The drawn blade goes on closing into the cane, and on the tick
its edge passes the cane's axis the cane is cut there, with
Shears.cut. Each cane is cut on its own tick: two canes in the mouth stand their own slices and are cut in turn, in the order the edge reaches them. - Meets. As the blades meet, every cane still between them is cut too: one the blade carried but never pressed onto the fixed blade, and one the open blade was in already when the stroke began, gone into rather than closed on.
A cane the blade is still carrying across is not cut before the blades meet, even when it has sunk into the blade past its axis: the solver lets a pushed cane sink in a little, and a cut there would be made in mid-air. Nothing is crushed between the blades and nothing the cane leans on is pulled from under it. As the shear opens, a standing slice rides the blade again once the blade has opened back past it, so every stroke starts with the blade whole:
stateDiagram-v2
direction LR
rides: rides the blade
stands: stands at a cane
rides --> stands: about to reach a cane held against the fixed blade
stands --> rides: the blade opens back past it
Shear.slices sets how many slices there are. --colliders shows them
at work: the magenta slices at the cane stand still while the steel blade
closes on through it.
The robot¶
Bumblebee in bumblebee.py is a dataclass of dimensions, and the URDF is
generated from it. The chassis is the Warthog as Clearpath's own
description
draws and weighs it; the first run downloads it from the
misina-lab-sim-assets
dataset on Hugging Face, and later runs find it in the local cache. Its links
are copied into the generated URDF with its rockers fixed: the real machine has
a differential bar between them that holds the chassis level, which the URDF
leaves out. The frame on its back and the slide along it are the paper's, drawn
from primitives, the rail out over the Warthog's left fenders, toward the row.
The arm is the stock UR10 that Isaac Sim's URDF importer ships, Universal
Robots' meshes and all: its links are copied into the generated URDF as they
are, standing on the carriage, so the arm is part of the robot's one
articulation and rides the slide. Isaac Lab's UR10_CFG would spawn it as an
articulation of its own, fixed to the world. Bumblebee.chain reads the same
joints out of that URDF for kinematics.py, which poses the arm with damped
least squares; the solver and the simulation come from one description.
Layout¶
| file | what it owns |
|---|---|
main.py |
the vineyard, the sky, the robot, the simulation loop |
bumblebee.py |
the robot: the Warthog, its dimensions, the URDF built from them, the actuators |
shear.py |
the bypass shear: its blades' geometry, their meshes and colliders, and Stroke, the blade closing through the canes; the greenhouse demo's module, kept the same |
kinematics.py |
forward and inverse kinematics of the arm on its slide |
pruner.py |
the ground truth off the stage, the rule, the cut sequence, the score |
driver.py |
driving the skid steer from stop to stop |
teleop.py |
the keyboard driving the base, jogging the mouth and working the shear |
video.py, record.sh |
the video: a vine pruned case by case, filmed headless, and its encoding |
DEVELOPMENT.md in ../isaaclab_demo covers the pinned Isaac Lab release,
which this example shares.