The world, understood.
Through knowledge and action.
Humanity has spent generations learning how the world works. A robot should be able to begin with that inheritance.
Our ambition is a world model that learns from human knowledge and physical experience, so robots can turn what we know into actions they can carry out.
From knowledge to actionLearn from the world.
And everyone who came before.
Language preserves explanations. Images reveal structure. Demonstrations show intent. Physical experience reveals what happens when a body acts.
We want machine learning to connect these sources in a model that predicts how a situation might change. Human knowledge gives the robot a starting point; contact with the world tests what it has understood.
Language · images · demonstrations
What might happen if I act?
See · reach · touch · measure
Many ways to learn.
One world to understand.
Explore the connection between human knowledge, a learning world model, and the robots that could carry that understanding into the physical world.
Knowledge is one source. Observation, simulation and embodied experience give the proposed model different ways to understand and question the world.
Research architecture · illustrative relationships, not a deployed learning system.
Predict. Act.
Learn what changed.
The intended loop connects inherited knowledge to the particular body, object and moment in front of the robot.
- Observe
What is happening?
Combine vision, joint state and touch. Keep confidence and missing information alongside the estimate.
- Rehearse
What could happen?
Predict the consequences of candidate actions. Account for contact, reach, force and the possibility of being wrong.
- Act
What should change?
Choose an action the mechanism can execute within its limits. Keep fast, local feedback responsible for the changing contact.
- Reconcile
What did we miss?
Compare prediction with measurement. Use the discrepancy to improve the estimate and inform the next learning experiment.
See from the body.
Understand the scene.
A rigid camera near the wrist would see the workspace as the arm approaches it.
The camera moves with the forearm, fixed to its structural chassis. It keeps one calibrated relationship to the mechanism. An optional stationary camera adds a wider view when fingers or objects obstruct the close view.
Sharp images require more than a steady mount. Exposure, focus, lighting and the arm’s measured pose must agree at the instant an image is taken. Global shutter prevents rolling distortion; it does not remove motion blur.
- Connection
- Shielded camera data runs back through the forearm to a desk computer. A separate wrist harness serves the hand’s sensors; paired tendons carry finger forces.
- Processing
- Check calibration, timing and visibility before interpreting an observation. Perception supplies bounded requests to a protected local controller.
- Current evidence
- An offline software contract exercises frame identity, image-quality inputs and calibrated viewing rays. Camera fit, live capture, trained inference and physical control remain in development.
Rehearse possibility.
Measure reality.
We are building an internal simulator to test our systems before they enter the real world.
The aim is to carry the actual mechanism into each experiment: its geometry, joint limits, actuator demands, contacts and uncertainty.
A useful simulation must make its assumptions visible. Friction, compliance, sensor noise and wear can turn an elegant virtual result into a failed physical task.
- Current foundation
- Revisioned engineering workflows, reviewable artifacts and bounded CAD geometry checks.
- Development direction
- Dynamics and contact models, repeatable task scenarios, and policy evaluation against measured hardware.
A general-purpose robot simulator is not yet demonstrated. Simulation results will guide hardware tests; they will not replace them.
The object moved.
Did the grip slip,
or did the skin deform?
The same apparent movement can require very different actions. Tightening the grip could save a tool and bruise a ripe ingredient.
The hand brings these questions together. Vision sees an object; touch reveals how it responds. Our first testbed is intended to make the difference measurable, one controlled interaction at a time.
Meet the first mechanical testbed