FINGR: Learning Dexterous Hand Control for Real-World Rubik’s Cube Solving

Video

Abstract

Manipulating a Rubik's Cube with one single dexterous hand is a challenging test of sustained, contact-rich control: the hand must execute successive layer turns while keeping the cube secure. Each turn requires some fingers to support the cube while others push a moving layer, release contact, and reset for the next move. To learn this coordination, we introduce FINGR (Future-supervised Interaction Network with Geometric Representations), a policy that combines finger-relative geometry with future interaction prediction. A shared point encoder expresses the cube relative to each fingertip and aggregates its points without depending on cubie indexing. Learned future tokens share the observation encoder and receive supervision for contact-force changes, layer-turn progress, and finger joint displacement at multiple time scales. The resulting representation conditions a flow policy that directly generates finger actions. On a real dexterous hand, our policy achieves around 99.0% success over 300 turn attempts, compared with 79.7% for the base flow policy. Integrated with grasping and table-assisted regrasping, the policy solves all ten scrambled 2×2×2 cubes in a mean complete-system time of approximately 137 seconds.

Method

Finger-relative geometry describes the cube from each fingertip. Predicting future contact, turn progress, and finger motion shapes the representation used by the flow policy.

Experiment

Real Robot Setup

Complete Solves Trajectory Visualization

Results

Layer-turn Success

FINGR · 99.0% success · 5.25 s per attempt

Complete Solves

BibTeX

@article{fingr2026,
  title={FINGR: Learning Dexterous Hand Control for Real-World Rubik's Cube Solving},
  author={Yutong Liang and Quanquan Peng and Matthew Kim and Xiaolong Wang},
  journal={arXiv preprint arXiv:2609.33973},
  year={2026},
  url={https://arxiv.org/abs/2609.33973}
}