The data collected was compiled into a dataset called Homes of New York (HoNY), which contains 13 hours of interactions in 22 different homes, with over 1.5 million frames. The Home Pretrained Representations (HPR) model, a ResNet-34 model, was trained on the HoNY dataset and used to initialize a robot policy for performing new tasks in novel environments. The model is available on Huggingface and can be used with Pytorch Image Models (TIMM). The research is detailed in a paper titled "On Bringing Robots Home", and the code is available on Github.
Key takeaways:
- An open-source framework for learning household robotic manipulation has been developed, with a success rate of 81% and the ability to learn a new task in 20 minutes.
- The Stick, a demonstration collection tool, was created to overcome the challenges in home robotics. It was built using a $25 Reacher-grabber stick, 3D printed parts, and an iPhone.
- The Homes of New York (HoNY) dataset was used, which includes 13 hours of interactions at 22 different homes in New York City, collected with the Stick. The dataset contains over 1.5 million frames and 5620 trajectories.
- The Home Pretrained Representations (HPR) model, pre-trained on the HoNY dataset, was used to initialize a robot policy to perform a new task in a novel environment. The model is available on Huggingface and can be used with Pytorch Image Models (TIMM).