BEHAVIOR is a human-centered simulation benchmark to evaluate embodied AI solutions

Embodied artificial intelligence (EAI) is advancing. But where are we now? We propose to test EAI agents with the physical challenges humans need to solve in their everyday life: household activities such as doing laundry, picking up toys, setting the table, or cleaning floors. BEHAVIOR is a benchmark in simulation where EAI agents need to plan and execute navigation and manipulation strategies based on sensor information to fulfill up to 1,000 household activities. BEHAVIOR tests the ability of agents to perceive the environment, plan, and execute complex long-horizon activities that involve multiple objects, rooms, and state changes, all with the reproducibility, safety, and observability offered by a realistic physics simulation.

The broader goal of BEHAVIOR is to fuel the development of general, effective EAI that brings major benefits to people’s daily lives – human-centered AI that serves human needs, goals, and values. BEHAVIOR achieves this by selecting activities from human time use surveys, and conducting large-scale preference surveys that ask people: “what everyday activities do you want robots to do for you?” Furthermore, these activities are defined based on the principles of participatory design: a team of crowdworkers and researchers work together to provide the knowledge and definitions for BEHAVIOR activities.

BEHAVIOR Benchmarks



BEHAVIOR-1K

  • 1,000 activities from a large-scale human preference survey
  • 8 scene types, 50 fully interactive scenes
  • 1,200+ object types, 5,000+ object models
  • Based on OmniGibson, powered by Nvidia's Omniverse
  • Supports flexible materials and deformable bodies
  • Supports realistic fluids and thermal effects
  • Requires Geforce RTX graphics card
  • Explore BEHAVIOR-1K


    BEHAVIOR-100

  • 100 activities from American Time Use Survey
  • Resident house scenes, 15 fully interactive scenes
  • 390+ object types, 1,200+ object models
  • Based on iGibson 2.0 and PyBullet
  • Human demostration dataset with 500 VR demos
  • Activity length of ~300-20,000 steps
  • Explore BEHAVIOR-100

    The BEHAVIOR Team


    Chengshu (Eric) Li Chengshu (Eric) Li
    Image placeholder Ruohan Zhang
    Image placeholder Josiah Wong
    Cem Gokmen Cem Gokmen
    Sanjana Srivastava Sanjana Srivastava
    Chen Wang Chen Wang
    Gabrael Levine Gabrael
    Levine
    Michael Lingelbach Michael Lingelbach
    Mona Anvari Mona Anvari
    Emily Jin Emily Jin
    Wensi Ai Wensi Ai
    Yunzhu Li Yunzhu Li
    Roberto Martín-Martín Roberto
    Martín-Martín
    Silvio Savarese Silvio Savarese
    Hyowon Gweon Hyowon Gweon
    Karen Liu Karen Liu
    Jiajun Wu Jiajun Wu
    Li Fei-Fei Li Fei-Fei

    Alumni


    Jiankai Sun Jiankai Sun
    Minjune Hwang Minjune Hwang
    Manasi Sharma Manasi
    Sharma
    Arman Aydin Arman Aydin
    Dhruva Bansal Dhruva Bansal
    Samuel Hunter Samuel Hunter
    Kyu-Young Kim Kyu-Young Kim
    Alan Lou Alan Lou
    Caleb Matthews Caleb
    Matthews
    Ivan Villa-Renteria Ivan
    Villa-Renteria
    Jerry Tang Jerry Tang
    Claire Tang Claire Tang
    Fei Xia Fei Xia
    Kent Vainio Kent Vainio
    Zheng Lian Zheng Lian
    Shyamal Buch Shyamal Buch