Task Description

The goal is to migrate events and modes from the MotionInput 3.2 repo to the 3.4 version.

We will first do this from the main branch, then the other branches of the MI 3.2.

Environment Setup

Install Python 3.11 and install all packages in requirements.txt. I strongly recommend using conda.

Modes

They define different game modes playable, such as Minecraft of Car Driving mode.

In the 3.2 version, they are all located in “data/controller.json” file:

...
"joystick": [
            "joystick_button_a",
            "joystick_stick_left",
            "joystick_triggers_left",
            "joystick_button_back",
            "joystick_shoulder_right",
            "joystick_stick_right",
            "joystick_shoulder_left",
            "joystick_triggers_right",
            "joystick_button_y",
            "joystick_dpad_right",
            "joystick_dpad_up",
            "joystick_dpad_down",
            "joystick_dpad_left",
            "joystick_button_x",
            "joystick_button_b",
            "joystick_button_start",
            "general_rocknroll_next_mode"
]
...

Each mode is associated with its name and a list of events, which can be triggered within the mode.

In 3.4 version, each mode is located within its own file in “data/modes”:

//Example Mode I made up, it would be located in e.g. file "test.json"

{
  "config": {
    "hand": "right"
  },
  "poses": [
    {
      "file": "hit_trigger.py",
      "landmark": "left_elbow",
      "pos": [140, 190]
    },
    {
      "file": "index_pinch.json",
      "action": {
        "class": "mouse",
        "method": "hold",
        "args": ["left"]
     }
    }
  ]
  }
}

In config, we define arguments that are passed into all the classes. Consider them like global parameters. If none are used, do not include the config.

In poses, we define a list of poses (they were called events in 3.2), which can be triggered by the mode.

Here, we first define an event, which runs a Python class that takes in two parameters, landmark name and position. The Python class manages everything about how the event is processed.

In the second element, we define a pose using a JSON file. This file has been created using a PoseRecorder and defines specific movement that is captured by the event. Such JSON events takes in only one parameter and that is action, which runs the specific method within a class if the pose is triggered.

Events (Poses in 3.4)

They capture specific movements that lead to some outputs. I have already shown how to define a pose in 3.4 in the Mode Section, so I will just show what an event looks like in 3.2. They are all located in the “data/events.json”:

"joystick_dpad_up": {
        "type": "JoystickButtonPressEvent",
        "args": {
            "button_name": "up",
            "gesture_type": "ring_fold"
        },
        "bodypart_names_to_type": { "Left": "hand" },
        "triggers": {
            "press": [ "GamingJoystick", "press_action" ],
            "release": [ "GamingJoystick", "release_action" ]
        }
}

Such an event is defined by its name and a number of properties: