hifi: #21202 $200 - If hand controller tracking is lost, slowly return hands to side

Currently, if Oculus rift touch tracking is lost (which happens a lot), hands snap instantly back to side, resulting in confusion and loss of held objects, etc. This can sometimes happen for a single frame.

Fix this behavior by making the rift controllers ease toward the rest position when tracking is lost, over a period of a few seconds. Interpolate 2% of distance remaining to the rest position for each frame over which tracking is lost, so that the motion is smooth and appealing.

Comments & Activity

  • 1 yr, 1 mnth ago

    #21202 created by PhilipRosedale Status set to Bidding.

  • 1 yr, 1 mnth ago

    #21202 updated by PhilipRosedale Changes: Summary changed. Notes changed.

  • 1 yr, 1 mnth ago

    #21202 updated by themelissabrown Changes: Assignee changed.

  • 1 yr, 1 mnth ago

    A bid was placed on #21202

  • 1 yr, 1 mnth ago

    themelissabrown accepted 200.00 from scotbayless on #21202 Status set to In Progress.

  • Currently MyAvatar::updateSensorToWorldMatrix calls MyAvatar::updatePalms and then sets the avatar controller LEFT_HAND and RIGHT_HAND matrices. While it's certainly possible to make updateSensorToWorldMatrix aware of the type of hand controller being used, it's almost certainly better to apply this fix regardless of controller type.

    Assuming for the moment, that it's acceptable to make this solution device agnostic, one of two approaches would seem to make sense.

    1) Do a simple lerp between the current hand matrix and the newly returned matrix. This has the advantage of needing zero awareness of external data elements such as the avatar's skeleton and is minimally math intensive.

    2) Do a slerp around the relevant joints axes of rotation. This could present a number of pitfalls, but could potentially produce a smoother result once those pitfalls are overcome.

    I recommend we try option 1, but let me know what you think.
  • @scotbayless - we don't want to add any extra latency to the incoming controller data (from the Vive or Oculus) before applying it to the avatar's hands. Is that what would happen with #1?

    My limited understanding (without looking more at the code) is that the oculus (and possibly other controllers, as you say) will sometimes lose/end tracking and return some indication of that. What is happening right now is that upon that event, the hands/arms instantly return to the neutral position at the avatar's side. What I'd like is for the loss of tracking to invoke a long relaxation period where the hands return to the sides. The use case, for example, would be the one where you turn around (Oculus) or put the controller in a desk drawer (Vive).
  • Anything that modifies the positional data being fed to the avatar's controller has the potential to introduce latency. It's really more an issue of how/when damping is introduced.

    It's certainly possible to introduce a special case when tracking is lost. However, a more general solution would be to lerp from current position to target position only in cases where positional delta exceeds a tunable threshold. Normal tracking would then be effectively exempted while any sort of 'impossible' movement would be damped.

    Given the tracking rate of the Oculus and Vive controllers vs the outer bound for hand movement speed, a controller should never move more than between 5 and 6 inches between samples. A threshold of, say, 10 to 12 inches for introducing the lerp should make normal operation completely immune to any damping while still making it possible to smooth things out when necessary.

    Having said that, if you'd rather go with a solution that only lerps in the event that tracking is lost, that's totally cool.

    Just let me know your pref and I'll get it into my branch.
  • When hand controllers are disabled, the input plugin will will return an invalid pose. You can check this by explicitly checking the valid flag on the controller::Pose object. This valid flag, in turn, is passed to the Rig object via the Rig::HandParameters structure. Within the Rig::updateFromHandParameters method, we change the IK target type based on this flag.

    For this project, the best approach would be to animate the position of the IK target to smoothly lerp from the last valid hand controller pose to the hand position from the currently playing pre-IK animation.

    The difficulty will be determining what the hand position of the currently playing animation is. Perhaps the best place to make this change would be within the AnimInverseKinematics node, during the evaulate call, it has access to both the desired IK target hand position, and the one in the underlying animation.

    Here are some potential approaches:

    1) the AnimInverseKinematics node can keep track of when IK target types change and smooth the IK targets internally.

    2) the Rig can recurse into the anim graph to find the main animation state machine (pre IK) to retrieve the hand pose to interpolate towards. Then just pass this position/rotation to the IK system.

    Hope this helps,
  • I think we're more interested in smoothing the IK when the inputPlugin returns "invalid" poses, not so much adding logic to detect invalid poses due to tracking glitches...
  • Yep. Very much appreciate the input. I was thinking in terms of staying as animation/rig/controller agnostic as possible, hence the suggested intervention at a lower level.

    But yes absolutely this approach should work nicely.
  • @scotbayless checking in to see how this is going
  • 1 yr, 1 mnth ago

    #21202 updated by themelissabrown Changes: Status set to Bidding.

  • 1 yr, 1 mnth ago

    A bid was deleted from #21202

  • 1 yr ago

    A bid was placed on #21202

  • 1 yr ago

    themelissabrown accepted 200.00 from LeonCrew on #21202 Status set to In Progress.

  • Hi @LeonCrew I tried to get the previous code checked in on this but the initial developer didn't have much. So, please feel free to create your own branch and get started. Let me know if you have questions.
  • 1 yr ago

    LeonCrew added a fee of $400 to #21202

  • @LeonCrew Feel free to ask questions here or in if you have any issues
  • Hi @LeonCrew Checking on this task. How's it going?
  • 10 mnths, 28 days ago

    A bid was deleted from #21202

  • 10 mnths, 28 days ago

    themelissabrown deleted a fee from LeonCrew on #21202

  • 10 mnths, 28 days ago

    A bid was placed on #21202

  • 10 mnths, 27 days ago

    themelissabrown accepted 200.00 from ctrlaltdavid on #21202 Status set to In Progress.

  • The following error was returned when making your pull request:
    A pull request already exists for ctrlaltdavid:21202.
  • 10 mnths, 22 days ago

    #21202 updated by themelissabrown Changes: Status set to Code Review.

  • 10 mnths, 20 days ago

    #21202 updated by themelissabrown Changes: Status set to Done.

Labels Saved!


Login to bid
Who Amount Done in ...
*name hidden*$ ***2 days