view
Tweet
hifi: #21612 *Vive Required* Adjust Camera behavior when "Use Vive devices in Desktop mode" is used

Dancers and musicians when performing need to see and can't wear an HMD which is why we support a head puck.

The problem is when using the head puck in desktop mode, the screen/camera tilts and rolls wildly as the head puck moves instead of being stable.

This is hard to see if you don't have head pucks, but the HMD can be a fairly close representation.

Reproduction:
In desktop mode:
1. Open Settings > Controller Settings
2. Enable "Use Vive devices in desktop mode"
3. Grab your Vive HMD with your hands and move it around. Notice that the whole screen shakes with the head movement.

Expected Behavior:
The vive hmd can still be used in desktop mode to control your avatar's head movement, but that the camera will not tilt wildly.
------------------------------------------------

This would just be a C++ change, no UI change needed.

Comments & Activity

  • 1 yr ago

    #21612 created by MiladNazeri Status set to Bidding.

  • 1 yr ago

    #21612 updated by MiladNazeri Changes: Summary changed. Notes changed.

  • 1 yr ago

    Menithal added a fee of $350 to #21612

  • 12 mnths, 3 days ago

    themelissabrown deleted a fee from Menithal on #21612

  • @Menithal please add as a bid rather than fees
  • @themelissabrown oh my bad, accidentally clicked on that late in the night :)
  • 12 mnths, 3 days ago

    A bid was placed on #21612

  • 12 mnths, 3 days ago

    MiladNazeri accepted 350.00 from Menithal on #21612 Status set to In Progress.

  • Okay, there is a much deeper issue here than just the "camera moving and rotating wildly", which I don't think I have the experience to correct / fix.

    So I will be forced to drop this work list ticket for someone else to take over, unless someone points me to the right direction in this case:

    https://youtu.be/CLrpPKvKfBM

    I had stabilized the camera by experimenting with temp removing the hmd check for Third Person view, the orientation of the camera. This was defined around 2401 of Application.cpp

    I managed to stabilize the view point when desktop tracking is enabled. I was going to create an Application wide flag that would be set by the plugin, but I ran into some a major issue, which probably also was causing the Camera issues in the first place:

    https://youtu.be/CLrpPKvKfBM?t=68
    The Avatar will still wildly twist as soon as soon as user faces towards their screen. (Direction which is configured via steamVR). Since the base line is the Camera using the Head orientation as reference, I think this is where the problems actually originate from, in the first place, not because of the Camera.
  • Menithal, if you aren't ready to drop this worklist just yet, would you be willing to push up a pr of your current work that I can label Do Not Merge to help in discussing your current approach?
  • I guess this issue is more complex than we once thought. The reason the IK is wildly twisting is that in HMD mode we specifically disable avatar rotations, as a hacky fix for having the third person camera swing behind the avatar as it rotates. See, MyAvatar::FollowHelper::shouldActivateRotation() in MyAvatar.cpp. You could try disabling that check. Then you will see the camera boom swing issue.

    Ideally the right fix would address both the camera rolling issue and the third person camera swinging issue.
  • @Menithal if you're not able to find a solution, I'll cancel this and return it to our fogbugz queue. As per tony, we'd need to address both issues and that might be more than you bargained for.
  • This is Tony's PR that I believe touches on the above issue:
    https://github.com/highfidelity/hifi/pull/12107
  • @Menithal what are your thoughts on this? Does the above PR help or do we need to take this off WL? You'll still be paid for the hours you've spent tracking it down.
  • @themelissabrown I have time this weekend to tackle this,
  • Ok I did a few quick tests https://youtu.be/vhZdiG9E1Cw and It looks doable to some degree now as the avatar is not folding: however the scope may have to change from above.

    I have an option, and would like some feedback in case there have been any changes to ideas, which I can implement over the week.

    I can try to stop some roll and tilt from occurring on to the Camera,
    but It may cause extended issues with view ability when the user looks up and down.

    I can clamp this to rotate to an min or max when looking up or down, but this does not solve the issue of the camera being able to move quite violently.

    There also is the possible issues of "what if there is something that is inbetween the performed and the camera" that can block it. It may need some tuning to get it right.

    This matches the worklist spec, but I'd need some information about how it should behave, and I think the end result will still cause the entire camera to rotate quite quickly as the user turns their head. is this swing intentional to the spec? etc Should the camera basically be an external camera that is like at an end of stick pointing at the user's head?

    However I think the simplest method would be to a create a Camera app instead that follows a damped point while follows the user and making sure the camera always stays infront of any objects, but for the desktop, where the camera still follows the user, but would dampen and try to find optimal view to user. I find this to be a bit easier to implement, and it could be good reference for others using the camera.

    This however does not match the worklist item on C++ change,
  • > I can try to stop some roll and tilt from occurring on to the Camera.

    I think we want to stop all rotation actually, but only in the case where the head is being tracked (via a puck) but you are still in desktop mode.

    > but It may cause extended issues with view ability when the user looks up and down.

    I think this is OK, as the user will be looking at a monitor, and likely pulled back quite a bit in order to see the full avatar on screen.

    > There also is the possible issues of "what if there is something that is in between the performed and the camera" that can block it.

    For now, I wouldn't worry about this case.

    The experience we want is similar to your video where you switched into independent camera mode. Except that we want the translation of the third person camera to follow the avatar as it's moves.

    > However I think the simplest method would be to a create a Camera app

    I don't think we need a sperate app for this, just special case the third person camera logic when the user is in desktop mode and the head is being tracked. i.e. the head input pose is valid.
  • > The experience we want is similar to your video where you switched into independent camera mode. Except that we want the translation of the third person camera to follow the avatar as it's moves.

    Ok, Will go forward with that.
  • 9 mnths, 14 days ago

    maromero3d uploaded an attachment to #21612

  • 9 mnths, 14 days ago

    maromero3d uploaded an attachment to #21612

  • @Menithal, checking in with you on this to see if there are any updates
  • @MiladNazeri I am gonna drop this worklist to focus on other things. Sorta run out of time. Ill let someone else do it, unless I have more time again in a few weeks.
  • 8 mnths, 25 days ago

    A bid was deleted from #21612

Labels Saved!

Bids

Login to bid
Who Amount Done in ...
No bids yet.
Login to add fees