Explanation

There are several reasons why the projected 3D skeleton or body segments may appear to be incorrect when reviewing a processed movement trial:

1. The GCVSPL Cutoff Frequency is too low for the movement.
   If the GCVSPL Cutoff Frequency is set too low for the movement contained in the videos, the filtered pose may be underfitting the movement and can cause the skeleton to be momentarily incorrect relative to the videos. This can show up as excessively smooth skeleton movements that do not fully capture the movements in the videos.
2. ‍The intrinsic lens calibration is inadequate.
   If the intrinsic lens calibration of your cameras provides relatively low coverage of the camera views or insufficient variation in chessboard angle during the lens calibration trial, it may not adequately adjust for lens distortion or other lens effects. This can result in non-linearities in the camera view(s), which can manifest as warping of the image around the outside of the camera view(s). If the skeleton is momentarily incorrect when the subject approaches the edges of one or more camera views, and/or the skeleton tracking gets worse the closer they are to the border of the camera view, then an inadequate lens calibration may be the cause.
3. The extrinsic chessboard calibration is inadequate.
   If the extrinsic calibration of your cameras has relatively high calibration error metrics (RMSE Diagonal, RMSE Angle), it may not provide reliable 3D reconstruction of predicted key points from the 2D camera views. This can result in the projected skeleton ‘drifting’ away from the participant as they move away from the calibrated capture volume origin. This drift can be further exacerbated by inadequate intrinsic lens calibration, which may further reduce the accuracy of the 3D projection when the subject nears the edges of the camera view(s). If the skeleton is momentarily incorrect when the subject moves away from the calibrated capture volume origin, an inadequate extrinsic calibration and/or intrinsic calibration may be the cause(s).
4. The participant’s body is not sufficiently visible for reliable construction.
   If the participant being tracked is momentarily occluded or contorted in such a way as to significantly reduce the visibility of one or more body segments, the keypoint detections and projected 3D reconstruction can become temporarily incorrect. This generally manifests as obviously incorrect reconstruction of the participant’s skeleton such as impossible body segment poses or movements, but it can also appear as believable movements that visibly disagree with the videos. If the momentarily incorrect body segment(s) are not clearly visible in three or more camera views when the tracking is incorrect, the visibility (or lack thereof) of the body segment may be the cause.
5. (OptiTrack Prime Color cameras) Frames dropped by the camera hardware were filled in using Dropped Frames: Last Frame.
   If there were any camera hardware issues that led to video frames being dropped during the recording of the trial, and the export setting Dropped Frames: Last Frames was used, the videos from cameras with dropped frames will have repeated identical frames for some duration of the video. That is, some videos may show the scene as perfectly stationary while the videos from other cameras that did not suffer dropped frames continue showing the movement. In this case, the person is usually tracked accurately based on the cameras without dropped frames, but the 3D reconstruction of the person’s movement will not align with the person in the videos with dropped frames. This is not incorrect tracking, but rather demonstrates the movement was tracked properly despite the videos showing the person at different instances in time.

Possible Solutions

Some possible solutions to momentarily incorrect skeleton tracking are as follows:

1. Increase the GCVSPL Cutoff Frequency.
   If the skeleton movement appears excessively smooth and is not fully capturing the movements in the video, try increasing the GCVSPL Cutoff Frequency to reduce the effect of the filter and allow the movement to be tracked more accurately. After adjusting the GCVSPL Cutoff Frequency in the Preferences window, you only need to run the Solve Skeleton analysis step to view the updated pose results.
2. Reprocess the lens and chessboard calibration trials.
   If the cause of the issue was an inadequate lens or chessboard calibration, the best approach is to reprocess both calibrations in an effort to improve their reults. Reprocess the lens calibration trial before reprocessing the chessboard calibration. Try using the Enhance Videos tool to improve the brightness, contrast, or white balance of the videos, or decreasing the Frame Grab Step value to increase the number of frames used to calibrate the lenses and camera system. After reprocessing the lens and chessboard calibrations, reprocess the movement.
3. Record a new chessboard calibration trial.
   If the results of the chessboard calibration trial are relatively low and you have tried enhancing the videos and adjusting the Frame Grab Step, you may need to record a new chessboard calibration trial, if possible.
4. Record a new lens calibration trial.
   If the results of the lens calibration trial are relatively low and you have tried enhancing the videos and adjusting the Frame Grab Step, you may need to record a new lens calibration trial. This necessitates removing the cameras from the capture volume, and is therefore a more significant undertaking that will also require a new chessboard to be recorded after the cameras are returned to their setup.