Ambient Lockit Star Configuration Sensor Timeout Issue

Hello,

I have a 10x sensor setup that has been running into an issue recently. I had the below set up working before:

but recently, i did an update to the depth kit version, now this set up runs at 3 fps, and is not synced with time. however, when i disconnected or shutdown the Ambient Lockit , it starts running a 25 fps but the sensors are not time synced. and when i switch the setup to a sensor only star config like the below:

everything runs fine with this setup.

the error I’m getting when using the Lockit is

image1090×619 38.3 KB

so even though the depth kit studio shows that all sensors are synced, its being timed out causing the low fps, and also not time synced.

any ideas what could be cause of this issue?

@BUTNV Can you provide some more information to help me reproduce the issue on our end?

• Which version of Depthkit were you using when the sensors were synchronized using Star Config (External Sync) and operating normally?
• Does this same issue occur with just two sensors (the minimum number needed to start the Depthkit Studio pipeline, both receiving sync from the Lockit)? If not, how many sensors are connected when the issue appears?

Hi Cory,

-0.7.1 was the last time we tested with a lockit and it worked

-The same issue does not occur with just two sensors, sync issues start appearing once we get to 5 sensors and get worse from that point after. at 10 sensors, at least 4 sensors have a half a second delay between each other. all the sensors are running at 1080p

-I did find a work around however, if we start the 10 sensor set up with the lockit being off , it will start showing the sensors are awaiting to by synced, i turn on the lockit, then turn it off immediately after the sensors are synced, it works without FPS issues and time sync is fine. I made sure to double check whether the sync cables are all in the right ports in both the kinects and the sync box, and doesn’t seem like there is any mistakes there. my guess it maybe with the new “determining sync topology” loading bar you added in version 0.8.1?
anyway it works for now but I’ll need to integrate audio into it later and i don’t think it will work if i leave the lockit off for the sync to work correctly. so any suggestions on better ways to fix this would be appreciated.

@BUTNV - Thank you for the additional information.

We did introduce some changes to how Depthkit detects sync topology between versions 0.7.1 and 0.8.0 to accommodate the introduction of support for the Orbbec Femto Bolt sensor for use with Depthkit Studio. With this release, the ability for Azure Kinect sensors to receive external sync signal from the Lockit sync generator was tested and verified.

I will try to reproduce your issue with this current Depthkit version once I am able to reconfigure our testing studio with more than 5 Azure Kinect sensors - likely toward the end of the month.

In the meantime, I am happy to hear you have a workaround which keeps the sensor frames in sync with each other. Because you are removing the external sync signal after the pipeline starts, my recommendation is to scrutinize how well the sync holds over time by capturing a long-duration capture, and include a clear sync reference (e.g. a clap visible to all sensors) at both beginning and end, then analyze the video from each sensor to ensure the sync reference falls on the same frame of each.

I’ll let you know once I have any further information. Please let us know if there are any updates on your end.

@BUTNV I am testing the external sync topologies you referenced above, and have validated that the latest version of Depthkit (0.8.1) starts up to 10 Azure Kinects in subordinate mode, and then when the external sync generator is connected/started, the sensors start streaming (and record) at full framerate.

Screenshot 2024-03-29 1431141920×1080 321 KB

It’s worth noting that the error you’re seeing is expected from some sensors occasionally when they drop frames (especially when not recording), but not as many and as often as you’re seeing. See the timestamps of the final 10 messages in the above screenshot.

That error may be the result of a bottleneck somewhere in the connection between the sensor and the CPU, for example using a slower USB port or an untested USB extender cable.

Can you provide more information on the following:

• Which color and depth resolution that each sensor is set to? The computer included in your Depthkit Studio Hardware package was tested at 1440p color up to 6 sensors, and 1080p color on up to 10 sensors. Are you able to stream and record with each sensor individually in the Depthkit Core (single-sensor) context?
• When streaming all sensors as well as recording, what percentages of the CPU and GPU are being utilized (as reported in Task Manager) when all sensors are set to 1080p?
• Have there been any other changes to your configuration? This could be anything from changing USB cables to running additional tasks on the computer.

Hi Cory,

sorry for the late reply and thanks for the response.

So i figured out the following can make it work:

-turning the lockit off
-starting depthkit streaming
-sensors are awaiting sync
-turn on the lockit as master (before i was just turning on the lockit and it was causing the de-sync and low fps)
-hold the green button until all sensors are synced
-streaming starts with no sync or fps issues.

however, I’m now getting this other issue where there is flickering of another kinect every few seconds. when i turn off the lockit while streaming, the flickering stops, when i turn it back on, it starts again. here is a link to a gif that shows what im talking about.

regarding your questions from your last response:
-all sensors are running at 1080p, and 640x576 depth
-CPU will be around 50-60% usage, GPU 70% usage
-same cables as before, only other application that we run on that machine is Spout to NDI for livestreaming

any suggestions regarding the flicker issue would be appreciated

@BUTNV - For the sync sequence, I recommend modifying your startup sequence as follows:

-turning the lockit on
-hold the green button until the Lockit is generating the sync signal
-disconnect the Lemo connector from the Lockit
-starting depthkit streaming
-sensors are awaiting sync
-connect the Lemo connector to the Lockit, thus providing the sensors with sync signal until all sensors are synced and start streaming

The intention behind this change is to avoid any unknown behavior caused by the Lockit starting generate sync signal while connected to the sensors. Connecting the Lockit after the Lockit is already generating ensures that the sensor will only receive valid signal to its Sync In port, and may eliminate the issue.

If that doesn’t resolve it, I suspect that the issue is occurring within the connection of one or two sensors.

• In the Core Record context, startup each sensor one at a time, ensuring the sensor is set to 640x576 NFOV Raw depth / 1920x1080p color resolution, and look for that same flickering.
• If you identify that one or two sensors specifically are flickering, trace their USB connections from the sensor to the computer, and reseat all of them. Ensure that the sensor is plugged into a verified USB port, as slower/incompatible ports can produce these results. You can also methodically narrow the source of the issue down by swapping the extender cables between two sensors and see if the issue stays with the sensor or moves with the cable. Do the same by trying to plug the flickering sensor into a different USB port.
• Be sure to test all sensors, as this may be occuring with more than one.
• Let us know your results.

@BUTNV Did my recommendation for a modified version of your startup sequence address the flickering issue? Let me know if the issue is still present - Otherwise, I will close the ticket. Thanks!

Hi Cory, going to need more time before going back to testing this as I’m currently busy with other work.

@BUTNV Thanks for the update - I’ll keep an eye out for more when you’re ready to resume testing.