[RESOLVED] Adding audio to WebXR-optimized geometry sequences for 8th Wall or Depthkit.js

Creating a new thread in response to @Terence 's question originally posted here as that thread is for general discussion of the new Depthkit version.

That’s great to hear Cory. What is the easiest workflow to trim captures and add audio prior to including the trimmed asset and audio in the WebXR 8thWall export function?

Embedding audio into the texture MP4 (part of the complete set of assets exported in the WebXR-optimized geometry sequence format) is covered in the Embedding audio in Depthkit WebXR texture video using Premiere section of our documentation.

There is a variation of this workflow where you can use Premiere to line up the audio, then export only the audio, then use FFMPEG to mux it into the original Depthkit MP4 to avoid any quality loss from decompression/recompression, but the one described in the the docs linked above is more user-friendly.

Thank you Cory

Great to now understand how to add and synchronise audio prior to inclusion in an WebXR export.

Two questions remain from my query:

1. How to clip captures at the start and end?
2. Did you include sound in Rong’s WebXR export?

Kind regards
Terry

@Terence Can you clarify what you mean by adding synchronized audio prior to inclusion in the WebXR format?

Clipping the Depthkit capture itself is done in Depthkit by using the In and Out markers to determine which section of the clip is exported.

Clipping the audio file is done in Premiere when combining the Depthkit texture video with the audio clip on a timeline, as mentioned in the linked documentation:

If necessary, trim and move the audio clip along the timeline until it is in sync with the video. Using a clap or slate as a sync reference is useful for this.

The Rong clip I sent does indeed include audio embedded in the texture MP4, which can be confirmed by opening the texture MP4 in a standard video player like VLC. I used the steps described in the documentation to produce that very asset.

It’s worth mentioning that I also needed to apply a fair amount of amplification/gain and noise reduction to the Rong clip using Premiere’s Audio Effects to get it to sound the way it does.

That’s very helpful Cory. Thank you so much.
We are ok from herein with what you have advised.

Kind regards
Terry

Great, marking as resolved.

Hi Cory

Please could you send me the completed project for Rong. I think you only sent the final WebXR export to Rafa? It will be a great example before we start our own post processing and WebXR export starting tomorrow afternoon.

Thanks again

Terry

@Terence, I have attached the Rong project JSON as I last left it. If you archive the original JSON and put this one in its place, the project should open with the surface reconstruction, texturing, and export settings I last used. Hope this helps.
dkproject.json (694.8 KB)

Great Cory. You are a star ⭐⭐⭐⭐⭐