Studio Mesh Exporter Issues

Hey ya’ll!

Running into some errors with the Studio Mesh Exporter workflow. I’m using the Phase 8 plug-in with Unity 2020.3.30f1. I’ve followed the guide here to get things setup. After the initial setup, I wasn’t seeing any errors - when I hit ‘Start Export’ the capture starts playing back choppily and then locks up and starts tossing a bunch of errors like so:

Mesh contains no data at frame 11
UnityEngine.Debug:Log (object)
Depthkit.GeometrySequenceExporter/<ProcessRecording>d__34:MoveNext () (at C:/Users/jesse/Downloads/Depthkit_Studio_Expansion_Package_Phase8_030722/depthkit.studio.meshsequence/Runtime/GeometrySequenceExporter.cs:238)
UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&)

and

ArgumentException: ComputeBuffer.GetData() : Accessing 37631952 bytes at offset 0 for Compute Buffer of size 28800000 bytes is not possible.
UnityEngine.ComputeBuffer.GetData (System.Array data) (at <3be1a7ff939c43f181c0a10b5a0189ac>:0)
Depthkit.MeshDownloader.OnGenerate () (at C:/Users/jesse/Downloads/Depthkit_Studio_Expansion_Package_Phase8_030722/depthkit.studio.meshsequence/Runtime/MeshDownloader.cs:69)
Depthkit.DataSource.Generate () (at C:/Users/jesse/Downloads/Depthkit_Studio_Expansion_Package_Phase8_030722/depthkit.core/Runtime/DataSource/DataSource.cs:261)
Depthkit.DataSource.Generate () (at C:/Users/jesse/Downloads/Depthkit_Studio_Expansion_Package_Phase8_030722/depthkit.core/Runtime/DataSource/DataSource.cs:270)
Depthkit.Clip.DoGenerate () (at C:/Users/jesse/Downloads/Depthkit_Studio_Expansion_Package_Phase8_030722/depthkit.core/Runtime/Clip.cs:529)
Depthkit.Clip.LateUpdate () (at C:/Users/jesse/Downloads/Depthkit_Studio_Expansion_Package_Phase8_030722/depthkit.core/Runtime/Clip.cs:729)

Here’s a video.

If I close down the project and reopen, I get these errors:

Shader warning in 'Depthkit Studio Mesh Source Extract Surface Compute': Program 'ExtractSurfaceFromVolumeWithUVs8x8x8', warning X4714: sum of temp registers and indexable temp registers times 512 threads exceeds the recommended total 16384.  Performance may be reduced at kernel ExtractSurfaceFromVolumeWithUVs8x8x8 (on d3d11)

and

Shader warning in 'Depthkit Studio Mesh Source Extract Surface Compute': pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them at kernel ExtractSurfaceFromVolumeWithUVs8x8x8 at C:/Users/jesse/Downloads/Depthkit_Studio_Expansion_Package_Phase8_030722/depthkit.studio/Runtime/Resources/Shaders/Includes/DepthkitStudio.cginc(51) (on d3d11)

Let me know if I could provide anything else that would help to diagnose the issue. Thank you!

Hi @JesseKirberger

Sorry for the trouble with the Mesh Exporter, and thanks for the thorough bug report log here.

From the video and logs, it looks to me like the Surface Buffer Capacity is set too high and as a result the export is eating all of your GPU memory and crashing.

To test this theory, replicate the steps in your video - but before hitting ‘Start Export’ press the “Set Surface Buffer Capacity” button, which will reset the number in that box from 200,000 to something much smaller.

A little explanation. The Surface Buffer Capacity sets the maximum number of triangles that can be rendered in each frame. Since the amount of data in each frame is variable, this is the upper limit. This number also represents an amount of memory that is being allocated, so left too high it may cause memory issues like what you are seeing in the logs there.

Note that the right buffer capacity is computed based on the current amount of geometry actually rendering in the frame the moment when you hit “Set Surface Buffer Capacity”, plus some padding. So make sure you are seeing a representative frame 3D rendered in the scene view when you hit it.

Let us know if this helps get your exports moving!

Thanks for the reply, @JamesGeorge!

Ah yes, this makes sense and reminds me that the initial bug I encountered was that hitting ‘Set Surface Buffer Capacity’ does nothing with the Image Sequence setup. It stays stuck at 200000.

I’m also not seeing any sort of representative frame rendered in 3D in the scene view. In the guide it says to disable the poster frame, but I’m not sure if that’s the issue or something more.

Hey @JamesGeorge - checking back in on this. Went through setup again and I’m still encountering the issue where I’m not seeing anything rendering in Scene view, and I can’t Set Surface Buffer Capacity. Let me know if you need me to do any tests/provide console output. Thank you!

Hi @JesseKirberger

To set the surface buffer capacity you need to make sure the poster frame (or another frame) is rendering and that the Unity game state is in Edit mode, not Play mode. The capacity only needs to be set once, for each clip.

So I suggest enabling the poster frame to get the 3D to preview render, then click “Set Surface Buffer Capacity”, then disable it again,.

Alternatively, if you have set up the clip to work in the the Unity timeline which does require disabling the poster frame, you can simply scrub the timeline to get a frame to show up that way, then click “Set Surface Buffer Capacity”

Let me know if this helps.

Hey @JamesGeorge

I am using the Unity Timeline, as per the Studio Mesh Exporter documentation. Scrubbing the timeline does not result in any frames rendering. The PNGs are all visible in the Timeline Image Sequence, but nothing appears in the Scene or Game views. Anything else I can try?

Hey Jesse, not sure if this will help but maybe you can try and toggle the gizmo button, and make the bounding box smaller so that only your captured volume is in view. And then press “Set Surface Buffer Capacity”

Good suggestion @GlennWustlich. @JesseKirberger, if you are seeing errors while scrubbing, then it could be that the buffers are overflowing immediately. Shrinking the bounds is a great way to prevent this.

With regards to getting a 3D preview- in the video you posted it shows the 3D previewing working just fine. Are you saying that you can no longer get it to show up at all? If so can you please post another video of the current error scenario?

I’m referring to this frame:

Screen Shot 2022-03-28 at 9.53.02 AM1292×586 106 KB

Thanks for the suggestion, @GlennWustlich. In that particular project I tried adjusting the bounding box size and hitting ‘Set Surface Buffer Capacity’, but since nothing was rendering it didn’t have an effect.

@JamesGeorge - It wasn’t tossing errors while scrubbing, it simply wasn’t rendering anything at all, so the ‘Set Surface Buffer Capacity’ button wasn’t doing anything.

That particular section of the video you’re referencing was only after hitting Play and then starting the mesh export process. When it was stopped, nothing was rendering in the preview window.

I tried starting a fresh project again and going through setup and now I am getting preview rendering when scrubbing the timeline, although I’m still experiencing some weird behavior. When I hit play on the Timeline (not hitting Play in Unity to enter Game mode, but hitting play on the Timeline UI) it starts playing back, but the image flickers randomly. I also get the following console error:

[SIS] Out of memory when loading images
UnityEngine.Debug:LogWarning (object)
Unity.StreamingImageSequence.ImageLoader:GetImageDataInto (string,int,Unity.StreamingImageSequence.ImageData&) (at Library/PackageCache/com.unity.streaming-image-sequence@0.14.2-preview/Runtime/Scripts/Loader/ImageLoader.cs:60)
Unity.StreamingImageSequence.StreamingImageSequencePlayableAsset:QueueImageLoadTask (int,Unity.StreamingImageSequence.ImageData&) (at Library/PackageCache/com.unity.streaming-image-sequence@0.14.2-preview/Runtime/Scripts/Features/SIS/StreamingImageSequencePlayableAsset.cs:334)
Unity.StreamingImageSequence.StreamingImageSequencePlayableAsset:ContinuePreloadingImages () (at Library/PackageCache/com.unity.streaming-image-sequence@0.14.2-preview/Runtime/Scripts/Features/SIS/StreamingImageSequencePlayableAsset.cs:302)
Unity.StreamingImageSequence.Editor.SISPlayableMixerEditorTask:Execute () (at Library/PackageCache/com.unity.streaming-image-sequence@0.14.2-preview/Editor/Scripts/EditorManagers/SISPlayableMixerEditorTask.cs:70)
Unity.StreamingImageSequence.Editor.EditorApplicationManager:EditorApplicationManager_Update () (at Library/PackageCache/com.unity.streaming-image-sequence@0.14.2-preview/Editor/Scripts/EditorManagers/EditorApplicationManager.cs:104)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

It seems to break further if I hit Play to enter Game mode, which is why I think I was having issues with the previous project not rendering at all. After hitting Play and then Stop, rendering is intermittent or non-existent in the preview window while scrubbing the Timeline or playing from the Timeline UI.

Here’s a video of the flickering.

However it does look like it’s exporting correctly now. Will the flickering cause any issues with the export? Otherwise, looks like it’s good to go. Thanks for assistance!

Hi @JesseKirberger

Set Surface Buffer Capacity
I recommend using the post frame. Enable the poster frame in Edit mode in order to get a 3D preview to render, then hit Set Surface Buffer Capacity button to configure the memory allocation.

Flickering
Hitting play on the timeline like you are doing in the video will cause flickering because the images are too big to load in time to keep up with the timeline playback speed. We don’t intend to support realtime playback from image sequences.

Please refer to the documentation on how to correctly export Studio Mesh Sequences, specifically to the last section where we specify that to perform an export, you first hit Play mode in Unity then hit Start Export on the Depthkit Geometry Sequence Exporter component.

@James -

Hitting play on the timeline like you are doing in the video will cause flickering because the images are too big to load in time to keep up with the timeline playback speed. We don’t intend to support realtime playback from image sequences.

No worries - just reporting back on the behavior to make sure nothing is amiss. My main point in mentioning this was to say that hitting Play mode in Unity and then stopping it seems to cause intermittent issues rendering the preview frame, and therefore the Set Surface Buffer Capacity functionality as well - in case anyone has a similar issue in the future.

Please refer to the documentation on how to correctly export Studio Mesh Sequences, specifically to the last section where we specify that to perform an export, you first hit Play mode in Unity then hit Start Export on the Depthkit Geometry Sequence Exporter component.

Yep, I’m aware of the export workflow. In the last post I said I was able to export successfully. I just wanted to confirm that the Timeline flickering wouldn’t cause any dropped frames on export or anything. Thanks for helping me through this!

@JesseKirberger OK great! Glad to hear you are exporting successfully. Thanks for working through the gotchas.