FPS freezes for a while every some time

Installation and usage on Windows / Installation unter Windows
azaz44
Posts: 32
Joined: 08 Jun 2022 06:56

FPS freezes for a while every some time

Post by azaz44 »

I run Heli-X on Windows, gaming laptop, VR, HP Reverb G2.
It is running smooth, but every couple of seconds it freezees for maybe a quarter of a second, and then it is smooth again.
So in general it works, but it is little annoying.

Anobody had similar issue maybe? I think my other software is fine, because I don't have it in other sims I use, but I'm open to any suggestions on tracing the problem.

I tried to change java process priority to high, but this didn't help.
azaz44
Posts: 32
Joined: 08 Jun 2022 06:56

Re: FPS freezes for a while every some time

Post by azaz44 »

I made a bit of investigation. I ran Process Monitor because I thought that these "glitches" happen because maybe Heli-X loads some files, like sounds or whatever. Especially because they happen more often right after you start it, and it gets more stable after a couple of minutes.

I didn't find any of that, but anytime a "glitch" (it freezes for a couple of frames) happened, there was "Thread Create" operation logged in Process Monitor.

Not every "Thread Create" resulted in a freeze. But every freeze resulted in "Thread Create" being logged.

is it maybe Java garbage collection triggering?

Screenshot 2022-07-12 181840.png
Screenshot 2022-07-12 181840.png (63.08 KiB) Viewed 4850 times
User avatar
Michael
Posts: 1825
Joined: 25 May 2008 16:11
Location: Germany
Contact:

Re: FPS freezes for a while every some time

Post by Michael »

Hi,

what you can try:

Try a small resolution and see if it works better.
Try without a model adn see if it works better.

in the startscripts, you could also play with java parameters:

-Xms512m gves ths initial stack size, which could be enlarged.

I am not sure if it is the garbage collection. In the start script you could try different options:

E.g. -XX:MaxGCPauseMillis=10

https://docs.oracle.com/en/java/javase/ ... 670B44E49C

Does it help?
azaz44
Posts: 32
Joined: 08 Jun 2022 06:56

Re: FPS freezes for a while every some time

Post by azaz44 »

Hi Michael. Excuse me late answer, I thought I'm subscribed to this thread and will get email notification, but I probably wasn't.

I was able to trace gc with -Xlog:gc option, and it looks like it's not GC. There's nothing about GC when "freezes" happen. And also sometimes GC happens, but then nothing freezes.

I also tried -Xlog:all, but this is logging stuff all the time and it's hard to browse.

But increasing initial heap to 2048 or 4096 might help, I increase it and will keep an eye on it.
azaz44
Posts: 32
Joined: 08 Jun 2022 06:56

Re: FPS freezes for a while every some time

Post by azaz44 »

I made quite some research on this topic today.

It looks like the reason is JIT compiler. I added logging around threads (-Xlog:thread*=debug) and any time I get this "freeze" and FPS drop, I get something like this in the console:

Code: Select all

[123.000s][debug][thread,smr     ] tid=35520: ThreadsSMRSupport::free_list: threads=0x0000018d0c5fd700 is freed.
[123.002s][debug][os,thread,timer] name='C2 CompilerThread2', exit-phase1=0, exit-phase2=0, exit-phase3=1, exit-phase4=2
[123.003s][debug][thread,smr     ] tid=35520: ThreadsSMRSupport::smr_delete: thread=0x0000018d0311f800 is deleted.
[123.004s][info ][os,thread      ] Thread finished (tid: 35520).
[123.120s][debug][os,thread      ] Thread 71272 stack guard pages removed: 0x000000038f000000-0x000000038f004000.
[123.121s][info ][os,thread      ] JavaThread exiting (tid: 71272).
[123.121s][debug][thread,smr     ] tid=71272: Threads::remove: new ThreadsList=0x0000018d1637dd00
[123.122s][debug][thread,smr     ] tid=71272: ThreadsSMRSupport::free_list: threads=0x0000018d0c7224c0 is freed.
[123.124s][debug][os,thread,timer] name='C2 CompilerThread1', exit-phase1=0, exit-phase2=0, exit-phase3=1, exit-phase4=2
[123.125s][debug][thread,smr     ] tid=71272: ThreadsSMRSupport::smr_delete: thread=0x0000018d140ad800 is deleted.
[123.127s][info ][os,thread      ] Thread finished (tid: 71272).
or this

Code: Select all

[111.440s][info ][os,thread      ] Thread started (tid: 75588, attributes: stacksize: default, flags: CREATE_SUSPENDED STACK_SIZE_PARAM_IS)
[111.441s][debug][thread,smr     ] tid=68832: Threads::add: new ThreadsList=0x0000018d04e60180
[111.445s][debug][thread,smr     ] tid=68832: ThreadsSMRSupport::free_list: threads=0x0000018d0aa3c730 is freed.
[111.446s][info ][os,thread      ] Thread is alive (tid: 75588).
[111.447s][debug][os,thread      ] Thread 75588 stack dimensions: 0x000000166f200000-0x000000166f300000 (1024k).
[111.447s][debug][os,thread      ] Thread 75588 stack guard pages activated: 0x000000166f200000-0x000000166f204000.
[112.795s][debug][os,thread      ] Thread 75588 stack guard pages removed: 0x000000166f200000-0x000000166f204000.
[112.795s][info ][os,thread      ] JavaThread exiting (tid: 75588).
[112.796s][debug][thread,smr     ] tid=75588: Threads::remove: new ThreadsList=0x0000018d00d03790
[112.801s][debug][thread,smr     ] tid=75588: ThreadsSMRSupport::free_list: threads=0x0000018d04e60180 is freed.
[112.802s][debug][os,thread,timer] name='C2 CompilerThread1', exit-phase1=0, exit-phase2=0, exit-phase3=1, exit-phase4=5
[112.803s][debug][thread,smr     ] tid=75588: ThreadsSMRSupport::smr_delete: thread=0x0000018d0783c000 is deleted.
[112.804s][info ][os,thread      ] Thread finished (tid: 75588).
and actually I was able to make Heli-X run smooth by adding this to java command line:

Code: Select all

-client -XX:CompileThreshold=500
both affect how JIT works. It basically makes Heli-X stable (not doing a lot of JIT) very quickly.

Can also debug JIT with this switch:

Code: Select all

 -XX:+PrintCompilation
 
Maybe you want to look at it. I think "-client" might be a better choice for Heli-X. The other switch might need some tweaking, 500 works for me although it is a bit aggressive.
azaz44
Posts: 32
Joined: 08 Jun 2022 06:56

Re: FPS freezes for a while every some time

Post by azaz44 »

More info :)

It seems adjusting JIT helped a lot, but it still has glitches sometimes. So I was maybe happy too early. What I found
  • first of all, problem also happens in non-VR. I didn't notice before, because I fly VR 100% of the time.
  • after adjusting JIT (like in previous post), and using `-Xlog:thread*=debug` it now doesn't show anything about threads in the console, when problem happens
  • however Process Monitor shows clearly, that there is thread activity in such moments
  • that's all I know as of now..
I was able to record it, see YT link:

https://youtu.be/24Q-wJp1xGE

at 0:24.

It looks like there is a 2nd problem, or there is one problem and server version of JIT was just triggering it more often.
Like if there would be some issue in Java anytime it creates a new thread. And with JIT adjustments to client version it just doesn't do this often (C2 server-version JIT was creating a lot of threads).

But of course I have no idea what are these threads which are causing issues..
azaz44
Posts: 32
Joined: 08 Jun 2022 06:56

Re: FPS freezes for a while every some time

Post by azaz44 »

Sorry for spamming, but I have more insight.

First of all, this is how it looks over time, just to have idea how often things happen. Every "Thread Create" was connected to FPS drop.

Screenshot0.png
Screenshot0.png (120.47 KiB) Viewed 4517 times

I was able to record thread activity and it seems anytime this happens, there's thread creation / destroying around these two:

Screenshot.png
Screenshot.png (120.47 KiB) Viewed 4517 times

If I suspend these threads, it seems problem is gone. I will test over time, but I was running Heli-X for 15 minutes with no FPS drop and no new "Thread Created" events in process monitor. And before it never happened that it would run 5 minutes without a problem.

This is a workaround of course and maybe introduces some problems, not sure yet. I don't have time to research more at the moment, will research later..
azaz44
Posts: 32
Joined: 08 Jun 2022 06:56

Re: FPS freezes for a while every some time

Post by azaz44 »

Not a lot of new knowledge, but I can say now with confidence that suspending "TpReleaseCleanupGroupMembers" threads on Heli-X makes Heli-X run smooth.

Also, what this function is doing is release a thread pool (Tp = thread pool).

I haven't found a way to automate this yet, so I do this manually anytime I start Heli-X.

Why would the process release a thread pool every couple of minutes - this looks wrong. Not sure if it's a problem in Java, or some library, no idea.
azaz44
Posts: 32
Joined: 08 Jun 2022 06:56

Re: FPS freezes for a while every some time

Post by azaz44 »

Just as confirmation, this really works. I always suspend these "TpReleaseCleanupGroupMembers" threads after starting Heli-X and this makes it run smooth. It would be great if this problem could be somehow addressed, because it's a super long procedure to get Heli-X running this way...
azaz44
Posts: 32
Joined: 08 Jun 2022 06:56

Re: FPS freezes for a while every some time

Post by azaz44 »

It's great to see 10.1 version!
But this problem is still there. I workaround it by suspending threads every time I start Heli-X, but this is very annoying.

I wonder if I can help somehow to track this? Or is this problematic to reproduce?
Post Reply