UMS getting stuck, some hard to spot bug

General discussion about Universal Media Server (no support or requests)
bodom
Posts: 17
Joined: Wed Nov 29, 2017 9:18 am

Re: UMS getting stuck, some hard to spot bug

Post by bodom » Sun Feb 03, 2019 2:39 pm

It happened again. I now have additional info.

The system gets unusable because it hits the threads limit (cgroups). The result is being unable to fork (so it is impossible to login too). Luckily, this time i had left a shell open :mrgreen:

Long story short, the Java process "java -Xmx768M -Xss2048k -Dfile.encoding=UTF-8 -Djava.net.preferIPv4Stack=true -Djna.nosys=true -classpath update.jar:ums.jar net.pms.PMS trace" has 10806 (!!!) threads. Apparently no clue from the logs.

Any ideas?

Edit: after raising the max threads number, UMS is back in working state (without restart), but it keeps leaking threads (10828 now). Soon or later it will hit the hard (32767) thread limit anyway and definitely crash the system.

User avatar
owlhuang
Posts: 5
Joined: Wed Aug 17, 2016 6:55 pm

Re: UMS getting stuck, some hard to spot bug

Post by owlhuang » Wed May 15, 2019 11:02 am

Just want to jump in and I hit the same resource leak. UMS cannot create any new threads and hangs the system. I am running 8.1.0 but I believe it happens on 6.8.0, as well. Since it is not the first time, I also have a log tracking the number of threads UMS is using (by "ps -eLf") It shows the UMS runs out of all 32767 native threads at 7-ish am. I'll keep monitoring the new version.

Code: Select all

2019/05/12 01:00:02-0700 UMS: Total of 59 threads
2019/05/12 02:00:01-0700 UMS: Total of 60 threads
2019/05/12 03:00:01-0700 UMS: Total of 641 threads
2019/05/12 04:00:01-0700 UMS: Total of 9044 threads
2019/05/12 05:00:02-0700 UMS: Total of 16577 threads
2019/05/12 06:00:02-0700 UMS: Total of 24139 threads
2019/05/12 07:00:02-0700 UMS: Total of 31713 threads
 UMS: Total of  threads
 UMS: Total of  threads
2019/05/12 20:00:02-0700 UMS: Total of  threads

Code: Select all

DEBUG 2019-05-12 07:04:02.315 [UPNP-AliveMessageSender] Sending ALIVE...
DEBUG 2019-05-12 07:04:13.910 [UPNP-AliveMessageSender] Sending ALIVE...
DEBUG 2019-05-12 07:04:25.386 [UPNP-AliveMessageSender] Sending ALIVE...
INFO  2019-05-12 07:04:26.745 [cling-2] Exception in thread "cling-2" java.lang.OutOfMemoryError: unable to create new native thread
INFO  2019-05-12 07:04:26.745 [cling-2]         at java.lang.Thread.start0(Native Method)
INFO  2019-05-12 07:04:26.745 [cling-2]         at java.lang.Thread.start(Thread.java:717)
INFO  2019-05-12 07:04:26.745 [cling-2]         at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
INFO  2019-05-12 07:04:26.745 [cling-2]         at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1025)
INFO  2019-05-12 07:04:26.745 [cling-2]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
INFO  2019-05-12 07:04:26.745 [cling-2]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
INFO  2019-05-12 07:04:26.746 [cling-2]         at java.lang.Thread.run(Thread.java:748)
DEBUG 2019-05-12 07:04:36.941 [UPNP-AliveMessageSender] Sending ALIVE...
DEBUG 2019-05-12 07:04:48.419 [UPNP-AliveMessageSender] Sending ALIVE...
DEBUG 2019-05-12 07:04:59.893 [UPNP-AliveMessageSender] Sending ALIVE...

Post Reply