UMS 7.0.1 Crashing on Pi with Raspbian Stretch Lots of mp3s

For help and support with Linux-specific issues
Forum rules
Please make sure you follow the Problem Reporting Guidelines before posting if you want a reply
wired4net
Posts: 5
Joined: Mon Apr 23, 2018 7:45 am

Re: UMS 7.0.1 Crashing on Pi with Raspbian Stretch Lots of mp3s

Post by wired4net » Tue May 15, 2018 1:24 pm

Again, thanks for the detailed reply -- and, of course, for your work in supporting this fine streaming server.

I restarted the test and after the activity settled down (actually the next day), tried to open one album (not in a playlist). This set off another long session of database activity where the program tried to add what seemed like every mp3 to the database (again?). The Memory Usage soon became complete (all 512Meg pegged), so you were right about the 1GB being kind of small. (I think I will move it back to 768MB and avoid streaming video and/or doing anything else while it is processing) There were many database errors adding occasional particular songs, and then finally I got a java.lang OutOfMemoryError.

I am not good enough with forum use to answer questions in-line, so here are some answers to your specific points:

There are clearly some "bad" mp3 files. That "Love is blind" song, for example, causes an error in the MediaInfoOnline. I processed it with MP3Diags and then it show as a correct file in MediaInfoOnline. This is my fault, but hopefully it is not responsible for any fatal errors in processing.

I am beginning to get the impression that this application is not set up to do all the pre-processing at the beginning so that, at least for playlists, you cannot expect it to respond promptly and easily to simply ask for a (previously processed) playlist to start streaming. Perhaps this is because, as you said, playlists are not always a high-priority feature.

My RPi3B system has a USB2-attached 2TB 2 1/2 inch 5400RPM drive in an enclosure. It is a "Seagate 2TB BarraCuda SATA 6Gb/s 128MB Cache 2.5-Inch 7mm Internal Hard Drive (ST2000LM015) ". It is mapped as a network drive to my Windows 10 PC, and copying files to/from it during UMS processing was a dumb idea.

The database is on the internal microSD card, which is I think a 32gb Class 10 card with about 9GB free right now. I would be happy to try moving it to the attached HD if you think it would help, and if you could tell me how.

Cheers to Norway! BTW, assuming English is not your native language, I wish I could speak (and write) ANY 2nd language as well as you do English.

Nadahar
Posts: 1238
Joined: Tue Jun 09, 2015 5:57 pm

Re: UMS 7.0.1 Crashing on Pi with Raspbian Stretch Lots of mp3s

Post by Nadahar » Wed May 16, 2018 1:26 am

wired4net wrote:
Tue May 15, 2018 1:24 pm
I restarted the test and after the activity settled down (actually the next day), tried to open one album (not in a playlist). This set off another long session of database activity where the program tried to add what seemed like every mp3 to the database (again?). The Memory Usage soon became complete (all 512Meg pegged), so you were right about the 1GB being kind of small. (I think I will move it back to 768MB and avoid streaming video and/or doing anything else while it is processing) There were many database errors adding occasional particular songs, and then finally I got a java.lang OutOfMemoryError.
This is confusing me. UMS will refresh every folder you enter from a renderer, but it shouldn't actually parse the files again, it should get their parsed values from the database if they exist. I don't understand why this would fill your memory, unless it was already very close to full. You might try to set the memory somewhat up again, but I'd be careful with setting it higher than what you actually have available. When I read your log I think I remember that you only had around 700 MB free while Java was "assigned" 768 MB. I honestly don't know how Java deals with that, but it sounds risky.

I don't understand why there would be database errors adding songs, I'd like to see those errors. Whatever is causing this might also cause the potentially unusual memory consumption.
wired4net wrote:
Tue May 15, 2018 1:24 pm
There are clearly some "bad" mp3 files. That "Love is blind" song, for example, causes an error in the MediaInfoOnline. I processed it with MP3Diags and then it show as a correct file in MediaInfoOnline. This is my fault, but hopefully it is not responsible for any fatal errors in processing.
It shouldn't pose a problem except that some warnings or errors might be logged. It's just useful to know what is caused by bad files and what is caused by bad behavior (by UMS) when we try to troubleshoot.
wired4net wrote:
Tue May 15, 2018 1:24 pm
I am beginning to get the impression that this application is not set up to do all the pre-processing at the beginning so that, at least for playlists, you cannot expect it to respond promptly and easily to simply ask for a (previously processed) playlist to start streaming. Perhaps this is because, as you said, playlists are not always a high-priority feature.
I still haven't dug into the playlist code, but they way I think it should work is that when you open a playlist UMS will "read" that playlist and build a "virtual folder" with its content. This should be (relatively) quick though. I suspect that some of the problem here is that the database is performing very poorly because of the storage medium.
wired4net wrote:
Tue May 15, 2018 1:24 pm
My RPi3B system has a USB2-attached 2TB 2 1/2 inch 5400RPM drive in an enclosure. It is a "Seagate 2TB BarraCuda SATA 6Gb/s 128MB Cache 2.5-Inch 7mm Internal Hard Drive (ST2000LM015) ". It is mapped as a network drive to my Windows 10 PC, and copying files to/from it during UMS processing was a dumb idea.

The database is on the internal microSD card, which is I think a 32gb Class 10 card with about 9GB free right now. I would be happy to try moving it to the attached HD if you think it would help, and if you could tell me how.
While this doesn't account for every error you see, it can explain the extreme slowness you experience. A Class 10 SD card can handle 10 MB/s while a modern SATA drive (not SSD) can handle somewhere around 200 MB/s. While your Seagate is no sprinter with its 5400RPM, it can probably outperform the SD card more than 10x. USB 2.0 is a serious limitation though, limiting transfer to around 35 MB/s.

Since 35 MB/s is still higher than 10 MB/s, it's worth a try to move the database. Sadly, because of the slow storage access, I don't think UMS will every perform anywhere near what it should otherwise be able to do with the CPU/RAM. I think I/O will always be the bottleneck here. You might have a better experience with UMS 6.8.0 that doesn't rely that much on the database, although it's hard to say.

To move the database, you need to use a different "profile folder". The "profile folder" is where UMS puts all its configuration and data. You can specify the profile folder when starting UMS, if none is specified it defaults to "~/.config/UMS". By starting UMS with "./UMS.sh profile=<folder>" you should be able to use any folder you want for this. To not having to start your configuration from scratch, you should probably copy your existing profile folder to the new location first.

I'm just thinking of another thing that might cause some serious slowness. UMS uses a temp folder (you can see where it's located in the log), and this normally ends up in the system's "/temp" folder. Since you have little RAM putting this on a RAMdisk isn't really an option. There's an undocumented option you an set in UMS.conf that will let you decide where the temp folder should be, "temp_directory = <folder>". You probably want to set this somewhere on the Seagate as well.
wired4net wrote:
Tue May 15, 2018 1:24 pm
Cheers to Norway! BTW, assuming English is not your native language, I wish I could speak (and write) ANY 2nd language as well as you do English.
Thanks, but it doesn't always work out for me. Sometimes I just can't figure out how to arrange a sentence so that it's readable for English speakers :roll: Speaking is also a whole different matter, I might know the words but my mouth doesn't really want to pronounce them :oops: I know that improves with a little bit of practice though, when I've been abroad where I've used English it takes about a week before my mouth starts cooperating.

Post Reply