2019.23 Locks up on Save

[rickswa]

Thanks dkulp.  So far going back to v2 has fixed the issue for me.

[brichi]

Is it normal even in .24 for V2 FSEQ to take up to 40 seconds to save vs V1 taking 4 seconds?

I have a 4 minutes sequence with video in it for my p10's and on V1 it saves instantly but when i switch to v2 it locks up for 40 seconds when it saves.

If this isnt normal i can upload the sequence so you can take a look

Im on 2 macbook pro's and mac mini, i5 processors and 16GB of ram, SSD drive with plenty of room

heres a video comparing the V1 to V2 save - https://youtu.be/LfAuL1I8AKo
from roughly 20 seconds to 1 minute xlights is completely locked up on saving

[Gilrock]

I really don't know the details of the V1 vs V2 but I thought V2 was using compression.  So saving V1 data all xLights has to do is write what's already in the render buffer to a file.  For V2 it would need to compress that data first and then write to file.  I would be curious to know the 2 different file sizes you end up with.  Now 40 seconds sounds like way too long to do the compression.  I would expect it to only take a second or two.

[brichi]

I really don't know the details of the V1 vs V2 but I thought V2 was using compression.  So saving V1 data all xLights has to do is write what's already in the render buffer to a file.  For V2 it would need to compress that data first and then write to file.  I would be curious to know the 2 different file sizes you end up with.  Now 40 seconds sounds like way too long to do the compression.  I would expect it to only take a second or two.

exactly Gil, thats why I wasn't sure if its really a matter of it being an issue with Xlights or simply the fact that V2 is compressing it down.

On V1 the FSEQ is 1.49GB and V2 compresses it down to 335MB so that will obviously take some time but not sure if 40 seconds is too much time

[dkulp]

The compression will definitely take some time, but that big of a difference is much larger than I would have expected.  On my "GreatestShow" sequence, it goes from 1.8 seconds to 8.2 seconds, but the size drops from 1.1GB to 93MB.   

That said, it's something we can easily change.   The zstd compression library supports over 40 different compression levels.   The version on the Pi/BBB supports levels from 0-25 and the version we use for xLights supports negative levels down to -20 or so.     Currently, the default is level "10" as that seemed (at the time) to be a good balance between compression and speed, particularly on the Pi/BBB I was testing it on.   However, it may make sense to drop it lower.   For example, dropping to "1" drops the save time down to 2.1 seconds, but increases the file size to 104MB, so 10% larger.  On the flip side, increasing the level to 15 makes it take 35 seconds to save, but only drops the file size to 89MB.   

[brichi]

The compression will definitely take some time, but that big of a difference is much larger than I would have expected.  On my "GreatestShow" sequence, it goes from 1.8 seconds to 8.2 seconds, but the size drops from 1.1GB to 93MB.   

That said, it's something we can easily change.   The zstd compression library supports over 40 different compression levels.   The version on the Pi/BBB supports levels from 0-25 and the version we use for xLights supports negative levels down to -20 or so.     Currently, the default is level "10" as that seemed (at the time) to be a good balance between compression and speed, particularly on the Pi/BBB I was testing it on.   However, it may make sense to drop it lower.   For example, dropping to "1" drops the save time down to 2.1 seconds, but increases the file size to 104MB, so 10% larger.  On the flip side, increasing the level to 15 makes it take 35 seconds to save, but only drops the file size to 89MB.

gotcha! I can try a few versions out for ya to see whats a happy medium, I may be one of very few that have large FSEQ files of that size so wouldn't want the compressed file size to get bigger just to make a few people happy.