Release 4.01.01. Import of LSP now matches to xLights effects.

[Gilrock]

scuba if you have somewhere you can host the files needed to do the import I'll download them and give it a try.

[logandc99]

Well I am totally blown away by how clever this software is. I just imported the megatree portion of my 52MB LSP sequence for Dance of the Sugar Plum Fairies which is made up of a fair portion of animations that I made using the animation tool in LSP, then tried the "promote node effects" and the megatree portion of the sequence came out absolutely perfect when I played it.
I did notice that I had to be a little patient with the computer. The first time I tried it, I wasn't sure if it was finished or not and started playing the sequence and errors appeared. The second time I let it do it's thing, went off to do something else and when I came back... voila! it was all there. Simply brilliant!

[scuba]

Gil
Please give this link a try:
https://www.dropbox.com/s/fnj7tb3880gof7q/Final%202013%20wizard%20of%20winter%2012-24.msq?dl=0

This is the .MSQ file for Wizards of Winter.  95 MB
John

[scuba]

Gil
This is a test sequence, Xlights files etc.  for the WIW import:
https://www.dropbox.com/sh/5hemo694oq2kf3s/AAAGzqh-g5H-pANH4ui_xlmBa?dl=0
John

[Gilrock]

I can't try this till tonight or tomorrow.  I can't see what's in your files right now but I will need your xlights_rgbeffects.xml file.

[dkulp]

Over 10GB to load that MSQ file as part of the import.    Works fine on my Mac, it definitely isn't going to work on the Windows builds.   Get a Mac.  :-)

Might need to flip it to using the pull parser (which we did for Vixen).   A pain to program and would be quite a bit slower due to having to parse the files twice, but would handle the larger files better.    That would just help for the import process though as it wouldn't hold the contents of the MSQ in memory like it does now.   It won't help at all with the actual xLights sequences in memory and the rendering of those.   Just importing the first 3 arches, promoting the effects (not many got promoted), and saving took 5 minutes to save and resulting in a 75MB xml file.


As far as I'm concerned, importing these into xLights like this is a very bad idea.   xLights is NOT designed for this.  The node (and strand)level effects are designed for handling relatively small number of individual items.   Maybe in the 100s at most.   It's a way to expand from the xLights "pixel effects" into allowing some level of control for AC and other single channel or simple RGB items.   It is definitely NOT meant for 1000's of pixels on large pixel based models.   It's a really bad idea to do that.   

If you insist on going down this path, you're quite likely going to be LESS happy with xLights than with LSP.   The performance is going to suck really really bad.   Memory usage will be outrageous.  It's not designed for this.   


Dan

[Gilrock]

I thought earlier in this thread the MSQ file was quoted as being 35 mb so I'm surprised that turns into 10GB.

Anyways I kinda agree with Dan that even if you could import everything as editable effects you're not going to want to mess with 1000's of individual effects.  I'm finding the same thing with my SuperStar sequences.  Even though I can import them and it seemed really cool I'm ending up with like 3,400 effects and thinking there's no way I'm going to ever want to manipulate all those effects.  So it's better to just import as a data layer and then use Nutcracker effects on a go forward basis.

[dkulp]

I thought earlier in this thread the MSQ file was quoted as being 35 mb so I'm surprised that turns into 10GB.

MSQ files are basically just ZIP files containing many XML files.   Since XML compresses very very well (and XML sequences compress even better since there are 1000's of identical strings), the MSQ is much smaller that the real data that would be loaded.   If you unzip the MSQ (rename to .zip and you can unzip it), you'll see that the XML in the Controllers directory is over 1.1GB in size.  Once loaded into the XML document structure in wxWidgets and all the strings parsed into unicode and such, it really is a lot of memory.  Since each node in the document is heap allocated and using pointers and such, it would be more on my machine due to the 64bit pointers, but it definitely is too much for the 32bit windows builds.

[gerry]

I concur with Dan as testing today via the promote nodes still gave me a large file.

I have been importing my LSP sequences via the data layer.
And in some cases , I have simply not exported some effects (LSP controller/model), from the LSP seq and therefore can do that component in xl.during the export process , u can select which items to export or simply clear effects from the controller before export.

Or I am guessing that you could have the best of both worlds..
Say I want to manipulate the LSP mega tree effects of a seq.
Export the LSP seq as a conductor seq without the Mega tree 'model'. Import via the data layer.
Then import only the mega tree as a effect import. Promote and manipulate.

I have tried this approach today and it works well.

[MrChristmas2000]

I have come to the conclusion that the only effects from LSP that would be worth fooling with in general is those for single DMX channel effects.

I am testing a process right now that will allow me to come close to building a new Nutcracker sequence with the same timing and as similar or better effects. It is essentially a two step process where I break out an element like my megatree into a separate LSP sequence then add a timing track in that sequence that follows the various Mega Tree effects. Then I build a timing mark excel spreadsheet that gives me start/end times for each of the different effects along with a description of the desired effect which can be converted to a timing track to import into Nutcracker. Then by opening my related Nutcracker sequence (the one where I have been importing the LSP data layer from LightElf.) and now import just that new LSP timing track and the newly built timing file I end up with a fairly good template to reconstruct the LSP effects in my Nutcracker sequence.

I hope all that makes sense. I will be completing my first run through this process over the next couple of days and will report back the results.

[logandc99]

Over 10GB to load that MSQ file as part of the import.    Works fine on my Mac, it definitely isn't going to work on the Windows builds.   Get a Mac.  :-)
Dan

I see in the above quote a comment about the process using a mac vs windows.
So is the formation of large files after importing an LSP sequence just a problem in windows or in general. I am using a mac and whilst I didn't import the  LSP sequence for all my models and just the effects on the megatree, the resulting fseq file was about 65.9MB and the XML file 48.2MB  (although admittedly the xml file was certainly bigger than the xml file for another sequence I have done so far wholly in Xlights which was only kbs in size). I was going to go ahead and import the rest of the effects of the LSP sequence since I was so impressed at how well the import process appeared to work but is this going to result in a very large xml file in the end and cause  problems in xlights even on a mac. I love the spped of xlights and certainly dont want to slow it down to LSP levels.
I thought the files sizes produced by the proces so far weren't too bad but then I really have not idea whether xlights works best with xml files measured in  kb vs something in the tens of MB. (i'm new to xlights so apologies if I have misunderstood the above discussion about importing from LSP).

[Gilrock]

The problem with imports like this is you are almost always ending up with 1000's of effects representing the data at the node level.  You kinda lose out on what makes xLights work well which is being able to represent an effect like a multi-color pinwheel with just a couple lines of XML text but it translates into possibly 1000's of nodes of data when rendered.  The same problem would exist if you tried to take our data output and import it back into effects.  It would be exponentially larger than the original XML that was capable of creating the same data.

What would really be the best thing is to figure out which effects are worth trying to copy from LSP and then providing you a way to create what you did in LSP and then you would get our awesome speed and small file sizes.

[logandc99]

Ok, thanks. Thats good to know. I was really only planning on bringing one sequence over from LSP largely because of the animations I had done in the LSP sequence and didn't want to or know how to recreate them in Xlights.

Slightly off the topic but is it possible to do animations for a matrix or megatree in xlights, like the ones you can do in LSP using the animations tool?
Is that what glediator does?

[MrChristmas2000]

What do you mean by the animations that you did in LSP?

Are you talking about the LSP 'Matrix Animator'?

What did you do with it that you want to do in Nutcracker?

Just curious how large is your matrix?

[logandc99]

Yes its the matrix animation tool. I have used it for my megatree for doing simple pictures that move across the tree or with the music. My megatree is made up of 12 x 5M 2811 strips giving me an effective matrix of 12 x 50.
As such, trying to put pre-made animated gifs on it doesnt work as the resolution of the tree matrix aint that good but by using the matrix animation tool in LSP I could do animations in a such away that they came up ok on the tree.