Making effects add-ins/plugin/extension

[MikeKrebs]

I would like to see effects become add-ins. I know it is probably major work, architecture re-write, could lead to stability issues, performance woes, viruses...

But. Think of the possibilities.
1. Now hundreds of new effects, enhancements,  and "I want it to do this" could be run without any effort on the developers.

2. By developing the addin architecture, Xlights/Nutcracker potentially makes it even easier for developers to create new or specific effects. One source file (RenderMyEffect) and one XML file (here is my meta information)?

3. Because of the cross platform nature, I am unsure how the source would turn into something that could be run. Maybe some automated "DLL" compiler could do that? Not sure how to resolve that but you probably have some ideas on that. (Unless someone has a really really fast script processor please do not even think about making the addin a script although that would fix the cross platform issue.)

4. This could lead to a repository of neat effects that would not make it into xlights.

5. Parameter prompting would have to be standard UI but if someone came up with a great idea for a different UI, that could be worked on later and built into Xlights if the idea was worthy. (Think of a pixel editor embedded into an effect for example).

[Gilrock]

Yeah I already had this idea.  We discussed it a couple weeks ago and I'm pretty sure its not going to happen.

[dkulp]

I'm not sure it *won't* happen, but it's not a supper high priority due to the amount of work that would be required to achieve it when all of us have other things we would like to get done.

Personally, I rather have people forking the github repo and submitting pull requests to add new effects.   If someone is interested in doing that, start with the  "spliteffects" branch.  That has my branch that I'm working on which is intended as a starting point.   On that branch, effects are much more "self contained".   To add an effect, you need to implement two classes:

1) A "FooEffect" class (.h and .cpp) which handles the rendering and additional information about the effect (should it participate in the "random" stuff, icons for toolbars, etc...)

2) A "FooPanel" class which is the panel for the effect dialog.  This is wxSmith/Codeblocks created.

Once that is done, it's one line addition to EffectManager to get it loaded and displayed on the toolbars and effects panel and such. 

You can take a look at something like the BarsEffect at:

https://github.com/smeighan/xLights/tree/spliteffects/xLights/effects

[Gilrock]

Yeah the more I thought about it 2 things came to mind that I don't like about the plugin idea.  One is that when sharing sequences the correct version of every plugin used would now also have to be included.  We have enough trouble now with people knowing the couple things needed.  The second thing is I don't want to be troubleshooting problems caused by plugins.

Anyone capable of writing an effect algorithm should be able to follow Dan's direction.  The only issue we have there which is soon a problem for all of us to consider is the toolbar is quickly becoming too large.

[MikeKrebs]

I respect the amount of effort this would require and the perceived difficulties that will arise. I also like that all the shared sequences I have used came across when fitted to my models without twiddling with them or having to "go get the plugin".

I will just point out that part of the reason for the request is that as I have been reading the forum requests I have seen the developers say things like show me the algorithm and I will consider it and that is not what I envisioned this to be so no I won't change it. (Aside...Kind of reminds me of "We will never need more than 640K")

So, if I decide I would like a new effect, I can develop it myself and it MIGHT make into Xlights or maybe it just won't be incorporated at all as being something not what someone envisioned.

So, then I am left with two choices. I can integrate into a local copy of Xlights that I will have to continually maintain as changes are made (and you guys are really fast at making changes!) or I can freeze my copy of Xlights and use it as is. Neither is a very good alternative.

Part of the overall plugin architecture is that eventually there would be a vetted and verified plugin repository as well as beta or personal repositories. The "standard set" of plugins would be downloaded during the install and the rest of the publicly available effects would be easily available.

To think of it in other terms...what if we considered the sequences and models to be only available to developers and you could only run the sequences on approved models (we'd call it Mr Christmas!). Sounds kind of ridiculous. I happen to think of the effects the same way. And the easiest way to get a bunch of new effects is to have an architecture that makes that a trivial function.

Are you saying yes to the plugin yet?

Since you have considered it before, just continue to be open to it and think about it some more. I really think it has great potential and the problems can be overcome. Xlights/Nutcracker version 2016 10 Cougar Taffy Pumpkin Pie arriving soon with new plugin effects architecture!

[MikeKrebs]

Yeah the more I thought about it 2 things came to mind that I don't like about the plugin idea.  One is that when sharing sequences the correct version of every plugin used would now also have to be included. 

Effects are tweaked in many versions of Xlights so the correct version of a plugin should not be much of an issue. It is pretty much the same idea. If you don't have the correct version of Xlights the effect may not work right. If you don't have the right version of the plugin, it might not work right.

Since the repository of "standard" effects would be public, it would be trivial to get the latest effect. It could be a rule of the way plugins work. The latest version is the "correct version". There would be no "run version .0553 to make it work". That doesn't mean that effect couldn't be split off at a particular version but that is part of the flexibility of the plugin system.

[Gilrock]

I have a long list of other things I'd rather do before that.  Sorry but in freeware you gotta find someone willing to do the work for free.  Since you act like you are qualified you are free to talk to Sean and start working in your own branch.  I don't run the show here I just help out on things I'm interested in.

[dkulp]

The main reason I don't see a "plugin system" being doable is that it REQUIRES a completely stable API.   Otherwise, all of the plugins themselves also need recompiling with releases of xLights which then defeats the purpose of having the plugins.  Just incorporate them into the main xLights so they are refactored as part of the changes.

I really don't see that happening.  I've been involved with xLights for 2.5 years now and I've pretty much completely re-written the rendering engine at least 4 times now and I'm working on a 5th.  Each change has provided fairly significant enhancements, but would completely change how any plugin would have interacted with it via any sort of API.   I have zero interest in trying to maintain some sort of stable API for any of it.

I'm MUCH MUCH more interested in getting potential developers building xLights itself and contributing via pull requests and eventually just getting them contributing directly to xLights as part of the team.  Having stuff "outside" of the main build kind of walls them off.    I  *am* speaking from experience here.   My official title at my day job is "VP of Open Source Development".   My job involves building communities around code. (primarily at Apache)    The size of the community of people doing animated christmas lights is relatively small, the subset of that using xLights is small, and the subset of those that have developer capabilities/knowledge to be able to write effects is even smaller.   I'd much rather get those few individuals to be contributing directly.   

[Gilrock]

That was another thing in the back of my mind.  When I used to play LOTRO it had a plugin system that was LUA based and every time they released a new version of the game it seemed to break half the plugins.

Also it's not true that we just change effects and break how they used to work.  That may happen in some cases but if you dig into the code you'll find a ton of code that handles upgrading older versions of effects to work with the latest version.

[MikeKrebs]

Thanks for considering the idea. I defer to your knowledge of how the program will work best.

I am not a good C++ coder and would be useful for simple maintenance functions only. Maybe I can help out in some other way. I will have to see how my interests progress. Maybe at some point I will develop a plugin um, er an effect for something I am considering.

I wanted to apologize if anyone thought I meant "we just change effects and break how they used to work". What I meant was that the effects included in Xlights change over time for some reason just as the plugin effects would. For example, it appears that RenderButterfly has had at least ten changes to it. Most of those appear to be for bugs and code standards but there are enhancements and fixes to the render algorithm itself. Probably a bad example but maybe I liked having red pixels in the bottom left corner of my matrix. Kind of a holiday ribbon thing. But that got fixed in some version and now Butterfly no longer does that. However if I was still running version x and copied a butterfly effect from a shared sequence, the red pixels would still be there. The effect may or not work. That was all I was saying.

I am satisfied you gave it some thought. Thanks for writing this wonderful piece of software. Without it, I would not have had a show this year and for this year, the 37 effects were plenty. ;-) I don't really need a hundred.

BTW, is there a list detailing things you would to see changed or enhanced? I will admit to not making it very far going through all the forums and github itself, but I have not found a "here is current work and next projects list".

[Gilrock]

There is a list of issues/enhancements on Github that the users have entered.  The developers just email each other.  Its very rare to hear any ideas we haven't thought about.

[sean]

mike; we welcome the discussion. before xlights 4 (released march  2015), we had xlights 3 (released jan 2013). I had discussions with Matt Brown back in 2013 about making a plugin architecture. It just never got a high enough priority to ever get us to re-architect xLights. It would be nice and it also would require lots of time. Some other sequencer people contacted me in 2013, they wanted us to build xlights like a sequencer engine with calls that they could make. Why do we each duplicate the code for displaying text, for example? With open source the developers prioritize the things that help our show and the things that we think we help the larger community. For example all of the sequencers we support in conversion, most of us do not use those sequencers.

We have made  a list of 100 enhancements/new features we would like to see in xLights 2016, that list is over 100 items. One item was plugin architecture. I dont think it will make the cut over all the other items. Other items on our list include better ability to share sequences, automatically push all your fseq files to  the fpp, new effects, improved performance, UI improvements.

please pull the "spliteffects" branch and see if you can build it. Dan has made it easier for people to add an effect with the re-factoring of all the effect code. Before it was monolithic and there was more interaction between effects. I will probably teach a class on creating your own effects. I am the gatekeeper for releases.

thanks for the interest, a open source project needs community involvement.

[bpducman]

I hate to sound like the old goat that I am
but
KISS

[Steve Gase]

I've been trying to get up to speed with the effects creation, especially BEFORE the vegas event so that I can have some basis before the classes and be able to ask more in-depth questions.

I've installed the tools, etc. 
I've watched the videos and read the pdf.

unfortunately sean's videos and pdf are outdated... written before 2015's rewrite, and not covering the newest 2016 version.

is there a description of the process to add effects?  something brief would be great... 

after a while hacking at the file level I finally got something to show up in the gui, but I'm sure there MUST be an easier way to do this.  I couldn't guess at the resource editor changes through code::blocks.

...and to create/edit xpm files, is the only choice to set up a linux dev box?

[keithsw1111]

Xpm files can be created using gimp then you edit them in a text editor to fix the variable name.