CrazyTalk - an option to Papagayo for Panels

[Steve Gase]

I was holding back on this advice since it would give me a leg up on the rest of you fantastic designers...

At the Expo, Charles Belcher shared with us the name of the software he used to do animation of his house in the projection mapping talk.

CrazyTalk7 is an affordable software tool...  http://www.reallusion.com/crazytalk/default.aspx

You can take a picture of a person -- or an drawing, lifelike, or a simple outline of a snowman...  and specify the location of the face, the mouth, and eyes... and the tool will "animate" the picture.  It is creepy in how life-like it can be.

The software can listen to your voice, and move the mouth accordingly.  you can animate the facial features, and add emphasis with the face and body motions.  there is a LOT that it can do.

Originally, I was looking for a way to create monster faces for halloween that I would place on a panel... for christmas I would do the same with santa, frosty, and other characters.  I never spent the time to get the papagayo stuff to work, and to delve into the creation of video content.

This software makes it so easy.  watch the tutorials on youtube to see.

to have your character lipsync to music, you need to record your own voice singing to the song as you listen on headphones to the original song.  The software matches your voice to the the facial movements.  when you complete the animation, swap our your voice track with the original music.

try it out.

in return for this suggestion, I hope everyone shares their cool sequences so that I can use them in my own show!

http://www.reallusion.com/crazytalk/default.aspx

https://youtu.be/VXRjX5bNGKA?t=305

https://youtu.be/eHVs5zr6MvE?t=626

[Steve Gase]

I will tie together another related discussion... this allows you to integrate the CrazyTalk mp4 file as frames that can be imported into XL3 or XL4...

For lower-resolution "faces", I suggest you use animated/drawn characters which are more "flat" and have less detail.  They'll appear well in the lower-res panels.  You still get the benefits of a life-like mouth and eye movement, and the auto-lipsync to voice.

Create a temporary directory , and copy the movie file there .
then use the ffmpeg utility to break it up into frames. Example:

ffmpeg -s 30x50 yourmovie.mp4 f-%d.png

This will create frames f-1.png, f-2.png .etc. Your original movie has been resized to 30x50 pixels. (so u need to change this value based on your matrix size)

Note: ffmpeg is run from the DOS command line .
Example:

C:\
cd \tempdir

ffmpeg -i yourmovie.mp4 -s 30x50 f-%d.png

This will create a number of files 'f1-' onwards

now , drop the picture effect and browse and point to the first file ie
"f-1.png", using the pictures effect

The picture effect will then pick all the frames.

Try with a small movie segment ie 10 secs long

Note: Only 'something-1' will work ie the application looks for the '-1' and then only picks the others.

[flyinverted]

This is beyond awesome!  Thank you Steve G!

I've been manually making a talking Santa on my tree for in-between-sequence snippets.   Ditching that method now.

- The other Steve G.

Hey Gil,
Think you can put this in 4.2 ?   

[arw01]

Steve did you just buy the $29.99 version of the software?

[mararunr]

Bugger. The secrets out.  I was in that class as well and have been playing with Crazy Talk as well.  It's pretty cool.  In my case we are going to try and take Jeff Holmes (of Lights, Music, Magic - talk a couple courses at Expo) snowman show narrator idea and add it to our show (along with singing faces on a panel, wreath, and couple other spots).

[Steve Gase]

Steve did you just buy the $29.99 version of the software?

I got the bundle.  My kid is really into animation, and thought the extra pieces would be appreciated.  My wife even wants to play with it.

I think the basic product would be fine for our hobby activities based on what I'm seeing.

[plaberge]

Like you Steve, I was never that keen on Papagayo. CrazyTalk, however, I could get into. Thanks for posting about this. More pressure to get my P10 panel done...

Paul.

[Gilrock]

Not sure that would be any good for like a 12x50 tree.  For the panels it might be cool though.  I think I liked the animator software better than the talking head one.  But I could see myself wasting all my free time trying to make something good in that program.  Have fun.

[arw01]

Steve did you just buy the $29.99 version of the software?

I got the bundle.  My kid is really into animation, and thought the extra pieces would be appreciated.  My wife even wants to play with it.

I think the basic product would be fine for our hobby activities based on what I'm seeing.

Just the crazy talk bundle Steve?  Or went for the animation bundle as well?  Got a couple of college kids so who knows what they might like, one learning Japanese and maybe that would be fun to play with in that language and help teach the Japanese kids English.

[Steve Gase]

Only the crazytalk7 bundle so far.  The other ones look tempting but i have a show to build.    

the animated dance competition looks fun... maybe with that i could express my hidden dance talent which still has not manifested in the real world.

[thevikester]

Haven't dug into this, but is it compatible with the holiday coro faces...

Sent from my SM-N900V using Tapatalk

[Steve Gase]

Haven't dug into this, but is it compatible with the holiday coro faces...

Sent from my SM-N900V using Tapatalk

I think the value is with rgb pixel panels, and not strings of AC lights, or coroface lights.

With a coroface you have a 'hard-coded' shape to the face -- a monster is a monster and will never be mrs claus... (well, i hope not... )

this software does generate the phonemes in a track as it processes the audio... you might be able do find a way to export this and pull it into a papgayo-like solution to match up your strings and the expressions.

where I think crazy talk excels is that it creates video from a static picture drawing and audio.  (text-to-speech is included as well)  with the video, you play it like a movie.  so, within the same sequence you could have dozens of different faces take their turn on the panel.  you can even do multiple faces in a conversation between each other!

[sean]

i bought crazy talk 7 two years ago, i loved that it had the ability to animate a face without breaking down any lyric.
It is like dragon talk. Feed it a mp3 and it will find teh words and phonemes automatically.

I stopped using it when i found that it has no export capability. Their files are internal, proprietary.
It is great if you want to create animated movies to play on a matrix.

i wanted to use it to replace papagayo, no such luck.
I tried both version 6 and version 7.

[markrvp]

Holy thread resurrection, Batman!!

Sean mentioned you can't export these, but I assume they can be saved as video files?  And if so, they can be converted to play on a matrix with ffmpeg?

If they can't be saved as video files, could they be captured with Camtasia or another screen recording program?

[Gilrock]

Holy thread resurrection, Batman!!

Sean mentioned you can't export these, but I assume they can be saved as video files?  And if so, they can be converted to play on a matrix with ffmpeg?

If they can't be saved as video files, could they be captured with Camtasia or another screen recording program?

Yes Camtasia can capture your screen...  I think the point here was you can't export the phoneme timing but that program would be worthless if you couldn't create a video from your work.