Alright, so at this point I gave up on the 5mm panels I had and I bought the 10mm Indoor panels from Ray Wu.
(https://www.aliexpress.com/store/product/P10-rgb-SMD-3in1-full-color-indoor-LED-Display-screen-unit-board-32-16pixels-320mm-160mm/701799_866772137.html)
Hooked them up and everything is almost perfect, but here's a video at what's going on...
https://www.youtube.com/watch?v=ubukU2EnegE
There's pixels lighting up in front of the one's that are supposed to light up... it's kind of odd and I'm not sure why that might be happening. I've double checked that it's the correct scan rate and correct setup on the FPP, I've restarted the BBB / Octoscroller a few times... Rebooted the computer, restarted Xlights, reconfigured it in both Xlights and in FPP. I tried running FPP in Bridge mode as well as a Standalone player just playing the .fseq and it does the same thing in both instances. It's most noticeable with just standstill text, but there's flickering with all effects and all colors as well.
Any ideas?