One of the things I changed a few years back was a reduction in the number of Remotes I used. 5 years ago, I ran every FPP based controller as a Remote. There were 9 of them, all getting their Sync packets via WiFi.
There were a few too many times where I noticed some of them being just a bit behind, then catch up, and not always at the same point. It annoyed me to the point that I rethought my layout.
Now, I only have 3 Remotes, mostly based on the Area of the display they are in. One of those Remotes drives a very large P5 panel (48 panels) so it does nothing else. The other 2 remotes don't actually have any Pixels attached. Their job is to output E1.31 or DDP information to the controllers in their area. This has resulted in a much better synchronized display.
Oh, and those Remotes use CAT5 inputs, using a Gigabit switch in the garage. I've don't use WiFi except for testing. I don't mind cables in the yard. It impresses the people who see it in daylight.