Hi
First up do some testing on dry land
If you can get someone to help by standing near the boat and watching the servos makes it easier,with the tx aerial up walk away from the boat (motor not running) working the controls as you go.see how far away can get.
If its only 20 to 50 feet check the RX aerial isnt broken or has been cut ,batts are fully charged and giving out correct voltage,tx batts are fully charged if you can try to borrow another TX I have seen a faulty TX cause issues out of teh box.If you can walk far enough away you cant see the boat then try with the motor running, blip the throttle as you walk away if it starts to play up,check that you have the correct type (resistor) spark plug,check the plug gap,check that the flywheel isnt hitting on the coil pickups,check for any metal to metal joints that might be rattling things like rudder push rod connection at rudder(this can cause RF interference) .If that all checks out and it only plays up on the water then check where the rx aerial is in relation to the water surface get it up as high as you can,Ive never heard of FM having issues like the early spectrum 2.4 dsm1 radios but you never know.
cheers
Wozza
Ps I also forgot if your radio system runs xtals check they are not loose or damaged,mainly in the RX. I always put some tape over mine as I had one fall out in a nitro heli once the end result wasnt pretty lol