Respectfully, they're probably right with respect to canoeing.
It varies with fish to fish and also to the level and frequency with which they are exposed to the potential scare and also to the extent said stimulus is followed by an unpleasant event.
While it might be the case that fish watch a diver with apparent indifference, it's absolutely the case that some fish species vanish as the least background bank-side movement or disturbance - chub, for example, are very shy, melting away at a careless movement, carp that are not habitually exposed to anglers, feral one might say, are likewise and often the only sight you get is the swirl as it bolts. Perch, large ones, are also wary of bankside movements and sudden surface disturbances. Pike will generally mosey off if they spot an angler and if you move too fast, will also bolt, often too fast to see exactly where, just leaving an eddy of silt.
I've in all these cases seen the fish and seen them remove itself from my presence. I've even snuck up on said carp/chub/pike and then suddenly waved my arms...off they go...I've done the same experiment by stamping my foot and that'll do it as well.