That's how it is

Well some fish can be really hard to catch without the right equipment (and even then) but I'd say it's generally not too difficult, the easier fish pose no problem once you understand the mechanic.

I'd even go so far to say that fishing is the most busted activity in single player. It takes little effective investment (time stops during the minigame and you can sustain your energy level with the stuff you catch) and can really kickstart a farm if you get a good chest