Since you have a family that size a fish and ski might be a good model for you.

I don't mean to be presumptuous but I would assume the wife would be joining the boys on some of these family type summer fishing/recreational trips?

If so, you need to start considering a slightly used boat if $20k is your budget.  20 thousand won't buy much fish and ski new.  An 18 foot boat is a minimum for that many in a family and once the kids start getting a little older that 18 foot boat will seem way too small.

I'd be looking for something in a fish and ski in a 19 1/2 - 20 foot.  Once you get down to the 18 foot range in those types of boats there isn't much room for the fishing part.  Both front and rear decks wind up being microscopic with most of the boat devoted to floor space and those swivel captains chairs.  There also isn't a whole lot of room in that size F & S for storage either.  Buy lightly used and get as big as you can.  

maybe you want to look at a pontoon boat, they work fine for fishing, & are perfect for the fanily. consider how many times your wife and kids are going to say "I goota go pee" most pontoons have enough storage space for a porta potty!lol