Length of cast and length of rod also could cost you some fish on the set.

Try to not setting the hook at all.  Bass don't give buzzbaits little taps, they're trying to kill it.  By the time you set the hook, it's had it for a second or two.  It will set the hook on itself, especially if you're using braid.  

Try to use a side sweep and just start reeling.  I've found that method works well with other topwaters too.