I can tell you from what I see on the susky with smallmouth is fish of the same size generally congregate together. One time you might get smaller fish and the next time in that same area bigger fish will be there.

Lack of food and a broken predator line often does this in my experience. 

 

Environmental factors should always be considered as well