Hey Mike-
I don't have a specific numbers suggestion for you, but I have a short article for you to read:
The Secret of Quality Over Quantity and Why Offering Less Styles Is More How To Start A Clothing Line From Scratch Everything on Fashion Industry Design Daily Definitions & Industry News
The reason why I am referring it to you is because I just had the same conversation with our consultant in LA 2 weeks ago. He consults for several brands and has worked in the apparel business for quite a awhile- as a consultant, a sales rep, and a company vp. He told me that as my business grows, the best thing I can do is to keep it focused. This article says something similar.
Hope it helps!