I think, to an extent, what fortunato and Beoran are saying is that you should trim down the forums if you hope to get a healthy number of visitors. It's going to be hard to get people to stick around when the first thing they see is like 20 different sub-forums with little to no activity. People are sheep, if there are no posts and the place looks empty, people will also not post and abandon the site before they've even given it the chance.
By trimming down the forums to the bare necessities, you will give focus to those who visit, which in part will encourage activity, which as a result will give the impression that the forums are active and further propagate your success as each newcomer further develops your forums.
Afterward, depending on the community you've attracted, you can start bringing back some/all of those sub-forums.