Licenses
What licenses are required? Get them. Even the troublesome local licenses.
Cities have their own requirements regarding the filing fee that goes with your business license. It usually not that expensive - $20 - $50 depending on what type of business you're operating. If you are a contractor most states have a contractor's board.
In order to truly grow your business you must commit to doing everything by the book. The last thing you need is a stop work order due to some licensing issue.
Registering
If you have not registered your business with your secretary of state, you may only do business under your personal name. You may not legally use your businessname without first registering it as one of the many business entities recognized by your state and the IRS.
See this article about registration.
Permits
Permits are generally job specific.
If you need them get them. Stop playing around and operate your business legitimately.