While I have never started my own billing company, I agree with Merry and shanbull...I would avoid taking out a start-up loan if you can. I base my opinion on having worked for two people who started billing companies, and I came on with each of them shortly after they each got their first client. Here is what they did:
Person A took out a hefty start-up loan, got a business coach, rented office space, made a website, and did all kinds of marketing. Nothing happened. They FINALLY got a client, then things went south with that client after a few months and they ended the contract. They got another client, and then they ran out of money, because they spent so much on marketing and the issues with the first client, and the new client they had wasn't generating enough income for the business to stay afloat. Person A was out of business and out of money within a year. And still had that hefty loan to pay back.
Person B started their business with no loan, working out of their home. No office, no business coach, no website, no marketing. What they had, which costs nothing, is contacts. They had worked for a doctor, and parlayed that into their first client, and from that client's referrals, got two more clients. Person B is two years into their business, doing well, with more clients on the horizon. And no loan to pay back.
I also work for a third billing company, very established and successful, who did what Person B did.
So from what I can tell, getting started isn't about loans and money...it is about contacts and referrals, which are free.