It isn't impossible!
Initially you have 2 choices - to start a business or to buy an existing one.
Either way you will need capital, and the decisions to be made are pretty similar as to those you would have to make in the UK.
All businesses have to be registered with the local Chamber of Commerce. I would strongly advise seeking advice from an accountant as to the best way to set up your business. Talk also to the tax office (Hôtel des Impôts) - they tend to be very helpful. You will certainly need a French accountant for preparing your accounts, as the structure of French accounting is quite different, although the principles are as in the UK.
Once the business is formed, you will nedd a bank account
There are various types of business structures in France
Entreprise Individuelle - self employed. Liability is unlimited, so suitable only for ventures with a modest turnover. No start up capital is required to form the Entreprise.
SARL Société à Resposabilité Limitée - the equivalent of a Limited Liability Company. There is a minimum capital requirement (currently 7500€, although only 20% needs to deposited up front; the rest can follow over 5 years) and more stringent accounting auditing requirements.. For sole traders, there is the EURL, Société Unipersonelle à Resposabilité Limitée, an SARL adapted to a sole trader, with a reduced capital requirement.
SA Société Anonyme - the equivalent of a plc. A higher capital requirement is required and accounting and auditing requirements are yet more stringent. Unlikely to be of interest to a start-up business venture.
As in the UK, VAT (TVA) registration is obligatory once turnover exceeds a certain sum.
Employment Issues
If you employ somebody in France, you will be bound by French employment law, which is very strict.
The 35 hour working week applies, as does a minimum wage (SMIC) - currently 7.19€ per hour - and employees are well protected under the law. Social Security costs add about 45% to the wage bill. The protection, together with the high costs, means many small entreprises cannot easily afford to emply others.