Non-bank foreign exchange companies offer currency exchange and international payments to private individuals and companies. These are also known as "foreign exchange brokers" but are distinct in that they do not offer speculative trading but rather currency exchange with payments (i.e., there is usually a physical delivery of currency to a bank account).
To use Gold CFD as an example, at the time of writing, to purchase an ounce of Gold you would need to spend 1,200 USD. However, with a leverage rate of up to 1:20 (which means a trader could trade up to 20 times the value of what they deposit), a trader could trade on the full value of an ounce of gold (equivalent to 1,200 USD), for a deposit of just 60 USD.
Unlike stock markets, which can trace their roots back centuries, the forex market as we understand it today is a truly new market. Of course, in its most basic sense - that of people converting one currency to another for financial advantage - forex has been around since nations began minting currencies. But the modern forex markets are a modern invention. After the accord at Bretton Woods in 1971, more major currencies were allowed to float freely against one another. The values of individual currencies vary, which has given rise to the need for foreign exchange services and trading.