At the moment the community rents venues for social gatherings
Muslims who rent halls for social gatherings and prayers have set up a charitable trust to raise money for a permanent Islamic centre.
Members of the West Wales Islamic Cultural Association hope to establish a community centre in Carmarthen.
They have identified a building and raised more than £60,000 towards their £150,000 target.
Spokesman Khalid Latif Khan said: "For the people who have made Carmarthen their home, they need a place."
He added: "We have now got a sizeable Muslim community in Carmarthen - many have been here for a good five or 10 years and most are British now.
"Our aim is a bit more than a mosque - we need a community centre so we can do all activities."
The association has 40 registered families and more than 70 members, most of which either work at West Wales General Hospital, or in the restaurant trade.
But Mr Khan said it was difficult to know exactly how many Muslims lived in the area.
Since the association started Sunday classes for children, numbers have grown from a handful to more than 30.
The nearest mosques to Carmarthen are in Swansea, Lampeter and Llanelli. A new centre would also give the community a place to pray.
Mr Khan, a GP registrar, said: "We pray five times a day - it is an obligation on every adult Muslim - so we need a place where we can do that.
"There are also special occasions. Twice a year we have the Eid festival, and especially in the month of Ramadan every night we pray - so with all these activities we need a place for ourselves.
"On top of that we are also thinking about our women folk because sometimes they have problems with the language - it takes time - so they need some kind of social meeting place."
He added: "We have to give our future generations the right knowledge of what Islam is about - they should know that even living in Britain they can be good Muslims."
He said the association had identified a suitable building in central Carmarthen and it was "more or less" halfway to its target.
The association has received support from some of the more established Muslim communities in larger cities.