Edwin Poots has been dropped from the Northern Ireland Executive after a reshuffle.
As minister for arts and culture he attacked the formation of a gay rugby team in the province.
In February Mr Poots said he “cannot fathom why people see the necessity to develop an apartheid in sport.”
He has been kicked out of government after just a year in office as part of a reshuffle by new First Minister Peter Robinson.
Gregory Campbell will replace him as culture minister.
Mr Poots has a record of insulting gay and lesbian people.
As a councillor in Lisburn, County Antrim, he tried to ban same-sex couples from holding civil partnerships in the town hall.
He said that the Ulster Titans rugby team, which is based in Belfast, is as unacceptable to him as a all-black rugby team.
“To me it’s equally unacceptable to produce an all-homosexual rugby team and I find it remarkable that people who talk so much about inclusivity and about having an equal role in society would then go down the route of exclusion.”
As minister responsible for arts and culture in Northern Ireland’s devolved administration he had to give a grant to Belfast Pride, which got him into trouble with some of the DUP’s more strident supporters, who regard the event as a “celebration of sodomy.”
Nigel Dodds has been appointed as finance minister and Sammy Wilson as environment minister.
There have been questions about whether it is correct for ministers such as Wilson, Dodds and Robinson to retain seats in Westminster while being in government in Northern Ireland.
Traditional Unionist Jim Allister MEP, who opposes the DUP, said:
“In a year we have moved from three double-jobbing MPs serving also as ministers to five.
“We have recently seen DUP MPs absent from key votes in the Commons on the Lisbon Treaty and the Embryology Bill.
“No-one can adequately do two, never mind three jobs – MP, Member of the Legislative Assembly and minister.”
A spokesman for the Ulster Unionist Party raised similar concerns.
“If we look at the re-shuffle within the DUP which saw the appointment of Sammy Wilson and Gregory Campbell to the Executive, I would simply raise concerns as to their ability in carrying out the job,” he said.
“This re-shuffle means that four out of five Minister within the DUP are now holding three or more positions between the posts of MP’s, MLA’s, Ministers and Councillors.
“This poses the question, how can they possibly attempt to carry out these specific roles to the best of their ability?”