The most senior gay Conservative MP, Alan Duncan, has apologised for charging the taxpayer for more than £4,000 of gardening and more than £1,400 in mortgage interest payments for a home he originally purchased outright.

The Shadow Leader of the House of Commons, who oversees his party’s expenses policy, was one of many Labour and Conservative MPs whose expense claims were revealed last week by the Daily Telegraph.

In March 2007, Mr Duncan, the multi-millionaire MP for Rutland and Melton, was warned over his expenses after he attempted to submit another claim for gardening bills.

His claim for £3,194 gardening which was not accepted by Commons authorities. The fees office wrote to Mr Duncan suggesting that the claim might be considered not to be “within the spirit” of the rules.

In a statement, he said: “I understand that people are really incredibly angry about this issue. I personally am very sorry about it. I think all MPs should be ashamed and every MP should apologise for the system that they’ve allowed to grow up without realising that this system is completely unfit for the modern age.

“I have always been in favour of complete transparency. I was the first MP ever to publish his expenses in full – which I did in 2004. In January I forced a U-turn from the government which ensured that these receipts would be published and not exempted from Freedom of Information laws. I have always tried to set up a model office with the highest possible standards.

“Our responsibility now is to do our best to restore the trust of the public which has been so badly undermined. The simple truth is that MPs should not determine their own pay or allowances. It should be decided by Sir Christopher Kelly’s Committee on Standards in Public Life, and the sooner the better.”

Other openly gay politicians such as Lord Mandelson and Labour MP Ben Bradshaw have also come under fire for their expense claims.

Lord Mandelson, the Business Secretary, was found to have billed taxpayers £3,000 for work on his constituency home less than a week after standing down as MP for Hartlepool.

Health Minister and Minister for South West England Ben Bradshaw claimed for the full mortgage of a home he only half owned.