Openly gay MP and Shadow Leader of the Commons Alan Duncan promised last night he would pay back the £4,000 he claimed on expenses for gardening.

Speaking to the Evening Standard, the multi-millionaire MP said: “I value my reputation and the integrity of Parliament and the Conservative party. I’m not happy to be thought a crook. I would far rather pay it back voluntarily.”

Earlier this week, he apologised for charging the taxpayer for more than £4,000 of gardening and more than £1,400 in mortgage interest payments for a home he originally purchased outright.

Mr Duncan, who oversees his party’s expenses policy, was one of many Labour and Conservative MPs whose expense claims were revealed last week by the Daily Telegraph.

In March 2007, Mr Duncan, the multi-millionaire MP for Rutland and Melton, was warned over his expenses after he attempted to submit another claim for gardening bills.

His claim for £3,194 gardening which was not accepted by Commons authorities. The fees office wrote to Mr Duncan suggesting that the claim might be considered not to be “within the spirit” of the rules.

Openly gay MP Ben Bradshaw will not be paying back his claims, his spokesman said this morning. Mr Bradshaw claimed for the full mortgage of a home he jointly owned with his civil partner.

In an interview with PinkNews.co.uk on Monday, he claimed the Daily Telegraph had been “homophobic” in the way it reported his expenses, saying he had been in the same circumstances as David Cameron, who had not come under fire over his mortgage payments.

As yet, there has been no suggestion that openly gay minister Lord Peter Mandelson and Tory MP Nick Herbert will pay back their expenses.

Lord Mandelson billed taxpayers for £3,000 for work on his constituency home less than a week after standing down as MP for Hartlepool.

Mr Herbert claimed £10,000 of £14,700 of stamp duty on a new constituency home in March 2006. Since then, he has claimed the full £1,893 monthly interest charge on the house’s £465,000 mortgage, despite his partner’s name also being on the property’s deeds. He had a civil partnership with Jason Eades, a solicitor, in January this year.

In a statement released on Monday, Mr Duncan apologised over the expenses scandal, saying: “I understand that people are really incredibly angry about this issue. I personally am very sorry about it. I think all MPs should be ashamed and every MP should apologise for the system that they’ve allowed to grow up without realising that this system is completely unfit for the modern age.

“I have always been in favour of complete transparency. I was the first MP ever to publish his expenses in full – which I did in 2004. In January I forced a U-turn from the government which ensured that these receipts would be published and not exempted from Freedom of Information laws. I have always tried to set up a model office with the highest possible standards.

“Our responsibility now is to do our best to restore the trust of the public which has been so badly undermined. The simple truth is that MPs should not determine their own pay or allowances. It should be decided by Sir Christopher Kelly’s Committee on Standards in Public Life, and the sooner the better.”