Ex-minister Crispin Blunt has confirmed he was arrested on Wednesday on suspicion of rape and the possession of controlled substances.
In a statement, the Reigate MP said he had twice been interviewed by police in connection with this incident.
Mr Blunt said he originally raised the incident with police three weeks ago, concerned about "extortion".
He said he was cooperating with Surrey Police's investigation and was confident he would not be charged.
The police were unable to say if the controlled substances were drugs, as tests are being carried out, according to PA Media.
The Conservative Party withdrew the whip, meaning Mr Blunt will sit as an independent MP, after he confirmed he had been arrested. He has been asked to stay away from Parliament.
Mr Blunt was elected as Conservative MP for Reigate, in Surrey, in the 1997 general election, defeating the incumbent Sir George Gardiner, who had switched from the Tories to the Referendum Party.
He served as a justice minister from 2010 to 2012, in the whips' office when the Conservatives were in opposition and chaired the influential Foreign Affairs Committee.
In April, he apologised for defending former MP Imran Ahmad Khan, following his conviction for sexually assaulting a teenage boy. Mr Blunt had called the verdict a "dreadful miscarriage of justice" in a statement, but retracted his comments.
A month later Mr Blunt announced he will be standing down as an MP at the next election.
In a statement, Mr Blunt said: "It has been reported that an MP was arrested yesterday [25 October] in connection with an allegation of rape. I am confirming that MP was me.
"The fact of the arrest requires a formal notification of the Speaker and then my chief whip.
"I have now been interviewed twice in connection with this incident, the first time three weeks ago, when I initially reported my concern over extortion.
"The second time was earlier this morning under caution following arrest. The arrest was unnecessary as I remain ready to cooperate fully with the investigation that I am confident will end without charge.
"I do not intend to say anything further on this matter until the police have completed their inquiries."
A police spokesman said: "We can confirm a man in his 60s was arrested yesterday morning (25 October) in Horley, on suspicion of rape and possession of controlled substances.
"He has been released on conditional police bail pending further inquiries."
The Conservative Party has declined to comment.
This is the latest sexual misconduct allegation to hit the party.
A separate Conservative MP, in his 50s, was arrested in May 2022 on suspicion of rape, sexual assault, abuse of position of trust and misconduct in a public office.
The MP is currently on bail until mid-February 2024.
Sign up for our morning newsletter and get BBC News in your inbox.
Gun battles near major Gaza hospital while thousands flee south
'We walked into a wasteland' – BBC in Gaza with Israeli forces. Video
A turning point in Myanmar as army suffers big losses
A turning point in Myanmar as army suffers big losses
He had a call from Israeli intelligence. 'We have the order to bomb. You have two hours'
Dead bodies and tanks on road south as people flee
Hezbollah warns of regional war if Gaza bombing goes on
Why 'I don't recall' is a common strategy on the stand
China's 'communist spies' in the dock in Taiwan
Musicians behind bars in Belarus crackdown on dissent
Afghanistan's dazzling run captivates world cricket
'Don't mess with Claressa' – a fighter as hard as Flint
The employees secretly using ChatGPT
Why aliens may be weirder than we think
Africa's safari trail with no tourists
© 2023 BBC. The BBC is not responsible for the content of external sites. Read about our approach to external linking.