The Bishop of Gloucester, Michael Perham, has been questioned by police over allegations of indecent assault on a girl under 18 and a woman.
Bishop Perham, aged 66, who was due to retire later this year, was not arrested but interviewed under caution. A Metropolitan Police statement said: "Enquiries continue."
The Diocese confirmed that he had "stepped back" from ministry.
The alleged incidents concern a period of time between 1980 and 1981 when Perham was a curate at St Mary's Addington in Croydon, south London.
The Gloucester diocese, which in two previous statements made no reference to police enquiries, said today, Tuesday: “This is a police matter. We have no further comment to make.”
A police spokesman said: "A 66-year-old man today attended a police station in Gloucestershire by appointment and was interviewed on suspicion of indecent assault of a child aged under 18 and indecent assault of a second female aged over 18."
The incidents are being investigated by officers from the Metropolitan Police Sexual Offences, Exploitation and Child Abuse Command.
Bishop Perham is from the Church of England's liberal catholic wing. He was a member of the Crown Nominations Commission that chose the latest Archbishop of Canterbury Justin Welby. His area of speciality is liturgy and he was a main influence over Common Worship, which succeeded the Alternative Service Book. A special service was due to take place in November to celebrate his ministry.
A previous Bishop of Gloucester, Peter Ball, is currently facing charges of misconduct in public office and of indecent assault on a boy and a young man.