Mr Young became a board member on the new Office for Students (OfS) watchdog on 1 January but had faced calls to quit after a series of controversial comments on social media surfaced.
Writing in The Spectator, the free schools pioneer said he “unreservedly” apologises for things he said when he was a “journalistic provocateur”.
“The caricature drawn of me in the last seven days, particularly on social media, has been unrecognisable to anyone who knows me. I am a passionate supporter of inclusion and helping the most disadvantaged, as I hope my track record of setting up and supporting new schools demonstrates,” he said.
“But some of the things I said before I got involved in education, when I was a journalistic provocateur, were either ill-judged or just plain wrong – and I unreservedly apologise.”
Mr Young was named as a board member by Universities Minister Jo Johnson, who has now responded to his resignation.
“Toby Young’s track record setting up & supporting free schools speaks for itself,” Mr Johnson said.
“His decision to stand down from the OfS board and repeat unreserved apologies for inappropriate past remarks reflects his character better than the one-sided caricature from his armchair critics.”
Mr Young’s offensive Twitter posts include a 2012 tweet when he wrote watching Prime Minister’s Questions: “Serious cleavage behind Ed Miliband’s head. Anyone know who it belongs to?”
A post he wrote in 2009 read: “What happened to Winkleman’s breasts (sic) Put on some weight, girlie #comicrelief.”
A petition was set up calling for him to be sacked and has gathered around 220,000 signatures.
Mr Young has also been criticised for his views on disabled people.
Education select committee chairman and Conservative MP Robert Halfon, who has cerebral palsy, referenced articles written by Mr Young during an urgent debate in the Commons on the issue.
“What I’m more concerned about is some quite dark articles where he talks about the disabled, where he talks about the working classes, and much more significantly in 2015 – and I have the article here – on what he calls progressive eugenics,” said Mr Halfon, adding he finds the views “incredibly dark and very dangerous”.
Following Mr Young’s appointment, senior Labour figures, including Shadow women and equalities minister Dawn Butler and Angela Rayner, the shadow education secretary, had written to the Prime Minister calling for her to remove him from the board of the new higher education watchdog, accusing him of “virulent misogyny”.
However, while Theresa May said she was “not impressed” with the obscene tweets, she said he would only lose his job if he used such language again.
Last week, amid the controversy, Boris Johnson defended Mr Young.
“Ridiculous outcry over Toby Young. He will bring independence, rigour and caustic wit. Ideal man for job,” the Foreign Secretary write on Twitter.
In the article announcing his resignation, Mr Young praised the newly appointed Education Secretary Damian Hinds.
“I know that he is equally passionate about helping children from the poorest backgrounds to succeed in life. His job is one of the most significant in Government. I cannot think of a more important mission,” he wrote.