Caitlyn Jenner felt people "hated [his] guts" when he came out as transgender.
The 70-year-old reality star claimed it was much "harder" to transition in 2015 than to train to win gold in the decathlon at the 1976 Olympics and he didn't feel "accepted".
Speaking on BBC's 'Don't Tell Me the Score' Podcast, Caitlyn - who was previously known as Bruce - said: "I trained 12 years for the games. I trained 65 years to transition in 2015. It was harder to do, it was less accepted.
"'Everyone loved the games, a lot of people when they see you transition hate your guts. Look at the quotes on Instagram. By far that was a lot more difficult."
But the 'I Am Cait' star is proud of his accomplishments.
He quipped: "I have the great double. Olympic decathlon champion and Glamour's Woman of the Year."
Although Caitlyn is happier since transitioning, he admitted it hasn't been easy and he doesn't always feel accepted by the trans community.
He said: "Has it been easy? Not even close. I've raised about 2.6 million dollars, given it away to trans organisations. A lot of them are extraordinarily appreciative of it.
"On the other hand they're having their big fundraiser and say, 'Oh please don't show up you're too controversial.' "
The 'Keeping Up With the Kardashians' star insisted his plans to transition didn't cause the end of his marriage to Kris Jenner as they simply grew apart, and despite previous tensions between them, they are on good terms now.
He said: "Kris and I went our separate directions. It wasn't for me to transition or anything to do with that. We changed apart. There was no argument, no nothing, we did our divorce agreement in one day.
"We both knew it was the right thing to do. I leased a house in Malibu to get out, and yeah we're on good terms today."