"Britain's Got Talent" once almost lost hosts Ant McPartlin and Declan Donnelly, all because of Simon Cowell's ridiculous behavior.

Simon has been one of the key mainstays of "Britain's Got Talent," but so are Ant and Dec. However, the three faced an ugly situation in the past which frustrated the comedic duo and almost sent them away.

In an excerpt from Ant and Dec's autobiography "Once Upon A Tyne" posted by The Sun, the two revealed how disheartened they felt over the record executive's timekeeping issue.

Per the two, waiting for Simon on "BGT" drove them nuts before adding how they truly felt on the show.

"A couple of hours after we arrive at the theatre, we focus on one of the key elements of any BGT audition day: waiting for Simon Cowell to turn up," Ant and Dec penned on the book. "Once we've got started on that, we'll often film interviews with some of that day's acts, then have a bit of food, then get back to waiting for Simon."

The waiting game seemed endless, and it was one of BGT's biggest issues since the beginning.

Given the fact that Ant and Dec started working with him and BGT since 2007, no one can truly tell how many times these instances happened.

Simon himself contributed to the book and proved that it happened.

According to the 60-year-old TV personality, him being late on set "used to be an issue to the duo." However, Ant and Dec jokingly said, "We've still got a real issue with him being late and it still drives us nuts."

Simon Also Saved Ant and Dec

Despite being a headache, Simon still did his part and stopped the two from leaving the show.

In the same autobiography, Ant and Dec recalled how frustrated they were upon seeing that large parts of their footage being cut out of the main show.

This pushed them to tell ITV bosses that they were thinking of leaving the show. Fortunately, they sat down with Simon in LA back in 2012 as he tried to win them back.

"After filming interviews and interacting with every single act, as well as making jokes, giving reactions and everything else we always do, we found that when the show went out on TV, we hardly seemed to feature in it at all," the famous hosts said in the book.

This made them feel like they were wasting their time doing the auditions since all of it was being cut. In the end, the experience left them feeling like they "were being sidelined."

The pair then revealed that Simon made the necessary changes in the next seasons of "BGT," and they eventually changed their minds about leaving the show.

Simon will be temporarily out of the "BGT" cameras, as he is taking all his time recovering after he suffered a back injury and underwent a successful six-hour operation.

READ MORE: Simon Cowell Injury Update: Music Mogul Permanently REPLACED in BGT?