In the past couple of years, the royal family and Kate Middleton's parents got along so well. 

Prince Charles invited Carole and Michael Middleton for joint gatherings at Birkhall, an estate in Balmoral, which was inherited by Prince Charles from the Queen Mother. 

It is where the Middletons would enjoy a weekend with Kate and Prince William, before and after marriage. 

But despite their seemingly good relationship, rumors started surfacing that Prince Charles was engaged in a feud with Carole Middleton over the time spent with their mutual grandchildren, especially Prince George. 

According to a 2016 Daily Mail article, Carole and Michael Middleton got more than their fair share of time with the future king. 

"Charles feels very much that the Middletons get more than their fair share of time with George and Charlotte. In comparison, he sees them very little, and he bears a bit of a grudge about that," royal expert Katie Nicholl revealed. 

According to several courtiers, Carole Middleton also has a "grandiose presence" when she is at Kensington Palace, feeling rather possessive of Prince William, Kate Middleton, and the Cambridge children. 

An unnamed source revealed to the British tabloids that Prince Charles said, "They never let me see my grandson." 

"Charles wants to see his son and daughter-in-law and spend time with his grandchildren alone. He has always wanted to mentor George, who is, after all, the heir to the throne, and sees it very much as his role."

Nicholl further explained that the Prince of Wales would want his grandchildren to come to Scotland to learn how to shoot, hunt, and fish, as it is important to Prince Charles. 

"He wishes the family would do more traditional things like a holiday in Scotland." 

On Prince George's third birthday in 2016, which was organized by Carole and Michael, they, of course, invited Prince Charles and Camilla Parker Bowles. 

The morning of the party, while the Middletons were preparing the children's food and games, Prince Charles was at a meeting in Clarence House. 

It reportedly upset Prince Charles that the Middletons were the ones organizing the party. 

"Charles feels rather left out. He gets very little time with his grandchildren, and I know he gets upset about it because he has said," said the source. 

A royal insider told the Daily Star, "Charles believes Kate's mom, Carole Middleton has taken over. And he is said to have blamed 'the March of the Middletons' for his exclusion."

While Prince Charles and Carole Middleton don't get on, the same can't be said about her and Queen Elizabeth II. 

In the past, the monarch drove Carole around the grounds of Balmoral Castle, and they bonded over their love of horse riding. 

The Queen even invited Carole and Michael during the 2012 Ascot and rode in one of Queen Elizabeth II's carriage on Ladies' Day. 

During Her Majesty's Jubilee Celebrations, the Middletons were also front and center, and royal reporter Emily Nash said how the two mothers seemed very comfortable around each other. 

"We've seen the Queen driving Carole Middleton around at Balmoral, pointing out sites to her; they're obviously very comfortable in each other's company."

READ MORE: Sussex Heartbreak: Royal Family Snub Meghan Markle, Prince Harry's on Special Day