A judge in Brazil has some doubts about the story that U.S. swimmers Ryan Lochte and James Feigen were held at gunpoint during a robbery in Rio last week, and has ordered that the two swimmers surrender their passports and remain in Brazil pending an investigation into the alleged crime.

According to reports, a judge believes the story was fabricated, and ordered on Wednesday morning that the men turn over their passports as an investigation goes underway. However, by the time authorities arrived in the Olympic village to seize the passports, both had already moved out, and Lochte was reportedly already back in the U.S. on Tuesday. It is unclear if Feigen had also returned to the U.S. yet, or how their not being there will affect the investigation.

Sources told People Magazine that authorities have doubts about the story Lochte told regarding being held up, and they don't believe the swimmer's version of events, hence the investigation.

"They don't believe his version of events, so there will be an inquiry," the source said. "This was the most high profile incident that has happened here in Brazil, so the government is really taking it seriously. The USOC is cooperating fully, of course, but they want to talk to Ryan. They're not happy at all that this has given Rio a black eye, and they are vowing to get to the bottom of it. It's a mess right now."

Lochte claimed that he, Feigen and two other American swimmers were held at gunpoint by robbers who posed as cops and pulled them over in a taxi. However, no witnesses have emerged to corroborate the claims, not has the taxi driver, and the swimmers were allegedly unable to provide details that were considered key during their interviews with police.

Lochte has not spoken out about the incident since the order to seize his passport and keep him from leaving the country.

Tags: ryan lochte Rio Olympics