There has been so much talk as to whether or not Britney would return for the third season of X Factor US that no one has even questioned whether the former American Idol big wig would return to the Fox show, so when it was announced that Simon Cowell wouldn’t be returning to the hit reality show, it took most people by surprise.

What does that mean for the future of the show? Well for now, no one really knows. In fact, insiders are saying that this came so far out of left field that they don’t even have a back-up plan in place for replacing Simon.

“Most people assumed that Britney would somehow get out of her contract after the first year, but they aren’t prepared for Simon’s exit,” said an insider from the show.

Show runners are assuming that both LA Reid and Demi Lovato will return for the third installment of the popular Brit show, but the insider claims that producers will be actively be looking for three replacement judges, just incase they’re blindsided again.

What do you think of a Simonless X-Factor? Will it be as good without him?