This happens if someone's game crashes to the desktop as it loads it. The pre-game screen doesn't have the player ID yet, so the person has no way to rejoin. There is no system currently for finding a replacement player, so the game just ends after a few minutes.
I've unfortunately had this...