This happens if someone's game crashes to the desktop as it loads it. The pre-game screen doesn't have the player ID yet, so the person has no way to rejoin. There is no system currently for finding a replacement player, so the game just ends after a few minutes.
I've unfortunately had this happen a lot due to a friend's computer. We think it's cause of his model of AMD graphics card, but we aren't sure.