There's nothing you can do outside of the obvious: pay attention to your surroundings.
Only go to games where you trust that the game runner knows the people they are inviting to the game.