In contrast to variants of a roguelike, the term vanilla refers to the unmodified original game (that is, NetHack itself in this case).
The most plausible etymology is that vanilla is the most standard and basic flavour of ice cream.