Most people accept that system does matter. But so does setting, at least as much (it must be true because Troy Costisick has said it before). Setting is tenously defined as the diegetic (in-game) context of the actual play.
How does setting matter? Well, I’d start with players often having preferences to some direction, and away from some other directions. I am a dark fantasy junkie, for example, but dislike running Cyberpunk. A friend has an unexplained dislike towards guns in gaming. These tastes are overwhelmingly subjective. Conclusion: Don’t play in setting someone hates. Do play in settings people are ambivalent about, because trying new things is useful.
On more technical note, setting is a large contributor in the sort of events that can take place and, hence, stories that can be told. Rarely do they completely rule out genres, but they often suggest and facilitate certain broad ways to play. If vampires are a major theme in the game, most likely possibilities are horror of them chasing you, horror of being them, high-action or gritty vampire hunting, or the political etc. ramifications of supernatural beings. Comedy, for example, is not ruled out, but neither is it particularly made easier by the vampires.
I am assuming that the reader is a GM designing a setting for gaming purposes. If publication is in mind, this rpg.net article is probably useful. If world-building in and of itself is the goal, with a distinct possibility of someday gaming in the setting, the Campaign Builders’ Guide is a useful resource (it is good if you intend to game, too).
The most important thing is not to overdesign. Nobody but you is really interested in the fine details of the kingdom’s dressing habits during the summer solstice, unless they are somehow very interesting. And if everything is full of interesting detail, the setting is utterly overwhelming to anyone trying to learn it. Further, such a setting will feel cluttered.
The other most important thing is to not under-design. It is very possible to start with freeform or very light system and more detail as play goes on, but starting with next to no setting is hard. The first reason is that players need something to inspire their characters. “You can play anything!” is far from useful. Second reason is that improvisation and keeping the game consistent are hard without a baseline.
So, one requires a suitable level of detail to create the optimal setting. The actual amount is, of course, a factor of group’s playstyle. GM, if any, needs to know enough to set up the game. Depending on play style, this may be a situation (the orcs are attacking the village where you have lived your entire lives; D&D-esque fantasy) or a location (the city is large, approximately medieval, and limited by two rivers and the ocean; create shady characters; little if any magic). This needs to be communicated to players. Again, depending on play style, GM may need enough material to prepare an adventure, or to build a relationship map, or map a dungeon, or whatever. When player characters are created, again depending on group style, additional material such as NPCs (contacts and relationships of the PCs), houses, organisations, cities, monsters, and so forth, may be required or created.
That’s all well and good for starting the game, but to keep it running smoothly, further information may be necessary or at least useful. One option is to be creative and create more-or-less original and new material. A second, far more economic, method has been adequately explained by Chris Chinn over Deeper in the Game, but I can do a summary of the piece: Apply real world stuff, or other known material (Star Wars, Tolkien, D&D, Cthulhu mythos, …). A culture that is “like ancient Romans” or a religion that is “Christianity with different symbols” are both very easy to use in play. Another powerful method is taking or making up an arbitrary game element and creating an intuitive explanation for it. For geography, “archipelago”, “great plains”, “huge delta with rainforests” or an overall map are likely sufficient for quite some time.
Some of the most important roles of setting simply snuck in: The role of PCs, the things they can change, and the things that can affect them. These are often emergent qualities, but sometimes part of the concept. They should be thought about, either way. Simply saying “You can play anything!” is, again, not useful. Some examples are either necessary or damn useful. A setting inspired by vikings might have the coming of Ragnarök as an immutable factor (there is no way to prevent it) that pushes the PCs around by threatening that which they hold in value. Or it may be something that needs to be stopped, NOW. The game will have a different feel in both cases.
For more detail on elements that compromise a setting, see Troy Costisick’s relevant article. And for good list of things to think true and potentially write down, see another article by the same author. Both are highly recommended.
How much detail should one create? For me, the sweet spot is in just enough to improvise all sorts of fun details, but not so much that I have to reference anything or fear making big mistakes. Your mileage may vary.