Way back when I started this series of thoughts, I got some very useful comments on the initial post, which I'm going to pick up on now that we've covered fluff as a key ingredient to a setting. Specifically, I'm thinking about open-endedness, and development/evolution, which means it's 'Mike waffles and ponders' time!
There's a tricky line to draw here. In an ideal world, story development is/should be a given. It's about the fluff - your setting moves along - much as (say) the VBCW setting does - so that there are always new things happening, but not so fast that people don't have time to play with what's available. If we look at the real-world setting of WW2 as an example, in six years an awful lot happens, factions rise and fall, technology advances...
Not all settings get this last bit right. Battletech (I quote this a lot, because as big, sweeping, SF settings go, it's one of the biggest and for the most part best) doesn't, in my opinion. The arrival of the Clans is a massive 'outside context problem', which isn't, in and of itself, an issue - it gives the existing factions new adversaries, the chance for new Alliances. But unfortunately at around about the same time, the Clans bring new tech with them, and the existing factions develop new tech, and it's game breaking tech, and they got that bit badly wrong. One of the core unwritten precepts of the original 3025/6 tech Battletech was that, to sum it up crudely, the only things that could blow a Mech's head off in one shot could only do so at short range. With the advent of the 3050-era tech, and extended range PPCs and Ultra AC-20s (for example), death from long range was suddenly much more likely. To be fair, this is actually a rather dumb piece of brokenness in the rules about the amount of armour and internal structure 'Mechs can have: the rules were originally designed around 3025-era balance, for which they work beautifully.
40K, of course, does this far worse, in that it doesn't really even pretend that what it's doing is advancing the setting - it's just pure 'cheese' - changes to the rules, because it can, and because it wants another pull on the direct line from little Johnny to daddy's plastic.
It's an interesting problem, though. You can make the argument that WW2 has this: bring a Tiger into an early war scenario, and it's massively unbalancing. And if you want to 'follow along' with WW2 as the campaign advances, you wind up replacing all your armies by the end of the war to keep with the story. But at least with any self-respecting set of rules, the game doesn't go out of whack if your forces are from the same period with the setting.
War, though, is a great driver for technology. You can't escape how much military technology advanced during both world wars and then slowed down a large amount thereafter. If your setting is at all high-tech, it's difficult to avoid (Moore's Law, anyone! :D) the fact that technology will and should get better over time unless you've hit some arbitrary, physical or scientific limit (the speed of light, the size of an atom of silicon, the strength/height of a normal human, etc).
Having waffled my way through that (*grin*) I think what I'm saying is that advancing the 'story' of a campaign is great: giving people new opponents and/or new contexts in which to fight enhances the odds they'll stick with your setting, that magic that is 'replay value'. And sure, everyone likes 'the new shiny': the trick is not to do new in-game tech and weaponry for the wrong reasons, not to make it so awesome that you can't, effectively, play the game without it. Introduce it slowly, make it scarce, make it have disadvantages - even if these boil down to 'it's so expensive you can only have 1', or 'you can have that Tech 3 energy weapon, but you need to mount it on a slow, massive chassis that can't use other tech because it needs all the power...'
In short - it's as much a game-design balance problem as anything else. If your developing story starts to lead you towards the new shiny, you need to consider long and hard what it does to the game.