Well of course you should try them all, if you have the time and feel like it. A quick game with Civ1 should make it clear how identical it is to Civ2, only a lot older. Since I haven't played Civ4 or Civ5 I can't recommend you where to start if you don't want to go in order, other than sticking to Civ2.
I think all Civ games specially cause very positive and very negative reactions with many different people. Case in point EOF and I don't agree in the least. Features in Civ games are very much a matter of taste. If it were possible to statistically extract some meaningful information from reviews, it would still be faster to try them yourself. (And the later installments have demos available.)
I like Civ2 better than Civ1. It's totally true that it's a re-make with nothing new substantial to the core game, and the downsides of the original are due to its age. But it's very annoying how random battles are, that's fixed in Civ2.
Civ3 introduced a lot of new ideas, but the problem is that I didn't like most of them or how they were implemented. The only one I like without reservation is bombarding, but it's too weak in that game. Culture-mandated frontiers are political nonsense. Insurgence is too strong, no matter how many military units you try to use; the only solution is to exterminate the populace; again "culture" has an undue importance. The trade system is terrible in my opinion: there are few kinds of resources and they are ridiculously scarce (even horses for example); and Civ's trademark AIs who are never interested in co-existing peacefully make you grab them all as your only alternative.
Alpha Centauri is the most innovative. It introduces a lot of new features that I love, but were never revisited in any other Civ game. For example units are designed (and can be customized by you) as a combination of means of transportation, weapon, etc., which are what must be researched. And the infamous Gengis Khan AI of civ games is totally fixed in Alpha Centauri, it's possible to have long-lasting alliances.
I'm very curious about Civ5's strategic combat system. But again I haven't played Civ4 or Civ5.
Also (starting remarkably early in the series with Civ2) you can get additional fun with scenarios and mods. You can find lots in
civfanatics.com. I have only tried Civ2 scenarios: the original game lets you build scenarios and design maps, but the expansions (
Fantastic Worlds and
Multiplayer Gold) allow more things to be modified, for example units, city improvements and technologies. It's amazing what some people have done with this. In particular I played a couple of Scenarios by a history buff called John Ellis and they were amazing, not only because of the historical setting: he takes advantage of Civ2's mechanics to improve them, and combat makes much more sense and is finely tuned. For example Cavalry has high attack and low defense, but also low hit points, meaning that if it doesn't happen to win the first charge it gets vulnerable and will probably be killed immediately. Infantry has similar attack and defense like in the original game, but also high hit points, so they're tougher to kill regardless of luck. Siege engines have, besides low movement, insane attack and firepower, meaning that only the heaviest fortification has a chance of not being destroyed, but very low hit points, so if they don't succeed at first they just break down.