CoinResource looks at the variables that drive coin valuation.
Popularity or collector demand
First, a coin series needs to be popular among collectors, this will help guarantee value in the future. This is not however, the only driver of value. Popularity also guarantees that there will be buyers willing to pay money for the coins. Some coin series are popular because of their "looks" (Saint Gauden’s Double Eagle), some are popular due to their rarity (Trade Dollars, Flowing Hair Dollars, or Barber Halves), and some are due to their uniqueness (State Quarters and the new Buffalo Dollar). Collectors tend to favor certain series and this drives the values up. What would you rather collect: Morgan dollars or Susan B. Anthony dollars?
The "grade" or condition of a coin
Simply put: higher grade coins are harder to find. The minting, storage and transport process used for manufacturing coins causes a lot of incidental damage to the coins surface. Damage ranges from nicks, dings, and scratches -- to general wear. The higher the grade, the more rare the coin. The strike of a coin is important, too. The strike, if strong and distinct, can add value to a coin. Weak struck coins are less desirable due to the design being less impressive. Proof-like condition adds value as well, since this is a more rare condition for a coin. Fields that have DMPL (deep mirror proof like) qualities are even more rare since the conditions for striking such a coin do not occur very often (the die must be new, the press set just right). So the grade and eye-appeal of the coin are an important measure of its value. There is even a 70 point scale for grading coins that is called the Sheldon scale or mint state scale. And, there are many books on this subject too.
Scarcity or rarity
The more rare or scarce coins tend to have higher values. Scarcity is determined by the mintage per year, per mint mark, and per grade, plus any special "varieties" or "types". [Note: For instance, a Susan B. Anthony dollar from a proof set in 1981-S (San Francisco mint) is fairly common, however, a variation of this coin (called Type II) has a "clear s" due to a die variation. This coin is much more rare and is worth 20 times more money.]
There are guide books that show all the mintages, dates, mint marks, and variations of coins as well as their values. These guides can help you understand how these factors work together to change a coins value.
More valuable coins have lower mintage numbers. For example, many modern commemoratives are fairly inexpensive, however the "Community Service" commemorative is very expensive because it has a low mintage and is an attractive and popular coin. You need to study the mintage and prices of a coin series to understand the relationship. This valuation is measured separately for each series, there is no one rule for all coins.
Common date coins with a large mintage are less likely to rise in value as compared to a more rare date coins with low mintage. Coins that exist in very scarce populations are considered highly desirable, for instance a proof Morgan dollar is very rare compared to a common date Morgan and the proof may be worth 50 to 100 times more.
One-of-a-kind and unique (or exceptionally rare coins) are among the most expensive. The 1933 Gold Saint Gaudens Double Eagle is the only 1933 Double Eagle that is legal to own, it therefore is exceptionally rare and expensive (expected to sell in the millions).
Inherent metal value (or "melt value")
Gold, platinum, and silver content coins have an inherent base value due to their precious metal content. This is called their "melt value." Many older coins are worth more than their melt value, but this at least sets the base price of a coin.