Dr Stan Humphries, Zillow's Chief Economist, has a great post over on ZillowBlog which I'll republish below. It dissects a very important issue that you all face every day in your business: how to talk about what's happening in the real estate market.

Unfortunately, the most commonly cited measures of the state of the real estate market are median sale price data from NAR or a local MLS, and the the S&P Case-Shiller Index. In the past, we've discussed differences between the Zillow Home Value Index and Case-Shiller.  And we've been particularly vocal about why median sale price is a terrible measure of what's happening to real estate values.

I cringe everytime I see a headline citing "home price declines" when they mean "median sale price" declines. A subtle, but very important distinction. Take for instance this recent Bloomberg News headline: "California Home Prices Decline 41% on Foreclosures". The article cites California Association of Realtors data: "The median price for an existing, single-family detached home in California sank to $247,590 in February from $418,260 a year earlier." Sigh. The true change in the value of all homes -- not just those that sold in the period  -- is represented by the Zillow Home Value Index. Our most recent data on home values says that the ZHVI declined 21% year-over-year in Los Angeles and 17% in San Francisco, for example. Why such a big discrepancy between the ZHVI and median sale price as reported by CAR? Simple: the CAR data is being weighed down by the fact that a larger percentage of the homes that are now selling are low end, particularly at foreclosure. This drags down median sale price data in a way that the Zillow Home Value Index does not.

Bottom line: methodological flaws in the median sale price and the Case-Shiller Index make them appear overly negative, which is bad for all of us.

Without further ado, here are Stan's thoughts on the topic:

The latest Case-Shiller numbers for the month of January were released a couple days ago, and they showed a year-over-year decline of almost 19% in the composite index of 20 markets. That’s a whopper of a decline. Could the real estate market really be that bad? Well, as it turns out, it really depends on what you consider to be the “market.” According to Standard & Poor’s, the Case-Shiller Index is “designed to measure increases or decreases in the market value of residential real estate.” It’s important to note, however, that “market value” according to Case-Shiller includes all arms-length sales of homes, even those of foreclosed homes. It’s really an indicator of the change in prices of homes regardless of the circumstances under which they are sold. What won’t surprise many people, however, is that there’s actually a very large difference in prices between foreclosure and non-foreclosure homes. For example, Figure 1 below shows the difference between the median sale price of foreclosure and non-foreclosure homes in the San Francisco Bay Area. As you can see, these are two very distinct markets. The median price of foreclosures in December 2008 was 47% that of non-foreclosure homes (this ratio reach its zenith of between 75% to 90% during the height of the market between 2003 and 2006). And in December, foreclosure transactions represented 60% of all real estate transactions recorded in the San Francisco metro region, meaning that any measure that includes both types of transactions is significantly influenced by foreclosure transactions.

A measure of real estate appreciation built using non-foreclosure transactions (like the Zillow Home Value Index) is essentially looking at the change in the value of homes making up the black line in Figure 1. By including foreclosure transactions in such a measure (as Case-Shiller does), you’re also looking at the depreciation of homes that were previously in the set of homes making up the black line, but went into foreclosure, thus becoming part of the set of homes making up the red line. Understandably, price depreciation is quite high for these homes given that they move from one (higher) market to another (lower) market rather than simply moving within the same market. Note that even indexes based entirely on non-foreclosure transactions are influenced by foreclosure transactions to the degree that foreclosure sale prices influence non-foreclosure sale prices. They just don’t consider foreclosure sale prices directly. To get some sense of the difference that including foreclosures can make on a measure of appreciation, we compare in Table 1 the Case-Shiller Index to the Zillow Home Value Index (ZHVI) since the latter does not include foreclosure sales in its calculation (note that the inclusion of foreclosure sales does not account for all the differences between the two indexes). The Case-Shiller numbers are uniformly lower than the ZHVI, particularly in areas with either high rates of foreclosures or in areas where there is a large difference between the median prices of foreclosures and non-foreclosures (indicated by a lower ratio of foreclosure to non-foreclosure prices).

Unfortunately, in combining both foreclosures and non-foreclosures into a single metric, you’re not really getting a good insight into either market. In the current climate, you’re underestimating the decline in value of foreclosed homes and overestimating the decline in value of non-foreclosure homes. More importantly, from a consumer perspective, homeowners probably infer that home price indexes are a general indication of the real estate appreciation that they might realize if they were to sell their own home. In interpreting appreciation information from this perspective, it is likely that homeowners assume implicitly that they would sell their home on the open market, not have it foreclosed upon. For homeowners thinking in this way, Case-Shiller is not a good measure for them to use because the assumptions used to interpret the data do not match the assumptions used to create the data. So, when you think of your “market,” if you think about what has happened to the price that your home might fetch on the open market (and you don’t intend to foreclose), things aren’t as bad as the Case-Shiller Index would lead you to believe. It’s closer to what the Zillow Home Value Index indicates for your area (available right down to your ZIP Code).