Climate Change Models "Validated"

There is a breathless report on Science Daily announcing "Climate Models Look Good when Predicting Climate Change". Sounds wonderful. But at the bottom of this article they note:

Although model-based projections of future climate are now more credible than ever before, the authors note they have no way to say exactly how reliable those projections are. There are simply too many unknowns involved in the future evolution of climate, such as how much humans will curb their future greenhouse gas emissions.

Isn't that a little worrisome that the authors caution that "they have no way to say exactly how reliable the projections are" for the climate models? Exactly how do you "validate" a model when you can't know how reliable the projections produced by the model are? The point of a model is to give you a tool to project data into the future. There is something fishy here.

So I went to the actual scientific report on which this news story is based.

As good scientists, the authors note limitations of their study (I've bolded key statements):

Several important issues complicate the model validation process. First, identifying model errors is difficult because of the complex and sometimes poorly understood nature of climate itself, making it difficult to decide which of the many aspects of climate are important for a good simulation. Second, climate models must be compared against present (e.g., 1979-1999) or past climate, since verifying observations for future climate are unavailable. Present climate, however, is not an independent data set since it has already been used for the model development (Williamson 1995). On the other hand, information about past climate carries large inherit uncertainties, complicating the validation process of past climate simulations (e.g., Schmidt et al. 2004). Third, there is a lack of reliable and consistent observations for present climate, and some climate processes occur at temporal or spatial scales that are either unobservable or unresolvable. Finally, good model performance evaluated from the present climate does not necessarily guarantee reliable predictions of future climate (Murphy et al. 2004). Despite these difficulties and limitations, model agreement with observations of today‟s climate is the only way to assign model confidence, with the underlying assumption that a model that accurately describes present climate will make a better projection of the future.

I'm bothered by the fact that this report does not quantify the "improvements" in climate model predictions. Instead the conclusion is qualitative (again I bold certain parts):

Current models are certainly not perfect, but we found that they are much more realistic than their predecessors. This is mostly related to the enormous progress in model development that took place over the last decade, which is partly due to more sophisticated model parameterizations, but also to the general increase in computational resources, which allows for more thorough model testing and higher model resolution. Most of the current models not only perform better, they are also no longer flux corrected. Both – improved performance and more physical formulation – suggest that an increasing level of confidence can be placed in model based predictions of climate. This, however, is only true to the extent that the performance of a model in simulating present mean climate is related to the ability to make reliable forecasts of long-term trends.

Is it me, or is there something a little odd about "validating" models without quantifying the improvement? Don't get me wrong. I do believe the models are more complex, use more computation, etc. But this report doesn't clarify how much more accurate the models have become. Have bad models become simple poor? Or did poor models become OK? Or maybe OK models are no good? Or, if we are really lucky, good models became excellent? These are all qualitative improvements, but without quantification, how am I decide which it is?

I think Freeman Dyson has a good grasp on the state of climate modeling. Dyson is not anti-science. He just wants more realism in applying the tools of modeling.