The building performance gap: Are modellers literate?

One of the most discussed issues in the design community is the performance gap. In this research, we investigate for the first time whether part of the gap might be caused by the modelling literacy of design teams. A total of 108 building modellers were asked to comment on the importance of obtaining and using accurate values for 21 common modelling input variables, from U-values to occupancy schedules when using dynamic simulation to estimate annual energy demand. The questioning was based on a real building for which high-resolution energy, occupancy and temperature data were recorded.

A sensitivity analysis was then conducted using a model of the building (based on the measured data) by perturbing one parameter in each simulation. The effect of each perturbation on the annual energy consumption given by the model was found and a ranked list generated. The order of this list was then compared to that given by the modellers for the same changes in the parameters. A correlation analysis indicated little correlation between which variables were thought to be important by the modellers and which proved to be objectively important. k-means cluster analysis identified subgroups of modellers and showed that 25% of the people tested were making judgements that appeared worse than a person responding at random.

Follow-up checks showed that higher level qualifications, or having many years of experience in modelling, did not improve the accuracy of people’s predictions. In addition, there was no correlation between modellers, with many ranking some parameters as important that others thought irrelevant. Using a three-part definition of literacy, it is concluded that this sample of modellers, and by implication the population of building modellers, cannot be considered modelling literate. This indicates a new cause of the performance gap. The results suggest a need and an opportunity for both industry and universities to increase their efforts with respect to building physics education, and if this is done, a part of the performance gap could be rapidly

Practical application: In any commercial simulation, the modeller will have to decide which parameters must be included and which might be ignored due to lack of time and/or data, and how much any approximations might perturb the results. In this paper, the judgment of 108 modellers was compared against each other. The results show that the internal mental models of thermal modellers disagree with one another, and disagree with the results of a validated thermal model. The lessons learnt will be of great utility to modellers, and those educating the next generation of modellers.