In order to answer the question raised, the authors undertook a pilot study within a Victorian inner city LGA. The objective of the study was to develop a method of assessing non-profit driven facilities in LGAs, taking into account the strategic aim of service delivery. The aim was not only to develop a new model to measure facility performance according to strategic objectives, but also to provide a system that would allow informed, transparent and accountable decisions about property assets to be made.

In line with this reasoning, the project involved an examination of community facilities to ensure their strategic and effective use. The underlying purpose was to ensure that facilities were held for strategic rather than political reasons, and to identify facilities that were under-performing in a number of key areas important to the LGA and the community. The focus of the facility evaluation was on service provision as a return on investment, with particular regard to the authority's key result areas (KRAs).

The involvement of stakeholders was essential to the success of the project. As discussed, authorities tend towards a division of roles between facilities management and service providers. The pilot study involved representatives from each of these groups to canvass different views and to maximise transfer into management systems at the completion of the project.

Design

It has already been mentioned that performance measurement in a strategic environment must take into account financial as well as non-financial measurements. The approach developed for this study may be termed the service balanced scorecard (SBS) (see Figure 1).

The SBS has its origins loosely in a tool from the general field of performance measurement. The balanced scorecard developed by Kaplan and Norton (1992) is a management tool that allows organisations to assess their overall performance in relation to strategic aims. It has become popular with industry because of its ability to combine a diverse set of performance measures that are aligned with the corporate mission. The central tenet is quite simple: performance must be measured against corporate aims balancing financial and non-financial perspectives.

The balanced scorecard departs from traditional approaches to performance measurement in another way. Perhaps as a result of their financial basis, traditional measures are geared towards control; the balanced scorecard, however, is goal oriented and has vision and strategy at its centre.

This study adopts the balanced scorecard's fundamental principle - namely, that performance must be assessed against the organisation's strategic aims - and applies it specifically to property. However, the implementation is quite different, in that the balanced scorecard scores by management opinion on a range scale, whereas the approach adopted here is to remove the reliance on management opinion (and potential bias) and seek independent quantitative and qualitative indicators of facility performance that have an open range.

Another approach to real estate performance measurement using a scorecard system that integrates strategic aspects of real estate management has previously been developed by Apgar (1995a). However, his method differs from the SBS approach as it restricts itself to property and building related metrics, and does not include outcome oriented performance indicators (Apgar, 1995a, b; Apgar and Bellew, 1995).

The SBS selects measures of performance to specifically reflect facility performance from a services context. Performance is evaluated from four different perspectives, balancing financial and non-financial measures to assess how property supports the delivery of services through the organisation's facilities.

The SBS differs from Kaplan and Norton's model in another significant way. While they consider the generation of appropriate strategic performance measures to be a senior management function, the SBS takes a stakeholder approach. LGAs have a number of powerful stakeholder groups, with sometimes diverging interests and knowledge as to what constitutes facility performance. In order to provide a successful set of metrics, the SBS must account for the viewpoints of all of these stakeholder interests[2]. The main stakeholder groups are:

• the community, who are the end users (or customers) of the facility;
• the service providers, who manage and provide services through the facility (the provider side of the client-provider split);
• facilities managers, who are concerned with the physical and financial running of facilities (often included with the provider side in organisational management, but in fact providing a service through facilities to the service providers); and
• management, who make decisions about facilities and services, and who must be accountable to the community and to other levels of government. This is the client side of the client-provider split, and usually represents the policy development components of the management of the LGA.

Figure 2 illustrates the mechanism required for maximum stakeholder input in developing performance indicators for each of the SBS's four perspectives. In designing facility performance measures, all stakeholder views should be considered equally; thus a great emphasis is placed on independent, quantifiable metrics.

In a further departure from traditional attempts to measure facilities performance, data is not collected by facilities managers only - rather, all stakeholders provide information in a centralised database linked to the council's asset register. In this way, existing information about facilities can be linked to the new set of data required for the SBS, thereby minimising duplication of data entry.

Once data collection is completed, each building is evaluated for each of the four perspectives: services, community, building and financial. From this the SBS develops a profile of each facility's performance which can be compared to like facilities within the organisation in a process similar to internal benchmarking. Balancing and evaluation of the four perspectives depends on the organisation's strategic objectives and is achieved through a process of weighting the four areas.

The successful implementation of the SBS depends on:

• metrics that reflect corporate aims and objectives;
• metrics that reflect stakeholder needs;
• data collection that is consistent and regular to a centralised database; and
• performance indicators that are checked at regular intervals to ensure that they continue to be reflective of the organisation's strategic aims.

Differential weighting of measures is a policy task that should be conducted by senior management with reference to the organisation's business goals. The weighting should reflect how important each of the four perspectives is in relation to the organisation's business aims, as such decisions influence how the performance of a facility is evaluated after the data collection has been completed (e.g. financial 20 per cent, building 20 per cent, services 30 per cent, customer 30 per cent). Potentially this weighted evaluation could also affect the design of new facilities.

When implementing the SBS, the organisation must decide on a suitable set of performance indicators that reflect its corporate mission for each of these perspectives. It must therefore take into account stakeholder interests and consider that each stakeholder group has a view of the needs of the facility that incorporates different aspects of the four perspectives. For this pilot study, the stakeholders participated in a series of unstructured interviews designed to derive appropriate metrics to be tested on real facility data. The organisation's published KRAs were used as triggers for these sessions.

In consultation with the LGA, a range of facilities were chosen for assessment with the SBS, including art galleries, reserves, community centres, playgrounds, elderly citizens' centres, child care centres, maternal and health care centres, club rooms and libraries. The resulting metrics combine qualitative and quantitative measures of facility performance. Taken together, these provide a comprehensive performance measurement system.

It was determined that the desired attributes of a facility that performed well in terms of service delivery were:

• delivers a range of services;
• has a high number of users;
• is used by a range of community sectors (for example, elderly citizens, ethnic groups, children);
• receives good community support;
• provides services suited to the community in terms of noise, zoning, accessibility, hazards, traffic and demographics;
• is open for as long as possible and used by the community during the majority of its opening hours; and
• is financially viable.

From these desired performance outcomes, a set of corresponding metrics was designed. After an initial period of testing, the system and the metrics were reviewed and revised in accordance with stakeholder feedback.

Results

Anecdotal evidence suggests that facility managers normally undertake facilities performance evaluation. However, for this study, the managers of the services being provided through the facilities were asked to provide the information about facility performance. The data was entered directly into a Microsoft Access database linked to the LGA's asset register. The database interface was designed like a questionnaire, and managers were prompted by questions regarding their facility at each data entry point. In this way, data could be collected in a central database, making use of already existing information on council's property[3].

The following is a description of the indicators used grouped by the four perspectives of the SBS. A discussion of the results obtained is included in each section (see also Figure 3).

Financial perspective

Initially, metrics in the financial perspective were broken down into:

• financial data relating to the physical operation of the building; and
• financial data relating to the provision of services through the building.

This was done because stakeholders felt that it was important to account separately for the cost of running the building and the cost of running the service. This financial information was then calculated as a ratio of the gross building area (GBA).

However, the first review of the pilot model found that, in collecting data about facilities, it was difficult to separate costs associated with the building and the service. Furthermore, it was decided that it was not useful to measure cost as a ratio of the GBA, as services were provided largely from existing building stock and as space rationalisation was not considered a high priority. As a result, the financial indicator was pared back to reflect only the total cost to council of providing the facility.

Building perspective

The building perspective brings together several tangible aspects of facility performance.

Utilisation rate

The utilisation rate assesses how well the facility is used in terms of time. This score does not include space-related measurements. Data collected are:

• the possible number of hours per week the facility could be open (maximum 168);
• the number of hours per week the facility is actually open out of the number of hours it could be open (maximum 168); and
• the number of hours per week the facility is actually used by the community out of the number of hours it is actually open (maximum 168).

From these figures, the utilisation rate was calculated as a score out of ten.

The review of the pilot showed that council considered it more useful to graph the three components of utilisation as a bar chart. In this way, the utilisation rate did not merely indicate whether or not a facility was performing well from the point of view of utilisation, but it was immediately obvious which aspect of utilisation was the cause of good or bad facility performance. The utilisation rate was revised to be recorded accordingly (see also Figure 4).

Building condition and trend indicators

The building condition indicator assesses the physical condition of the facility, the associated risk, and its suitability in physical terms for the services it delivers.

Managers were asked to rate for the following factors on a five-point scale ranging from "very poor" to "very good":

• Condition: the physical condition of the facility;
• Suitability: the suitability of the facility to the service it provides; and
• Risk: the risk associated with the facility.

Averaging these three scores derives the building condition indicator.

The building condition indicator is supplemented by the building condition trend indicator, which measures expectations of how the facility will develop in each of the three areas over the next five years on a similar five-point scale ranging from "get much worse" to "get much better". Averaging these three scores derives the building condition trend indicator.

In combination, the building condition and building condition trend indicators show the state of current building factors and how these are expected to develop over the next five years.

Services perspective

The aim of the services indicator is to determine how well the facility delivers services to the community in line with the LGA's objectives. The services indicator is a composite score that is derived from data relating to:

• the number of distinct individuals (users) who use the services and the facility annually;
• the number of distinct visits to the facility made annually by its users;
• the number of distinct services provided through the facility; and
• the number of community sectors (for example, children, ethnic groups, disabled persons, the elderly) that use the facility.

The above data was then measured as a ratio of the facility's GBA.

The review of the initial measure showed that the LGA did not consider it useful to measure service performance as a ratio of space used. Some services (e.g. recreational facilities) require larger amounts of space to operate well. Because the services indicator was initially calculated as a ratio of service delivery to GBA, the relationship between service measurements and GBA was an inverse one. For example, if comparing the service indicators for two facilities, both of which have similar figures for service performance, but one of which operates from smaller premises, then the smaller facility appears to be performing better from a service perspective. LGA stakeholders felt that this distorted an evaluation of service performance, and preferred to measure service delivery as an absolute number independent of GBA.

The measure was consequently revised to reflect only service performance in line with the LGA's strategic aim, disregarding the GBA.

Another challenge encountered was that community groups often do not keep adequate records of their membership and usage patterns, and consequently the data provided is sometimes inadequate, thereby limiting the accuracy of the performance evaluation.

Community/customer perspective

Two sets of metrics make up the community/customer perspective, both of which include indicators of expected future developments.

Community access and trend indicators

The community access and trend indicator measures how well the facility and its location are matched to community needs and the services provided, and how accessible the facility is to the community.

Managers were asked to evaluate the facility on a five-point scale ranging from "very good" to "very poor", for each of the following six factors:

(1) Accessibility. How accessible is the facility to the community and staff in terms of parking, public transport, etc.?

(2) Demographics. How well are the services provided through the facility matched to the demographics of the area in which it is located?

(3) Zoning. Is the zoning of the area in which the facility is located suited to the services it provides?

(4) Noise. Do noise levels in the area in which the facility is located impact on the service delivery?

(5) Traffic. Does the amount of traffic in the surrounding streets affect the service delivery?

(6) Hazards. Do hazards affect service delivery through the facility?

Averaging the scores for the above six factors derives the community access indicator.

In a second step, the community access indicator is complemented by the community access trend indicator, which measures how the facility would be expected to perform over the next five years. Managers were asked to estimate on a five-point scale ranging from "get much worse", to "get much better" how the facility would develop in relation to each of the six factors discussed above over the next five years.

In conjunction, the community access and trend indicators provide a snapshot of how the facility is currently performing, and the anticipated direction of change over time in relation to its suitability and accessibility to the local community.

Community support and trend indicators

The community support indicator measures the level of community involvement in terms of volunteer contribution. This score is considered to be an important measure of how well the service and the facility are accepted by the community, that is, how much ownership the community feels it has over the service and the facility.

Managers are asked to assess the community's volunteer contribution on a five-point scale ranging from "very poor" to "very good".

The community support trend indicator supplements the community support indicator. Managers were asked to estimate how the community's volunteer contribution would develop over the next five years on a five-point scale ranging from "get much worse" to "get much better".

After the initial piloting period it was found that "volunteer contribution" was considered to be a confusing concept that was difficult to measure. Initially this was designed to gauge the level of community involvement; however, it was felt that "volunteer contribution" was not an adequate reflection of the community's acceptance of a council service. Aspects that were considered to be equally important were flow-on effects in terms of community building and the building of informal networks within the community. This confusion highlighted the need for concise descriptions and definitions that are understood by all parties contributing data to the service balanced scorecard.

Evaluation

Once all relevant data was collected, facilities' performances were evaluated from a set of metrics centred on the desired performance outcomes.

Based on the data collected above, facilities were evaluated by comparing the performance of facilities of a certain type with one another, as well as comparing facilities against the averages for all facilities surveyed in the study. In this way it can be established how one asset group is performing against all assets, and a ranking for facility performance within the asset group may also be established (see also Figure 5).

An overall evaluation of how well a facility is performing against stated objectives depends on the weighting, mentioned earlier, that is attached to each of the four perspectives. This process of weighting the importance of each of the perspectives is conducted by senior management with reference to council's strategic goals. The weighting should be reflective of the importance of each of the four perspectives in relation to the organisation's business aims. The weighting influences how the performance of a facility is evaluated after the data collection has been completed.

For example, the LGA may decide that the services perspective is the most important of the four perspectives, on an equal par with the community perspective, followed by the building perspective and finally the financial perspective (see also Table I).

Consequently, a facility (or group of facilities) that is performing well on the financial and building perspectives, but not so well on the services and community perspectives, would be considered to be performing below its capacity. Conversely, a facility that rates well on the community and services perspectives, but that was losing money on the financial perspective, would still be considered to be performing well.

Further research

In its present state of development, the SBS allows performance to be evaluated only through a process of internal benchmarking of facilities. However, the research reported here is currently being expanded through cooperative support from other Victorian councils to further refine the indicators and method of the SBS. The expanded project develops a variation of the SBS called Logometrix (local government facilities - strategic performance measurement). It will develop and test the technique further and ultimately lead to a tool incorporating a centralised database that will allow councils to benchmark the performance of their facilities against other councils.

There is also considerable potential to apply the technique in the corporate environment. Here the balanced scorecard is being adopted; however, there is interest in the application of this new approach because it allows for independent indicators and open-ended scales of assessment.

Conclusion

The service balanced scorecard, piloted in this study, assists LGAs to remove the environment of distrust and to provide information to stakeholders that will empower management to make strategic decisions about the future of facilities.

The SBS provides a new method for evaluating facilities linked to an authority's strategic aims. Performance measures derived using a stakeholder approach provide measures that are meaningful across business units and management levels. The advantage of this approach is that facilities are considered in line with the LGA's overall objectives, and performance indicators are reflective of the facilities' desired function. This means the SBS is not merely a tool that measures performance in building related terms, but it also provides the LGA with metrics that are reflective of the facilities' service function, and therefore a means of making accountable decisions.

The involvement of the stakeholders in the development of the tool and the designed independence in scoring the indicators all act to reassure that strategic planning is based on sound strategy rather than political machinations.

The benefit of this approach is that future facility-related decision making has a greater chance of receiving support from those it is intended to serve - the community.

Notes

1 The term "metrics" here requires comment. This is not a dictionary term (even in Barron's dictionary of real estate terms), but rather a US derived colloquialism of such prevalence in the industry that its use has become preferred to terms such as benchmarking and performance measurement. The term is used here because it implies measures for performance evaluation, usually with a strategic intent.

2 For a stimulating discussion on performance measurement using a stakeholder approach, see Atkinson et al. (1997).

3 Deakin (1997a, b) has written about the issues relating to the management of property data for local government in a strategic environment. He points to the lack of appropriate database systems for local government, which is partially to blame for the fact that data collection in relation to property performance remains largely restricted to asset registration and valuation information.

---

 

Figure 1 The SBS

Figure 2 Balancing stakeholder views

Figure 3 The SBS in action

Figure 4 Utilization rate

Figure 5 Financial indicator