Historic Buildings & Places Data Management Tools: Defining and Visualizing Casework Impact
| Sponsor: | Historic Buildings & Places |  Bainbridge, H. (2024). HB&P Students [Photo]. |
| Sponsor Liaison: | Ross Anthony, Liz Powers | |
| Student Team: | Mona Blake, Sarah Fenton, Parker Langa, Michael Nixdorf | |
| Abstract: | HB&P is a National Amenity Society that assists in the preservation of built heritage in England and Wales. HB&P has faced pressure to highlight its impact for the sake of acquiring more membership and sharing updates with trustees. Our project goals were to define and create an impact assessment for HB&P, identify trends within their data, and create recommendations for future data analysis. To achieve these goals, we interviewed HB&P trustees and other amenity society representatives, surveyed HB&P membership, and compiled and analyzed casework data. From this we created a comprehensive impact analysis which showed a positive impact while using interview and survey feedback to advise HB&P on future data collection. | |
| Link: |
Historic Buildings & Places Data Management Tools: Defining and Visualizing Casework Impact Report |
|
Executive Summary
Historic Buildings & Places (HB&P) was established in 1924 and is a member of the Joint Council of National Amenity Societies (JCNAS) (HB&P, 2021). HB&P works with local planning authorities (LPAs) in England and Wales on a variety of preservation initiatives, including applications to have a site listed or de-listed in the National Heritage List for England (NHLE). Listed buildings have additional legal protections, the most relevant being the required consultation of National Amenity Societies (NAS) on Listed Building Consent (LBC) applications (Planning (Listed Buildings and Conservation Areas) Act, 1990). LBC applications make up the majority of HB&P’s casework and are submitted to LPAs when the renovation of a listed building involves an aspect of demolition. National Amenity Societies do not have the power to halt a construction process, but offer letters of support, objection, or advice to LPAs while they are making their decision.
HB&P advises on applications and advocates for heritage preservation. Our team organized their casework data, looked for trends, and demonstrated the impact of their organization’s casework. We defined impact in terms of their casework and identified criteria that can be used in its assessment. The organization has been unable to conduct data analysis in the past due to limited resources; therefore, we wanted to make it as easy as possible for staff to continue analysis after the end of our project. Our final objectives were: (1) Define impact in terms of HB&P’s casework; (2) Identify trends within casework for their current and historic data; and (3) Provide recommendations for future data collection and analysis. To do this, we used a mixed methods approach of surveys, semi-structured interviews, and data analysis.
To address our objectives, we used mixed methods of surveys and interviews. By using surveys and semi-structured interviews, we were able to collect perspectives from a variety of stakeholders. Surveys were geared towards HB&P members, who contribute financially to the organization, and individuals that have an interest in built heritage preservation. HB&P’s marketing team distributed the survey electronically through the HB&P members email list and posted on their website. We also conducted semi-structured interviews with trustees, who are HB&P volunteers with more knowledge on built heritage and involvement in the organization.
To help define impact, we asked stakeholders about their values and perspectives on impact for HB&P casework and their input on mock visualizations our team had created. Their feedback on impact helped us define goals which informed data categories we should analyze, and their feedback on mock visuals provided insight on which graphs we should create from our analysis. Our interviews with other NAS caseworkers also guided our analysis as it allowed us an understanding of systems used across the sectors. A flow chart outlining the ways our methodology informed our objectives is seen in Figure 1.
Figure 1: A flow chart highlighting the methods we used and their relationship with our findings.
Trustees and survey respondents were both highly interested in the outcomes of applications. However, HB&P has not tracked this data previously, as decisions are posted on individual LPA websites and can undergo multiple rounds of appeals or re-applications. Therefore, as part of our impact objective we were able to determine whether LPAs referenced HB&P’s advice in their decision for non-pending LBC applications in the 2023-2024 fiscal year.
For our second objective, the casework analysis data came from physical documents starting in the 1950s, multiple Excel sheets of HB&P’s consultations, and the JCNAS database of applications and responses from different National Amenity Societies. From this data, we conducted in-depth data analysis for different application types, NHLE status, building types, and regional distribution for both the number of applications and responses. For each category, we divided responses by applications to see the percentage of cases that received answers from HB&P to identify if there are points of focus or overlooked categories within their casework.
To address the final objective, we centralized HB&P’s historic data, created manuals and templates of our analysis, and provided additional categories for them to track on Excel which fall under the recommendations block in our methods flow chart (Figure 1). These recommendations came from our own experience while conducting data analysis, survey responses, and semi-structured interviews with HB&P trustees and caseworkers from other National Amenity Societies. We asked in both our survey and trustee interviews for additional information stakeholders would like to see from HB&P’s casework, and we asked other NAS employees which metrics are tracked in their databases. When organizing their data, we also identified missing fields that we believe HB&P would benefit from collecting. We created an infographic for HB&P members with a summary of our data analysis and provided a template to our sponsors for them to update as an annual impact report. The manuals contain procedures and instructions so that their analysis can be updated as casework continues.
Our findings identified members and trustee’s goals and desired impacts of HB&P, which types of data visualizations are most accessible, and the results from data analysis of HB&P’s current and past records.
Goals and Impact
We found that stakeholders wanted HB&P’s casework to address a diverse range of cases. Every trustee we interviewed mentioned the importance of working with cases from different site types, grades, and regions and addressing a broad range of application types. From a multi-select question, 46% of survey respondents chose working with different building types as an important metric to evaluate casework impact. Additionally, a majority of members wanted publications that show casework by building type (67%) and regional casework distribution (51%). Both members and trustees also indicated an interest in HB&P’s work focusing on continued use of historical sites, with 72% of survey respondents selecting it as an area of interest. In interviews, some trustees also mentioned that they were willing to sacrifice some of a site’s historical authenticity to extend its building’s life span. Both groups also mentioned that they wanted HB&P’s work to support LPAs, with one trustee highlighting that empathetic responses are more likely to be well received and the advice followed.
Visualization Types
We found that stakeholders wanted to see clustered columns, heat maps, and composition graphs. Across multiple data categories, 58% of members and 83% of trustees preferred clustered column charts with one trustee saying that clustered columns provide consistency to visualizations since they work for each data category we graphed. Other than clustered column charts, 43% of members wanted heat maps for regional distribution data and most trustees found composition graphs, such as pie charts, helpful for illustrating data with one dominant response.
Data Analysis
Within regional distribution, western regions receive the most responses, while the North East and the Greater London Area receive the least, but this is in part due to the varying number of listed sites per region. Staff mentioned concerns about HB&P’s work focusing on London, so this highlights regional diversity within casework. For site type, grade, and application type the percentage of consultations responded to for each category was relatively constant; however, one category generally received most of the responses. The constant percentage of consultations responded to highlights that HB&P addresses a wide range of cases across multiple categories. Across site type, grade, and application type around 5% of consultations received responses. For site type, the majority of HB&P’s responses were for domestic sites and Grade II buildings. For application type, most responses were for Listed Building Consent applications; however, two outliers within consultation response percentage were de-listings and pre-applications. About 40% of de-listings and 20% of pre-application requests received responses, but this is due to those categories having fewer overall applications. Finally, we found that 70% of response types were letters of advice. This aligns with HB&P’s goal of supporting LPAs, since their focus isn’t on writing objections.
Outcome Assessment
One of the primary goals of this research was to understand multiple factors of the outcome of HB&P’s casework. Next, we developed a system of comparing response letters with LPA decisions to determine HB&P’s influence on casework outcomes. Based on criteria we created based on our findings, we found that the majority of analyzed responses had a positive impact with 44% as fully positive, 14% as slightly positive, and only 27% as negative (Figure 2).
Figure 2: The breakdown of casework outcome impacts for the 2023-24 fiscal year.
After determining stakeholders’ goals for HB&P, which visuals were the most accessible, and trends of HB&P’s casework responses and outcomes, we created a list of recommendations for future work.
We developed five recommendations for HB&P curated from insights from the survey, interviews with HB&P trustees and caseworkers from other NAS, and analysis of HB&P’s data. These recommendations aimed to address our three objectives of (1) defining casework impact, (2) identifying trends and casework, and (3) providing guidelines for future data collection and analysis. Our first four recommendations address Objective 3, offering new data fields and ways to continue and expand upon our analysis. Recommendations 3 and 4 also address Objective 1 by demonstrating casework impact. Our final recommendation addresses Objective 2 by providing a way to simplify regional analysis.
- We recommend that HB&P conduct and publish a periodic analysis of their casework data. This type of publication is accessible and can be used to increase visibility and awareness on HB&P’s casework. HB&P would benefit from a frequent and regular publication, but due to limited resources we recommend an annual report. We have created an infographic template and manuals demonstrating the process of streamlining, analyzing, and presenting the data. We recommend the utilization of these resources during future implementation.
- We recommend the collection and detailed analysis of additional data fields to increase the analysis capabilities of HB&P. Recommended data fields include additional building and site characteristics, the communal value of sites, build and renovation dates, increased regional breakdowns, casework by LPA, casework overlap with other NAS, and whether a case is voluntary or obligatory. Analysis of these data categories will inform HB&P on the diversity of their casework, ensuring they dedicate their resources across a wide range of case types as well as provide a multitude of perspectives on HB&P’s casework impact.
- We recommend the addition of more qualitative analysis of the impact HB&P has on perception of heritage within England and Wales and their casework’s effects on a community. Our team has primarily analyzed quantitative data and conducted surveys and interviews with members of the heritage sector. We suggest devoting resources to the surveying of communities on the effect of HB&P’s casework on preserving the history and culture of their towns.
- We recommend the detailed analysis of Change of Use (CoU) casework, by type, region, and over time. This category of casework represents the evolution of the heritage sector and shifts in community values. Analysis of CoUs can give insights into both the past and prospective future on the prioritization of distinct types of buildings and uses across England and Wales.
We recommend the creation of mapping software to generate customizable maps by LPA. Data for these maps is available on the JCNAS database. The use of this type of software will allow quick regional analysis and the reallocation of time and resources to other areas in the future.