Choosing the design

The first consideration was how to distribute the documents. At some point I’ll look into creating a full vector store and frontend, but initial investigations showed that this could easily turn into a very enjoyable and time-consuming diversion from more useful activities. So next I looked at ways to distribute it via a Claude Project or a Gemini Gem.

The initial idea was to generate a bundle of concept files in line with Google’s new Open Knowledge Format (OKF), that we could have the Project/Gem reference. But it turns out that input files have to be uploaded by hand in such cases - OKF really seems designed for exclusive use by agents. NotebookLM can look at a folder… but has no threaded conversations. Every option for providing an encapsulated solution had its own limitations.

So the eventual fallback was to provide single markdown files for each input document, albeit with each clause properly tagged using ideas borrowed from OKF. This gave me 10 files that I could share - within Gemini’s upload limit for gems. Then my colleagues could upload them into whatever tool they chose.

What to ingest

The next question was to select what information I should feed into this resource. The goal here is to provide an ongoing resource for directors to understand their responsibilities and options for taking action on various matters. This led to the following basic list of inputs:

• Contractual Documents and regulations - leases, service agreements, articles of association and codes of conduct
• Title registers - records of property coupled with listings of relevant contractual agreements that touch the property
• Relevant legislation - not the actual text of legislation, but links to the correct version
• Instructions - instructions to the LLM on how to reason using the information (e.g. quote specific lease clauses)

I decided to exclude a few types of information.

First, documentation around building safety compliance - inspection results, regulations, and so on. Inspection results are transitory; the report comes, and remediations take place. Likewise, building regulations seem to be in constant flux at the moment, so since I did not want to adopt curating such information as an ongoing task, they had to be excluded from scope.

Second was spatial information such as maps and title plans. The idea is to allow directors to recieve accurate answers to their questions, and LLMs are currently limited in their spatial reasoning to support this. I decided to extract certain spatial information such as floor numbers for flats into a text file for accurate answers.

Last of all, it was tempting to provide information on the history of each flat and leaseholder as I’ve been in the block the longest and have had the most interaction with the other leaseholders. But brief reflection showed what a data protection nightmare this could become.

Cleaning the documents

Before I could start ingesting, the files themselves needed cleanup. For the leases, I had a number of scanned PDFs. Our services agreement and the RICS code of conduct werer proper PDFs with text, and the titles were also PDFs of text. I used Claude Code to clean the files, which worked - mostly. There were small errors here and there, important for a legal document, so this highlights the need to check the outputs in such cases.

Defining the document ingestion - contracts and guidelines

The largest group of documents - Contractual Documents and regulations - have a fairly standardised structure. The first section contain definitions of specific terms used in the document. The rest of the document is divided into chapters, sections, different documents call them differently but each contains a number of paragraphs that may have sub clauses.

%%{init: {'flowchart': {'rankSpacing': 90}}}%%
flowchart LR
  A[Document] --> B[Definitions]
  A --> C[Chapter]
  A --> D[Chapter]
  A --> E[Chapter]
  A --> F[...]
  D --> D1[Paragraph]
  D --> D2[Paragraph]
  D --> D3[Paragraph]
  D1 --> D11[Sub-Paragraph]
  D1 --> D12[Sub-Paragraph]
  D3 --> D31[Sub-Paragraph]
  D3 --> D32[Sub-Paragraph]
  D3 --> D33[Sub-Paragraph]

There was a concern over ingesting the lease - was it the same for each unit? As a check, I obtained someone else’s lease and checked it against my own. Apart from some fascinating (but trivial) discrepancies in the scanning, the language was identical - giving me the confidence that I could use my lease as the reference.

Defining the document ingestion - titles

Title registers are the other type of document to ingest, and they are also pretty standard in their structure.

• Each has three sections:
  • The property register lists the property and any transfers
  • The proprietorship register lists who owns what and any agreements that affect disposal
  • The charges register defines agreements that touch the property in question

Each section is a list of numbered and dated entries.

%%{init: {'flowchart': {'rankSpacing': 90}}}%%
flowchart LR
  A[Title Register] --> B[A. Property register]
  A --> C[B. Proprietorship register]
  A --> D[C. Charges register]
  B --> B1@{ shape: docs, label: "List of entries"}
  C --> C1@{ shape: docs, label: "List of entries"}
  D --> D1@{ shape: docs, label: "List of entries"}

This one had me thinking for a long time. Arguably each separate entry is its own concept. But a given entry often does not give much information. For example:

This being the case, all that the AI could do is list an entry. I decided to treat each section as a concept for the purposes of OKF.

Defining the document ingestion - Legislation

In theory I could ingest various relevant legislation, but in practice the UK government is excellent at making legislation available online. For example, here is the legislation defining the right to manage. The key thing to be wary of to use the current version of any act or statutory instrument, as they do get amended over time - the current template articles of association for an RTM company have gone through 4 iterations. The other caveat here then, is the need to be aware of freshness and provenance.

So, each piece of relevant legislation becomes a separate concept (and hence file) in the bundle, where we can track the link to the actual resource along with relevant metadata.

Defining the document ingestion - the instruction file

The last piece of the puzzle was the instruction file. Easy enough to create a first cut, but how to check operation? Here I defined half a dozen theoretical test questions, some based on queries that have already arisen, some theoretical. For example, could a leaseholder install plugin solar panels on their balcony?

Using these test cases, I was able to iterate on what instructions file would give the best result. The key points:

• It’s a search-and-citation aid, not a lawyer.
• It answers only from the documents it’s given.
• Every statement is cited.
• It surfaces differences but doesn’t reconcile them.
• It doesn’t draw conclusions the documents don’t.
• Statute law is pointed to, not paraphrased.
• It flags its known limits instead of answering past them.
• Settled matters get written up.
• The tone is plain, factual and brief.

Creating the files

So with the document structure understood, what remained was the processing to generate the output files. To do this, I iterated on a processing prompt for each file, tweaking after each output run. For example, the prompt for the single-file individual lease is

  _One file, 'sample-lease.md', of 140 sections: "Definitions" and "Particulars" (no reference), then each clause, sub-clause and general-words section with its clause
  number as its reference, clauses 3, 4 and 5 listing their sub-clause numbers too, then each schedule paragraph with its paragraph number. Parts B, C and E of the Sixth
  Schedule have no reference. Links such as "Clause 7.10" and "Paragraph 3 of the Seventh Schedule" point to '#clause-07-10' and '#schedule-seventh-03'._

This illustrates the judgement required — clauses 3–5 keep their sub-clause numbers, these sections have no reference.

Conclusion

This effort took a lot longer than expected… but the largest part of it was to understand the structure of the documents and then embed that understanding in the config files. Now the directors and I have a much more useful set of documents that we can use as we take the block forward. It was also a good exploration of where AI is a force multiplier, and where it can be a trap.

Anonymised outputs and associated code are at https://github.com/peterharrad/RHDirectorAssistant

Getting the Data

I moved in 7 years ago, and had documents mostly covering that period. The first hiccup was that halfway through, our existing managing agent was bought by another company, with a different approach to presenting the accounts. The 2021 accounts were a bare ledger dump and there was no 2023 budget, so figures for those years come from the comparative columns of the following year’s accounts. The apparent change of agent is a rename rather than a handover — the same company throughout — so the comparatives are the same organisation restating its own figures, not a successor reconstructing a predecessor’s. Where I could check a comparative against the original, the carry-over was exact. All in all, I had actuals for 2019-2025, and budgets for 2020 and 2022-2026. It felt like this was enough to start with. The accounts and budgets from 2024 also had commentaries, which opened the door to some extra analysis - to be discussed later.

Getting the financial data

The early accounts to 2022 were scanned images. To start, I simply instructed Claude to extract the P&L for each year, from each file. Claude did an excellent job (zero transcription errors), but was most apologetic - explaining that the scanned files didn’t separate out P&L into the different budget schedules that the block had. Which, on checking, turned out to be complete rubbish. Lesson - don’t just check the figures that AI brings you, check if it simply didn’t read far enough in the document. Fixed by rerunning the command but specifying the precise pages in the scanned document to look at.

The next issue was that the two sets of accounts had slightly different categories - e.g. the old one separated out insurance types, the new one aggregated them. Decision: the newer agent was the more pertinent data, so I had the AI aggregate the older data accordingly.

Lesson learned: check the data that AI extracts, but also check availability, despite what the AI tells you.

Cleansing the commentary

With a post like this, it’s natural that I might be expected to share the source data. Which means that I needed to be aware of confidentiality. In particular, where people’s names and contact details were shown, as well as specific quotes that contractors provided, these had to be redacted. There was also a question of whether anything in the commentary could be sensitive to the block itself.

So I had the AI create three scripts:

• redact_docx.py - a simple matching script to remove names and contact information. No reason to feed spammers.
• score_sensitivity.py - a heuristic to score each line on potential sensitivity based on what it discussed and produce a file for me to inspect
• sanitise_docx.py - a script to clean out any contractual figures and remove any lines that the sensitive line analysis had thrown up

Simple enough, no? And if I hadn’t intended to share the scripts it would have been fine.

redact_docx.py helpfully listed the names and details for it to redact in the comments that had been generated at the top, and then listed the names in an array. Meanwhile, after I asked for sanitise_docx to remove a specific line that related to security, it had helpfully explained this removal in the comments and the code to do so was helpfully named in way that made it clear what information was being removed. Oops. So a couple of quick manual edits were in order. In practice, the time taken was probably equal to redacting the information manually… but this was useful learning for a future project that I’ll be writing about.

Lesson learned (or, rather, reinforced): Check the scripts that AI generates, not just the outputs. In retrospect, I should have explicitly told it to obfuscate the names of the relevant variables and functions.

Financial Analysis

Two questions stood out for me in the service charge - which areas had the greatest growth, and which areas showed the greatest variance of actual versus budget?

Asking about growth areas produced an impressive set of graphs, with two standing out as particularly useful. As shown below, general maintenance, electricity, and plant and machinery all stand out as significant items and topics that the directors need to dig into. It’s noticeable that reserve contributions are incredibly volatile - it seems like both agents used it as a smoothing function for the budget totals.

Variance tells a similar story. It seems clear that maintenance is not only one of the biggest contributors to service charge growth, but also one of the most unanticipated

Commentary Analysis

Likewise, there were two areas that interested me in the commentary. Specifically, did the commentaries tend to claim success or failure on specific topics, and did the language become more handwavy and evasive for the same topics. As an experiment, I decided to perform this via the Claude Code interface, not the Jupyter notebook.

For the analysis of claimed success and failure, the commentary is negative 6 times more often than it’s positive - perhaps understandable when trying to justify spiralling service charges. What’s more interesting is the topic breakdown: Insurance is the most consistently negative presentation, while energy costs are presented as a relative success.

More interesting was the evasive language analysis. A simple lexical analysis script produced the following, rather questionable results. Boilerplate and FAQ scored second in the list - and the top category only covered 58 words. Time for a rethink.

A quick examination showed that the boilerplate was repeated across all five files, biasing the result. Likewise, the single comment on general maintenance scores highly - but could just be due to chance.

Rebuilding the measure to deduplicate repeated text and drop thinly-populated topics moved boilerplate FAQ from second place to ninth, below almost every substantive topic — the original score was largely an artefact of the same standard passages recurring each year. On the cleaned figures electricity and energy tops the table at 3.8, with lifts and water and pumps also high; all of which show significant budget variance in the financial analysis. General maintenance and major works disappear from the analysis, which in some ways is telling - large areas of spend were simply skirted over.

Conclusions

The analysis has highlighted some important areas for us to focus on. We need to dig into the electricity and understand what’s driving the growth and unpredictability. The repeated unexpected variance in maintenance highlights that an attempt to understand expected lifetimes of different systems is long overdue.

Some other useful lessons fall out for future analysis using these tools.

It’s often said that you should treat AI like an intern whose work needs to be thoroughly checked. Two examples came up in this effort. The AI did an excellent job at extracting data from the image files… yet needed to be told which page to look at. At the same time, the redaction scripts smoothly removed sensitive data - while exposing it in the source code. I’ve read that models are trained to write clear, self-documenting code — but this pushes in the opposite direction to confidentiality.

A more subtle point is the initial problems with the evasive language. What could be a useful metric was initially badly biased due to repeated text and small sample size.

A fourth point stands out - the tendency of Claude Code to attempt to use its initiative. Ask for a matrix of evasive words in the commentary by topic, and you get an entire essay opining on the results. Ask the Jupyter notebook to analyse variance of budget v actual, and get a swathe of tables and graphs. Not a problem, but a subtle trap to be wary of when using AI for this kind of work - the specific outputs can end up guiding you down a path instead of you guiding the outputs. Last of all, while this happens whether working via the Jupyter notebook or the Claude Code interface, it manifests in different ways due to the different constraints of the two interfaces. The Jupyter notebook throws a bunch of tables and graphs at you, while the Claude Code interface waxes lyrical with commentary.

Building the base of support

We needed the support of 39 units - at least 50% of the block had to be members of the RTM Company before we could claim. The immediate problem that we faced, was how to contact and get in touch with a group of leaseholders who did not live in the block, and whose contact details were unknown. From the start I decided against asking the incumbent managing agent - there was essentially no chance that they would help, and tipping them off might cause them to start their own campaign to undermine ours. To start with, I used my existing network. There was already a small community of resident leaseholders held together via an email group and a WhatsApp group. I’d personally managed to add several people to this community by actively engaging with unknown residents - purely because you never know when a connection might be mutually beneficial.

This gave us a starter group of 12 - with 39 needed and ideally up to a dozen more. My working assumption was to treat the remaining units as rented out. Now, one option for the next step might have been to download the titles for the remaining units, and write to the registered addresses.

I chose not to do this as the next step for several reasons:

• The title registers cost £7 a time
• People often forget to update their contact details on title registers when they move, and a look at property sales data showed that the majority of the units had not changed hands in 20 years
• Letters are common and what with junkmail, the inclination is to dismiss an unexpected letter - “how can I get rid of this?”. A mailshot, personally signed by another resident, stands out
• Landlords are often in touch with their tenants and some even have cordial relations in place

So instead, I chose to do a personal mailshot - printing out 65 leaflets to place in each mailbox.

Selling with the mailshot

The mailshot was essentially a sales pitch - as a tenant, why should you care, and why should your landlord care? In a case like this, you need a few carefully chosen talking points. Too many, and it feels like some hard-sell sales pitch on late night television. Too few, and you fail to persuade.

So I chose to make the following talking points:

• If you are a tenant, rising service charges put financial pressure on your landlord to raise rents. Meanwhile, problems external to the flat are not fixed, affecting your quality of life
• If you are a landlord, the value of your investment is being slowly eaten away

And on each leaflet was a request to pass on my contact details to the owner of the property.

65 leaflets went out; over the course of a month, 13 new contacts reached out and stated their support for the effort. With something like this, you have to accept that some people will be slow to respond.

25 supporters, 39 needed.

Moving on to referrals

Some people are natural connectors, and if you can convince them of your case, they will put you in touch with other potential supporters or sources of contacts. This effort proved no different. Two of the landlords had lists of other leaseholders from previous efforts to rally the community. 21 more supporters stated their support. 46 supporters with 39 needed. But ideally we wanted more.

Another leaseholder was in contact with the director of the lettings agent who handled most of the lettings in the block - passed my details, they called me and after I convinced them of the cause, they contacted all their landlords that I was not in touch with. 12 more supporters - 58 total, with 39 needed.

8 more were to join us over the coming months, but we had enough at this point.

Establishing stakeholder credibility

At this point, we had commitments from enough leaseholders and questions of stakeholder management came to the fore. There were two specific groups to consider.

The first and most important stakeholder was the part-time caretaker. While not living on site, they knew a lot of what went on at the block, and it was only a matter of time before they heard about the effort and started questioning what it would mean for them. So I sat down with them and explained very carefully what we planned and how it would actually empower them. I took care to visit them regularly and update them on progress.

The second stakeholder management question was how to keep the support of the group of supporters that I’d rallied. For historical reasons, certain leaseholders were deeply suspicious of each other, and I needed to avoid being seen to take sides.

I foresaw two separate problems:

• Having gained support, I needed to keep it until the claim went in. In particular, I needed to build up trust for when it came time to join the company
• They might be concerned that I was making decisions to benefit myself to their own detriment

Each required its own separate strategy.

To keep people invested, I sent out regular email newsletters, explaining progress and what the next steps would be.

To avoid the suspicion of this being my one-man feathering of the nest, I established an advisory committee made up of volunteers to act as a sounding board, with its own separate email and WhatsApp groups.

This immediately provoked a challenge from a couple of leaseholders, who while unwilling to join this group felt that one who had would subvert the effort in order to reintroduce short-term letting in violation of the lease. I found myself spending significant time reassuring them that I was not inclined to allow this and in fact would have a statutory duty to enforce the lease provisions.

Calling in a professional

So now it was time to form the company and start the claim process. The RTM process is notoriously full of pitfalls, and if the freeholder or incumbent managing agent decides to challenge, working through the courts can take years. You have to invite all leaseholders to join the company; get a single thing wrong and the claim is invalid. You have to notify all parties, and if you get anything wrong, the claim is invalid. And then the freeholder can demand a mountain of background paperwork, and if there are any discrepancies or errors, the claim is invalid. With all these pitfalls, we felt it was necessary to engage advisors. There are a range of companies that will help you through the RTM process, charging a wide range of fees.

We considered the key factors; the desire to ensure success so that we could immediately start work on reducing the service charge, and the fact that service charges were already in the mid 4-figures for most leaseholders. Given these, we engaged Shula Rich, one of the most recognised names in RTM advice provision. A key decision criterion was her stated approach of merely amending claims that were rejected for trivial reasons and resubmitting. This significantly reduced the risk of needing a lengthy and expensive challenge at the First Tier Tribunal, the section of the courts that deals with such disputes.

Choosing the replacement

At the same time, we needed to choose a replacement managing agent. Some RTM efforts manage the block themselves, but for us, a 77-flat riverside block with 4 entrances and a caretaker needed a professional company.

We created an ITT and sent it to a number of candidate companies after a couple of review iterations with the advisory committee. We then had each member of the advisory committee score the companies across five weighted categories, creating a shortlist of three. Detailed examinations and follow up questions produced a front runner, who we eventually engaged to take over. As with our advisor, their stated experience in taking over from the incumbent was a key factor.

The outcome

The claim was posted on the 4th of June. What with a weekend, deemed delivery was the 9th of June. The RTM regulations stated that the counterparties had a month to respond, so once we factored in another weekend and some leeway, we agreed a cutoff for response of the 13th of July. Again driven by the RTM regulations, the handover can only happen a minimum of 3 months after the response deadline. I chose to move it from Tuesday to Wednesday (to leave two clear working days between the weekends), so - the 14th of October.

As expected, the freeholder immediately responded with a request for evidence - we had to hand over no less than 79 pages of documentation for them to pick through for discrepancies. On the 8th of July, the freeholder sent over their formal acceptance of the claim. Contrary to anecdote, they were highly professional at every step.

How much will the service charge come down by? We’ll know once we have the new budget in place, but the consensus is that RTM cuts charges from 10% to 30% or even much more. A highly comparable block has roughly half the service charge levels.

What I wished I’d done

I should have identified and recruited directors much earlier in the process. As it is, it sometimes feels like my fellow directors are only now finding their feet with the claim already accepted.

I also should have anticipated the tensions between certain leaseholders and worked to pre-handle them. The concerns raised could have been anticipated.

Conclusion

A presales or business transformation effort requires the ability to craft an argument, reach stakeholders, manage stakeholders and navigate process and regulatory landscapes. All of these came to play in this effort. Going from 12 initial supporters to 66 needed a combination of cold approach, and networking - but just as in business, referrals win out every time. Looking to prehandle stakeholder objections was the right call, even if I failed to anticipate a key one - at the very least, the patterns put in place brought enough credibility to ride out that objection. Identifying the right suppliers with a traceable set of decisions also built trust. Last of all, simply being able to comprehend the process and describe it intelligently meant that the necessary notices did not provoke panic when they landed in leaseholders’ mailboxes.