Recently, my research on integrated care in Tower Hamlets with The Health Foundation was published in BMJ Open. The results are summarised in the diagram (you can click on the image to make it larger). Essentially, we found that the intervention was leading to increases in elective inpatient admissions and GP contacts in the first year. However, the actual aim of the intervention was to reduce emergency inpatient admissions.
This project was particularly interesting to me because of the novel methods we used. The study is a quasi-experiment, which essentially means that it is not a randomised controlled trial (RCT), but attempts to simulate one. Instead of randomly allocating patients to a treatment, it uses a large amount of data on patients characteristics in an attempt to find control patients (those who did not receive the treatment) who are similar to the treatment patients prior to the intervention start date. Thus, by tracking what happens to the two groups of similar patients, researchers hope to identify the difference caused by one group being enrolled on the intervention. This is called a matched control cohort study.
One of the problems with these methods is that after matching the two groups can often look dissimilar at the aggregate level. This is called imbalance (a video on assessing balance is available here). For example, even though individual patients are closely matched one to another, there may be systematic differences overall that cause one group to have a different average age (or any other characteristic) than the other group. In this study, we used a method called genetic matching to improve on standard matching. Genetic matching uses a machine learning algorithm to iteratively alter the parameters used in the matching model in order to improve the balance of the end result.
In our study, we found that the balance was pretty good after genetic matching, but we wanted to try to improve it further. We did this by also using a method of post-match adjustment called entropy balancing. This method takes the matched data and weights the individual cases slightly. By weighting them up or down, the overall effect is to improve balance to the point where the two groups are practically identical in the aggregate. In simulations, this method has managed to replicate the results of RCTs closely when RCT data has been altered to create non-random selection into the treatment group.
Our study did not find the results that Tower Hamlets were hoping for. We hypothesised that there were two main reasons for this. Firstly, the intervention had only just been implemented, and so may have not been working as efficiently as it would a few years down the line. Secondly, the patients in Tower Hamlets receiving the intervention generally come from very deprived backgrounds and tend to be very unwell. The intervention may have initially led to increased use of services because these patients had substantial healthcare needs that were not previously being met. Simply, they needed more healthcare, and were now getting it. Maybe two or three years later, their use of emergency services may have gone down, and they may have benefited from decreased mortality.
There were problems with our study too. Initially, we had wanted to look at three London boroughs (Tower Hamlets, Newham and Waltham Forest). But we failed to get access to the data we needed from the other two boroughs. This meant that the statistical power of our study was much smaller than we had hoped. This means that we could not reliably identify small effects on healthcare use because our confidence intervals were very wide. Also, we couldn’t find matches for many of the integrated care patients in Tower Hamlets (30% of them were not matched). This is because Tower Hamlets were being highly selective in their enrolment of patients — the sickest patients were almost all enrolled, and so there were not suitable control patients to match to them. Thus, our results only apply to the 70% of integrated care patients we matched (who were less sick than those who were not matched). It is quite possible that the effect of integrated care for the sickest patients could have been different to the effect we found for the 70% we matched.