Transcript: Building a London academic-policy ecosystem
Sarah Chaytor 0:05
Hello, welcome to CAPE cast, the podcast on the Capabilities and Academic Policy Engagement of (the) CAPE project. I’m Sarah Chaytor, one of the project leads for CAPE and also Director of Strategy and Policy at UCL and CAPE is a partnership between UCL and the Universities of Cambridge, Manchester, Northumbria, and Nottingham and we’re funded by Research England to explore ways of strengthening the engagement between universities and public policy.
And today, we’ve got a really exciting episode, we are joined by three wonderful people Michelle Reeves, who is the Senior Manager heading up the strategy team in the GLA. Sarah Jasim, who is a CAPE Policy Fellow and also holds a post in the care policy evaluation Centre at the LSE, and in the NIHR funded applied research collaboration with North Thames. And CAPE policy fellow Ilias Krystallis, who is also an associate professor at the UCL Bartlett School of Sustainable Construction.
So today we’re going to be thinking about academic policy engagement in the context of London, the UK’s capital city, home to millions of people. Every time I look, I think the number has gone up by about half a million of people. And with a really complex University ecosystem of more than 40 Higher Education Institutions. London has also equally complex governance system. With both the Greater London Authority and a number of individual Borough Councils across London. Michelle Ilias and Sarah have all been really instrumental in developing a new emerging partnership between the higher education sector and the GLA in London that’s really looking at how we can better embed evidence in policymaking and use academic research and expertise to help to illuminate and tackle some of the pressing problems that London is facing.
So we’re going to start with asking Michelle to just give us a sense of what the policy ecosystem in London looks like from her perspective.
Michelle Reeves 1:54
Thanks, Sarah. So my sense of it is that we have got a really strong policy ecosystem. There’s a really a rich policy discourse, and a number of leading policy organisations and think tanks providing robust, authoritative and influential research and policy analysis. So you’ve got the Centre for London, Institute for Government, Centre for Cities, and many more. We’ve also got a really influential policy institutes within universities. So Policy Institute at King’s, you’ve got, you know, UCL’s one public policy department, and actually the GLA has its own policy analysis function called City intelligence.
So this, there’s definitely no shortage of policy organisations. There’s no shortage of policy events to attend. And it’s very hard as a policymaker myself to keep up with the numbers of policy reports. And there’s also of course, events like the London conference, which is held annually by the Centre for London, which brings together diverse stakeholders, both political stakeholders, and researchers and representatives from Borough Councils and for city government. I think at the same time, however, policy ecosystem is messy. It is also fragmented. And I think, in certain circumstances, you know, we do see a duplication of work. Actually, my team conducted some research in April ’22, with some of the key influential policy organisations across the Capital. And I think some of the feedback from them was that there didn’t feel like there was necessarily many opportunities for the different policy organisations to work together on a particular project or a particular policy theme. And potentially, there was a need for more opportunities for convening across the policy ecosystem. So yeah, sort of huge, huge plethora of different organisations.
Also, you’ve got organisations like the business of cities, like UCL’s IPPO who are also looking at the global politics and policy and the lessons that we can learn from them. So yeah, so, you know, hugely rich, but also opportunities, I think, for bringing together those diverse policy organisations. I think one of the things that’s happened over what’s been a positive development is during COVID, ironically, because I think what we then saw are sort of coming together of public policy, business organisations, civil society, and communities, all working together to address the impacts of the pandemic and to share research and intelligence and data and be part of that was due to the work of the London Recovery Board. But I think one of the legacies of the Recovery Board has been that there’s developed a really strong partnership working ethos across those different policy players and research organisations. And that’s now being taken forward by the London Partnership Board, which is, instead of looking at the pandemic is now looking at how we tackle London’s long term challenges.
And I think there is a sense also that those different sectors want to work together and provide a uniform London voice and particularly around policies around gaining additional powers for London Government. Also around the levelling up debate, I think, where London has potentially been seen as you know, working at the expense of other regions, we’re really clear that there is an interdependent relationship between London and the regions, so a real coming together. And so I think LRAPP, the London Research and Policy Partnership can play a really strong role in providing that sort of one stop shop and that convening space to bring together researchers and policymakers.
Sarah Chaytor 5:13
Just before we come on to LRAPP, I wonder if you could also just give us a sense of some of the key priorities for the GLA and the challenges they’re currently facing. And where it is given that sort of rich and diverse but but perhaps complex and a bit messy ecosystem you’ve outlined where it is that you think evidence bringing evidence to bear on those policy challenges can make a difference.
Michelle Reeves 5:34
Okay. So I think in certainly in this Mayor’s term, Mayor Khan, I mean, he’s been really clear, he’s got five key priorities. And they’ve also been embedded in the COVID recovery programme. So obviously, green recovery, just transition to net zero by 2030 is obviously one of Sadiq’s key priorities, also providing opportunities for young people, I think, particularly given the impact the negative impact on young people in terms of education, mental health. And so looking at how we can deliver services and support to young people through mentoring… Keeping London safe, a key priority for the mayor, it’s seen as being one of the greatest concerns of Londoners, and polling this certainly saying that that’s sort of one of the top priorities for them. And so there’s been a lot of work establishing policies around violence against women and girls, addressing serious use violence from things like that through the violence reduction unit.. again, affordable housing, physical development, and then getting London back on its feet following COVID. And also the impacts of Brexit.
So those have been consistently, his priorities and more immediately, obviously, affordable housing continues to be a real challenge. In addition to that, obviously, what we’ve seen post COVID Is been the cost of living pressures on London is, and really, the Mayor is looking at in London with more widely looking at how we can address that, and what support we can give an example of this is, you know, universal free school meals. But we’ve also provided support, capacity building and [unintelligble], the voluntary community sector that’s actually working with those communities that are most impacted. And again, I think continuing the work in terms of getting London on its feet, restoring tourism, looking at inward investment. And again, as I say, addressing the impacts of Brexit in terms of things like addressing skill shortages.
Sarah Chaytor 7:16
So those are all pretty wicked problems. They are complex, they are systemic, they are deep rooted, you know, what are the realistic ways, I suppose, in which research evidence and expertise can really help to help with those? We’re not talking about finding solutions that can be implemented next week? I don’t think
Michelle Reeves 7:35
No, I mean, I think I mean, evidence can help in a huge number of ways, you know, from sort of helping to define the problem and the challenges to sort of developing, you know, modelling, projections, forecasts that help us to anticipate future challenges, help us with our horizon scanning… You know, again, looking backwards, sort of trend analysis, help us to identify trends understand the drivers of impact for some of these challenges. Again, one of the big areas that I think probably there needs to be more work done on these what works. So you know, what interventions work, in what circumstances, how we can learn lessons and share success factors, I think one of the challenges of being in large organisations is about being able to develop and maintain a sort of organisational memory so that you’re not always reinventing the wheel. So you’re taking the best of what previous projects and programmes but then you actually, you know, can then innovate and develop those. And so that’s something that we are continually trying to address.
I think, in terms of new evidence. I mean, I think what we’ve seen again, since COVID, is a real interest in policy innovation, and then broadening the use of different policy methodologies. So we’re now looking at areas that we’ve previously not really looked at. So behavioural science systems dynamics, agent based modelling sort of urban analytics, and then increasingly, using data science and data visualisations. So yeah, there’s a whole range of areas where evidence can help and expertise can help and, and that’s why developing long term strategic relationships with research community is really, really vital.
Sarah Chaytor 9:09
It’s really interesting. And it takes us quite nicely on I think, to the creation of LRAPP, the London Research and Policy Partnership. I’m going to ask Michelle to just outline briefly how that came about. And then I’m going to turn to Sarah and Ilias who through their CAPE Fellowships, have been doing some really deeply embedded work with our LRAPP, looking at how it can work, what it needs to tackle and how it can be progressed, and I suppose scaled across, ideally, the whole of the London University sector. But Michelle, if you would first just give us a sense of what drove LRAPP?
Michelle Reeves 9:41
Yes, really quickly. I mean, the impetus from the Mayor’s office has been through an appetite to put London as at the heart of City Hall policymaking to enable Londoners and London stakeholders to sort of engage at all stages of the policymaking sort of cycle if you like. Certainly when I set up the strategy team in the GLA, we undertook a strategic review and I think one of the key gaps was being able to bring in more external perspectives into our policy formulation. I think that there’s been, you know, a sense from, again from from the mayor’s office, that there is a real opportunity to harness the breadth and depth of expertise and research that resides in London’s hmm, these you know, I mean, London is one of the world’s capital, academic sort of centres and it makes sense to really try and address that and use that to support our evidence based policymaking. So in that sense, I mean, those were key impetus is for the mayor’s office and for London Government more broadly, to seek to develop that collaboration with the academic community. I think from what we’ve gathered from the academic community, I think there’s been a real appetite to work on real policy challenges, and it’s applied the research that they’re undertaking on those challenges. And so it’s very much saw it as a win win relationship. You know, there is mutual benefit to be had across both communities.
Sarah Chaytor 10:58
Brilliant, and turning them to Ilias and Sarah, I wonder if you could just give us a sense of the sorts of things you’ve been doing with LRAPP under your CAPE Policy Fellowship.
Sarah Jasim 11:08
Yeah, sure. So Ilias and I have been in role as CAPE policy fellows embedded within Michelle’s team. And also as part of the LRAPP executive team since March 2022. It’s been over a year now. And when we came into roll, we looked at the literature, looked at what had been done. And there’s great work, and kind of maybe more on an international scale and a national scale, looking at the UK and globally, what happens in different disciplines, how academic researchers and policy professionals engage, how they form partnerships, and so on and so forth. But for this London context, I think we really wanted to begin to understand what was working and maybe what had happened, what people had been involved in before.
So we use two methods of kind of looking at that. So two data collection approaches. So we sent out a very kind of brief post survey in April 2021, we got quite a lot of responses. But that was just really just to get a snapshot of who has been involved, which disciplines, what kind of experiences and then that helped us kind of further our thinking, but also begin a summer long of kind of these conversations that we had with academic researchers, and also policy professionals, but also other stakeholders too. So we speak to people like London hire other types of organisations Centre for London, think tanks that were also in this sort of research policy ecosystem is based in London, really to understand from a broad perspective, what was working, what challenges had been faced, what the barriers were, what the facilitators were that kind of helped this. But our key kind of interest was long term partnership, long term working. So we found that there have been lots of examples of pocket ad hoc work maybe centred around personal relationships, but what actually led to long term working, because that’s the point at which our wrap can really harness these relationships. So it’s probably spoken enough. So I’ll give Ilias a chance.
Ilias Krystallis 13:19
Yes, so with Sarah, we have been working quite extensively on this piece of work, and Michelle and her team, as well as Ben Rogers, who is the other counterpart from LRAPP has really helped us identify key people and connecting us with other organisations to try to understand, from a baseline point of view, what’s happening in London and the engagement between the two communities, academics and the government. So within the first month of our fellowship, we started by trying to understand what’s going on, what are some examples of engagement? What practices do work? Well, what was the approach and we found that there are pockets across the London ecosystem of good partnerships, we have good examples, but these were sort of isolated cases and not really connected together in a systematic way. So after we finished with our interviews, which we have collected more than 100, we then started to analyse our data and try to find patterns to make sense of how we can develop an approach or a proposition for establishing a long term partnership for London. And Sarah and I have been focusing on two different elements. I focused more from an organisational point of view, how an HR app as an organisation, as an ecosystem, to be more accurate, can take this journey and mature to become the service that provides a platform for collaboration between these two communities because we have I mean, from the literature we have a great examples of well established partnerships, but we don’t actually know their journey from you know, starting from the very beginning and becoming what
They have become now where everybody knows that and is aware of. So we did this bit and we looked at how the governance changes from one form to the other, what sort of capabilities are developed within this journey? What sort of funding is required, where’s funding coming from what models are more appropriate for the party to seek to establish itself, and then Sarah, extensively at the several engagement practices that have been proven after speaking to our interviewee. So from that we develop the big typology of several engagement practices that we’re now testing some of them to an extend to see, you know, which engagement practices more appropriate for what kind of purpose. So this is what we have been doing now, as the next phase of our fellowship is to run several pilots and test their efficacy.
Sarah Chaytor 15:51
I mean, that’s really interesting, I think listening to the three of you, it’s really clear that what LRAPP is both is in itself, I think, but also the broad context, the landscape in which it’s working, you know, is something that’s very complex, that is operating at multiple levels, and is also very dynamic. You’re not working in a fixed state, you’re working in a really sort of evolving ecosystem, which I think is interesting. And it’s also interesting that you’ve kind of all in different ways highlighted these areas of best practice or areas of things that might be working, but a sense that they’re not actually systematically joined together, that this isn’t kind of something that’s embedded in practice, across that research policy interface within London. I think that’s interesting, because there are lots of parallels there to the national picture. But I do also wonder, as we’re seeing more investment in kind of regional policy engagement initiatives now around the UK, whether there is a particular challenge around just sheer complexity and scale within London, and what that means when you talk about LRAPP becoming a sort of a platform collaboration, what are some of the challenges associated with that platform in? Yes. And then also, in terms of the typology of different engagement practices, you mentioned, Sarah wonder if you have an even an early hunch about what’s the most effective in what context and how you think our app is likely to develop? I appreciate this as ongoing work. But if you can give us any sort of sneak previews, which I hereby grant you licence to rollback from in six months time, if you need to, that would be really interesting.
Sarah Jasim 17:22
Sure, absolutely happy to kind of office and sneak preview insights. So the typology that we’ve been thinking about begins to sort of categorise this situation into the different levels, you have the system level, you have the organisation level, you have maybe the discipline level, or project. And that’s kind of a bit fluid, because sometimes we see interdisciplinary projects such as housing and health, for example. And then you have the very individualistic personal relationships. So it’s about looking at which practices are appropriate at those different sort of four levels. And we collected like over 180 187, something like that knowledge brokerage practices from just across London, and we still didn’t get a chance to speak to everybody. But I think what’s really interesting about this work is that you need a multi pronged approach. And therefore, that’s where something like the London Research and Policy partnership can really come in, because you almost need a bit of an infrastructure that has an oversight, that has convening power, and that has membership sort of organisations tapped in. So you can have a bit of that system level knowledge, but as well as bringing those people into the room or bringing them around the table, but also an ongoing knowledge of what happens at the other three levels. So the organisation or the project or discipline, and the individual as well. So I hope that sort of begins to forge your thinking.
Sarah Chaytor 18:50
Yeah, I mean, it’s particularly interesting, I think, because it touches on what feels like a sort of perennial question in knowledge mobilisation, which is how much actually it comes down to kind of relationships. And at the moment, most of those relationships in reality, I think function at the individual level in the way you’re sort of describing rather than at the institutional level. So I think thinking then about the role LRAP plays, which is not quite an institution, but I suppose is of that nature and thinking about what you were saying earlier, as in terms of the platform, do you see a way to move from that kind of collection of different individual relationships for different purposes to that sort of holistic platform that presumably needs to allow space for those, those individual level relations, but also provide something something bigger, something more comprehensive?
Ilias Krystallis 19:37
You’re absolutely right. We have very good evidence that suggests that these relationships, these connections between individuals happening at the individual level and that they will always happen on the individual level. But what’s more, perhaps challenging is, if we treat London if we see it as an ecosystem, can we provide an ecosystem solution and that’s where Rob has the big channel.
If you like have come in via the ecosystem platform who enables greater collaboration between the individuals from policy and individuals from academia. And you know, they’re of course, they’re these people are free to engage, to talk to each other drop an email, but nothing stopping them. But there are also many academics as well as policymakers who haven’t got the chance to establish relationship personal relationship with our counterparts. And LRAPP is the platform who could enable this. So that’s one that if there’s no connection, then that error could serve as a platform, but also how much more productive it will be if all of these pockets of relationships that have been around in London, could all come together under this ecosystem approach. And we suddenly learn from each other or, you know, develop new links with each other. So the outputs will be multiple, I think,
Sarah Chaytor 21:02
yeah, I think that’s right. I think it’s just so interesting. And so important, actually, the LRAPP has been conceived of as a sort of joint enterprise, a joint initiative between the GLA, and the University of London, sort of on behalf of the broader higher education sector. I do also wonder if there is a question about how we collectively between us almost cascade out now that joint enterprise, we’ve got some really key kind of institutional and leadership counterparts, but actually that broader cascading that you’re talking about earlier, so that everybody feels it’s something that is useful for them is really interesting to think about. I want to finish early by asking you all what you think the key next steps will be in terms of developing our app and developing this London academic policy ecosystem, and reflecting and perhaps in particular, on what you think the next big challenges, what’s the thing you’re going to be bashing your head against over the next six months, but also perhaps a bit more optimistically? Where you see, you know, the ultimate success line? Or what’s the real outcome that this is all in aid of? Michelle, can we start with you?
Michelle Reeves 22:06
Yeah, I mean, I’m actually also as well as them and helping to co develop LDAP, and more so myself undertaking a policy fellowship. And I think one of the things that’s been really exciting, and I’ve been really enjoying is the engagement with different research institutions. Obviously, my fellowship is limited to UCL. But I think it really convinces me even more than I’m already convinced that actually, through LRAPP, the two communities can do great things together, there’s such an appetite for collaboration, I think in lots of ways, you know, there’s this challenge about, you know, we’ve always wanted that to be inclusive, to be a broad church to involve, certainly, in the first instance, all of the London higher education institutes and potentially beyond. And there’s just a challenge, because actually, even just delving into UCL, there’s so many different Policy Institute’s doing different work all incredibly excited. And so I think that is a challenge in itself, just the breadth and getting across making different universities aware of what our reps trying to do, and looking at how we can sort of, you know, boost our mutual work together. So that’s really exciting. And there is a huge appetite. And I think there’s a real untapped potential for collaboration. I mean, we’ve been lucky in our app to have won a small seed funding award through the ESRC, HRC. And Innovate UK to develop a local policy innovation partnership, and we’re one of 10 sort of partnerships or consortium who have been able to win this award. And so in the immediate next steps, we are actually just delivering that programme for that phase one. And we’ll get an opportunity to bid for a phase two programme of activities for the next three years. So we are actually very focused on completing our phase one. And then using the outputs from the insight from that to inform our face to bid which will be a three year programme of activities. And we focus this around just transition to net zero and looking at how we potentially can use our LPIP to accelerate home retrofitting in London, which, of course, is a massive challenge for London and also particularly pressing given that the mayor has a target of net zero by 2030, which is rapidly approaching. So I think immediately our focus will be on that. But I think the key thing to say though, that is really important for us that the L PIP is not the only thing that l rap is doing. We are continuing to develop those relationships across the two communities, we will look into hold roundtable to really expand that knowledge networks across all the policy areas. And that’s been going really well actually in terms of just talking to other policymakers about different policy areas, whether that’s from no health inequality, or you know, sport and innovation or looking at AI policy. So there’s enormous potential to look a whole range of policy areas but obviously, I think what we want to do with LPs to use it to help accelerate the development and the movement of LRAPP from what is the stages at the moment, which is really, as I said, is very much founded on personal relationships are probably more reactive, probably looking at more short term policy interventions, but to move it through to a stage where it’s much more around policy, collaboration, policy partnerships, and then ultimately, the goal is that, you know, we become a sort of help to convene a network of networks in London. And for me, I feel like this can be the engine for regional collaboration and innovation, and really dynamic engine for London. So I’m hugely excited about it.
Sarah Chaytor 25:36
I should just clarify that LPIPS is the local Policy and Innovation Partnerships, which is a new research council funded scheme. And what’s particularly interesting I suppose to reflect on for me is that LRAPP has provided almost a platform from which the proposal was developed. And so what you can start to see is that, as these different interventions come into the space, they are mutually reinforcing, and the one can amplify the other. So it’s interesting to see how they can work together in that sense, even though they are, as you say, distinct things. I think it’s also interesting that the LPIP in London has provided a way of identifying a really core kind of policy focus, a policy challenge area for both the LPIP to address but also, as you say, to think about how the broader LRAPP work feeds in. Ilias, what are your thoughts on where next LRAPP, what the challenges are, what what the outcomes should be?
Ilias Krystallis 26:29
I think, in general, we are on the right track with the LPIP which provides us a testbed to focus on one particular area, as you said, which is around retrofitting of London homes. And we’re looking to scale that. So for now, we’re focusing on retrofit, which is one policy challenge, but is a massive challenge, both from a policy point of view, but also from a research point of view. And in terms of innovation and businesses, as well as the supply chain. As you said at the beginning, it’s a wicked problem that we’re working on from that there is parallel work that’s gonna be going on, as Michelle said, But ideally, doing this work, there is an opportunity to replicate and scale this work that we’re doing this approach that we are developing into other policy areas. So I think we’re on the right track.
Sarah Chaytor 27:14
which is great to hear. And, Sarah, I hope you share really, yes, there’s optimism.
Sarah Jasim 27:19
Yes, of course, I think everybody sort of touched on really, really key things. And I think for me what I think the next steps are for our app is really using its power. And it’s grown in momentum. I mean, from when Helios and I joined just policy fellows, and everyone seems to recognise the name, people kind of come up to me and talk about it, which is really, really exciting. And now it’s about bringing all of those communities in London together. So the sort of civil society, the business, the academic research, also non academic research, as well as policy professionals. And as a researcher, I’ve often seen that done quite tokenistically so researchers need to kind of disseminate and they need to get their messages out there. So that’s when they may be turned to non academic groups. But it’s about building those connections in investing in those relationships, making sure true partnership occurs. And I really think that our app is such a great kind of convening platform to enable member institutions, but also researchers like myself, policy professionals, that sort of civil society organisations and businesses, and also just those who are less known. So I think the inclusion and diversity piece is really important. And often, those relationships are really anchored in large institutions that are quite powerful, that have really good existing relationships that we must remember, London is a really great, fantastic and diverse place. And it’s made up of so many different people, big institutions, small institutions. And yeah, I should just be inclusive for everyone completely agree. So I mean, it’s making me reflect again, listening to your how relatively modest interventions can actually make quite a big difference. So you know, thinking about it from the CAPE perspective, funding the fellowships that you and Ilias have been taking up, it’s quite a small part of what CAPE has done, but in terms of what it has enabled you to both do directly and the other broader things it’s stimulated. It’s been really impactful. I think. And I think that’s particularly important in response to your last point about making sure that this academic policy engagement space isn’t just one for the institutions with the broader shoulders, actually, those of us that have broad shoulders should perhaps be thinking about where we can invest, where we can make those interventions that offer those broader scalable benefits. And I think we’ve seen that as well with posting Michelle on her policy fellowship in UCL. It’s lovely to see we’ve got a little interchange ecosystem going on here. But it’s enabled you to make broader connections, which I hope will start to go out and beyond UCL itself. I think we’re out of time. Is there anything anybody else wanted to add? Briefly?
Michelle Reeves 29:54
Yeah, briefly. I mean, I think I wanted to say that with LRAPP and just going back to Sarah’s point about diversity. I think one of the things that LRAPP can really do, and it’s something that we sort of started to think about in London Government, and when I say London Government, I mean GLA London councils and Boroughs is about moving policy formulation from a sort of almost an in house practice to along that continuum towards co-production, you know, and real genuine co-production, you know, with civil society, with communities, as Sarah said, so that actually policymaking itself becomes a place where it’s lived experiences, you know, has the equivalent of, you know, assessing, you know, the usual data sources, and really meaningfully engaging with Londoners, and making sure that their insights do help to shape those policies. And we saw a bit of that actually, there was a bit of a proof of concept on the hoof, you know, it happened during COVID. Because people did do that. And we did listen to London is a problem because it was about unprecedented circumstances. And previously, before COVID, you know, I had been in within GLA trying to bring together policy of engagement. And, you know, there wasn’t a huge amount of buy in for it, but obviously, going through COVID, we actually have to do that in real life. And now, what’s happened is that now the thought of policy and engagement together, it’s now accepted. And so I think what we’ve tried to do is to move the dial a bit more so that ordinary people there in London is can see their needs and aspirations reflected in policy, because they’ve actually helped to co design it. So that’s fantastic. I mean, that’s a real step forward. And I think when people start to see themselves reflected in policies, and their lives and their aspirations resonating, then I think, again, maybe it’s a little bit optimistic, but I just feel like there’s such a apathy. And it’s such a, I suppose, disappointment in politics. But I think just in terms of Everyday Democracy, if people see themselves and their lives reflected, then they’re more likely to believe in politicians and policymakers that actually we are working on their behalf. And that’s exactly what we’re supposed to do.
Sarah
That’s a really important consideration for universities as well in terms of the broader role they play in society. So that’s a great note to leave it on and let’s hope that that’s the direction in which LRAPP will take us and good luck to Sarah and Iliasas for the next phase of your fellowship and those really interesting pilots. We’re all looking forward to seeing the results of those. Thank you all for your time today. It’s been such an interesting conversation. I’m really grateful to you for all taking the time and coming and sharing your reflections and insights and thank you to everybody for listening and goodbye.