We revere evidence-based practice. But what if the evidence base itself is not inclusive?

Photo by Mario Gogh on Unsplash

Are you engaged in research? Take one of the projects you’re involved in…

  • Is the team lead a white person or a person of color? How about the rest of the team, who are you inviting on it?
  • Who are you studying; do they have a real say in the research agenda?
  • Who do you cite; do you actively search for and include scholars from underrepresented backgrounds?
  • Similarly, who do you invite to give presentations?

I’m talking to myself as much as I’m talking to you; there are so many areas where I should have been more inclusive (more on that later). This Wednesday/Thursday we hold a session (2 time zone options) to brainstorm small steps that we can already take to improve what we do.

Today, I wanted to talk about what motivated our team to organize this session: what we learned from our project on child trauma treatment guidelines.

Clinical practice guidelines are documents that inform clinical practice in several ways. They tell us what current evidence-based practice is, what the best trauma treatments are. They inform practitioners, prospective clients, professional associations, policy makers, program managers, regulatory agencies, and insurers, to name a few.

So they are important and influential. They can support equity and inclusion by means of their recommendations.

While doing an initial exercise of comparing recommendations across various guidelines, we got interested in children’s representation:

  • whether children had had a voice in the guideline process (as a ‘consumer’) and
  • whether their diversity of cultural backgrounds had been recognized and represented

The full – though short 😊 – commentary is available here (open access). Here is a quick overview of what we found: 

Voice:

As far as we could tell from the 14 sets of guidance documents we considered, children or young people had not been actively involved in the development process of the guidelines:

They will have had a voice via the input from child-focused practitioners and researchers but we didn’t find reports of facilitated discussions or consultation with children and/or young people allowing their voice to be heard directly regarding proposed guideline questions, procedures or recommendations.

Cultural background:

For this, we looked at the evidence base that underpinned the International Society for Traumatic Stress (ISTSS) guidelines. These guidelines have a global outlook and its committee has followed one of the most rigorous approaches to guideline development.

As a start, we looked at where the studies that were selected for inclusion in the evidence base (only randomized controlled trials in this case) had been conducted. They came from 22 different countries. If we considered representation of high-income countries versus low- or middle-income countries in the trials, we saw an overrepresentation of the former:

If we scale it to actual population size (not taking into account that populations in LMICs are at higher risk of, for example, natural disaster and conflict exposure), it should have looked like this, just to achieve parity:

It also seems essential to look at who led the research, to get a sense of how attuned they may have been to local issues. We did very basic searches for this, looking up author portraits and biography/CV information online, so it should only be taken as a very rough indication:

While the above focused on the trials as unit of analysis, it’s also important to look at the numbers of individual children within the trials. Sample sizes are often used to give studies a smaller or larger weight in the meta-analyses that underpin guidelines. Because studies led by researchers of color or based in LMICs were generally smaller than those led by white authors based in HICs, this picture was even starker:  

Only 7% of the children in the global evidence base underpinning the guidelines were described by a lead author of color or based in a LMIC.

This means we have a very strong white, Western lens. It may mean that important community insights are not integrated in the research questions, design, inclusion/exclusion criteria, adaptations to treatment approaches, analyses, and eventual publication.  

So we concluded with a set of recommendations for committees developing future clinical practice guidelines:

  • “Establishing a guidelines committee that includes young people and parents as members, including from disadvantaged groups and cultures within the remit of the guidelines, with explicit attention to making sure that power dynamics are managed to really hear those committee members’ views.
  • Including historians, sociologists, anthropologists, educationalists, public health economists and local ethicists in the committee or its advisory team, to understand the local history, culture, norms, strengths and vulnerabilities of the population, and make considered decisions in the context of this information.
  • Reviewing one or more recent guidelines (e.g. ISTSS, NICE) and their underlying systematic reviews, assessing potentially relevant studies for cultural competence and representation of children’s voice. This includes asking questions such as ‘Do the researchers recognize their own cultural framework and its influence on the research approach?’ ‘Were cultural brokers, including young people, involved in the analysis and interpretation of the data?’ (cf. Riggs et al., 2014).
  • Gathering already available local evidence on needs and opportunities, and where possible, local treatment studies and reports of cultural adaptations (see Bernal et al 2009), with attention for evidence that has been communicated in a local language only.
  • Including equity as a standard agenda item throughout the guideline development process, from setting priorities to evaluation and implementation (see Akl et al 2017).”

These are fairly big steps however, aimed at committees and people in positions of influence. So we thought about what we can all do already, irrespective of how ‘senior’ one is within the academic context.

And that’s where the invitation for a conversation on making our daily research practice more inclusive came from (join us this Weds/Thurs!).

In the next few weeks, I’ll share a more personal reflection on my journey of making my research more inclusive. I’m learning as I go…

If you’re after further reading, I’ve found these two articles insightful and inspirational:

And please share this post or our commentary with anyone who you think may benefit from it; we want to stimulate conversation and constructive action towards more inclusion!

1 thought on “We revere evidence-based practice. But what if the evidence base itself is not inclusive?

  1. Pingback: Small steps towards the big goal of inclusion | Trauma Recovery Lab

What's your view?

This site uses Akismet to reduce spam. Learn how your comment data is processed.