Embedding inclusion in pharmaceutical design

Recently, I went to a barbecue restaurant with my partner – it had been a long time since we were last able to go out for dinner, so it felt like a special treat. When I went to the bathroom to wash my hands, everything was touchless: there were automatic faucets, hand dryers, soap dispensers… even an automatic paper towel dispenser if you prefer that to a hand dryer.  However, as soon as I tried to use any of them, they would not work. No matter what I did, I just could not get the sensor to detect my hands and dispense water or soap. Then, someone else came into the bathroom and tried to wash his hands and everything worked perfectly.  What was the difference? His skin was white, while mine was brown.

Bias by design

If you are a person of colour, this is probably an all-too-familiar experience – and it happens because of a completely preventable mistake. The reason the sensors in those automatic taps do not work as well for people of colour as they do for white people is because the pictures the machine uses to learn when to dispense the soap or water are made up mostly of light-skinned hands. One study found these training sets contain 3.5 times more pictures of light-skinned hands than dark-skinned ones.¹ As a result, the machine learns that it should only really dispense soap for light-skinned people.

This is annoying for people of colour, but the real problem occurs when we use those same object-detection systems in more dangerous contexts, such as self-driving cars. A study at Georgia Tech University in the US found that many of the autonomous vehicles being developed that used these systems were significantly worse at detecting and so stopping for dark-skinned pedestrians.² This could lead to more Black and brown people dying.

This type of bias does not just weave its way into tech products, but in all kinds of products we build.  Take seatbelts, for example. Seatbelts are legally required to be built into cars in most countries and they are rigorously tested on crash-test dummies to ensure they adequately prevent injury and death for those who get into car accidents. However, those crash-test dummies are built based on the average height and weight of men. This is a problem, as women are often lighter and shorter than the average man and, as a result, women are 73 percent more likely to be injured and 17 percent more likely to die in an automobile accident than men.³

The lack of inclusion in our design processes are not just an inconvenience, they are having serious consequences for the public.

Pharma is not immune

For decades when developing and testing medications, scientists – and the pharmaceutical companies they worked with – disregarded the idea that drugs might affect men and women differently. Women were excluded from drug trials for decades with no evidence-based reasons to do so.  As a result, the dosage amounts for medications have been incredibly gender-biased, based on male-dominated clinical trials. This has led to women suffering from much worse side-effects than men.

One study, conducted by the University of California – Berkeley and the University of Chicago in 2020, found that across the 86 drugs they looked at (all approved by the US Food and Drug Administration [FDA]) women suffered worse side-effects in more than 90 percent of cases.⁴ These side effects were not just headaches and nausea (though those were some of them), but major issues like seizures, hallucinations and cardiac anomalies.

The study notes that even extremely common drugs like the sleep medication Ambien have different effects on men and women – Ambien “lingers longer in the blood of women than of men, causing next-morning drowsiness, substantial cognitive impairment, and increased traffic accidents.”  It was not until 2013 that people noticed and dosage recommendations decreased for women.

There are numerous other examples of gender bias in research, ranging from under-representation of women in HIV medication trials⁵ and long-acting anti-psychotic medication research⁶ to failures to report on gender differences in therapeutic responses.⁷ One review exploring gender bias in trials of antibodies to combat multiple sclerosis stated: “in most cases, the primary and secondary endpoints are not analysed according to patient sex, despite the fact that international recommendations include this as a minimum requirement for ensuring scientific validity and obtaining appropriate results for extrapolation to the wider population.”⁸

Racial biases are also evident, especially in Alzheimer’s research where a study found as few as 3.2 percent of patients are non-White, when Black and Hispanic individuals are known to have higher rates of dementia.⁹ COVID-19 research has also been identified as having a high risk of bias, with one review finding 80.8 percent of 713 peer-reviewed original articles about COVID-19 medical research have high risk of bias.¹⁰

Why is this happening?

Two main issues underpin these mistakes in the way we develop products, including pharmaceuticals:

1. Our teams are not diverse enough. When a lab or development team is made of people who are all from a similar background, it is much more likely they will have a blind spot when it comes to people different from them, including racial, gender and other differences. Non-diverse teams are just less likely to notice when a clinical trial group just is not properly representative of the population.
2. Our teams are not inclusive enough. Even when the team is very diverse and representative, often those from marginalised or non-dominant groups do not feel psychologically safe. They do not feel comfortable speaking up, voicing dissent or expressing a different point of view.  This prevents them from being able to actually say something if they notice a bias, blindspot or mistake.

What can we do?

The problem is that our default methods are not equitable. But if we are conscious and deliberate about inclusion and diversity in the process we use when designing new products – medications, medical devices or any treatment method – we can reduce the likelihood that our default biases will be reflected in what we have created.  Here are three ideas you can implement into your process:

1. Add an inclusion review to your product development process, similar to a legal review. The inclusion review should be completed before a product goes to market and could simply be a check by someone who is not on the development team to ensure any inclusion blind spots are covered.
2. Leverage the diversity of our teams by adding “inclusion checks” at different points of the development process. In this scenario, teams would actively ask everyone to specifically comment on whether they can see any potential bias in the way the research is being conducted.
3. Add questions at the beginning and end of a product’s lifecycle to notice and reflect on one’s own identities and biases – and those of the team as a whole. By doing this at the beginning, researchers can put themselves in an inclusive mindset as they create their products, and asking such questions at the end acts as a retrospective learning exercise to improve future design.

These are just a few ideas teams could implement to make better and more inclusive products.  Whatever you choose, notice that none of these solutions are passive; to stop making these very preventable mistakes, we need to take active steps and build them into the way we do our work every day.

References

1. Research Reveals Possibly Fatal Consequences of Algorithmic Bias [Internet]. School of Computer Science. 2019 [cited August 2021]. Available from: https://www.scs.gatech.edu/news/620309/research-reveals-possibly-fatal-consequences-algorithmic-bias
2. Samuel S. Study finds a potential risk with self-driving cars: failure to detect dark-skinned pedestrians [Internet]. Vox. 2019 [cited August 2021]. Available from: https://www.vox.com/future-perfect/2019/3/5/18251924/self-driving-car-racial-bias…
3. Mohn T. Dummies Used In Motor Vehicle Crash Tests Favor Men And Put Women At Risk, New Report Says [Internet]. Forbes. 2019 [cited August 2021]. Available from: https://www.forbes.com/sites/tanyamohn/2019/10/28/…
4. Zucker I, Prendergast B. Sex differences in pharmacokinetics predict adverse drug reactions in women. Biology of Sex Differences [Internet]. 2020 [cited August 2021];11(1). Available from: https://bsd.biomedcentral.com/articles/10.1186/s13293-020-00308-5
5. Curno M, Rossi S, Hodges-Mameletzis I, Johnston R, Price M, Heidari S. A Systematic Review of the Inclusion (or Exclusion) of Women in HIV Research. JAIDS Journal of Acquired Immune Deficiency Syndromes [Internet]. 2016 [cited August 2021];71(2):181-188. Available from: https://journals.lww.com/jaids/Fulltext/2016/02010/…
6. Santos-Casado M, García-Avello A. Systematic Review of Gender Bias in the Clinical Trials of New Long-Acting Antipsychotic Drugs. Journal of Clinical Psychopharmacology [Internet]. 2019 [cited August 2021];39(3):264-272. Available from: https://journals.lww.com/psychopharmacology/…
7. Phillips S, Hamberg K. Doubly blind: a systematic review of gender in randomised controlled trials. Global Health Action [Internet]. 2016 [cited August 2021];9(1):29597. Available from: https://www.tandfonline.com/doi/full/10.3402/gha.v9.29597
8. Alonso-Moreno M, Ladrón-Guevara M, Ciudad-Gutiérrez P. Systematic review of gender bias in clinical trials of monoclonal antibodies for the treatment of multiple sclerosis. Neurología [Internet]. 2021 [cited August 2021];. Available from: https://www.sciencedirect.com/science/article/pii/S0213485321000086
9. Gottesman R, Hamilton R. Recruiting Diverse Populations in Clinical Trials. Neurology [Internet]. 2021 [cited August 2021];96(11):509-510. Available from: https://n.neurology.org/content/96/11/509
10. Raynaud M, Zhang H, Louis K, Goutaudier V, Wang J, Dubourg Q et al. COVID-19-related medical research: a meta-research and critical appraisal. BMC Medical Research Methodology [Internet]. 2021 [cited August 2021];21(1). Available from: https://link.springer.com/article/10.1186/s12874-020-01190-w