The Mathematics of Gender Bias

A couple weeks ago, at Music City Agile I attended a session on combating biases presented by Neem Serra. There was a slide presented that had data from simulations on how gender bias during promotions affects the number of women at different levels of an organization. This was something that seemed to be simple to model and hopefully prove out. Around the same time, my wife became more active in the Women In Leadership events that my company regularly organizes. I have definitely experienced a raising of consciousness about the obstacles faced by women in IT and wanted to put some numbers behind this newfound consciousness.

Most of the conversation about gender bias centers around the pay gap. This is a legitimate concern and a good deal has been written about it. In this post, the exploration is centered around a slightly different question - What is the effect of gender bias in promotions through the hierarchy in an IT organization? In order to do this, we are going to set up a very simple mathematical model. We are going to start with the assumption that we are looking at an organization that has 5 levels of hierarchy, ie, line level folks are at level 1 and C-Level execs at level 5. Let us also say that in this organization, every couple of years 10% of the workforce at every level gets promoted to the next level. We are going to start with the case that the lowest level of the hierarchy has 50,000 women and 50,000 men. For the first run of our model, we will assume there is no (0%) gender bias shown during promotions. So, our starting model has the following assumptions -

• Organization has 5 levels of hierarchy. 
• 50,000 Women and 50,000 Men at the entry/line(first) level.
• 10% of employees at every level receive promotions.
• 0% promotion decisions have Gender Bias

The resulting numbers would look like this -

Level	Women	Men	Promotions	Percentage Of Women	Percentage Of Men
1	50000	50000	10000	50%	50%
2	5000	5000	1000	50%	50%
3	500	500	100	50%	50%
4	50	50	10	50%	50%
5	5	5	1	50%	50%

What we see in this case is that when there is no bias involved, the percentage of women at every level equals the percentage of men. With every round of promotions, the same number of women and men get promoted.  Let us change our model to include some degree of bias. We will start with the case that there is a 10% gender bias. This would be reflected in 10% of promotion decisions involving women and 5% overall showing the bias. In other words, 5% of the promotions that should have gone to women, are actually handed to men due to conscious or sub-conscious bias. We are going to factor the bias in for promotions at every level. This time our model has this one parameter changed, and hence the following assumptions -

• Organization has 5 levels of hierarchy. 
• 50,000 Women and 50,000 Men at the entry/line(first) level.
• 10% of employees at every level receive promotions.
• 5% promotion decisions have Gender Bias

This has the following effect on the results -

Level	Women	Men	Promotions	Percentage Of Women	Percentage Of Men
1	50000	50000	10000	50%	50%
2	4500	5500	1000	45%	55%
3	405	595	100	41%	60%
4	36	64	10	36%	64%
5	3	7	1	33%	67%

The percentage of women at every subsequent level gets worse. When we finally reach the fifth level, we have only about 30% women holding office. The change is drastic compared to the base model with no bias. This should not be a surprise as in many organizations, the ratio of women in the highest echelons seems to be lower than at the entry level positions. If we change the bias to be affecting 10% of promotion decisions as opposed to 5%, we get the following results -

Level	Women	Men	Promotions	Percentage Of Women	Percentage Of Men
1	50000	50000	10000	50%	50%
2	4000	6000	1000	40%	60%
3	320	680	100	32%	68%
4	26	74	10	26%	74%
5	2	8	1	20%	80%

Once we change the bias to 10%, by the time we get to the fifth level of the organization, the percentage of women drops dramatically to close to 20%. Having 5 levels of hierarchy is actually a great simplification for most organizations. Usually, there are 7 or more levels separating the C-Level executives from the line level employees. The more levels we add, the more drastic the decrease in percentage becomes.

Another simplification in our modeling is assuming equal numbers of men and women in the industry. According to CNET, the percentage of women working in the tech industry is about 30. Our models, till now, have assumed an equal (at least at the starting level) ratio of men and women. What if we change the number of level one employees to reflect this. Also, let us assume bias being a factor in 10% of the promotions. Our model setup is now the following.

• Organization has 5 levels of hierarchy. 
• 30,000 Women and 70,000 Men at the entry/line(first) level.
• 10% of employees at every level receive promotions.
• 10% promotion decisions have Gender Bias

This renders the following results -

Level	Women	Men	Promotions	Percentage Of Women	Percentage Of Men
1	30000	70000	10000	30%	70%
2	2400	7600	1000	24%	76%
3	192	808	100	19%	81%
4	15	85	10	15%	85%
5	1	9	1	12%	88%

The number of women at the top level in this type of an organization drops drastically to 12%. Incidentally, the research at Paysa.com suggests that the percentage of female C-Level executives is close to 12% and the percentage of tech directors (Encapsulated by Level 3 in this case) is close to our model's 19%. 

The numbers from our model matching the industry numbers is partly coincidental. We have made numerous assumptions in our modeling. The 10% promotion rate, same bias at every level, the number of levels in an organization are all assumptions in the model. These numbers do point to the existence of gender bias in the tech industry. Based on our simplifications, it seems that about 10% of the promotion decisions are affected by this bias. You can see what a drastic effect this has on an organization's makeup. 

It would be interesting to see how individual organizations stack up. How consistent is the percentage of women as you move up the rungs of the corporate ladder? How about your organization? Does it seem that the ratio of women to men seem to get smaller the further up the organization you go? The culture of organizations is a product of the culture of the individuals that make up the organizations. As individuals it is up to us, to educate ourselves and reduce the role of bias in our own thinking. Neem Serra, whose session inspired me to play with the mathematics of bias, in her session also mentioned Harvard's Project Implicit. Project implicit helps you test for different kinds of biases. So far I have been brave enough to test myself only for gender bias and am pleased to say that the results said that I was unbiased in that regard. Take the tests for yourselves and see where you land on the bias spectrum. If there is work to be done, maybe spending some time with the folks you might hold a bias against would help.

A final note - This mode of modeling (applying bias as a percentage) and interpreting the results (tracking changes in percentages of members at every level) can be applied to any type of bias. This does not have to be restricted to gender and can be expanded to test for biases with regards to race, ethnicity, sexual orientation etc.

Hackathons : Paving The Desire Path

Desire Path

There is a very interesting Urban Planning concept called "Desire Path". The idea is pretty simple. People walking through a park will take the shortest or easiest routes. This is regardless of whether this path is paved or not. Eventually, due to erosion from the foot traffic the path will become visible. The designers can choose to ignore this or make this path official by paving it. Paving the desire path is a great design decision (most of the time). That is probably the reason behind the Lean and Agile UX communities having embraced this concept. If the user keeps going to the same part of your app over and over again, and it takes 4 clicks to get there, provide them a shortcut in the next release that makes it one click. Twitter making hashtags and @ mentions official features is another example of this. As Twitter users started communicating through hashtags and @ mentions, Twitter saw the desire path and not only made it official, but enhanced their use.

What does this have to do with hackathons?

My company has a great tradition of doing two hackathons per year. We call these events 48 hours as developers have 2 days to build anything they want. The projects could be product enhancements, internal tools, automation of processes, even apps to decide where to go for lunch. The results are often amazing. It is incredible that when left to their own accord, how creative and productive developers can get.  There have been entire features that have been built in 2 days that have eventually made it into the product. In fact, 48 hours is looked upon as a great way to get product innovation and ideas from the folks who are knee deep in the code every day.

In our latest hackathon, a few developers from one of our teams submitted and completed their 48 hours projects.  One of the projects also won an award at the closing ceremony for 48 hours. A small team of developers was able to build an award-winning feature in 2 days. Here is the interesting part of it though - The average time it takes for the team these developers belong to, in order to complete a feature, is over 200 days. In fact 2 days is close to a tenth of the time it takes for the team to complete user stories. A lot of this difference is due to the processes that the team has in place. There is definitely some factor of time spent exercising quality control for a production quality feature. I would argue though, that most of it is due to the processes that the team follows.

During hackathons, developers are forging a desire path. This is the ideal process they would like to follow, instead of the process that has been paved for them.

During hackathons, developers are forging a desire path. This is the ideal process they would like to follow, instead of the process that has been paved for them. What is interesting is that our teams have a lot of liberty to adjust and change their processes. Many a time the paved process path, has been created by the team itself.  They have full control over branching strategy, testing policies, definitions of done and to a good extent technologies they can use. For some reason though the hackathon desire path does not match the paved daily process path.

Paving The Desire Path

In my previous life, I am proud to have led a team that won every hackathon that it presented in. You can see their happy faces in the picture below. 

I cannot take any credit for their success as I did not write a single line of  code or test for these projects. But I did notice that they were doing a few things that were different from our "paved" process. Every time I would pick one of the things, for example, using slack to collaborate, when our company itself was not using it and make it a part of our daily process. The team(as enlightened as they were) would catch on to the trend and start incorporating other things that worked for them into their daily work. This would often mean removing unnecessary process. I clearly remember our prototype and product UI code merging and becoming the same soon after a hackathon where our UX designer had actively paired with one of our developers for two days. The team (with a slight push from me) paved the desire path that they were travelling.

Hackathons for me, as a manager, became not just celebrations of how good our developers were, but also retrospectives on our process. 

This does not mean that the two processes became the same. They just became more alike. Hackathons for me, as a manager, became not just celebrations of how good our developers were, but also retrospectives on our process. We as a team, sometimes consciously and other times, sub-consciously learned from our hackathons, both in terms of the processes we should use(or discard) and the technologies we should use(or discard). Our hackathons were often proof of concepts that with some modifications made it to the product itself. It allowed our engineers to play with technologies that get the job done the fastest and then incorporate them into our product.

Since our 48 hours event is very developer focused, the managers and team leads, usually continue doing their daily jobs as usual. Due to the fact that not everyone is participating, "official work" still happens. Stand-ups, meetings and other day-to-day activities are still going on for the teams.  I think this is a great missed opportunity. Team Leads and managers should be able to observe the team's behaviour during these hackathons. They should then encourage the team to behave in similar ways during their everyday work. Any organization that invests in hackathons, should look at this as a learning opportunity and a retrospective on their process. What process do the developers follow when they are most productive and have cycle times that are 100 times shorter? How far away is our current process from that and what are the tweaks that we can make to bring the two closer?

Hackathons reveal your team's desire path. Make it official! Pave It!