10 dimensions of making data science work

Original article was published by Goda Ramkumar on Artificial Intelligence on Medium


10 dimensions of making data science work (part 5)

Culture : Dimension #5

Culture by Sumit Dutta

Here is part 5 of the series of 10 dimensions of making data science work. Culture is probably the least clear and most important dimension of making data science work. Still, it is being discussed after covering other dimensions of Expectation in part 1, Strategy in part 2, Roles in part 3, and Collaboration in part 4 because the culture is a by-product of how these dimensions have been catered to with an additional garnishing of attitude and awareness.

For most startups, culture begins with the founder and the very first seeding team. As the definition of culture could mean different things for different people, it is helpful to define what I mean by culture in the context of data science. The three questions that determine how the culture pans out over time and how truly data-driven an organization can be.

– How are decisions taken?

Decisions are taken based on gut pretty much in every start-up in the beginning. That’s how it starts but whether it stays that way or not is what determines the importance given to collecting data, making it accessible, and relying on the same to take decisions.

– What drives prioritization?

Whether the highest priority in the company is who” wants something done vs “what” needs to be done based on metrics, root-cause analysis, strategic what-if analysis, and impact estimation is what determines how motivated people are to pitch in with new ideas and make it work.

– What questions are asked when results from data are counter-intuitive?

When an intuitive hypothesis or experiment fails and counter-intuitive results are in front of you, whether your immediate response is — “Let us go deeper and find out why and make amends” vs “This is impossible. My gut is right, let’s go ahead with it” is what determines if data science is a thing to do vs thing that drives the business. This also determines whether data starts showing “what people want to see” vs “what is”.

Culture: How we do vs What we do

Here are 5 stages in which culture evolves in a startup. Actions taken at every stage almost determine the next stage deterministically and hence early stages have a bearing for a long time that becomes more difficult to change later.

Stage 1: Intuition — During the early stages of a start-up when achieving product-market-fit is the goal, intuition is the only way to prioritize goals and take decisions as there is hardly any data that could help except a few dip-stick checks and anecdotal evidence to back them up. There is usually no structured and measured approach to validate any hypothesis and that is very much acceptable.

Stage 2: Experiments for hypothesis validation — This is the stage when the very first analytics team starts taking shape and the best use of this team would be to bring trackability to business. Key L0 and L1 metrics are measured on a daily basis and the variations in them help ask questions of why to understand what drives these metrics. This understanding leads to ideas and hypotheses that an idea should work. The ideal way to sow the seed for a culture that makes data science work in the future is to start ad-hoc experimentation and measurement of impact to scale up or scale down ideas. Most start-ups do experiment but the key differentiator would be what is done with the result of the experiment. “Null hypothesis is true is a result” — this should be written on the walls at this stage to encourage people to share results of experiments that did not give the intended result. A good result is not where the metric always moves up, but one where you know why it moved the way it did and this belief is what lays the foundation brick for the culture.

Stage 3: Sizing and Prioritization— By this stage, most start-ups would have grown to a size where everything you think of cannot be done overnight over a call and people need to be given the clarity of goals to make it work. This is also the stage where more people join the growing organization and figure a way to fit into the culture. So it is important to do things right at this stage as that is the invisible message written on the walls that people coming in pick-up and adapt to. Hence sticking to a process of

  1. Estimate Size of an opportunity
  2. Write down an idea proposal with a clear method for experiment and measurement prior to execution
  3. Execution
  4. Retrospective of all ideas executed including those that did not give intended results to understand and drive the next actions

would create a practice that becomes the norm for people to follow and create the culture that prioritizes and makes decisions based on “what is” in the data.

Ultimately it all boils down to whether we ask “Which metric and by how much” before starting the work and “What did we learn” after completing the work

Stage 4: Clarifying the output and rewarding the inputs — Though Stage 3 ensures that “what” needs to be done is decided in a data-driven manner, it still leaves room for “how” it is done during the execution phase. Most cultural issues in organizations arise when the individual goals and not aligned with organization goals or the goals themselves are not clear at both levels. This needs a three-part solution of

  1. Defining the goals with system thinking that brings together all parts that need to make a whole. Basically the definition of OKRs based on the prioritized “whats”
  2. Creation of pods for the objectives cutting across functions blurring any territories within the organization and clarifying what each member’s role and responsibility within the pod is
  3. Measuring the quality of inputs rather than the impact of output from the pod and rewarding the members of the pod. This will reiterate the point that a good outcome is not where the metric always moves up, but one where you know something new about why it moved the way it did that you did not know before

Stage 5: Belief — If the previous stages indeed take shape to create the culture of making data science work, the organization definitely reaches a stage of belief in the power of data science, the power of data-driven decision making, and most importantly the power of continuous learning. Data becomes a key part of every conversation right from top leadership townhalls to team review meetings. Post this stage Data Science can never become just a thing to do and only remains a way of doing things across the business.

Soon, we will cover the sixth dimension of Discovery in part 6 of this series.

Content jointly created with Anand Sharma and Maneesh Mishra while at xto10x