Data-Driven "Frenzy"
Data-Driven "Frenzy"
"Data! Data! Data! I can’t make bricks without clay!" – Sherlock Holmes
In today’s post, we are going to discuss the shift to data-driven decisions in software engineering or, how I like to call it, the “Data-Driven Frenzy”.
Jokes aside, data is good! Data-Driven decisions… even better. There’s also a fine line between following the data and “fixating” on data, but I will save that topic, for a future post 😏.
Data-Driven engineering
Data-driven software engineering is an approach that uses data as a primary source of information and feedback for software engineering activities. This data can come from a variety of sources, such as user behavior, system metrics, code quality, testing results, and business outcomes. The goal of data-driven software engineering is to measure and analyze the data to gain insights, identify problems, test hypotheses, and evaluate solutions.
(via LinkedIn Advice, see Reference 1 in the end of the post)
Data-Driven engineering can help an organization to achieve improved customer satisfaction and loyalty, enhanced competitive advantage and innovation, reduced costs and waste, increased agility and adaptability, and improved collaboration and communication, among others.
At the same time, there are challenges that come with the Data-Driven engineering approach. The data sources must be carefully chosen in order to provide accurate and useful data. Moreover, these data should be connected with the right KPIs in order to mean something for the team or the organization. Finally, in order for a Data-Driven culture to be sustainable, the data collection needs to be curated and kept up to date.
Data-Driven leadership
For a leader, the use of data can be a “double-edged sword”. On the one hand, they might help solve problems and identify ways to maximize productivity. On the other hand, they could lead to compounding of the problem or resentment in the team.
Data-Driven leadership can, in general, make use of internal or external data. Data like team velocity, average story points per sprint, open / closed issues rate, are considered internal data and are gathered by the team. Average rating of an application on the Apple Store, number of complaints to support from clients / users, number of uninstalls after the introduction of a new feature, can be considered external data (in the sense that they are the result of the product’s end users).
Using data the right way, can help a leader to better understand the team and user base. Moreover, make better sense of the team’s output and be in a position to visualize this output or the performance of the team when / where needed. Data-Driven leadership can also help in cross-team collaboration, since the analysis of the data can reveal gaps or overlaps in the work of two different teams working on the same domain or having dependencies with each other.
Having said that, Data-Driven decisions should not be considered an opportunity to minimize or eliminate conversation. Especially on fully-remote teams, synchronous communication is one of the most important and, in my opinion, most effective ways, of getting things done. Moreover, basing decisions solely on data would be similar to judging a team’s performance by counting lines of code delivered or number of commits. Finally, a leader should always be vigilant to avoid stepping on the mine of metrics being seen as a “performance monitoring” mechanism by its team.
NOTE: this part was highly influenced by “6 Reasons Why Data-Driven Engineering Teams Build Better Software”; see Reference 2 in the end of the post
Resources of the day
Now that you know (although I am pretty sure that was the case before you read my post 😁) how important data is to your role as a leader, here are a couple of resources that can help you grow as a Data-Driven leader.
The engineering leader’s guide to data-driven leadership
In this 9 part guide from the good people at LeadDev, you will find useful information on what you should be measuring as a data-driven leader, how companies like Netlify, use metrics to measure their engineering processes, how to achieve better team performance, how to improve planning and more.
Available for free here.
Leveraging ChatGPT for data analysis and visualization
ChatGPT is now part of every professional’s life. Don’t deny, I know you probably use it (like I do) more than you care to admit. There are several ChatGPT prompts in the WWW ready to be re-used. Following, I am sharing some interesting links with prompts related to data analysis and visualization:
Did you try them? Share your experience or feedback by leaving a comment. Did you find other prompts or links with prompts that were more helpful? Leave a comment and let us know 🙇.
Going “hardcore”. Learning Modern Statistics with R
I have a soft spot for books available for free, online! Books are a means to character formation, and libraries are cradles of culture. We were so lucky that so much valuable information has become available for free with the evolution of the WWW.
“Modern Statistics with R” by Måns Thulin is available for free online, under the Creative Commons CC BY-NC-SA 4.0. license. The author was kind enough to include examples and more than 200 exercises with worked solutions.
I highly suggest you have a look, even if you are not that familiar with the R language.
Hat tip to Professor Angelis for sharing this.
That was it for today! I hope you enjoyed this article of the Software Advent Calendar 2023 series! See you tomorrow 🖖.
References