WHO are the people that make Data Science possible?
aka the Data Science Shop crew
TL;DR
Data Science is a collaborative effort, not a one-person show. Like a high-end restaurant kitchen, it requires a diverse team of specialists working together.
The five key roles in a Data Science Shop are Data Scientists (head chefs), Data Analysts (sommeliers), Data Engineers (suppliers), Machine Learning Engineers (equipment specialists), and Project Managers (restaurant managers).
Each role has unique responsibilities and skill sets, but they all work in harmony to create Data Products, with the exact team composition depending on the specific project needs.
There are lots of myths and misconceptions about Data Science – an important one around the idea that it’s some sort of magical black box operated by a single all-knowing genius. But the truth is, building Data Products that solve real business problems is a team effort that requires a diverse group of specialists working together in perfect coordination.
Think of a restaurant kitchen: you wouldn’t expect one chef to handle everything from prepping ingredients to plating the final dishes, would you? That’s why there’s a whole crew of talented individuals, each with their own area of expertise, collaborating to create a delicious meal.
This is just like the Data Science Shop where each member of its talented crew brings their unique skills to the table, working in sync to create Data Products that are as impressive and valuable as a gourmet meal. Data Scientists design the recipe, Data Engineers gather the ingredients, Machine Learning Engineers optimize the cooking process, Data Analysts perfect the flavors, and Project Managers ensure a smooth dining experience.
Take a look at this short animation that summarizes this introduction to the Data Science Shop crew:
Let me make a more formal introduction to each one of these five key players in a Data Science Shop:
meet the Data Scientists
Data Scientists are like the head chefs in the Data Science Shop, crafting the recipes (or prototypes) that will eventually become full-fledged Data Products. With their deep knowledge of statistics, coding, data visualization, and machine learning, they dissect business problems and map out the approach to solving them.
The day-to-day tasks of Data Scientists involve modeling data using statistical or machine learning algorithms (including AI) to build prototype solutions. As the architects of Data Products, Data Scientists determine the appropriate scientific approach and craft the initial models that form the core of the solution. They also play a crucial role in translating information, requirements, and implications between the technical team (Data Analysts, Data Engineers, Machine Learning Engineers) and the business stakeholders, ensuring that the scientific approach aligns with business needs.
To perform these tasks effectively, a Data Scientist needs a versatile skill set. An advanced command of statistics is crucial for thinking critically about data and evaluating its appropriateness for a given task. Coding skills are necessary for building repeatable and automated processes. Advanced graphing skills enable Data Scientists to produce bespoke data visualizations. A deep understanding of when and how to apply various statistical and machine learning algorithms is essential, including a clear grasp of their limitations and the difference between using them for explanation (interpreting the past) or prediction (projecting into the future). Finally, outstanding communication skills are vital for a Data Scientist to effectively convey complex concepts to non-technical audiences and to serve as a translator between the technical team and business stakeholders.
meet the Data Analysts
Data Analysts are the sommeliers of the Data Science Shop, tasting and analyzing the data to identify insights and trends that could impact the final product. They play a distinct role, focusing on extracting meaningful insights from data to inform business decisions. While Data Scientists build technical solutions, Data Analysts are the domain experts who pair business context with their interpretation of the data.
The primary day-to-day tasks of Data Analysts involve obtaining, visualizing, and analyzing data. They identify and interpret trends and patterns in the data. A key aspect of their role is communicating these insights effectively to business stakeholders, equipping them with the data-driven insights they need to make informed decisions.
The most crucial skill for a Data Analyst is a deep knowledge of the business domain and the company’s operations. Without this context, it would be impossible to identify which trends in the data are truly meaningful for the business. Compared to other roles in the Data Science Shop, Data Analysts require less technical depth. While they need to be proficient in extracting data from databases and creating visualizations, they don’t need advanced skills in programming or machine learning. Instead, their value lies in their ability to understand the business, ask the right questions of the data, and communicate insights effectively to non-technical stakeholders. Strong communication skills are therefore another essential component of a Data Analyst’s skill set.
meet the Data Engineers
Data Engineers are the essential suppliers in the Data Science Shop, responsible for ensuring a continuous flow of high-quality ingredients (data) - the fuel that powers Data Products. As master architects, they develop and maintain the systems and infrastructures that securely gather, store, prepare, and deliver data seamlessly to where it's needed.
Their crucial day-to-day tasks of Data Engineers include building and maintaining data architectures for ingesting, storing, and transforming data with high levels of security and quality. They construct data pipelines to transport data between different locations, improve and scale data transformations prototyped by Data Scientists, and orchestrate processes for seamless automation in data handling. Data Engineers also create environments where these processes are built, tested, and integrated into functioning data architectures, effectively managing all aspects of data within the Data Science Shop to ensure it’s ready for use in creating valuable Data Products.
To accomplish these tasks, a Data Engineer must possess a range of technical skills. Advanced programming and software engineering skills are a must, as most of the data architecture runs on code. Experience with cloud computing and distributed systems is crucial, as many components of the architecture would likely be on different computing devices or networks. Expertise in designing databases and data architectures is also essential. Beyond the technical abilities, a Data Engineer must be an effective communicator, able to collaborate with both technical and non-technical stakeholders to ensure the data infrastructure meets the needs of the business.
meet the Machine Learning Engineers
Machine Learning Engineers are the kitchen equipment specialists of the Data Science Shop, responsible for optimizing and fine-tuning the equivalent of ovens, mixers, and blenders (the computational processes) to guarantee the optimal performance of Data Products. While Data Engineers supply the fuel (data), Machine Learning Engineers manage the fine-tuned engines that run on it.
The primary tasks of Machine Learning Engineers involve optimizing performance of heavy computing processes, making them quick and efficient. They take algorithms and prototyped solutions from Data Scientists, improving and scaling them for production. Machine Learning Engineers create processes for automating storage and access of machine learning algorithm outputs, orchestrate seamless automation in computational processes, and build environments where algorithms are developed, tested, and integrated into functioning computing architectures. In essence, they ensure that the data processing “equipment” in the Data Science Shop runs at peak efficiency, maximizing the value extracted from the data.
To excel in this role, a Machine Learning Engineer must possess several key skills. Advanced programming and software engineering skills are essential, as most computational processes run through code. Experience with cloud computing and distributed systems is crucial, as the computing processes will likely rely on a network of computers. A strong foundation in optimization mathematics and at least an intermediate knowledge of the computational underpinnings of machine learning algorithms is necessary to ensure that computational power is used efficiently. Moreover, Machine Learning Engineers must be skilled in communicating with both technical and non-technical audiences to effectively collaborate and convey complex concepts.
meet the Project Managers
Project Managers are the restaurant managers in the Data Science Shop, serving as orchestrators in the complex process of Data Product development. They are responsible for the execution and delivery of Data Products, which can range from simple to highly complex, they ensure that each product is built according to specifications, on time, and within budget.
The day-to-day tasks of Project Managers involve developing actionable timelines, breaking these down into discrete tasks, and assigning them to appropriate team members. Project Managers build feasible plans, coordinate the entire Data Product development process (aka the Data Product Cycle), and critically, monitor and mitigate risks that arise during development. They keep everything on track, ensuring that Data Products are delivered according to plan, or if necessary, updating stakeholders’ expectations. In essence, Project Managers are the linchpins that hold the entire Data Science Shop operation together, orchestrating the symphony of data transformation from concept to finished product.
To accomplish these tasks, a Project Manager needs a blend of technical and soft skills. They must have sufficient technical depth to be able to communicate effectively with all members of the Data Science Shop in their own terminology. Strong planning skills and the ability to use technical tools to build actionable plans are essential. Leadership skills are crucial for keeping the team motivated and working together smoothly. Project Managers must also be skilled negotiators to overcome any technical obstacles that may impede the completion of the Data Product. Finally, outstanding communication skills are vital for a Project Manager to translate complex technical concepts to non-technical stakeholders, and to serve as an effective liaison between the technical team and the business side.
Come together, right now…
Each member of this talented crew brings their unique skills to the table, working in harmony to create Data Products that are as impressive and valuable as a gourmet meal.
Now, just like a restaurant, not every Data Science Shop needs to have all five of these roles filled at all times. The necessary crew members depend on the specific Data Products being cooked up and the business challenges they’re designed to solve. Sometimes you might just need a few specialists, other times the whole team.
But no matter the size of the crew, each member plays a crucial part in the multi-step process of building a Data Product. Data Scientists might sketch out a prototype solution, which Data Engineers then scale up and integrate into the shop’s data systems. Machine Learning Engineers optimize those computational processes, while Data Analysts provide insights from the data itself. And throughout it all, Project Managers keep everything organized and on schedule.
It's that symbiotic collaboration between these five roles that allows the real magic of Data Science to happen. Just like a Michelin-star restaurant dish is the result of many talented hands working together, a successful data product requires the combined expertise of this whole crew.
So, the next time you hear someone talk about Data Science like it’s the work of a single genius operating behind a curtain of complexity, just remember – they’re forgetting about the full cast of specialists who make it all possible. Data Science is very much a team sport.
A helpful note: In this post, I aimed to present a clear categorization of roles in a Data Science Shop. The goal is to separate and define the different skills, tasks, and outputs needed to create Data Products. Many Data Science Shops use these exact titles for their staff, but some use different job titles – some quite creative. These alternative titles often indicate combined roles – where one person performs parts of different jobs. This tends to be more common in startups and in smaller or less established Data Science Shops. In a few other cases, alternative titles reflect the very specific needs of the Data Products that are being built.
Did we spark your interest? Then also read:
What is Data Science? to learn more about the practical way understand Data Science: focus on its outputs (Data Products)
What is the Data Science Shop? to learn more about the roadmap for the operation of Data Science in a business