A Discussion: Top Challenges of Moving Data Science to Production.

8 min readMar 9, 2023

We know MLOps and Production ML are hard.

But what are the common challenges that plague teams and practitioners (regardless of industry or background) when it comes to deploying data science models to production?

Sabrina Aquino from Shakudo organized a Twitter space called “ MLOps: Solving the Complexity of Moving Data Science to Production” where a group of us got to answer common questions and talk about the best practices (as well as anti-patterns) we’ve experienced as data scientists, MLOps engineers, and DevRels.

This is a summary of the conversation as well as links to the polished & copyedited conversation.

Question 1: What are some of the most common organizational challenges deploying ML Models?

Answered by Mikiko, Chanin, Pau, & Tawanda

The TL;DR:

Organizations typically approach the development of data science models by either using all-in-one solutions such as Amazon Sagemaker or Google Vertex, or by using a heterogeneous team.
Common challenges that organizations face include the need for high-quality data, ensuring the statistical properties of training data match production data, and understanding the complexities of exploratory work versus production work.
Additionally, organizations must consider the applicability domain, sharing data and code, and reproducibility when deploying models.
Finally, managing and keeping models running, available, and performing as expected is a challenge that requires expertise and a clear understanding of the paradigm.

Additional Notes

Organizations typically approach the development of data science models by forming a team of experts from various disciplines, such as software engineering, DevOps, and data science.
This team is responsible for managing the data science model from development to production.
Furthermore, sharing data and code between teams can be a challenge, as well as ensuring the model is reproducible and can be interacted with using the reader’s data.
Finally, automating the deployment of models from offline to online and understanding the difference in metrics between offline and online models is also a challenge.

Question 2: How can organizations get running with ML?

Answered by Mikiko, Christine, & Chanin

The TL;DR:

Organizations should :

understand their ML Ops maturity and be honest with where they’re going;
not try to build what other companies are building, and should instead focus on getting their fundamentals down.
be problem-oriented and focus on solving the bottlenecks in their production and deployment stack.
bridge the knowledge gap between the data science and engineering teams.
be clear on the necessary tech stack and deployment approach for their own models.

Additional Notes

It’s important to acknowledge the size and resources of the company, use the tools that the team is used to, and start with simpler models before building up a long-term system.
They should communicate and learn more about what it takes to deploy models and consider low-code technology and interactive model applications.
Additionally, organizations need to be clear on the tech stack and deployment approach for their unique needs. With the right steps, organizations can successfully deploy and develop their data science models.

Question 3: How can organizations approach tooling?

Answered by Christine, Pau, Chanin, Mikiko

The TL;DR:

When choosing the right tools and technologies for data science models, there are key considerations to keep in mind. These include compatibility with existing tools, open source activity, ease of adoption, and understanding the paradigms behind the tools.
It’s also important to consider the current stack, existing know-how, and the human expertise available to the organization.
Additionally, organizations should look at the ease of use and adoption, as well as the ability of the tool to support specific tasks, and techniques, or integrate with other tools.
Finally, organizations should ensure they are choosing tools that meet their needs, not just using technology for technology’s sake.

Additional Notes

Consider features and capabilities that support data science projects and goals, as well as the tool’s ability to support specific tasks, and techniques, and integrate with other tools.
Enablement and adoption of tools are often overlooked, and organizations should provide support and growth opportunities to data scientists and other users.

Speaker Question: What are some MLOps organizational best practices and pitfalls you’ve observed?

Answered by Mikiko

The TL;DR:

MLOps is an essential part of any organization’s workflow, and there are a few best practices that can help ensure its successful implementation.
The first is to get a serving and deployment component set up, and the second is to make sure it works properly.
It’s important to never neglect the dev-to-prod pipeline, and consider moving away from local development.
Finally, ML Ops engineers should aim to automate as much of the process as possible and focus on areas where data scientists and engineers can add unique value.
Additionally, it’s important to encourage product and domain knowledge, creativity, and innovation.

Key Quotes

“The reason they’ve published that blog post is because the architects have fought so long and hard to get that stuff up and running that they’re patting themselves on the shoulders going: ‘Job well done. You survived, you’re not dead. You’re still alive to fight another day.’”
“You have to get out of the way of your ego as an ML ops engineer, as a data engineer, as a tool developer.”
“You have to be okay with the fact that one day you built something so cool that you’re just not gonna be there anymore and then you can move on to helping people solve other problems.”

Speaker Question: ML and Data Science Challenges in Bioinformatics

Answered by Chanin

Chanin has an undergraduate degree in biology and self-taught himself coding, starting with C and Java before eventually finding success with Python.
He transitioned outside of academia and bioinformatics and is currently working in tech as a senior developer advocate at Snowflake for Streamlit.
He is also known as the DataProfessor on Youtube to his 142K subscribers.

The TL;DR:

Data in the field of biology can be high-dimensional and difficult to analyze.
Exploratory data analysis and machine learning techniques have been used to identify relevant subsets of data and to describe data samples.
It is important to use interpretable features when building models, and algorithms such as random forests can help to determine which features are most important.
Bioinformatics has benefited greatly from the use of machine learning, with applications such as creating molecules from scratch and designing drugs.
Data science techniques are helping bioinformaticians to analyze and interpret this data, uncovering unique linkages and patterns in the process.

Key Quotes

“Because there’s no concrete way of doing things. It really depends on the particular training of the researcher or the nature of the data itself. We become more like a data artist. “
“Applying our own knowledge or expertise in the field to dabble and through serendipity, we might discover some unique insights.That’s part of researching and being a data scientist. It’s uncovering unique linkages or patterns.”

Speaker Question: ML and Data Science Challenges in Reinforcement Learning

Answered by Pau

Pau is an experienced reinforcement learning expert, with a background in big data and a variety of projects under his belt.
He is the author of the Hands-on Reinforcement Learning Course, and regularly shares weekly Data Science and Machine Learning content.
He is an active contributor to the field, with interesting threads and content posted daily.

The TL;DR:

Reinforcement learning is a flexible and powerful tool that can be used to solve a variety of complex problems such as playing the game of Go or personalizing a user’s gaming experience.
It is not as widely used as supervised machine learning.
Reinforcement learning faces two main challenges: lack of data and bridging the gap between simulations and the real world.
To overcome these, researchers often build simulations as a proxy for the real world and fine-tune models when deploying them in the real world.
Despite these challenges, reinforcement learning still presents great potential for solving a variety of tasks and problems.

Key Quotes

“So far reinforcement learning has been the tool to solve problems that were out of reach. For example, in simulated environments. In the game of Go, DeepMind became famous because their model Alpha Go is able to play this game like no one before.”
“What you want to do, for example, is maximize the profit and loss over a longer period.”
“Lack of data is the main impediment because in simulated environments, you can’t simulate the data, right? Shortage of data is something that academics face when they work in simulated environments.”

Speaker Question: Challenges with Big Data

Answered by Tawanda

Tawanda is an expert in the field of big data with experience leveraging and understanding the challenges of working with it.
He is also an entertaining presence on Twitter, often posting funny memes and brightening up the day.
Based in Harare, Zimbabwe, Tawanda is into data analytics and also works as a software engineer, primarily with C# and .Net.

The TL;DR:

Organizations often face a variety of challenges when dealing with Big Data, such as a lack of understanding of the benefits, velocity, volume and variety of data, and storage issues.
However, there are tools and techniques which can help.
For example, streaming tools like Kafka and RabbitMQ can help manage data velocity, while compression techniques can help with storage.
Big Data can also be used to drive business value and improve decision-making, as seen with Netflix and Spotify’s recommendation systems, and Path AI Freedom’s diagnostic tools.

Big Thanks to Christine Yuen (Co-Founder, Shakudo) & host Sabrina Aquino!

About Christine Yuen (Co-Founder, Shakudo)

Christine has extensive experience in AI research and development, and is currently the head of engineering for Shakudo, allowing AI teams to take their ideas from concept to production.
She has a background in computer science and machine learning and has been doing both research and engineering work.
Her current focus is on DevOps, but she still finds AI and machine learning to be fascinating.

About Sabrina Aquino (Host, Dev Rel)

Sabrina is a developer advocate at Shakudo, a platform designed to make the complexities of building, deploying, and maintaining data models easier.
She is a developer, Twitter Spaces host, technical writer, social media and community manager, and is a contributor to humor Twitter.
Shakudo was created by data scientists who wanted to make the data science workflow as simple and painless as possible.
You can find her at: