When you buy through affiliate links in our content, we may earn a commission at no extra cost to you. Learn 
When the AI race peaked last year, many organizations across various industries invested heavily in developing their own AI models. However, many have faced significant challenges in translating their experimental AI models into real-world applications.
The journey from controlled machine learning experimentation to production-ready artificial intelligence systems is often fraught with obstacles, including data silos, complex deployment workflows, and governance issues.
Despite these hurdles, recent data suggests a promising shift in the AI landscape. According to recent reports, the number of AI models being put into production has risen.
Specifically, there is a staggering 1,018% increase in AI models registered for production compared to the previous year.
This growth outpaced the 134% increase in experiments logged, indicating a maturing AI ecosystem where more projects are being moved beyond the experimental phase.
To understand how organizations scale their AI models from experimentation to production, Techopedia sat down with Databricks CIO, Naveen Zutshi.
Naveen Zutshi is the Chief Information Officer (CIO) at Databricks. He was previously the CIO at Palo Alto Networks for the last six years, where he was responsible for corporate-wide analytics & AI, applications, global infrastructure, operations, and corporate services.
Prior to this, Naveen was SVP, Infrastructures & Ops at Gap, responsible for Corporate & E-Commerce infrastructures, overall security, and global operations.
Naveen has also worked in a SaaS startup as VP, Engineering, and was previously in various IT management roles at Cisco.
Key Takeaways
- Naveen Zutshi, CIO of Databricks, discusses the challenges of moving AI models from experimentation to production.
- Successful AI scaling requires clean data, strong SDLC processes, and user adoption strategies.
- Governance and data quality are critical for effective AI deployment in enterprises.
- Emerging technologies like advanced language models and data innovation will drive AI scaling.
- Balancing a high-pressure tech role with personal life involves blending work with personal interests.
- Show Full Guide
Determining Readiness for Production Deployment
Q: To have a reliable AI model, how long do you think a model should be in experimentation before being moved to a production environment?
A: It depends on the intended use rather than the model. So, for instance, you could have ChatGPT as your model, but you also need to think about the AI product you want to build; how long will the AI product be in the experiment before it moves into production?
For example, if we’re talking about ChatGPT, the experimentation time would vary based on the AI product we’re building.
We typically look at factors like click-through and open rates when testing AI email generation models. It’s crucial to compare the AI’s performance against human results to ensure equivalent outcomes.
We also run rigorous A/B tests and continuously refine the model based on user feedback and performance metrics. This includes reducing hallucination rates and improving accuracy over time. Monitoring these factors helps you build confidence in the model before moving it to production.
Key Factors for Successful AI Model Scaling
Q: What factors contribute to the successful scaling of AI models from experimentation to production environment?
A: I think key considerations for scaling AI experiments involve starting with a frontier or open-source model and adding a chatbot layer.
However, based on our experience and customer interactions, understanding the end-to-end user experience is crucial. When building an AI solution, consider how users will interact with it and how their behavior might change. Implementing a robust software development lifecycle (SDLC) process is essential for training, improving, and managing models and data effectively.
Data governance is also critical. Ensuring data sources are well-governed and protected through measures like access control lists (ACLs) is vital for maintaining data privacy. Additionally, controlling intellectual property (IP) and preventing data leakage is essential for many companies.
Accuracy is another key factor. While some level of hallucination might be acceptable in casual interactions, it’s unacceptable in business settings. Techniques like retrieval augmented generation can help improve accuracy by leveraging existing data.
Finally, broader governance aspects, such as AI policies, security, and bias mitigation, should be considered for responsible AI deployment.
Technical Challenges in Transitioning AI Models to Production
Q: What do you see as the biggest technical challenges organizations face when moving AI models from experimentation to production?
A: I think the biggest challenge is threefold.
First is Data. Having access to clean data sets, both structured and unstructured, that are well governed, and well managed is key. You may have a lot of data, but the real question is, how clean is that?
Then ensure you follow a really strong SDLC process to scale the models, scale the steps required for the models, and ensure that is done well.
And the third, often overlooked, is change management and behavior with the people who are going to use the models.
“Make sure you are focusing a lot on improving or educating the user base, making sure they’re comfortable with the use of those models, that they feel it is beneficial to them in their work, and they’re going to adopt it and give you feedback on what can be done to improve it.”
So having a very close partnership with your business, and not treating it just as a technology problem, is really important as well.
Strategies for Data Quality and Governance
Q: Businesses are pushing for wider access to data and AI tools. What strategies do you recommend for maintaining data quality and ensuring proper governance?
A: To enhance data quality, we need a robust data management strategy. This includes establishing a centralized repository like a lakehouse for structured, semi-structured, and unstructured data, implementing strong access controls, and defining clear data metrics.
Collaborating closely with business stakeholders to identify data champions and stewards is crucial for data cleansing and governance.
It’s essential to view data quality as an ongoing process that requires continuous attention.
Future Technologies Shaping AI Scaling in Enterprise
Q: Looking ahead, what emerging technologies do you think will have the most significant impact on scaling AI in the enterprise over the next few years?
A: I see three key areas driving AI scaling in the enterprise:
- Model Advancement: The landscape is shifting from a few dominant models to a wider range of high-quality options, including open-source large language models (LLMs). This increased competition will benefit businesses by offering more control, flexibility, and cost-effectiveness.
- Model Capabilities: We’re moving beyond simple chatbots to more complex agent and tool models capable of handling intricate tasks. These models can autonomously complete multiple steps, such as booking travel arrangements, etc,.
- Data Innovation: While traditional data has reached a point of diminishing returns, new frontiers like chain-of-thought data are emerging. Capturing and utilizing this data will enable models to better understand human reasoning, leading to more sophisticated problem-solving capabilities.
Work-Life Balance in a High-Pressure Tech Role
Q: How do you balance the demands of your role with your personal life?
A: It’s quite hard. I think your work follows you. For me, I blend my work with my personal life. Find time to do what I love. But I’m not often disconnected. I’m always connected.
Another thing is that everyone is always connected in this 21st century. But I love hiking. And so when I hike, and sometimes when there is no cell service, I get a chance to really be with my family and be present.
References
- State of Data + AI | Databricks (Databricks)
