Astonishingly, as many as 87% of companies that experiment with AI fail to put their model into production — meaning they never see a return on their investment. And if you’re working in the public cloud, there’s another hurdle to watch out for: the risk of vendor lock-in. So how do you get your AI into production — and profitability?
“To build and deploy your own AI model, you typically go through three stages,” explains Indy Van Mol, AI infrastructure expert at Piros. “First, the development and training of the AI model, then scheduling — where you put the model into a workflow — and finally, inference, where you put it into production.”
If you’ve completed these three steps, you’ve already come a long way…and yet things often still go wrong.
The vendor lock-in trap
“These steps, all the way up to the inference phase, can be completed on the platforms offered by public cloud services,” says Indy. “Major players such as Microsoft Azure and AWS offer platforms on which you can develop and refine AI models until they’re ready for production, but the problem is that you end up in a vendor lock-in situation.
“Being stuck with one provider can introduce a host of issues. After all, complexity increases at the moment of inference, and it’s entirely possible that your platform will fall short at this point. But by then you’re locked in and can no longer switch to another platform. And that’s when production grinds to a halt.”
There is a solution
Red Hat OpenShift AI offers a solution to this dilemma, providing some fundamental features that make a world of difference. “When you start building an AI model with OpenShift AI, you write the code in Jupyter Notebook. You then test that code on a GPU, which might be on a server at Microsoft Azure, AWS, or Google Cloud Platform.”
The situation becomes more complicated when you have multiple developers simultaneously working in Jupyter Notebook and wanting to test code. “Then you want to use the GPUs, which are scarce and expensive, as efficiently as possible. In OpenShift AI, you can do that because you work with pipelines, which don’t test on a fixed GPU but rather use capacity on a GPU that’s available at that moment. It doesn’t matter which provider the GPU is with. OpenShift AI is cloud agnostic.” Et voilà: you’re no longer tied to a specific provider.
“You can even house the GPUs for your AI models on-premise again, in whole or in part,” explains Indy. “More and more companies are choosing to build and run their AI models on-prem so that they can better protect their data. With Red Hat OpenShift AI, you can optimally use your local servers with GPUs for your AI models.”
The crucial step
The inference phase remains the most common point of failure for companies that don’t work with OpenShift AI. “Suppose you have about 20 AI models, all of which have to run on a GPU. But like we said, GPUs are scarce. Using OpenShift AI, you can host all your models together, optimally distributed over the available GPUs. Then you can easily scale up if necessary.”
In other words, OpenShift AI doesn’t just support model development (from scratch or based on existing models), it helps you to get across the finish line into production, where AI delivers real added value. That’s the difference between the 87% who get stuck in development hell and the 8% who successfully operationalize AI.”
More possibilities
And last but not least, OpenShift AI carries over many components from the open-source platform and industry pioneer Kubeflow. Indy Van Mol: “You don’t have vendor lock-in in terms of features either,” says Indy. “In fact, OpenShift AI’s features are adaptable and follow the latest open source trends, with full support from Red Hat and the open source community and in partnership with hardware manufacturers such as NVIDIA.”
Are you eager to know more about getting your AI model into production? Contact us for a no-obligation appointment.
Do you want to know more about our services? Discover them here.
