Almost every week, I am contacted by a number of vendors who try to sell a product, a demo, or or even a vision on the applications of agile in AI! Of course, both words, “Agile” and “AI” are still pretty powerful buzzwords and they are unlikely to go out of fashion any time soon! It seems peculiar that the “Agile” buzzword has outlived many of the other buzzwords like ”B2B”, “B2C”, “Mobile” or “Social” etc., I guess it is now time to see how it can apply to AI?
Can an agile methodology be applied to AI? After all, AI is all about experimentation/ exploration, so how can an agile methodology be applied to it? The point is, all experimentation will need to drive value and produce some business value and return on investment (ROI). So, how can an agile framework help with that?
As always, a single agile methodology will probably fail on its own as it needs to be tied to organizational interdependencies, strategies and even cultures. It is also all about a shift in mindsets which can be really hard in some circumstances.
In this article, I will discuss a unique agile methodology that can work well within different organizational boundaries and industries. This methodology was born and was put to practice by working very closely with Data Scientists, Data Engineers, AI Consultants, Information Technology Engineers and last but not least, a number of lines of business (LOB).
Our Goal
The goal was to considerably reduce the Model Development Lifecycle (MDLC) timelines for building AI Machine Learning (ML) and Natural Language Processing (NLP) models. Time to market is essential and taking too long seemed like a missed opportunity for all!
According to Harvard Business Review, it takes between 18 to 32 months to build impactful Machine Learning models. It goes without saying that this kind of timelines are simply unacceptable regardless of the industry you work in. By the time you are ready to go live with a model within this timeframe, your model would probably be outdated or would need to go through a complete refresh. It is also possible that the original hypothesis may entirely change and or may no longer be applicable!
AI ML MDLC Phases
Generally, there are 5 phases to an MDLC as follows:
What is wrong with the diagram above? If you are involved with AI and MDLC, you have probably already guessed it. It is suggesting a sequential/linear approach. It is common knowledge that AI model development needs to be iterative, so let’s try to fix that!
Whilst this diagram suggests an iterative approach in different phases, it is still suggesting hand offs from one phase to another and potentially from one team to another. Hand offs are generally expensive and time consuming. Data Science is iterative in nature, i.e. you would have to go through several takes until you reach a desired state where your model can better predict, prescribe or suggest an action that is needed. So, how can you make sure these iterations are time bound and allow overlaps between different phases?
A unique Agile AI Methodology
The answer, as usual, is not that simple, but it is doable if and only if you have a proper methodology and a solid framework that is acceptable and can be adopted by a majority of your stakeholders, I say majority because without applying a proper adoption and change management strategy any approach would probably be dead on arrival! Such methodology will also need to be organizationally agnostic, in other words, as changes occur within the organizations, your methodology and framework should still work regardless!
Which Agile Methodology?
So, which agile methodology should be used used? Scrum, Lean, Crystal…. The list goes on. The reality is, none of the methodologies above can help on their own. Before using a specific agile methodology, a solid supportive framework needs to be developed within which an agile methodology can be applied to.
What Framework?
The answer to this is almost always the same. A solid and versatile framework is needed that not only focuses on an agile “delivery methodology”, but also on a number of Business Value Accelerators (BVA) that can help to accelerate MDLC. Using say, “Scrum” alone without a proper underlying framework will not be of much use. We built a framework that helped us gain significant acceleration married with an agile methodology. We call this framework Flash AI.
Flash AI is supported by an iterative architecture that allows a continuous and iterative training of the models whilst presenting the results to the business in a “show and tell” manner until the optimal results are delivered. This iterative approach is a “soft launch” of the model that will live in a real production environment and in a time bound manner producing real business results. More to come on the Soft Launch approach in part II of this article.
The underlying engine behind Flash AI is a number of BVA with 5 major components:
1- Hypothesis Value Acceleration
A true data science hypothesis does not immerge easily unless it is defined within the context of solving a business problem. There are several techniques and methods that can help to clearly define data science hypothesis by always keeping an eye on the business outcomes and a true Action Oriented Architecture. Design Thinking is the one I have used on a number of AI use cases and have benefited from the clarity it offers including the ability to build visual prototypes in the first iterations of discovery phase (Sprint 1-2). These prototypes help the ultimate users with a crisp visualization and a true business value realization offered by the model output!
2- Data Ingestion Accelerators
Tools that allow you to accelerate your data ingestions and expedite building data pipelines faster. If data governance and compliance is paramount to your lifecycle, it is important to build approval workflows and include your key stakeholders such as, legal, business operations, information security and data owners very early in the process, perhaps as early as sprint 1-2. A great tool (framework) in this area is the very notion of a “Data Foundry” that allows recurring ingestion and data reuse as needed. Once the roles within a given MDLC are identified, Role Based Access Controls (RBAC) can also expedite data ingestion and onboarding provisioning. There are many other examples within this component that I will cover in the part II of this blog.
3- Model Development Accelerators
Tools that allow you to develop your AI models quicker. These tools allow your data scientists to quickly build benchmarks to compare their models with and make enhancements as necessary. Note that the model you are trying to build has probably been built before and you do not necessarily need to build every single model from scratch. Data Robot and H2O Driverless AI are two great tools that can help data scientists to develop AI models faster.
A global Feature Store can also help you build and reuse your features across different AI models. There are several tools in the market or you can build your own from scratch!
4- Model Deployment Accelerators
It is important to note that model deployment is not necessarily all inclusive of MDLC, rather an SDLC process that requires tight partnership with your technology teams (see my blog where MDLC meets SDLC). Deployment accelerators can significantly reduce the model deployment timelines, and on some occasions (depending on model output type) down to 4 weeks, now, that’s just MAD (model auto deployment)!
MAD is a framework that enables automatic deployment, running and monitoring of AI ML and NLP models. Once again, there are several DevOps tools in the market for this purpose or you can build your own in conjunctions with the available tools in the market to meet your needs.
5- Model Integration/Business Consumption Accelerators
Tools and techniques that allow you to consume the results of your AI models faster and ensure that they are driving the business results defined within the original hypothesis. It is of ultimate importance to ensure the very business value of AI models are realized by taking them through an iterative pilot phase. A “show and tell” framework which allows us to build models as they are being trained, produce results and take such results to the business through a pilot (more details in part II of this blog)
6- Business Process Accelerators
At the very center of the Flash AI framework there exists an agile component that can help to streamline the delivery processes across the MDLC. This is where you focus on methods by which you can DELIVER faster. This can be achieved by breaking down the activities within each phase into specific units of time, say 2 week sprints in conjunction with a number of pre-defined user stories that will help to save time for the scrum masters and product owners.
The Agile Blueprint
The diagrams below depicts a high level agile POD for ML and NLP models. Using these blueprints along with the BVA described earlier can lead to significant MDLC acceleration across, as high as 37% in some AI use cases.
In part 2, I will be discussing the BVA in a more granular details.
Ramin Mobasseri 