Deploying a Model Into Production
Deploying a Model Into Production
This information is a generalization, so it may no be accurate for every case.
Process:
exploratory analysis -> model development -> API development -> code review -> unit test creation -> load testing -> deploying to production
Make a Production-suitable Model
Starts as an analysis, then becomes something bigger.
- The model needs to be used by other programs. Typically done with an API.
- Test the model to handle many different inputs (especially ones that it doesn't expect). Make unit tests.
- Deploy the model to a test environment and try to break it, then make it stronger for production.
Data Collection
You'll need a pipeline to get live data - this will likely be very different from where you got the historical data from. You'll need to do leg work and find out if you can get the data you need live (and what format the model will get it in). It's not uncommon for models to fail during this step.
Model Build
Maximize model accuracy and simplicity. You need a model that works and one you can troubleshoot in the future.
API
Document the living crap out of the API. It should be easy for anyone at any time in the future to understand thoroughly. Not just you at this moment in time.
Use an OpenAPI document.
Testing
The API needs to handle any input and not fail. You want to prevent crashes before they happen. Input tests and unit tests are your friend( even though they're annoying).
Deploy the Model
Host the API on server with a virtual machine (VM) or Docker.
Run load tests. If it fails, fix it.
Maintain the Model
Set up alerts based on logging and telemetry records. Some companies have the DevOps team manage all APIs.
Retrain the model periodically. Either do it on a regular schedule, or automate the retraining process.
You'll probably make improvements over time - be careful of scope creep and make sure any updates are worth the effort.
Sources: 1