Inference tutorial

mpVision SDK allows yout to integrate your existing ML models into a an inference engine that can be used to run predictions on images. To facilitate this integration, the SDK offers a suite of functions designed to encapsulate your inference code into mpVision inference engine.

To implement your custom inference engine only thing you have to do is to utilize functions provided from SDK to retrieve inference datasets, report outputs to store inference process results in your model inference code. That set of functions will enable you to wrap your process into an isolated Docker image forming an inference engine that is pluggable to mpVision application. When you done implementing and testing your training process, using the testing interface of mpVision you will be able to configure scheduler to use registered Docker image to initiate inferences processes within mpVision ecosystem.

Retrieving inference request

Whenever user or a scheduler initiates an inference process, mpVision will create an inference request and will queue it for processing. To retrieve that request, you can use mp_sdk.inference.get_request function. It will return you an instance of InferenceRequest object that contains all necessary information about the request and corresponding images.

from mp_sdk import inference

inference_request = inference.get_request()

Then you can access model file that is associated with the request, as well as images that are requested to run inference on:

from mp_sdk import inference

inference_request = inference.get_request()

model_file = inference_request.model_file

my_model = load_model(model_file)

while inference_request.has_next_image():
    image = inference_request.next_image()
    # Run inference on image
    results = my_model.run(image)
    # Report results
    image.save_inference_results(results)

Reporting inference results

After running inference on an image, you can report results back to mpVision using save_inference_results method of InferenceImage object. This method will store results within mpVision database and will make them available for further processing.

from mp_sdk import inference

inference_request = inference.get_request()

model_file = inference_request.model_file

my_model = load_model(model_file)

while inference_request.has_next_image():
    image = inference_request.next_image()
    # Run inference on image
    results = my_model.run(image)
    # Report results
    image.save_inference_results(results)

Supported result types and specification can be found in the InferenceImage class documentation.