Machine Learning made Easy :Segmentation using Huawei ML Kit and Kotlin

[leifu]

Introduction

Image Segmentation – This service can extract segments from image. In this service static image form camera and gallery both are supported and it also supports dynamic camera streams.

Internally it uses Mean Intersection-Over-Union which is a common evaluation metric for semantic image segmentation.

Once segmentation is done it can create coordinate array of each element like human, plant animal etc. Once you get this array you can replace any value with other one to create some wonderful results.

Here we will learn on how we can extract a user from an image, change his background and create a new wonderful result using Huawei ML Kit.

I will divide this process in 4 steps

1. How to choose an image from gallery or camera
2. Once we get the result how to send it to Huawei ML Kit
3. How to complete the processing of image with desired result
4. How to showcase newly extracted image onto the screen

Have you noticed how easy it is to select an image and change its background as per our choice.

One more thing to notice here is how sharp the selection was and which results to how clear the extracted image is looking.

Once we get the segmented image we can create different results as well.

Before we create MLImageSegmentationAnalyzer and as we need to tune our settings to get a desired result, Hence below settings are been changed in order to extract fast human body segment from the image.

Once this MLImageSegmentationSetting object is created for tuning we will pass this to MLImageSegmentationAnalyzer.

setExact(False) : True is whether you want fine segmentation and False is fast segmentation.

setAnalyzerType(MLImageSegmentationSetting.BODY_SE G) : Setting the mode to identify and extract body from image.

setScene(MLImageSegmentationScene.ALL) : On a whole we can extract 4 scenes from image which are mentioned below with their key as well.

MLImageSegmentationScene.ALL: All segmentation results are returned (pixel-level label information, human body image with a transparent background, and gray-scale image with a white human body and black background).

MLImageSegmentationScene.MASK_ONLY: Only pixel-level label information is returned.

MLImageSegmentationScene.FOREGROUND_ONLY: Only a human body image with a transparent background is returned.

MLImageSegmentationScene.GRAYSCALE_ONLY: Only a gray-scale image with a white human body and black background is returned.

For our example we are using MLImageSegmentationScene.ALL as we want all the scenes from the imag

Finally above created MLImageSegmentationSetting object we will be providing to MLImageSegmentationAnalyzer

MLImageSegmentationAnalyzer analyzer = MLAnalyzerFactory.getInstance().getImageSegmentati onAnalyzer(setting)

We can also create MLImageSegmentationAnalyzer object by calling below code as well.

MLImageSegmentationAnalyzer analyzer = MLAnalyzerFactory.getInstance().getImageSegmentati onAnalyzer();

But in this code customization is very limited.

Next step is to create an MLFrame by below code and ptovide your previously fetched image in bitmap format.

MLFrame frame = MLFrame.fromBitmap(bitmap);

On analyser object we will be calling asyncAnalyseFrame(frame) and providing MLFrame which we recently created.

This will yield you a Task object, on this object you will get 2 callbacks.
onSuccessonFailure

You can save the new resource from onSuccess() and stop the analyzer to release detection resources by analyzer.stop() method.
I hope you liked this article. I would love to hear your ideas on how you can use this kit in your Applications.

In case you dont have a real device you can check out this article

For more information, you can visit https://forums.developer.huawei.com/forumPortal/en/home