Object Detection

The Object Detection component enables the user to detect and draw a bounding box around each object of interest in the image and assigns them a class label.

Setup

  1. Make sure Intellibot Machine Learning Plugin is installed. Refer Installing Plugins documentation for details.

  2. In Toolbox, expand Machine Learning. Drag and drop Object Detection connector to Global Objects.

  3. Save the project on your machine. The ML model will be generated in the same directory.

Configuration

  1. To configure Object Detection, expand Global Objects, right click on Object Detection and select Configure.

  2. The Object Detection Model screen opens. User has the option to build a new model or work on an existing pre-trained model. The user has to select any of the pre trained model and test the accuracy of the test images by clicking on Test tab.

    Screenshot

  3. To work on a new model the user must select New Model and click OK.

  4. The user will be exposed to the Object Detection Model screen which has the Configuration, Train & Test tabs.

  5. Under the Details tab, assign a name to the model. The Description field is optional.

Model Training

  1. The Train tab exposes images for training. Click on the green + sign, to expose to Image Group. Right click on Image Group to upload sample images to train the model. Double click on the image to get it under Annotation heading.

    Screenshot

  2. The next step is to press the Alt key and select area which the user wants to detect.

  3. Name the selected image under class (label) name (example- stamp, and sign). The class label will populate on the extreme right end corner under Classes Tab.

    Screenshot

  4. Once all the image classes have been created, click on the Train button.

  5. The Get Status button will show the training results with the accuracy percentage.

  6. Depending on the model accuracy the user can decide to tweak the Object Detection Model settings and retrain the model.

Model Testing

  1. Click on Test tab which will help the user to test the created model.

  2. Click on Import button to add test image(s).

  3. Click on Test button. The object detection result will be displayed as a box around each object of interest in the image and a class label will be assigned.

Using the Model

  1. Double click on Object Detection under Global Objects to expose the various components under Methods in Object Explorer.

  2. Drag and drop the ProcessImage component to the design surface which takes an image as an input and detects the object of interest in the image like stamp or signature.

    Screenshot

  3. Connect this component to your automation flow.