Object Recognition using Cognitive Service Vision API

Introduction

Microsoft has come up with Cognitive Services. These are a set of machine learning algorithms which has been developed to solve problems in the field of Artificial Intelligence (AI).

In the last article we looked at how to use Face API for reading the facial expression. This article deals with Object Recognition using Cognitive Service Vision API.

An important work that the Vision API does is that it not only tries to identify object from and image but also tag and categorize an object based on what it can identify inside that image.At present the API recognizes nearly 2000 distinct objects and classifies them in 87 groups.

Let's do the experiment

Step 1

Let us first get the Computer Vision API key(s)

Step 2

Open Visual Studio 2017 and fire a WPF application

Step 3

Make the design as under

<Window x:Class="WpfApp2.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:local="clr-namespace:WpfApp2"
        mc:Ignorable="d"
        Title="MainWindow" Height="463.039" Width="525">
    <Grid>
        <Button Content="UploadPhoto" HorizontalAlignment="Left" Margin="110,113,0,0" VerticalAlignment="Top" Width="100" Click="btnUploadPhoto_Click"/>

        <Image x:Name="imgFacePhoto" Stretch="Uniform" Margin="0,0,-702.333,49.667"/>
        
        <TextBlock Name="txtDescription" HorizontalAlignment="Left" Margin="10,157,0,0" TextWrapping="Wrap" VerticalAlignment="Top" Height="240" Width="412"/>

    </Grid>
</Window>

Step 4

From the Nuget package manager console, fire the below

PM > Install-Package Microsoft.ProjectOxford.Vision

Step 5

In the code behind, the most important part is the

private readonly IVisionServiceClient visionServiceClient =
           new VisionServiceClient("Service Key", "https://westcentralus.api.cognitive.microsoft.com/vision/v1.0");

The above line helps to connect , authenticate and query webservice at the remote endpoint.It has a method AnalyzeImageAsync that accepts image as a stream along with the information that has been sought and return a Task of type AnalysisResult

Task<AnalysisResult> AnalyzeImageAsync(Stream imageStream, IEnumerable<VisualFeature> visualFeatures = null, IEnumerable<string> details = null);

The code snippet will clear the way of invocation to this function

 private async Task<AnalysisResult> AnalyzeImage(string imageFilePath)
        {
            // The list of Face attributes to return.
            IEnumerable<VisualFeature> visualFeatures =
                new VisualFeature[]
                {
                    VisualFeature.Adult,
                    VisualFeature.Categories,
                    VisualFeature.Color,
                    VisualFeature.Description,
                    VisualFeature.Faces,
                    VisualFeature.ImageType,
                     VisualFeature.Tags
                };

            // Call the Analyze Image API.
            try
            {
                using (Stream imageFileStream = File.OpenRead(imageFilePath))
                {
                    AnalysisResult analysisResult = await visionServiceClient.AnalyzeImageAsync(imageFileStream, visualFeatures);
                    return analysisResult;
                }
            }
            // Catch and display all other errors.
            catch (Exception e)
            {
                MessageBox.Show(e.Message);
                return new AnalysisResult();
            }            
        }

Now finally, we have received the image analysis information. Next task is to read the information by using some methods as described under

private string ObjectDescription(AnalysisResult result)
{
	StringBuilder sb = new StringBuilder();

	sb.AppendLine("Information about the Object");
	sb.AppendLine("--------------------------------");

	//Read Captions
	sb.AppendLine("Object Description : " + result.Description.Captions[0].Text);

	//Read Object Category
	sb.AppendLine("Object Category : " + result.Categories[0].Name.Replace("_", " "));

	//Read Tags
	sb.AppendLine("Other Probable Objects => ");
	result.Tags.ToList().ForEach(i => sb.AppendLine("\t\t" + i.Name));

	return sb.ToString();
}

Step 6

Now let's run the application.

As can be figured out that the vision API has correctly identified the object i.e. mountain.

Let's try with another

Again the image is close though not precise.

Reference

1. Azure Cognitive Services
2. Computer Vision API

Conclusion

In this article we learnt Object Recognition using Cognitive Service Vision API with an example.Hope this helps. Thanks for reading.Zipped file attached.

Disclaimer: The image used in this article is for demo purpose only. They might be respective owners copyright content.