Combining Azure Vision Default and Custom Image Tags with an Azure Function

Published 6/4/2025
azureazure-functionscomputer-visionapiimage-tagging

Azure Vision has two APIs for returning Image Tags:

• Azure Computer Vision API – for general-purpose tagging
• Azure Custom Vision Prediction API – for domain-specific tagging

If you want to return a list of tags that contains tags from Azure’s default trained model combined with your own custom tags from your own trained model you need to make two API calls.

The Solution

I created an Azure Function that will take in an imageUrl parameter and combine the results of the two calls into one output result.

đź”— GitHub Repo: AzureVisionCombineTags

Configuration

There is a “Minimum Confidence” level setting that can be adjusted for each API. Depending on how well you train your model you may need to tweak the settings.

All the settings need to be configured as Environment Variables and you need to restart the Azure Function for them to take effect. View the ReadMe.MD or GetDefaultAndCustomImageTags.js file for detailed comments on the variables.

Required Environment Variables

# Computer Vision API
CV_ENDPOINT=your-computer-vision-endpoint
CV_KEY=your-computer-vision-key
CV_MIN_PROBABILITY=0.5

# Custom Vision Prediction API  
CP_PREDICTION_KEY=your-prediction-key
CP_ENDPOINT=your-custom-vision-endpoint
CP_MIN_PROBABILITY=0.7

API Usage

An API key is required to call the Azure Function. Regardless of what you name your API Key, you need to pass it as an HTTP Header via HTTP POST with a variable name of x-functions-key.

Request Format

Method: POST
Header: x-functions-key: your-function-key
Body:

{
  "imageUrl": "https://statcdn.fandango.com/MPX/image/NBCU_Fandango/230/687/AvengersAgeofUltron_Trailer1.jpg"
}

Performance Considerations

You may also want to increase the timeout for your function to run as it is dependent on the two external services. The function makes sequential calls to both Azure Vision APIs, so response time is the sum of both services plus processing overhead.

Use Cases

This solution is particularly useful for:

  • Content Management Systems - Automatically tag images with both general and specific categories
  • E-commerce Platforms - Combine product-specific tags with general image attributes
  • Digital Asset Management - Comprehensive tagging for searchability
  • Integration with Sitecore Content Hub - See my related posts on BYOAI integration

This Azure Function serves as middleware to solve the limitation of Azure Vision’s separate APIs. Check out my related posts on AWS Rekognition and Content Hub integration for more cloud-based image tagging solutions.