Skip to content

[Priortity queue] Removing Local Execution #424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
v-federicoar wants to merge 3 commits into
base: priority-queue-update
Choose a base branch
from
Open

[Priortity queue] Removing Local Execution #424

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
29a0fb2
Addressing Pull request comment
v-federicoar Dec 19, 2025
04243ff
Update priority-queue/bicep/main.bicep
v-federicoar Dec 19, 2025
28ecf6f
Update priority-queue/Readme.md
v-federicoar Dec 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 14 additions & 59 deletions priority-queue/Readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ This directory contains an example of the [Priority Queue pattern](https://learn

This example demonstrates how priority queues can be implemented by using Service Bus topics and subscriptions. A time-triggered Azure Function is responsible for sending messages to a topic, each with a priority assigned. The receiving Azure Functions read messages from subscriptions that have the corresponding priority.

For local execution, the sample demonstrates the producer/consumer model, where each consumer processes only one type of message based on its priority.

In the Azure Deployment, the _PriorityQueueConsumerHigh_ Azure function could scale out to 200 instances, while the _PriorityQueueConsumerLow_ Azure function could scale out only to 40 instances. It simulates high priority messages being read from the queue more urgently than low priority messages.

The Azure deployment also demonstrates operational aspects of applications running on Azure. Monitoring tools are essential to understand how the sample operates. Azure Functions in the solution **must** be configured to use the diagnostics mechanism. Otherwise, trace information generated by the example will not be visible.
Expand Down Expand Up @@ -47,80 +45,37 @@ Install the prerequisites and follow the steps to deploy and run an example of t
az group create -n $RESOURCE_GROUP_NAME -l $LOCATION
```

1. Deploy the supporting Azure resources.

```bash
CURRENT_USER_OBJECT_ID=$(az ad signed-in-user show -o tsv --query id)
SERVICE_BUS_NAMESPACE_NAME="sbns-priority-queue-$(LC_ALL=C tr -dc 'a-z0-9' < /dev/urandom | fold -w 7 | head -n 1)"

# This takes about two minutes
az deployment group create -n deploy-priority-queue -f bicep/main.bicep -g $RESOURCE_GROUP_NAME -p queueNamespaces=$SERVICE_BUS_NAMESPACE_NAME principalId=$CURRENT_USER_OBJECT_ID
```
## Deploy the example to Azure

1. Configure the samples to use the created Azure resources.

```bash
# Retrieve the primary connection string for the Service Bus namespace.
SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE="${SERVICE_BUS_NAMESPACE_NAME}.servicebus.windows.net"

sed "s|{SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|${SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|g" ./PriorityQueueSender/local.settings.template.json > ./PriorityQueueSender/local.settings.json
sed "s|{SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|${SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|g" ./PriorityQueueConsumerHigh/local.settings.template.json > ./PriorityQueueConsumerHigh/local.settings.json
sed "s|{SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|${SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|g" ./PriorityQueueConsumerLow/local.settings.template.json > ./PriorityQueueConsumerLow/local.settings.json
```

1. [Run Azurite](https://learn.microsoft.com/azure/storage/common/storage-use-azurite#run-azurite) blob storage emulation service.

> Azure Functions require an Azure Storage account as a backing resource. When running locally, you can use Azurite, the local storage emulator, to fulfill this requirement.
Alternatively, you may configure the AzureWebJobsStorage setting to use a real Azure Storage account if preferred.

1. Launch the Azure Function PriorityQueueSender to generate Low and High messages.

```bash
cd ./PriorityQueueSender
func start
```

1. In a new terminal, launch the Azure Function PriorityQueueConsumerLow to consume messages.

```bash
cd ./PriorityQueueConsumerLow
func start -p 15000
```

> Please note: For demo purposes, the sample application will write content to the screen.

1. In a new terminal, launch the Azure Function PriorityQueueConsumerHigh to consume messages.

```bash
cd ./PriorityQueueConsumerHigh
func start -p 15001
```

> Please note: For demo purposes, the sample application will write content to the screen.

## Deploy the example to Azure (Optional)

This Bicep template sets up the core infrastructure for a priority-based message processing system using Azure Functions. It creates a secure Storage Account, an Application Insights instance for monitoring, and uses a previously created Service Bus namespace to enable communication between the sender and consumer functions. The deployment includes three Azure Function Apps: one sender and two consumers, each with different scaling limits to simulate message prioritization.
This Bicep template sets up the core infrastructure for a priority-based message processing system using Azure Functions. It creates a secure Storage Account, an Application Insights instance for monitoring, and uses a Service Bus namespace to enable communication between the sender and consumer functions. The deployment includes three Azure Function Apps: one sender and two consumers, each with different scaling limits to simulate message prioritization.
The funcPriorityQueueConsumerHigh function can scale out to 200 instances, allowing it to process high-priority messages quickly. The funcPriorityQueueConsumerLow function is limited to 40 instances, handling lower-priority messages with less urgency. All function apps use the FlexConsumption plan and are connected to Application Insights for diagnostics and monitoring. Role assignments are configured to securely grant access to the Service Bus and Storage resources using managed identities.
All Azure Function Apps share the same Storage Account and Application Insights instance (It is essential to understand how the sample operates), which centralizes observability and logging.

```bash
SERVICE_BUS_NAMESPACE_NAME="sbns-priority-queue-$(LC_ALL=C tr -dc 'a-z0-9' < /dev/urandom | fold -w 7 | head -n 1)"
# This takes about three minutes
az deployment group create -n deploy-priority-queue-sites -f bicep/azure/azure-function-apps.bicep -g $RESOURCE_GROUP_NAME -p serviceBusNamespaceName=$SERVICE_BUS_NAMESPACE_NAME
az deployment group create -n deploy-priority-queue-sites -f bicep/main.bicep -g $RESOURCE_GROUP_NAME -p serviceBusNamespaceName=$SERVICE_BUS_NAMESPACE_NAME
```
After deploying the infrastructure, you need to publish each Azure Function to its corresponding Function App using Azure Functions Core Tools:

```bash
# Construct the fully qualified namespace (hostname) for the Service Bus.
SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE="${SERVICE_BUS_NAMESPACE_NAME}.servicebus.windows.net"

sed "s|{SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|${SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|g" ./PriorityQueueSender/local.settings.template.json > ./PriorityQueueSender/local.settings.json
sed "s|{SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|${SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|g" ./PriorityQueueConsumerHigh/local.settings.template.json > ./PriorityQueueConsumerHigh/local.settings.json
sed "s|{SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|${SERVICE_BUS_FULLY_QUALIFIED_NAMESPACE}|g" ./PriorityQueueConsumerLow/local.settings.template.json > ./PriorityQueueConsumerLow/local.settings.json

cd ./PriorityQueueSender
func azure functionapp publish funcPriorityQueueSender
func azure functionapp publish funcPriorityQueueSender --dotnet-isolated
cd ..

cd ./PriorityQueueConsumerLow
func azure functionapp publish funcPriorityQueueConsumerLow
func azure functionapp publish funcPriorityQueueConsumerLow --dotnet-isolated
cd ..

cd ./PriorityQueueConsumerHigh
func azure functionapp publish funcPriorityQueueConsumerHigh
func azure functionapp publish funcPriorityQueueConsumerHigh --dotnet-isolated
cd ..
```

Expand Down
82 changes: 0 additions & 82 deletions priority-queue/bicep/azure/azure-function-apps.bicep
View file
Open in desktop

This file was deleted.

93 changes: 66 additions & 27 deletions priority-queue/bicep/main.bicep
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,21 @@ param location string = resourceGroup().location

@minLength(15)
@description('Service Bus Namespace Name.')
param queueNamespaces string
param serviceBusNamespaceName string

@minLength(36)
@description('The principal ID used to run the Azure Functions. In Azure, this should be the managed identity (system-assigned or user-assigned) of the Azure Function. When running locally, it should be your user identity.')
param principalId string
@description('Defines the name of the Storage Account used by the Function Apps. It uses a unique string based on the resource group ID to ensure global uniqueness.')
param storageAccountName string = 'st${uniqueString(resourceGroup().id)}'

@description('Sets the name of the Application Insights resource for monitoring and diagnostics. Like the storage account, it uses a unique string based on the resource group ID.')
param appInsightsName string = 'ai${uniqueString(resourceGroup().id)}'

var logAnalyticsName = 'loganalytics-${uniqueString(subscription().subscriptionId, resourceGroup().id)}'

var senderServiceBusRole = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'69a216fc-b8fb-44d8-bc22-1f3c2cd27a39'
) // Azure Service Bus Data Sender
var receiverServiceBusRole = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'4f6d3b9b-027b-4f4c-9142-0e5a2a2247e0'
) // Azure Service Bus Data Receiver
var senderRoleId = '69a216fc-b8fb-44d8-bc22-1f3c2cd27a39' // Azure Service Bus Data Sender
var receiverRoleId = '4f6d3b9b-027b-4f4c-9142-0e5a2a2247e0' // Azure Service Bus Data Receiver

resource queueNamespacesResource 'Microsoft.ServiceBus/namespaces@2025-05-01-preview' = {
name: queueNamespaces
name: serviceBusNamespaceName
location: location
sku: {
name: 'Standard'
Expand Down Expand Up @@ -186,24 +182,67 @@ resource diagnosticSettings 'Microsoft.Insights/diagnosticSettings@2021-05-01-pr
}
}

// Assign Role to allow sending messages to the Service Bus
resource serviceBusSenderRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, principalId, 'ServiceBusSenderRole')
scope: queueNamespacesResource
resource storageAccount 'Microsoft.Storage/storageAccounts@2025-01-01' = {
name: storageAccountName
location: location
sku: {
name: 'Standard_LRS'
}
kind: 'StorageV2'
properties: {
roleDefinitionId: senderServiceBusRole
principalId: principalId
principalType: 'User' // 'ServicePrincipal' if this was App Service with a managed identity
defaultToOAuthAuthentication: true
publicNetworkAccess: 'Enabled'
allowCrossTenantReplication: false
minimumTlsVersion: 'TLS1_2'
allowBlobPublicAccess: false
allowSharedKeyAccess: false
networkAcls: {
bypass: 'AzureServices'
virtualNetworkRules: []
ipRules: []
defaultAction: 'Allow'
}
supportsHttpsTrafficOnly: true
encryption: {
services: {
file: {
keyType: 'Account'
enabled: true
}
blob: {
keyType: 'Account'
enabled: true
}
}
keySource: 'Microsoft.Storage'
}
}
}

// Assign Role to allow receiving messages from the Service Bus
resource serviceBusReceiverRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, principalId, 'ServiceBusReceiverRole')
scope: queueNamespacesResource
resource appInsights 'Microsoft.Insights/components@2020-02-02' = {
name: appInsightsName
location: location
kind: 'web'
properties: {
roleDefinitionId: receiverServiceBusRole
principalId: principalId
principalType: 'User' // 'ServicePrincipal' if this was App Service with a managed identity
Application_Type: 'web'
}
}

module functionApp './sites.bicep' = [
for name in [
'funcPriorityQueueSender'
'funcPriorityQueueConsumerLow'
'funcPriorityQueueConsumerHigh'
]: {
name: name
params: {
location: location
functionAppName: name
storageAccountName: storageAccount.name
serviceBusNamespaceName: serviceBusNamespaceName
roleId: name == 'funcPriorityQueueSender' ? senderRoleId : receiverRoleId
appInsightsName: appInsights.name
scaleUp: name == 'funcPriorityQueueConsumerHigh' ? 200 : 40
}
}
]
0 priority-queue/bicep/azure/sites.bicep → priority-queue/bicep/sites.bicep
View file
Open in desktop
File renamed without changes.