Skip to content

Conversation

@sourabh1007
Copy link
Contributor

@sourabh1007 sourabh1007 commented Nov 8, 2024

Description

This PR introduces the following changes:

Network-Level Telemetry Implementation

  • Telemetry Updates:
    • Initialized network meter in DocumentClient.cs for client metrics.
    • Refactored ClientContextCore.cs to ensure consistent metric and trace collection.
    • Added models for NetworkMetricData and OperationMetricData.
  • Metrics Enhancements:
    • Populated telemetry dimensions with consistent naming in key files (e.g., IActivityAttributePopulator.cs).
    • Core metric generation logic in CosmosDbNetworkMeter.cs.
    • Refactored CosmosDbOperationMeter.cs and aligned packages (TracesStabilityFactory.cs).
    • Included region info in ClientSideRequestStatisticsTraceDatum.cs.
  • Test Coverage:
    • Added and updated tests in OpenTelemetryMetricsTest.cs.

Reference: open-telemetry/semantic-conventions#1495

Type of change

  • [] New feature (non-breaking change which adds functionality)

sourabh1007 and others added 30 commits October 18, 2024 11:04
more changes

wip

rewrite meters

refactor code

dimension names

test fix

contract update

import fixes

wip

added other metrics

code refactor

add docuemnattion

Delete Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/Microsoft - Backup.Azure.Cosmos.EmulatorTests.csproj

upodated contract
…ripts (#4813)

[INTERNAL] CI: Fixes emulator set-up to leverage central SDK teams
scripts

Observations
- Lessmsi: extraction taking longer (1M) then msiexec based model
- Starting emulator from ProgramFolder path is faster 
- Unnecessary exponential retry logic (current one)
# Pull Request Template

## Description

This PR adds optional attributes in the `VectorIndexDefinition` class to
support partitioned DiskANN. A typical index definition would be
something like the below:

```
{
    "indexingPolicy": {
        "automatic": true,
        "indexingMode": "Consistent",
        "includedPaths": [
            {
                "path": "/*",
                "indexes": []
            }
        ],
        "excludedPaths": [],
        "compositeIndexes": [],
        "spatialIndexes": [],
        "vectorIndexes": [
            {
                "path": "/vector1",
                "type": "flat"
            },
            {
                "path": "/vector2",
                "type": "quantizedFlat",
                "quantizationByteSize": 3,
                "vectorIndexShardKey": [
                    "/Country"
                ]
            },
            {
                "path": "/vector3",
                "type": "diskANN",
                "quantizationByteSize": 2,
                "indexingSearchListSize": 100,
                "vectorIndexShardKey": [
                    "/ZipCode"
                ]
            }
        ]
    },
    "vectorEmbeddingPolicy": {
        "vectorEmbeddings": [
            {
                "path": "/vector1",
                "dataType": "int8",
                "dimensions": 1200,
                "distanceFunction": "dotproduct"
            },
            {
                "path": "/vector2",
                "dataType": "uint8",
                "dimensions": 3,
                "distanceFunction": "cosine"
            },
            {
                "path": "/vector3",
                "dataType": "float32",
                "dimensions": 400,
                "distanceFunction": "euclidean"
            }
        ]
    },
    "id": "test_binary_vector_container_6",
    "partitionKey": {
        "paths": [
            "/pk"
        ],
        "kind": "Hash"
    }
}
```

## Type of change

Please delete options that are not relevant.

- [x] New feature (non-breaking change which adds functionality)

## Closing issues

To automatically close an issue: closes #4628

---------

Co-authored-by: Kiran Kumar Kolli <[email protected]>
Azure.Core:  Fixed upgrading azure core dependency to 1.44.1

Changes
- ResourceType: Conflict between Azure.Core and
Microsoft.Azure.Cosmos.Documents
- Microsoft.Bcl.AsyncInterfaces: Azure.Core needs at-least 6.0.0 (as
Azure core upgraded part of minor version, we are good as well)
- Microsoft.Azure.Cosmos.Encryption.Custom: Direct dependency on
Azure.Core removed (now its transitive)
- Microsoft.Azure.Cosmos.Encryption.Custom.Performance.Tests: `NU1903`
added Newtonsoft.Json dependency
- Performance project: `NU1903` Newtonsoft dependency upgraded
…onKeyStreamAsync API to GA (#4814)

DeleteAllItemsByPartitionKeyStreamAsync: Adds
DeleteAllItemsByPartitionKeyStreamAsync API to GA

Changes
- DeleteAllItemsByPartitionKeyStreamAsync: Marked virtual with default
NotSupportedException (Dependent libraries will continue to work)
- Encryption package related changes will be in a follow-up PR after the
package is published
Copy link
Member

@FabianMeiswinkel FabianMeiswinkel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sourabh1007 sourabh1007 added the auto-merge Enables automation to merge PRs label Dec 13, 2024
Copy link
Member

@FabianMeiswinkel FabianMeiswinkel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@microsoft-github-policy-service microsoft-github-policy-service bot merged commit bf16585 into master Dec 16, 2024
24 checks passed
@microsoft-github-policy-service microsoft-github-policy-service bot deleted the users/sourabhjain/otelnetworkmetrics branch December 16, 2024 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge Enables automation to merge PRs Telemetry

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants