VerifiedPermissions - Vague error returned when passing in attributes containing empty sets.

[elexisvenator]

Describe the bug

Calling AmazonVerifiedPermissionsClient.IsAuthorized or AmazonVerifiedPermissionsClient.BatchIsAuthorized requires a series of entities to be passed in. These entities contain attributes which are dictionaries of values similar to what you would find on dynamodb.

If you set an attribute's value to an empty set (which is a valid value) then an error is thrown that does not provide any information as to what went wrong. This is a problem as depending on your access model and policies the difference between an attribute with an empty set and a missing attribute may change the outcome of the access check.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

Running the same command in the console will not produce an error and will process the access check correctly.

(note nothing in this picture is sensitive data)

Current Behavior

The error is as follows (calling BatchIsAuthorized):

Error during execution
Amazon.VerifiedPermissions.AmazonVerifiedPermissionsException: The service returned an error with HTTP Body: {}
 ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.HttpWebRequestMessage.ProcessHttpResponseMessage(HttpResponseMessage responseMessage)
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception, Stream responseStream)
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext executionContext, HttpErrorResponseException exception)
   at Amazon.Runtime.Internal.ExceptionHandler`1.HandleAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.ProcessExceptionAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Program.<>c__DisplayClass0_0.<<<Main>$>g__BuggyAccessCheckScenario|8>d.MoveNext() in Program.cs:line 319

Reproduction Steps

Minimal reproduction (note that depending on your schema you may get a validation exception once this bug is fixed, but that is better than the current error)

var policyStoreId = "provide-own-policy-store-id-here";

var avpClient = new AmazonVerifiedPermissionsClient();
var request = new BatchIsAuthorizedRequest
{
    PolicyStoreId = policyStoreId,
    Entities = new EntitiesDefinition
    {
        EntityList =
        [
            new EntityItem
            {
                Identifier = new EntityIdentifier
                {
                    EntityType = "AccessManagement::User",
                    EntityId = "user::123456"
                },
                Attributes = new Dictionary<string, AttributeValue>
                {
                    // ! this line is the problem !
                    // commenting out the below line removes the error, so does adding a value to the set (eg Set = new List<AttributeValue> { String = "foo" } )
                    { "accessibleProcesses", new AttributeValue { Set = new List<AttributeValue>() } }
                }
            },
            new EntityItem
            {
                Identifier = new EntityIdentifier
                {
                    EntityType = "AccessManagement::Process",
                    EntityId = "abcd"
                }
            }
        ]
    },
    Requests =
    [
        new BatchIsAuthorizedInputItem
        {
            Principal = new EntityIdentifier
            {
                EntityType = "AccessManagement::User",
                EntityId = "user::123456"
            },
            Action = new ActionIdentifier
            {
                ActionId = "ViewProcess",
                ActionType = "AccessManagement::Action"
            },
            Resource = new EntityIdentifier
            {
                EntityType = "AccessManagement::Process",
                EntityId = "abcd"
            }
        }
    ]
};

var response = await avpClient.BatchIsAuthorizedAsync(request);

Possible Solution

After reading the sdk V4 blogs I think the issue is related to what was discussed in this paragraph
https://aws.amazon.com/blogs/developer/preview-1-of-aws-sdk-for-net-v4/

Collections null by default
In V3, properties of type List or Dictionary for requests and responses types were initialized to an empty collection. When making requests, the SDK would only send the property if the collection was not empty. This was done to avoid null pointer exceptions in code using the SDK but it has the following drawbacks:

• The SDK couldn’t always understand the user’s intent, especially when attempting to clear out a collection on an AWS resource.
• When getting a response back from a service, users couldn’t tell if the empty collection was due to the default initialization of the SDK or if the service actual sent back an empty collection.
• Collections were being created that were not even being used, which caused wasted allocations.

Given this, I am not sure how this issue should be fixed in V3. It's a bit more problematic than not understanding intent though as it causes invalid payloads.

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

AWSSDK.VerifiedPermissions 3.7.401.18

Targeted .NET Platform

.NET 9

Operating System and version

Windows 11

[dscpinheiro]

Yes, as you suspected this is happening because the SDK will initialize collections by default in V3. In that same blog post you linked, there's another section on how we're addressing that in V4 (currently in preview and being tracked in #3362):

To address this issue, V4 will no longer by default initialize collections to an empty collection. When using V4, you will need to initialize the collections when making requests. For a lot of code using the .NET SDK, this is already being done as part of constructing the type with an object initializer like the Lambda function code shown above. Collections properties in responses will need to be checked whether they are null to avoid null pointer exceptions.

This change is likely the most impactful change to existing code using the SDK due to it being a runtime change. To alleviate concerns and allow users of the SDK to upgrade to V4 even if they are unsure whether they will be affected by this change, we created the Amazon.AWSConfigs.InitializeCollections property. If you set this property to true you can revert to the collection initialization behavior of V3. The Amazon.AWSConfigs.InitializeCollections property was also recently added to V3 with the default value true. This allows V3 users to set the property to false and try out this new collections behavior before upgrading to V4.

If I run your reproduction code but set that flag to false, the API call succeeds (however, this config applies to all service operations, so if you use it you'll need to validate other places where collections in responses may be null):

AWSConfigs.InitializeCollections = false;
var avpClient = new AmazonVerifiedPermissionsClient();

// Same request as before (with an empty accessibleProcesses)
var response = await avpClient.BatchIsAuthorizedAsync(request);
Console.WriteLine(response.HttpStatusCode); // OK

[elexisvenator]

Unfortunately I am also working on a library, which means I cannot responsibly set a global flag in the awssdk that would affect other parts of the applications using my library. My workaround for now will be to say the library does not support empty sets as attributes, and internally remove any attribute item that is an empty set.

Agree that V4 will help and look forward to upgrading to V4 when its released.
Is there any way to set the flag for either a single request or a single awssdk request object? Or maybe a way to provide a more meaningful error message?

[ashishdhingra]

Unfortunately I am also working on a library, which means I cannot responsibly set a global flag in the awssdk that would affect other parts of the applications using my library. My workaround for now will be to say the library does not support empty sets as attributes, and internally remove any attribute item that is an empty set.

Agree that V4 will help and look forward to upgrading to V4 when its released. Is there any way to set the flag for either a single request or a single awssdk request object? Or maybe a way to provide a more meaningful error message?

@elexisvenator Good morning. Thanks for the response. The error is returned by the service due to empty body. Unfortunately, we do not have a request specific flag to flip the behavior in version 3.x of SDK.