Skip to content

[NEW EXAMPLE REQUEST] <<ItemUtils for dynamoDB sdkv2>> #889

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ajit-yantriks opened this issue Nov 12, 2019 · 6 comments
Closed

[NEW EXAMPLE REQUEST] <<ItemUtils for dynamoDB sdkv2>> #889

ajit-yantriks opened this issue Nov 12, 2019 · 6 comments

Comments

@ajit-yantriks
Copy link

ajit-yantriks commented Nov 12, 2019 •
edited
Loading

What are you trying to accomplish?

I have created a spring boot application. I am sending response in Map<String, software.amazon.awssdk.services.dynamodb.model.AttributeValue> . It shows No serializer found for class software.amazon.awssdk.services.dynamodb.model.AttributeValue error in postman when hitting endpoint

Which AWS services, if known, should this code example use?

https://aws.amazon.com/dynamodb/

Which AWS SDKs or tools, if known, should this code example use?

AWS SDK v2 dynamodb

Are there any similar code examples that could be leveraged?

https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/javav2/example_code/dynamodb/src/main/java/com/example/dynamodb/GetItem.java

Any other related details we should know about?

No
@scmacdon
Copy link
Contributor

This is an interesting use case. That is, working with the AWS APIs in a Spring app. I will work on this and post back my findings.

@scmacdon
Copy link
Contributor

scmacdon commented Nov 15, 2019 •
edited
Loading

Have you tried using Spring APIs and AWS APIs in an IDE such as the IntelliJ IDE?

When doing so, ensure that you have properly setup the POM file to include the proper dependencies. For example:

POM

I successfully got the AWS Dynamo API working within a Spring project. I built the application by using the IntelliJ IDE. You can see the following illustration that shows the application rendering a value retrieved from a Dynamo table:

pic11

We will look at adding this use case to AWS developer docs.

@ajit-yantriks
Copy link
Author

Sorry i could have provided you a reference code . so that you can understand what exactly i am doing.

I am using a spring-boot application with spring webflux . Please find the controller code below:

    @GetMapping("/v2/single")
    Mono<Map<String, software.amazon.awssdk.services.dynamodb.model.AttributeValue>> getDynamoDataV2(){
        Map<String, software.amazon.awssdk.services.dynamodb.model.AttributeValue> key_to_get = new HashMap<String, software.amazon.awssdk.services.dynamodb.model.AttributeValue>();

        key_to_get.put("id", software.amazon.awssdk.services.dynamodb.model.AttributeValue.builder()
                .s("ad2b9522-89fd-474e-83af-d49042c693f9").build());

        GetItemRequest request = request = GetItemRequest.builder()
                .key(key_to_get)
                .tableName("employee")
                .build();
        return Mono.fromFuture(dynamoDbAsyncClient.getItem(request)).map(resp->resp.item());
    }

here is the error stack trace:

org.springframework.core.codec.CodecException: Type definition error: [simple type, class software.amazon.awssdk.services.dynamodb.model.AttributeValue]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class software.amazon.awssdk.services.dynamodb.model.AttributeValue and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.Collections$UnmodifiableMap["firstName"])
	at org.springframework.http.codec.json.AbstractJackson2Encoder.encodeValue(AbstractJackson2Encoder.java:186) ~[spring-web-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
	|_ checkpoint ⇢ Handler com.example.dynamo.controller.DynamoController#getDynamoDataV2() [DispatcherHandler]
	|_ checkpoint ⇢ HTTP GET "/dynamo/v2/single" [ExceptionHandlingWebHandler]
Stack trace:
		at org.springframework.http.codec.json.AbstractJackson2Encoder.encodeValue(AbstractJackson2Encoder.java:186) ~[spring-web-5.2.1.RELEASE.jar:5.2.1.RELEASE]
		at org.springframework.http.codec.json.AbstractJackson2Encoder.encodeValue(AbstractJackson2Encoder.java:151) ~[spring-web-5.2.1.RELEASE.jar:5.2.1.RELEASE]
		at org.springframework.http.codec.EncoderHttpMessageWriter.lambda$write$1(EncoderHttpMessageWriter.java:125) ~[spring-web-5.2.1.RELEASE.jar:5.2.1.RELEASE]
		at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:118) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
		at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:67) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
		at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:121) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
		at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1592) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
		at reactor.core.publisher.MonoCompletionStage.lambda$subscribe$0(MonoCompletionStage.java:86) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
		at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[na:1.8.0_221]
		at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[na:1.8.0_221]
		at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_221]
		at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[na:1.8.0_221]
		at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallTimeoutTrackingStage.lambda$execute$2(AsyncApiCallTimeoutTrackingStage.java:69) ~[sdk-core-2.10.14.jar:na]
		at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[na:1.8.0_221]
		at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[na:1.8.0_221]
		at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_221]
		at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[na:1.8.0_221]
		at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.retryResponseIfNeeded(AsyncRetryableStage.java:136) ~[sdk-core-2.10.14.jar:na]
		at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.retryIfNeeded(AsyncRetryableStage.java:120) ~[sdk-core-2.10.14.jar:na]
		at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.lambda$execute$0(AsyncRetryableStage.java:107) ~[sdk-core-2.10.14.jar:na]
		at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[na:1.8.0_221]
		at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[na:1.8.0_221]
		at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_221]
		at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[na:1.8.0_221]
		at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$1(MakeAsyncHttpRequestStage.java:138) ~[sdk-core-2.10.14.jar:na]
		at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[na:1.8.0_221]
		at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[na:1.8.0_221]
		at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442) ~[na:1.8.0_221]
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_221]
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_221]
		at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_221]
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class software.amazon.awssdk.services.dynamodb.model.AttributeValue and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.Collections$UnmodifiableMap["firstName"])
	at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77) ~[jackson-databind-2.10.0.jar:2.10.0]
	at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191) ~[jackson-databind-2.10.0.jar:2.10.0]
	at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:403) ~[jackson-databind-2.10.0.jar:2.10.0]
	at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:71) ~[jackson-databind-2.10.0.jar:2.10.0]
	at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:33) ~[jackson-databind-2.10.0.jar:2.10.0]
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFieldsUsing(MapSerializer.java:817) ~[jackson-databind-2.10.0.jar:2.10.0]
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:641) ~[jackson-databind-2.10.0.jar:2.10.0]
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33) ~[jackson-databind-2.10.0.jar:2.10.0]
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[jackson-databind-2.10.0.jar:2.10.0]
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:400) ~[jackson-databind-2.10.0.jar:2.10.0]
	at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1429) ~[jackson-databind-2.10.0.jar:2.10.0]
	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:921) ~[jackson-databind-2.10.0.jar:2.10.0]
	at org.springframework.http.codec.json.AbstractJackson2Encoder.encodeValue(AbstractJackson2Encoder.java:181) ~[spring-web-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.http.codec.json.AbstractJackson2Encoder.encodeValue(AbstractJackson2Encoder.java:151) ~[spring-web-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.http.codec.EncoderHttpMessageWriter.lambda$write$1(EncoderHttpMessageWriter.java:125) ~[spring-web-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:118) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
	at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:67) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:121) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1592) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
	at reactor.core.publisher.MonoCompletionStage.lambda$subscribe$0(MonoCompletionStage.java:86) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[na:1.8.0_221]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[na:1.8.0_221]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_221]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[na:1.8.0_221]
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncApiCallTimeoutTrackingStage.lambda$execute$2(AsyncApiCallTimeoutTrackingStage.java:69) ~[sdk-core-2.10.14.jar:na]
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[na:1.8.0_221]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[na:1.8.0_221]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_221]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[na:1.8.0_221]
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.retryResponseIfNeeded(AsyncRetryableStage.java:136) ~[sdk-core-2.10.14.jar:na]
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.retryIfNeeded(AsyncRetryableStage.java:120) ~[sdk-core-2.10.14.jar:na]
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.lambda$execute$0(AsyncRetryableStage.java:107) ~[sdk-core-2.10.14.jar:na]
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[na:1.8.0_221]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[na:1.8.0_221]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_221]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[na:1.8.0_221]
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$1(MakeAsyncHttpRequestStage.java:138) ~[sdk-core-2.10.14.jar:na]
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[na:1.8.0_221]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[na:1.8.0_221]
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442) ~[na:1.8.0_221]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_221]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_221]
	at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_221]

I am using AWS V2 DynamoDbAsyncClient Also AWS V2 Does not have ItemUtils.toSimpleMapValue(....) so that i can convert Map<String,AttributeValue> to Map<String,Object>

@scmacdon
Copy link
Contributor

scmacdon commented Nov 18, 2019 •
edited
Loading

AN update - i am seeing an issue too with this code:

       DynamoDbAsyncClient client = DynamoDbAsyncClient.create();
       CompletableFuture<GetItemResponse> returned_item  = client.getItem(GetItemRequest.builder()
                .key(key_to_get)
                .tableName(table_name)
                .build());

I am going to keep working on this.

@scmacdon
Copy link
Contributor

scmacdon commented Nov 19, 2019 •
edited
Loading

Your issue was code was not correct. I tried to compile your example and it was not working. Here is code that successfully retrieves data from a Dynamo table using DynamoDbAsyncClient.

**
String myName="";
try {
DynamoDbAsyncClient client = DynamoDbAsyncClient.create();

        GetItemRequest request =  GetItemRequest.builder()
                .key(key_to_get)
                .tableName(table_name)
                .build();

        //Get result Set!
        java.util.Collection<software.amazon.awssdk.services.dynamodb.model.AttributeValue>  returned_item = client.getItem(request).join().item().values();

       //Convert Set to Map
        Map<String, AttributeValue> map = returned_item.stream().collect(Collectors.toMap(AttributeValue::s, s->s));
        Set<String> keys = map.keySet();
        for (String key : keys) {
            System.out.format("%s: %s\n",  key, map.get(key).toString());
            myName= map.get(key).s();
        }


    } catch (Exception e) {
        System.err.println(e.getMessage());
        System.exit(1);
    }

**

The following illustration shows this code running and successfully retrieving a value from the same table as shown above.

AsDynamoClient

Hope this helps you.

We will update the AWS code catalog too.

@ajit-yantriks
Copy link
Author

Thanks for response.

But You printing the Map<String,AttributeValue> in console not sending the data via REST Endpoint.

I had to write this below class to convert it into Map<String,Object> for Rest Endpoint

public class DynamoDBConverter {
    public static <T> Map<String, T> toSimpleMapValue(
            Map<String, AttributeValue> values) {
        if (values == null) {
            return null;
        }

        Map<String, T> result = new LinkedHashMap<String, T>(values.size());
        for (Map.Entry<String, AttributeValue> entry : values.entrySet()) {
            T t = toSimpleValue(entry.getValue());
            result.put(entry.getKey(), t);
        }
        return result;
    }

    public static <T> T toSimpleValue(AttributeValue value) {
        if (value == null) {
            return null;
        }
        if (Boolean.TRUE.equals(value.nul())) {
            return null;
        } else if (Boolean.FALSE.equals(value.nul())) {
            throw new UnsupportedOperationException("False-NULL is not supported in DynamoDB");
        } else if (value.bool() != null) {
            @SuppressWarnings("unchecked")
            T t = (T) value.bool();
            return t;
        } else if (value.s() != null) {
            @SuppressWarnings("unchecked")
            T t = (T) value.s();
            return t;
        } else if (value.n() != null) {
            @SuppressWarnings("unchecked")
            T t = (T) new BigDecimal(value.n());
            return t;
        } else if (value.b() != null) {
            @SuppressWarnings("unchecked")
            T t = (T) copyAllBytesFrom(value.b().asByteBuffer());
            return t;
        } else if (value.ss() != null) {
            @SuppressWarnings("unchecked")
            T t = (T) new LinkedHashSet<String>(value.ss());
            return t;
        } else if (value.ns() != null) {
            Set<BigDecimal> set = new LinkedHashSet<BigDecimal>(value.ns().size());
            for (String s : value.ns()) {
                set.add(new BigDecimal(s));
            }
            @SuppressWarnings("unchecked")
            T t = (T) set;
            return t;
        } else if (value.bs() != null) {
            Set<byte[]> set = new LinkedHashSet<byte[]>(value.bs().size());
            for (SdkBytes bb : value.bs()) {
                set.add(copyAllBytesFrom(bb.asByteBuffer()));
            }
            @SuppressWarnings("unchecked")
            T t = (T) set;
            return t;
        } else if (value.l() != null) {
            @SuppressWarnings("unchecked")
            T t = (T) toSimpleList(value.l());
            return t;
        } else if (value.m() != null) {
            @SuppressWarnings("unchecked")
            T t = (T) toSimpleMapValue(value.m());
            return t;
        } else {
            throw new IllegalArgumentException(
                    "Attribute value must not be empty: " + value);
        }
    }

    public static List<Object> toSimpleList(List<AttributeValue> attrValues) {
        if (attrValues == null)
            return null;
        List<Object> result = new ArrayList<Object>(attrValues.size());
        for (AttributeValue attrValue : attrValues) {
            Object value = toSimpleValue(attrValue);
            result.add(value);
        }
        return result;
    }
}

@pkgonan pkgonan mentioned this issue Aug 6, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Doug-AWS @ajit-yantriks @scmacdon