SQS listener in your Spring Boot project

I was adding SQS listener support to our spring boot based microservice and I realized that most of the examples online are for old versions of spring boot (2.x) or aws starter and I had few problems with dependencies etc so I wanted to prepare this small tutorial for people having similar experiences. The most complete one I could find was this one but lack of local running SQS is a downside.

In this post, I'm going to use localstack to emulate SQS locally. Localstack will be handy with integration tests as well but in this post I'll skip that part.

For dependencies we are going to add 1 bom and 2 usual dependencies:

    implementation(platform("io.awspring.cloud:spring-cloud-aws-dependencies:3.0.2"))
    implementation("io.awspring.cloud:spring-cloud-aws-starter")
    implementation("io.awspring.cloud:spring-cloud-aws-starter-sqs")

BOM (bill of materials) is to keep working dependency versions together.

The next thing is docker compose we will use so that we would have a local SQS. OFC you can choose to use a real instance as well but being able to run it locally is much more straightforward.

 version: "3.8"

services:
  localstack:
    container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}"
    image: localstack/localstack
    ports:
      - "127.0.0.1:4566:4566"            # LocalStack Gateway
      - "127.0.0.1:4510-4559:4510-4559"  # external services port range
    environment:
      - SERVICES="sqs"
      - DEBUG=${DEBUG-}
      - DOCKER_HOST=unix:///var/run/docker.sock
    volumes:
      - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

This is directly copy pasted from localstack website and there are many ways you can do the same thing. As many projects make use of Docker, I thought this version might become handy.

Next thing we will do will be to run this docker compose with:

docker-compose up

This should run localstack sqs on port 4566.

Next thing we would need is to define local sqs values in our application. Our application.yml will contain

 spring:
  cloud:
    aws:
      credentials:
        access-key: local
        secret-key: local
      region:
        static: 'eu-west-1'
      endpoint: 'http://localhost:4566'

In case you would need to setup this for your servers, you probably won't be using StaticCredentialsProvider that's been used there but some other AwsCredentialsProvider such as WebIdentityTokenFileCredentialsProvider. This would require you to define your own bean for this but there's no need for more override of autoconfiguration.

Next thing we will do is to define the service/component to listen to our sqs queue:

@Component
class MessageListener {

    @SqsListener("my-message-queue-name")
    fun receiveMessage(
        message: Message<CustomMessage>,
    ) {
        println("Message received from SQS listener. msg=${message.payload.msg}, code=${message.payload.msgCode}, headers=${message.headers}")
    }

    data class CustomMessage @JsonCreator constructor(
        @JsonProperty("msg") val msg: String,
        @JsonProperty("msgCode") val msgCode: Int,
    )
}

Usually we would expect that CustomMessage can be handled easily with object mapper we would have in the app context but for some reason it does not. This is the reason I specified Json creator and json properties. This post has some brief explanation regarding this issue. I have not tried to specified an object mapper and register with kotlin module, this can be tried to fix the issue as well. For the sake completeness, here's the exception I was getting and I fixed with json annotations:

Caused by: org.springframework.messaging.converter.MessageConversionException: Could not read JSON: Cannot construct instance of `com.sezin.sqsdemo.listener.MessageListener$CustomMessage` (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (String)"{"msg":"my message to you","msgCode":12345}"; line: 1, column: 2]

Now we have a running local sqs instance and a running app that listens to "my-message-queue-name" queue. So two more things are left. We would need to create our msg queue and send the message to test our implementation.

aws --endpoint-url=http://127.0.0.1:4566 sqs create-queue --queue-name my-message-queue-name

can be used to create our queue. OFC, another option would be to add this to an init script and include it into our docker compose but for the sake of simplicity, I did not include it.

Next thing we will do is to send a message to our local queue:

aws --endpoint-url=http://127.0.0.1:4566 sqs send-message --queue-url http://127.0.0.1:4566/000000000000/my-message-queue-name --message-body '{"msg":"my message to you","msgCode":12345}'

If everything run fine, then you should see your SQS listener method picking the message up and printing this:

 "Message received from SQS listener. msg=my message to you, code=12345, headers=..."

You can also prefer to use CustomMessage dto instead of Message or you can change the sqs listener behavior to batch message retrieval by expecting List<CustomMessage> as well.

For the code you can check my github repo.