SQS listener in your Spring Boot project

I was adding SQS listener support to our spring boot based microservice and I realized that most of the examples online are for old versions of spring boot (2.x) or aws starter and I had few problems with dependencies etc so I wanted to prepare this small tutorial for people having similar experiences. The most complete one I could find was this one but lack of local running SQS is a downside.

In this post, I'm going to use localstack to emulate SQS locally. Localstack will be handy with integration tests as well but in this post I'll skip that part.

For dependencies we are going to add 1 bom and 2 usual dependencies:

    implementation(platform("io.awspring.cloud:spring-cloud-aws-dependencies:3.0.2"))
    implementation("io.awspring.cloud:spring-cloud-aws-starter")
    implementation("io.awspring.cloud:spring-cloud-aws-starter-sqs")

BOM (bill of materials) is to keep working dependency versions together.

The next thing is docker compose we will use so that we would have a local SQS. OFC you can choose to use a real instance as well but being able to run it locally is much more straightforward.

 version: "3.8"

services:
  localstack:
    container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}"
    image: localstack/localstack
    ports:
      - "127.0.0.1:4566:4566"            # LocalStack Gateway
      - "127.0.0.1:4510-4559:4510-4559"  # external services port range
    environment:
      - SERVICES="sqs"
      - DEBUG=${DEBUG-}
      - DOCKER_HOST=unix:///var/run/docker.sock
    volumes:
      - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

This is directly copy pasted from localstack website and there are many ways you can do the same thing. As many projects make use of Docker, I thought this version might become handy.

Next thing we will do will be to run this docker compose with:

docker-compose up

This should run localstack sqs on port 4566.

Next thing we would need is to define local sqs values in our application. Our application.yml will contain

 spring:
  cloud:
    aws:
      credentials:
        access-key: local
        secret-key: local
      region:
        static: 'eu-west-1'
      endpoint: 'http://localhost:4566'

In case you would need to setup this for your servers, you probably won't be using StaticCredentialsProvider that's been used there but some other AwsCredentialsProvider such as WebIdentityTokenFileCredentialsProvider. This would require you to define your own bean for this but there's no need for more override of autoconfiguration.

Next thing we will do is to define the service/component to listen to our sqs queue:

@Component
class MessageListener {

    @SqsListener("my-message-queue-name")
    fun receiveMessage(
        message: Message<CustomMessage>,
    ) {
        println("Message received from SQS listener. msg=${message.payload.msg}, code=${message.payload.msgCode}, headers=${message.headers}")
    }

    data class CustomMessage @JsonCreator constructor(
        @JsonProperty("msg") val msg: String,
        @JsonProperty("msgCode") val msgCode: Int,
    )
}

Usually we would expect that CustomMessage can be handled easily with object mapper we would have in the app context but for some reason it does not. This is the reason I specified Json creator and json properties. This post has some brief explanation regarding this issue. I have not tried to specified an object mapper and register with kotlin module, this can be tried to fix the issue as well. For the sake completeness, here's the exception I was getting and I fixed with json annotations:

Caused by: org.springframework.messaging.converter.MessageConversionException: Could not read JSON: Cannot construct instance of `com.sezin.sqsdemo.listener.MessageListener$CustomMessage` (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (String)"{"msg":"my message to you","msgCode":12345}"; line: 1, column: 2]

Now we have a running local sqs instance and a running app that listens to "my-message-queue-name" queue. So two more things are left. We would need to create our msg queue and send the message to test our implementation.

aws --endpoint-url=http://127.0.0.1:4566 sqs create-queue --queue-name my-message-queue-name

can be used to create our queue. OFC, another option would be to add this to an init script and include it into our docker compose but for the sake of simplicity, I did not include it.

Next thing we will do is to send a message to our local queue:

aws --endpoint-url=http://127.0.0.1:4566 sqs send-message --queue-url http://127.0.0.1:4566/000000000000/my-message-queue-name --message-body '{"msg":"my message to you","msgCode":12345}'

If everything run fine, then you should see your SQS listener method picking the message up and printing this:

 "Message received from SQS listener. msg=my message to you, code=12345, headers=..."

You can also prefer to use CustomMessage dto instead of Message or you can change the sqs listener behavior to batch message retrieval by expecting List<CustomMessage> as well.

For the code you can check my github repo.

send your data async on kafka

For a project, I'm trying to log the basic transactions of the user such as addition and removal of an item and for multiple types of items and sending a message to kafka for each transaction. The accuracy of the log mechanism is not crucial and I don't want it to block my business code in the case of kafka server downtime. In this case an async approach for sending data to kafka is a better way to go.

My kafka producer code is in its boot project. For making it async, I just have to add two annotations: @EnableAsync and @Async.

@EnableAsync will be used in your configuration class (also remember that your class with @SpringBootApplication is also a config class) and will try to find a TaskExecutor bean. If not, it creates a SimpleAsyncTaskExecutor. SimpleAsyncTaskExecutor is ok for toy projects but for anything larger than that it's a bit risky since it does not limit concurrent threads and does not reuse threads. So to be safe, we will also add a task executor bean.

So,

@SpringBootApplication
public class KafkaUtilsApplication {
    public static void main(String[] args) {
        SpringApplication.run(KafkaUtilsApplication.class, args);
    }
}

will become

@EnableAsync
@SpringBootApplication
public class KafkaUtilsApplication {
    public static void main(String[] args) {
        SpringApplication.run(KafkaUtilsApplication.class, args);
    }

    @Bean
    public Executor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(2);
        executor.setMaxPoolSize(2);
        executor.setQueueCapacity(500);
        executor.setThreadNamePrefix("KafkaMsgExecutor-");
        executor.initialize();
        return executor;
    }
}

As you can see there's not much change here. The default values I set should be tweaked based on your app's needs.

The second thing we need is addition of @Async.


My old code was:

@Service
public class KafkaProducerServiceImpl implements KafkaProducerService {

    private static final String TOPIC = "logs";

    @Autowired
    private KafkaTemplate<String, KafkaInfo> kafkaTemplate;

    @Override
    public void sendMessage(String id, KafkaType kafkaType, KafkaStatus kafkaStatus) {
        kafkaTemplate.send(TOPIC, new KafkaInfo(id, kafkaType, kafkaStatus);
    }
}

As you can see the sync code is quite straightforward. It just takes the kafkaTemplate and sends a message object to the "logs" topic. My new code is a bit longer than that.

@Service
public class KafkaProducerServiceImpl implements KafkaProducerService {

    private static final String TOPIC = "logs";

    @Autowired
    private KafkaTemplate kafkaTemplate;

    @Async
    @Override
    public void sendMessage(String id, KafkaType kafkaType, KafkaStatus kafkaStatus) {
        ListenableFuture<SendResult<String, KafkaInfo>> future = kafkaTemplate.send(TOPIC, new KafkaInfo(id, kafkaType, kafkaStatus));
        future.addCallback(new ListenableFutureCallback<>() {
            @Override
            public void onSuccess(final SendResult<String, KafkaInfo> message) {
                // left empty intentionally
            }

            @Override
            public void onFailure(final Throwable throwable) {
                // left empty intentionally

            }
        });
    }
}

Here onSuccess() is not really meaningful for me. But onFailure() I can log the exception so I'm informed if there's a problem with my kafka server.

There's another thing I have to share with you. For sending an object through kafkatemplate, I have to equip it with the serializer file I have.

public class KafkaInfoSerializer implements Serializer<kafkainfo> {

    @Override
    public void configure(Map map, boolean b) {
    }

    @Override
    public byte[] serialize(String arg0, KafkaInfo info) {
        byte[] retVal = null;
        ObjectMapper objectMapper = new ObjectMapper();
        try {
            retVal = objectMapper.writeValueAsString(info).getBytes();
        } catch (Exception e) {
            // log the exception
        }
        return retVal;
    }

    @Override
    public void close() {
    }
}

Also, don't forget to add the configuration for it. There are several ways of defining serializers for kafka. One of the easiest ways is adding it to application.properties. 

spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer spring.kafka.producer.value-serializer=com.sezinkarli.kafkautils.serializer.KafkaInfoSerializer

Now you have a boot project that can send async objects to the desired topic.

spring annotations i never had the chance to use part 2: @ConfigurationProperties

Few days ago, I accidentally stumbled upon a Spring annotation from Spring Boot project while I was checking something else.

We all know how to bind property values with "@Value" to the classes and we all know that this can be quite cumbersome if there are multiple properties to bind. Spring Boot is here to help. You can use "@ConfigurationProperties" and bind multiple values quite concisely. We will give a prefix to differentiate other configs from ours. e.g. "@ConfigurationProperties(prefix = "jdbc")".
Any field this annotated class has is populated with property values from the property resource. For instance if it has a username parameter then property resource with "jdbc.username" key will populate this field. The most practical way of using this annotation is using it with "@Configuration".


You can check how we create the config class.

package com.sezinkarli.tryconfigprops;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;

import javax.annotation.PostConstruct;
import java.util.HashMap;
import java.util.Map;

@Configuration
@ConfigurationProperties(prefix = "jdbc")
public class JdbcConfig
{
    private String user;
    private String password;
    private String url;
    private String driver;

    public String getUser()
    {
        return user;
    }

    public void setUser(String user)
    {
        this.user = user;
    }

    public String getPassword()
    {
        return password;
    }

    public void setPassword(String password)
    {
        this.password = password;
    }

    public String getUrl()
    {
        return url;
    }

    public void setUrl(String url)
    {
        this.url = url;
    }

    public String getDriver()
    {
        return driver;
    }

    public void setDriver(String driver)
    {
        this.driver = driver;
    }

    public String getProperty(String key)
    {
        return propertyMap.get(key);
    }
}

And below you can check the properties we map from application properties

jdbc.user=myJdbcUser
jdbc.password=myPwd
jdbc.url=myUrl
jdbc.driver=myJdbcDriver

After that you can easily get these values by injecting the configuration class to somewhere.

@Service
public class YourService
{

    @Autowired
    private JdbcConfig jdbcConfig;
}

You can also check here for a working toy project using "@ConfigurationProperties".

spring boot presentation for ozgur yazilim ve linux gunleri 2015

I did a Spring Boot Workshop at Ozgur Yazilim ve Linux Gunleri 2015 (Open-source Software and Linux Days for Turkish-free people). You can view the presentation below.

head first elastic search on java with spring boot and data features

In this article I'll try to give you an easy introduction on how to use Elastic Search in a Java project. As Spring Boot is the easiest and fastest way to begin our project I choose to use it. Futhermore, we will heavily use Repository goodies of beloved Spring Data.

Let's begin by installing Elastic Search on our machine and run our elastic server for the first time.

I go to elastic-folder\bin and run elasticsearch.bat (yeah I'm using Windows) but no luck. I get this:

"Error occurred during initialization of VM

Could not reserve enough space for object heap

Error: Could not create the Java Virtual Machine.

Error: A fatal exception has occurred. Program will exit."

What a great start!

In my bin folder there's a "elasticsearch.in.bat" file. I set ES_MAX_MEM=1g to 

ES_MAX_MEM=512mb and voila it is fixed.

I start a new server without problem after that.

Now it is time to define the document we will index in elastic search. Assume we have movie information to index. Our model is quite straightforward. Movie has a name, rating and a genre in it. I chose "elastic_sample" as index name which sounds good as a database name and "movie" as type which is good for a table name if we think in relational database terms. Nothing fancy in the model as you can see.

@Document(indexName = "elastic_sample", type = "movie")
public class Movie {

    @Id
    private String id;

    private String name;

    @Field(type = FieldType.Nested)
    private List &lt Genre &gt  genre;

    private Double rating;

    public Double getRating() {
        return rating;
    }

    public void setRating(Double rating) {
        this.rating = rating;
    }

    public void setId(String id) {
        this.id = id;
    }

    public List &lt Genre &gt  getGenre() {
        return genre;
    }

    public void setGenre(List &lt Genre &gt  genre) {
        this.genre = genre;
    }

    public String getId() {
        return id;
    }

    public String getName() {
        return name;

    }

    public void setName(String name) {
        this.name = name;
    }

    @Override
    public String toString() {
        return "Movie{" +
                "id=" + id +
                ", name='" + name + '\'' +
                ", genre=" + genre +
                ", rating=" + rating +
                '}';
    }
}

For those who wonder what Genre is here is it. Just a POJO.

public class Genre {
    private String name;

    public Genre() {
    }

    public Genre(String name) {
        this.name = name;
    }

    public String getName() {
        return name;
    }

    @Override
    public String toString() {
        return "Genre{" +
                "name='" + name + '\'' +
                '}';
    }

    public void setName(String name) {
        this.name = name;
    }
}

Not it is time to create DAO layer so we can save and load our document to/from our elastic search server. Our Repository extends the classic ElasticserchRepository (no idea why it is search and not Search). As you probably know Spring Data can query one or more fields with these predefined methods where we use our field names. findByName will search in the name field, findByRating will search in the rating field so on so forth. Furthermore thanks to Spring Data we don't need to write implementation for it, we just put method names in the interface and that's finished.

public interface MovieRepository extends ElasticsearchRepository &lt Movie, Long &gt {
    public List &lt Movie &gt  findByName(String name);

    public List &lt Movie&gt  findByRatingBetween(Double beginning, Double end);
}

Our DAO layer will be called by a Service layer:

@Service
public class MovieService {

    @Autowired
    private MovieRepository repository;

    public List &lt Movie &gt  getByName(String name) {
        return repository.findByName(name);
    }

    public List &lt Movie &gt  getByRatingInterval(Double beginning, Double end) {
        return repository.findByRatingBetween(beginning, end);
    }

    public void addMovie(Movie movie) {
        repository.save(movie);
    }
}

Here is the main Class we will use to run our application. EnableAutoConfiguration will auto-configure everything it recognizes under our classpath. ComponentScan will scan for Spring annotations under the main Class' directory.

@Configuration
@EnableAutoConfiguration
@ComponentScan
public class BootElastic implements CommandLineRunner {

    @Autowired
    private MovieService movieService;

    private static final Logger logger = LoggerFactory.getLogger(BootElastic.class);

// add star wars and
// princess bride as a movie
// to elastic search
    private void addSomeMovies() {
        Movie starWars = getFirstMovie();
        movieService.addMovie(starWars);

        Movie princessBride = getSecondMovie();
        movieService.addMovie(princessBride);
    }

    private Movie getSecondMovie() {
        Movie secondMovie = new Movie();
        secondMovie.setId("2");
        secondMovie.setRating(8.4d);
        secondMovie.setName("The Princess Bride");

        List &lt Genre &gt  princessPrideGenre = new ArrayList &lt Genre &gt();
        princessPrideGenre.add(new Genre("ACTION"));
        princessPrideGenre.add(new Genre("ROMANCE"));
        secondMovie.setGenre(princessPrideGenre);

        return secondMovie;
    }


    private Movie getFirstMovie() {
        Movie firstMovie = new Movie();
        firstMovie.setId("1");
        firstMovie.setRating(9.6d);
        firstMovie.setName("Star Wars");

        List &lt Genre &gt  starWarsGenre = new ArrayList &lt Genre &gt();
        starWarsGenre.add(new Genre("ACTION"));
        starWarsGenre.add(new Genre("SCI_FI"));
        firstMovie.setGenre(starWarsGenre);

        return firstMovie;
    }

    public void run(String... args) throws Exception {
        addSomeMovies();
        // We indexed star wars and pricess bride to our movie
        // listing in elastic search

        //Lets query if we have a movie with Star Wars as name
        List &lt Movie &gt starWarsNameQuery = movieService.getByName("Star Wars");
        logger.info("Content of star wars name query is {}", starWarsNameQuery);

        //Lets query if we have a movie with The Princess Bride as name
        List &lt Movie &gt  brideQuery = movieService.getByName("The Princess Bride");
        logger.info("Content of princess bride name query is {}", brideQuery);


        //Lets query if we have a movie with rating between 6 and 9
        List &lt Movie &gt  byRatingInterval = movieService.getByRatingInterval(6d, 9d);
        logger.info("Content of Rating Interval query is {}", byRatingInterval);
    }

    public static void main(String[] args) throws Exception {
        SpringApplication.run(BootElastic.class, args);
    }
}

If we run it the result is:

015-02-28 18:26:12.368  INFO 3616 --- [           main] main.BootElastic: Content of star wars name query is [Movie{id=1, name='Star Wars', genre=[Genre{name='ACTION'}, Genre{name='SCI_FI'}], rating=9.6}]
2015-02-28 18:26:12.373  INFO 3616 --- [           main] main.BootElastic: Content of princess bride name query is [Movie{id=2, name='The Princess Bride', genre=[Genre{name='ACTION'}, Genre{name='ROMANCE'}], rating=8.4}]
2015-02-28 18:26:12.384  INFO 3616 --- [           main] main.BootElastic: Content of Rating Interval query is [Movie{id=2, name='The Princess Bride', genre=[Genre{name='ACTION'}, Genre{name='ROMANCE'}], rating=8.4}]

As you can see the interval query only retrieved Princess Bride. We did not do any configuration right? It is unusual. I have to share the huge configuration file with you:

spring.data.elasticsearch.cluster-nodes=localhost:9300
 # if spring data repository support is enabled
spring.data.elasticsearch.repositories.enabled=true

Normally you would use port 9200 when you query your elastic server. But when we programmatically reach it we are using 9300. If you have more than one node you would separate them with a comma and use 9301, 9302 etc as port numbers. Our pom file is no surprise either. Just elastic starter pom and we are set to go.

    4.0.0

    caught.co.nr
    boot-elastic-sample
    1.0-SNAPSHOT
    war

    
    
        org.springframework.boot
        spring-boot-starter-parent
        1.2.2.RELEASE
    

    
        
            org.springframework.boot
            spring-boot-starter-data-elasticsearch
        

    

    
    
        
            
                org.springframework.boot
                spring-boot-maven-plugin
            
        
    

As you can see thanks to Spring Boot and Data it is quite easy to work with elastic search. Lets check what we indexed from the server api as well. I'll use Sense -a chrome plug-in for elastic commands-.

Here's the result json:

{
   "took": 2,
   "timed_out": false,
   "_shards": {
      "total": 1,
      "successful": 1,
      "failed": 0
   },
   "hits": {
      "total": 2,
      "max_score": 1,
      "hits": [
         {
            "_index": "elastic_sample",
            "_type": "movie",
            "_id": "1",
            "_score": 1,
            "_source": {
               "id": 1,
               "name": "Star Wars",
               "genre": [
                  {
                     "name": "ACTION"
                  },
                  {
                     "name": "SCI_FI"
                  }
               ]
            }
         },
         {
            "_index": "elastic_sample",
            "_type": "movie",
            "_id": "2",
            "_score": 1,
            "_source": {
               "id": 2,
               "name": "The Princess Bride",
               "genre": [
                  {
                     "name": "ACTION"
                  },
                  {
                     "name": "ROMANCE"
                  }
               ]
            }
         }
      ]
   }
}

You can check out the whole project in the github.

access to dialectResolutionInfo cannot be null when 'hibernate.dialect' not set

While I was trying JPA stuff on Spring Boot I had a "Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set" exception. I was using "spring-boot-starter-data-jpa" and everything seemed to be in place but still I was getting this. Later I realized that I did not include my database's dependency. After I added it to my pom the problem was fixed.

  postgresql
  postgresql
  9.1-901.jdbc4

spring boot presentation

Yesterday I did a Spring Boot presentation at Sony Eurasia. Here are the slides:

spring social example on spring boot or how I stopped worrying and loved autoconfiguration

As of Spring Boot 1.1.0.RC1, autoconfiguration and the starter pom of Spring Social  is added which means that I won't have to add a hundred dependency to my pom and lots of meaningless Spring configuration will be handled for me. Let's see how it works on an example.

 I will implement a web application of two pages. One will show the given user's Twitter timeline and the other user's profile information. Here's my pom:

    4.0.0

    nr.co.caught
    BootTwitterJoy
    1.0-SNAPSHOT
    war

    
    
        org.springframework.boot
        spring-boot-starter-parent
        1.1.0.RC1
    

    
        
            org.springframework.boot
            spring-boot-starter-social-twitter
        

        
        
            org.apache.tomcat.embed
            tomcat-embed-jasper
        
        
            javax.servlet
            jstl
        

    

    
    
        
            
                org.springframework.boot
                spring-boot-maven-plugin
            
        
    

    
    
    
        
            spring-snapshots
            http://repo.spring.io/snapshot
            
                true
            
        
        
            spring-milestones
            http://repo.spring.io/milestone
        
    
    
        
            spring-snapshots
            http://repo.spring.io/snapshot
        
        
            spring-milestones
            http://repo.spring.io/milestone
        
    

As you can see, I have my starter-social-twitter dependency which gives me Spring Social and Web capabilities. I'll add jasper and jstl for my jsp pages to work. My repositories part is quite populated due to the milestone repositories.
Now we will add our Service to do Twitter method calls and a Controller for handling the requests. Our Controller is plain and simple:

@Controller
public class TwitterController {

    @Autowired
    private TwitterService twitterService;

 @RequestMapping(value = "/timeline/{twitterUser}")
 public String getUserTimeline(@PathVariable String twitterUser, Model model) {
        model.addAttribute("tweets", twitterService.getUserTimeline(twitterUser));
        model.addAttribute("user", twitterUser);

  return "timeline";
 }

    @RequestMapping(value = "/profile/{twitterUser}")
    public String getUserProfile(@PathVariable String twitterUser, Model model) {
        model.addAttribute("userProfile", twitterService.getUserProfile(twitterUser));

        return "profile";
    }
}

If the request comes with "/timeline/username", our controller will get the user timeline and if it comes with "/profile/username" it will get the user profile from TwitterService. Here's our TwitterService:

@Service
public class TwitterService {

   @Autowired
    private Twitter twitter;

    public List < Tweet > getUserTimeline(String twitterUser) {
        TimelineOperations timelineOps = twitter.timelineOperations();
        List tweets = timelineOps.getUserTimeline("@" + twitterUser);

        return tweets;
    }

    public TwitterProfile getUserProfile(String twitterUser) {
        UserOperations userOperations = twitter.userOperations();
        TwitterProfile userProfile = userOperations.getUserProfile(twitterUser);

        return userProfile;
    }
}

We have a Twitter object that'll be created thanks to Spring Boot's autoconfiguration. We just have to provide an app id and app secret key (a.k.a. consumer key and consumer secret) in our application properties and Boot will do the rest. I'm quoting Twitter object explanation from Spring javadoc: "This instance of TwitterTemplate is limited to only performing operations requiring client authorization. For instance, you can use it to search Twitter, but you cannot use it to post a status update. The client credentials given here are used to obtain a client access token via OAuth 2 Client Credentials Grant". If you try to do a status update, you'll get "org.springframework.social.MissingAuthorizationException: Authorization is required for the operation, but the API binding was created without authorization". For further Twitter functionality, we would need to provide access token and access token secret keys as well but as far as I know autoconfiguration would not handle these cases yet.
My JSPs:
profile.jsp:

<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<html>
<head>
    <title></title>
</head>
<body>
<img src="${userProfile.profileImageUrl}"/>  

Screen name: ${userProfile.screenName} 

Name: ${userProfile.name}      

Description: ${userProfile.description}   

Location:  ${userProfile.location}  

Followers: ${userProfile.followersCount}  

</body>
</html>

As you can see, profile takes the userProfile provided by our Controller and show the basic profile properties. timeline.jsp

<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<html>
<head>
    <title>Time Line for <c:out value="${twitterUser}" /> TimeLine</title>
</head>
<body>
<ul>
    <c:forEach items="${tweets}" var="tweet">
        <li>${tweet.text}
 at <c:out value="${tweet.createdAt}"/></li>
        <br/>
    </c:forEach>
</ul>
</body>
</html>

Tweets are shown with their text and creation date. My application.properties content:

# Config for JSPs
spring.view.prefix: /WEB-INF/jsp/
spring.view.suffix: .jsp

# SPRING SOCIAL TWITTER (TwitterAutoConfiguration)
spring.social.twitter.appId= someAppId
spring.social.twitter.appSecret= someSecretId

spring.view properties are for the jsp handling. spring.social.twitter properties can be obtained from http://dev.twitter.com. Just login there with your twitter account, create your app and get your api keys. Here's the result:

You can check the code at github.

exception after main class package change in spring boot

In my Spring Boot toy project ( web + mongodb), I want to change the structure of my packages. My old structure was like this:

Putting @ComponentScan annotation (without a base package) was fine for scanning all my classes. But when I update the structure and moved my main file (BootQeyfi) to launch package, I needed to update base package with
@ComponentScan(basePackages = "main") . After that definition Spring will scan the main as a base package and handle everything perfectly.

But of course that's not the case. I got the following exception which basically means "I could not autowire your repository class (your interface which extends MongoRepository)" 

Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'helloController': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private main.service.HelloService main.controller.HelloController.service; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'helloService': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: main.dao.ProductRepository main.service.HelloService.repository; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [main.dao.ProductRepository] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
 at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:292)
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1185)
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
 at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:304)
 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:300)
 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:195)
 at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:703)
 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)
 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
 at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:120)
 at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:648)
 at org.springframework.boot.SpringApplication.run(SpringApplication.java:311)
 at launch.BootQeyfi.main(BootQeyfi.java:27)
Caused by: org.springframework.beans.factory.BeanCreationException: Could not autowire field: private main.service.HelloService main.controller.HelloController.service; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'helloService': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: main.dao.ProductRepository main.service.HelloService.repository; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [main.dao.ProductRepository] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
 at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:508)
 at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:87)
 at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:289)
 ... 14 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'helloService': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: main.dao.ProductRepository main.service.HelloService.repository; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [main.dao.ProductRepository] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
 at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:292)
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1185)
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
 at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:304)
 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:300)
 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:195)
 at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1017)
 at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:960)
 at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:858)
 at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:480)
 ... 16 more
Caused by: org.springframework.beans.factory.BeanCreationException: Could not autowire field: main.dao.ProductRepository main.service.HelloService.repository; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [main.dao.ProductRepository] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
 at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:508)
 at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:87)
 at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:289)
 ... 27 more
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [main.dao.ProductRepository] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
 at org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoSuchBeanDefinitionException(DefaultListableBeanFactory.java:1103)
 at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:963)
 at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:858)
 at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:480)
 ... 29 more

To fix this problem, @EnableMongoRepositories must be added to your main class. My repository is under "main.dao" so my definition is like @EnableMongoRepositories(basePackages = "main.dao").

rocking with mongodb on spring boot

I'm a fan of Spring Boot and here's my mongodb example project on Spring Boot. Most of the mongodb example projects are so basic that you won't go far with them. You can search for plain Spring Data examples but they can get much complex than you'd like. So here's mine.

Here's the pom I'll use.

    4.0.0

    caught.co.nr
    boottoymongodb
    1.0-SNAPSHOT
    war


    
    
        org.springframework.boot
        spring-boot-starter-parent
        1.0.0.BUILD-SNAPSHOT
    

    
        
            org.springframework.boot
            spring-boot-starter-data-mongodb
        

    

    
    
        
            
                org.springframework.boot
                spring-boot-maven-plugin
            
        
    

    
        
            spring-snapshots
            Spring Snapshots
            http://repo.spring.io/snapshot
            
                true
            
        
    

    
        
            spring-snapshots
            http://repo.spring.io/snapshot
        
    

The only dependency I need is "spring-boot-starter-data-mongodb" which contains all necessary dependencies for a spring boot mongodb project. Next is the model for my collection. Document annotation points to my collection named "products". It is need only if your model name does not match your collection name. You can see a field annotation which maps the field name in the collection to the model's field name.

@Document(collection = "products")
public class Product {
    @Id
    private String id;
    private String sku;

    @Field(value = "material_name")
    private String materialName;

    private Double price;
    private Integer availability;


    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public String getSku() {
        return sku;
    }

    public void setSku(String sku) {
        this.sku = sku;
    }

    public String getMaterialName() {
        return materialName;
    }

    public void setMaterialName(String materialName) {
        this.materialName = materialName;
    }

    public Double getPrice() {
        return price;
    }

    public void setPrice(Double price) {
        this.price = price;
    }

    public Integer getAvailability() {
        return availability;
    }

    public void setAvailability(Integer availability) {
        this.availability = availability;
    }

    @Override
    public String toString() {
        return "Product{" +
                "id='" + id + '\'' +
                ", sku='" + sku + '\'' +
                ", materialName='" + materialName + '\'' +
                ", price=" + price +
                ", availability=" + availability +
                '}';
    }
}

Not we will need a DAO layer to manipulate my data. MongoRepository is the interface I should implement if I want to use autogenerated find methods in my DAO layer and I want that. Every field of my model can be queried with these autogenerated methods. For a complete list of method name syntax check here. My query below will take a sku name and search my collection for this name and return the matching ones.

public interface ProductRepository extends MongoRepository < Product, String >{
    public List < Product > findBySku(String sku);
}

Now I'll introduce a Service which will call my DAO interface. But wait a minute, I didn't implement this interface and wrote necessary code for fetching the models right? Yep, these methods are autogenerated and I don't need an implementation for this interface.

@Service
public class ProductService {
    @Autowired
    private ProductRepository repository;

    public List < Product > getSku(String sku){
        return repository.findBySku(sku);
    }
}

Next, lets launch our Boot example. Here's our main class:

@Configuration
@EnableAutoConfiguration
@ComponentScan
public class BootMongoDB implements CommandLineRunner {

    @Autowired
    private ProductService productService;

    private static final Logger logger = LoggerFactory.getLogger(BootMongoDB.class);

    public void run(String... args) throws Exception {
        List < Product > sku = productService.getSku("NEX.6");
        logger.info("result of getSku is {}", sku);
    }

    public static void main(String[] args) throws Exception {
        SpringApplication.run(BootMongoDB.class, args);
    }
}

If you have a connection to a mongodb instance and a sku matching to the name you searched than you should see one or more Products as a result. What we did was quite basic. What if I want more complex queries? For instance if I want a specific sku with an availability equal to "1"? I can't do it without using some @Query magic. So I'm updating my DAO class.

public interface ProductRepository extends MongoRepository < Product, String >{
    public List < Product > findBySku(String sku);

    @Query(value = "{sku: ?0, availability : 1}")
    public List < Product > findBySkuOnlyAvailables(String sku);
}

I provided a direct query for mongodb where sku in the signature of my method will be inserted to "?0" in the query and will be sent to mongodb. You can update your Service and then your main method to see if it works. You may not like writing queries which are not much readable if you're not very familiar with mongodb's syntax. Then this is the time for adding custom DAO classes. It's not possible to add and use methods other than the autogenerated ones to ProductRepository. So we will add few classes and have a nice featured methods. Our repository class was named "ProductRepository". We will add a new interface named "ProductRepositoryCustom" and a new method which will find available skus for the given name (twin of findBySkuOnlyAvailables method).

public interface ProductRepositoryCustom {
    public List < Product > findBySkuOnlyAvailablesCustom(String sku);
}
 

Then provide an implementation for this. Below you see that we inject ProductRepositoryCustom's mongotemplate and do stuff with it. We create two criteria. First one is for the sku name and the second one is for availability.

public class ProductRepositoryImpl implements ProductRepositoryCustom {
    @Autowired
    private MongoTemplate mongoTemplate;

    public List < Product > findBySkuOnlyAvailablesCustom(String sku) {
        Criteria criteria = Criteria.where("sku").is(sku).
andOperator(Criteria.where("availability").is(1));
        return mongoTemplate.find(Query.query(criteria), Product.class);
    }
}

The last step for custom implemetation is the update of ProductRepository class. As you can see below the only update I need is the addition of my ProductRepositoryCustom so we can link both of them together. All this naming can sound a little stupid. But notice that although the name of your custom interface is not important, a change in the name of the implementation will result in the throw of an exception:

Invocation of init method failed; nested exception is org.springframework.data.mapping.PropertyReferenceException: No property only found for type String! Traversed path: Product.sku.

To fix this make sure that the name of your implementation class is "ProductRepositoryImpl" which is the concatenation of the name of the interface that extends MongoRepository and "Impl".

public interface ProductRepository extends MongoRepository < Product, String>, ProductRepositoryCustom

If we add our new method to our Service layer:

@Service
public class ProductService {
    @Autowired
    private ProductRepository repository;

    public List < Product > getSku(String sku){
        return repository.findBySku(sku);
    }

    public List < Product > getAvailableSkuCustom(String sku){
        return repository.findBySkuOnlyAvailablesCustom(sku);
    }
}

Then update our main class' run method:

   public void run(String... args) throws Exception {
        List < Product > sku = productService.getSku("NEX.6");
        logger.info("result of getSku is {}", sku);

        List < Product > availableSkuCustom = productService.getAvailableSkuCustom("NEX.6");
        logger.info("result of availableSkuCustom is {}", availableSkuCustom);
    }

Again you must see something in the log :). You can check the whole project on github.

how to build your web application on spring boot and deploy it on heroku

Why would you need a tutorial for building your web application on spring boot and deploying it on heroku?
That's because our web application will serve JSP files and as Spring Boot's JSP support is limited we will have to use war files for deployment and it can get a bit tricky to deploy it on heroku. Furthermore, as the web application we will deploy is super concise you can use it on your own project by just editing it.

Our controller is quite basic. It will take the name of the user from url and print it on the next page. So calling "/hello/sezin" will print on "hello sezin" on the page.

package main.controller;

import org.springframework.stereotype.Controller;
import org.springframework.ui.ModelMap;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;

/**
 * @author sezin karli (skarligmail.com)
 * @since 3/19/14 3:42 PM
 *        User: Sezin Karli
 */
@Controller
@RequestMapping(value = "/hello")
public class HelloController {

    @RequestMapping(value = "/{user}")
    public String handleOne(@PathVariable String user, ModelMap modelMap){
        String helloToken = "Hello " + user;
        modelMap.put("token", helloToken);
        return "welcome-page";
    }
}

Our main Spring class is below. PORT attribute is taken from environment variables if it can't be found 8080 is used. This part is a requirement for Heroku.

package main;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;

/**
 * @author sezin karli (skarligmail.com)
 * @since 3/19/14 9:26 AM
 *        User: Sezin Karli
 */
@Configuration
@EnableAutoConfiguration
@ComponentScan
public class ToyProjectLauncher {

    public static void main(String[] args) throws Exception {
        String webPort = System.getenv("PORT");
        if (webPort == null || webPort.isEmpty()) {
            webPort = "8080";
        }
        System.setProperty("server.port", webPort);

        SpringApplication.run(ToyProjectLauncher.class, args);
    }
}

Our application properties. We need to show a folder for jsp files and if we don't want to print ".jsp" suffix for every return value in the controller we will also need the suffix parameter below.

server.port: ${port:8080}
spring.view.prefix: /WEB-INF/jsp/
spring.view.suffix: .jsp

Our jsp is as follows. We just print the token we get from the controller.

<%@ page language="java" contentType="text/html; charset=US-ASCII"
         pageEncoding="US-ASCII"%>



    
    
    <%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
    

If we analyze the pom below you can see that we have a spring boot parent which's needed to inherit defaults from spring boot. Then we add starter-web dependency because it will add web dependency template. Jasper and jstl dependencies are needed for JSPs to work. We would need spring-boot-maven-plugin if we want a fat jar which makes it easier to deploy.

    4.0.0

    nr.co.caught
    boot-toy-project
    1.0-SNAPSHOT
    war


    
    
        org.springframework.boot
        spring-boot-starter-parent
        1.0.0.BUILD-SNAPSHOT
    

    
        
            org.springframework.boot
            spring-boot-starter-web
        

        
        
            org.apache.tomcat.embed
            tomcat-embed-jasper
        
        
            javax.servlet
            jstl
        

    

    
    
        
            
                org.springframework.boot
                spring-boot-maven-plugin
            
        
    

    
        
            spring-snapshots
            Spring Snapshots
            http://repo.spring.io/snapshot
            
                true
            
        
    

    
        
            spring-snapshots
            http://repo.spring.io/snapshot
        
    

You can check GitHub if you need further details with the project. If everyting's fine you should be able to build it with "mvn package".
You must see your war file under target folder. Why a war file? Because Spring Boot supports JSPs only if the project package is war.
 Lets run our code by going under "target" and doing "java -jar boot-toy-project-1.0-SNAPSHOT.war". You should see your hello page if you type "localhost/hello/sezin".

Now we will continue by deploying it on heroku. I assume you already have a heroku user and heroku toolbelt already installed.

Open a command prompt and login to heroku by typing "heroku login". Type your heroku email and password. It says "could not find an existing public key. We try to generate one but get a "could not generate key: 'ssh-keygen' is not recognized" error (under windows 7).

There seems to be a problem.
Open a git bash and lets create an ssh key with 'ssh-keygen -t rsa -C "yourEmail"' command.

Lets login again and here we get "authentication successful".

After that we will try to push and deploy our Spring Boot Project.
Create a file named "Procfile" with the information below under our project folder

 web: java $JAVA_OPTS -jar target/boot-toy-project-1.0-SNAPSHOT.war

There's another way to deploy war files to heroku (heroku deploy:war), but I didn't manage to run my application with it so I'm teaching you what worked for me. :)

Now, create a project with "heroku create" command. We will get "Git remote heroku added".

Open git bash and enter "git push heroku master" under your project directory. Now we will push our code into heroku git and heroku will deploy it to our server.

The last line will show you on which address your site is available. Mine was "http://pure-shelf-8719.herokuapp.com/". I add few strings to test my hello page and go to "http://pure-shelf-8719.herokuapp.com/hello/world". You can see the awesome result below.

mongodb default configuration in spring boot

I'm trying to integrate mongodb to my spring boot application. I create a db aptly named "mydb" and was wondering how mongodb classes were able to connect to my database without any configuration and why they were not retrieving my data. Answer to both is of course "default configuration" for mongodb.

You can see mongodb defaults that spring boot uses:

host: localhost
port: 27017
db: test

My mongodb is at localhost:27017, but as I'm using "mydb" instead of "test" no data is retrieved. So I have to override the default configuration by adding

spring.data.mongodb.uri=mongodb://localhost:27017/mydb

to my application.properties file.

problem with displaying jsp on spring boot

I'm currently trying to learn Spring Boot (1.0.0 snapshot) in my spare time. I try to use spring-boot-starter-web for building a sample website but had problems when I try to use jsp for the view part of my mvc. Everything was in place.

• I had a jsp named "welcome-page.jsp" under "src/main/webapp/WEB-INF/jsp/" 
• In my controller I was sending the user to "welcome-page". 
• The path configuration was done under application.properties with
• spring.view.prefix: /WEB-INF/jsp/
• spring.view.suffix: .jsp

And yet it was not working. Nothing on logs too, even on debug level.
After some search and several trial/errors, I saw that it is impossible to use embedded tomcat for displaying jsps without adding several dependencies to my main pom. Next to the standard spring-boot-starter-web dependency, I had to add jasper and jstl dependencies as well. So long for leaving dependencies to starter templates.

   
 
        
            org.springframework.boot
            spring-boot-starter-web
        
        
            org.apache.tomcat.embed
            tomcat-embed-jasper
            provided
        
        
            javax.servlet
            jstl