elasticsearch for beginners part 1: how to create a document

In this Elasticsearch for beginners series, I will give you basics of Elasticsearch with real world examples. The first part is for teaching you how to create -that is index- a document in Elasticsearch.

I assume you managed to download the latest Elasticsearch zip to your local, extracted it and after that installed the marvel plugin and started a new node by running "elasticsearch" under your installation's bin directory.

Now it is time for some action. We can choose to give custom id to our document or leave it to Elasticsearch so we won't bother by generating a new one.

You can choose whatever content you want, but for keeping it simple I'll consider the usual user example. Our user will have a  username, first name, last name and a contact phone.

 {
   "username" : "kimchy",
   "fist_name" : "kimchy",
   "last_name" : "aloevera",
   "phone" : "00905332446578"
 }

Now lets open our Sense plugin. Go to "http://localhost:9200/_plugin/marvel/sense/index.html" after your node is up and running.

Before indexing anything I'll tell you that we will need an index and a type information.
Index is like a database and type is like a table if we use  RDBMS analogies.

Please don't get confused over the meaning of index. Indexing a document is creating a document, index information of a document is its database.

Lets insert our first document with custom id. We will use "elastic_101" as index and "user" as type.

PUT /elastic_101/user/1
 {
   "username" : "kimchy",
   "fist_name" : "kimchy",
   "last_name" : "aloevera",
   "phone" : "00905332446578"
 }

We can check if we are successful in indexing it.

The result to the following query

GET elastic_101/user/_search

is

{
   "took": 16,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 1,
      "max_score": 1,
      "hits": [
         {
            "_index": "elastic_101",
            "_type": "user",
            "_id": "1",
            "_score": 1,
            "_source": {
               "username": "kimchy",
               "fist_name": "kimchy",
               "last_name": "aloevera",
               "phone": "00905332446578"
            }
         }
      ]
   }
}

Now lets index a new document with autogenerated id.

POST /elastic_101/user/
 {
   "username" : "sezinkarli",
   "fist_name" : "sezin",
   "last_name" : "karli",
   "phone" : "00905322426528"
 }

Again let me check the result with

GET elastic_101/user/_search

Result is:

 
{
   "took": 40,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 2,
      "max_score": 1,
      "hits": [
         {
            "_index": "elastic_101",
            "_type": "user",
            "_id": "AVAFxb36kyTg7_oSOIVN",
            "_score": 1,
            "_source": {
               "username": "sezinkarli",
               "fist_name": "sezin",
               "last_name": "karli",
               "phone": "00905322426528"
            }
         },
         {
            "_index": "elastic_101",
            "_type": "user",
            "_id": "1",
            "_score": 1,
            "_source": {
               "username": "kimchy",
               "fist_name": "kimchy",
               "last_name": "aloevera",
               "phone": "00905332446578"
            }
         }
      ]
   }
}

As you can see both users have a username, first name, last name and a phone. Notice that if for kimchy is 1 which is the one we gave it and the other id is "AVAFxb36kyTg7_oSOIVN" which is generated by Elasticsearch.

In this part of my Elasticsearch tutorial, we learnt how to create documents with custom or auto-generated id. You can have more in depth information in my Elasticsearch in Action course and this link will make a discount of 60%.

spring annotations i never had the chance to use part 1: @primary

Today I remembered an old friend of mine (@primary) with whom we met from tutorials to tutorials. You know that in Spring @Autowired annotation works by type, that is if Spring finds an eligible bean that matches in terms of type it will inject it in. Let's see it on an example.

Assume I have two Singer classes; OperaSinger and MetalSinger.

@Component
public class MetalSinger implements Singer{

    @Override
    public String sing(String lyrics) {
        return "I am singing with DIO voice: "+lyrics;
    }
}

public class OperaSinger implements Singer {
    @Override
    public String sing(String lyrics) {
        return "I am singing in Bocelli voice: "+lyrics;
    }
}

They both implement the Singer interface.

public interface Singer {
    String sing(String lyrics);
}

And lets define a SingerService and inject the Singer bean inside.

@Component
public class SingerService {
    private static final Logger logger = LoggerFactory.getLogger(SingerService.class);

    @Autowired
    private Singer singer;

    public String sing(){
        return singer.sing("song lyrics");
    }
}

What do you think; which Singer will be injected inside? Here's the result:
 I am singing with DIO voice: song lyrics.

This is because OperaSinger is not defined as Component or Service so Spring does not have a clue about it. If we add @Component annotion to it:

@Component
public class OperaSinger implements Singer {
    @Override
    public String sing(String lyrics) {
        return "I am singing in Bocelli voice: "+lyrics;
    }
}

Than I'll get this exception:

 org.springframework.beans.factory.NoUniqueBeanDefinitionException: No qualifying bean of type [main.service.Singer] is defined: expected single matching bean but found 2: metalSinger,operaSinger

The reason is quite plain to see. If I have more than one bean with same type, and if I use @Autowired annotion which binds type I'll have this exception. Spring does not have a clue which Singer it should use.

Let's favor a music genre and tell Spring to use OperaSinger as Primary.

@Primary
@Component
public class OperaSinger implements Singer{

    @Override
    public String sing(String lyrics) {
        return "I am singing in Bocelli voice: "+lyrics;
    }
}

If we do the SingerService call we will get:

"I am singing in Bocelli voice: song lyrics"

That is because we choose OperaSinger as Primary which means "if you get confused with types you better use this one". Another approach would be the use of qualifier names which directly maps names to beans.

elastic search crash course

Check this Elasticsearch crash course video out. It's not only for newbies but also for people with Elasticsearch experience.

custom domain with github pages

I recently bought a domain name from 1and1.com. I was planning to host my static files on github pages. After I checked my page on github I saw that responsive design works. But when I do the config (on 1and1) for forwarding my domain name (sezinkarli.com) to my github page (sezinkarli.github.io), I saw that responsiveness is gone. Nothing changes if I change the screen size. After a little bit of google search, I learnt that I'll need a proper redirect for this to work. There are several steps for using your custom subdomain (www.sezinkarli.com in my case) for your github page:

1. Add a file named "CNAME" to your root in your github page. Its content must be your subdomain url. "www.sezinkarli.com" in my case.
2. Define a subdomain for your domain. www.sezinkarli.com in my case.
3. Configure a CNAME in your subdomain's DNS and point it to your github url (sezinkarli.github.io in my case).

Let me show you these settings ((2) and (3)) for my domain company.

First create s subdomain.

Choose a prefix for your subdomain.

Click on your newly created subdomain.

Edit your subdomains DNS settings.

Choose to add a CNAME and put your github pages url as alias.

spring boot presentation for ozgur yazilim ve linux gunleri 2015

I did a Spring Boot Workshop at Ozgur Yazilim ve Linux Gunleri 2015 (Open-source Software and Linux Days for Turkish-free people). You can view the presentation below.

head first elastic search on java with spring boot and data features

In this article I'll try to give you an easy introduction on how to use Elastic Search in a Java project. As Spring Boot is the easiest and fastest way to begin our project I choose to use it. Futhermore, we will heavily use Repository goodies of beloved Spring Data.

Let's begin by installing Elastic Search on our machine and run our elastic server for the first time.

I go to elastic-folder\bin and run elasticsearch.bat (yeah I'm using Windows) but no luck. I get this:

"Error occurred during initialization of VM

Could not reserve enough space for object heap

Error: Could not create the Java Virtual Machine.

Error: A fatal exception has occurred. Program will exit."

What a great start!

In my bin folder there's a "elasticsearch.in.bat" file. I set ES_MAX_MEM=1g to 

ES_MAX_MEM=512mb and voila it is fixed.

I start a new server without problem after that.

Now it is time to define the document we will index in elastic search. Assume we have movie information to index. Our model is quite straightforward. Movie has a name, rating and a genre in it. I chose "elastic_sample" as index name which sounds good as a database name and "movie" as type which is good for a table name if we think in relational database terms. Nothing fancy in the model as you can see.

@Document(indexName = "elastic_sample", type = "movie")
public class Movie {

    @Id
    private String id;

    private String name;

    @Field(type = FieldType.Nested)
    private List &lt Genre &gt  genre;

    private Double rating;

    public Double getRating() {
        return rating;
    }

    public void setRating(Double rating) {
        this.rating = rating;
    }

    public void setId(String id) {
        this.id = id;
    }

    public List &lt Genre &gt  getGenre() {
        return genre;
    }

    public void setGenre(List &lt Genre &gt  genre) {
        this.genre = genre;
    }

    public String getId() {
        return id;
    }

    public String getName() {
        return name;

    }

    public void setName(String name) {
        this.name = name;
    }

    @Override
    public String toString() {
        return "Movie{" +
                "id=" + id +
                ", name='" + name + '\'' +
                ", genre=" + genre +
                ", rating=" + rating +
                '}';
    }
}

For those who wonder what Genre is here is it. Just a POJO.

public class Genre {
    private String name;

    public Genre() {
    }

    public Genre(String name) {
        this.name = name;
    }

    public String getName() {
        return name;
    }

    @Override
    public String toString() {
        return "Genre{" +
                "name='" + name + '\'' +
                '}';
    }

    public void setName(String name) {
        this.name = name;
    }
}

Not it is time to create DAO layer so we can save and load our document to/from our elastic search server. Our Repository extends the classic ElasticserchRepository (no idea why it is search and not Search). As you probably know Spring Data can query one or more fields with these predefined methods where we use our field names. findByName will search in the name field, findByRating will search in the rating field so on so forth. Furthermore thanks to Spring Data we don't need to write implementation for it, we just put method names in the interface and that's finished.

public interface MovieRepository extends ElasticsearchRepository &lt Movie, Long &gt {
    public List &lt Movie &gt  findByName(String name);

    public List &lt Movie&gt  findByRatingBetween(Double beginning, Double end);
}

Our DAO layer will be called by a Service layer:

@Service
public class MovieService {

    @Autowired
    private MovieRepository repository;

    public List &lt Movie &gt  getByName(String name) {
        return repository.findByName(name);
    }

    public List &lt Movie &gt  getByRatingInterval(Double beginning, Double end) {
        return repository.findByRatingBetween(beginning, end);
    }

    public void addMovie(Movie movie) {
        repository.save(movie);
    }
}

Here is the main Class we will use to run our application. EnableAutoConfiguration will auto-configure everything it recognizes under our classpath. ComponentScan will scan for Spring annotations under the main Class' directory.

@Configuration
@EnableAutoConfiguration
@ComponentScan
public class BootElastic implements CommandLineRunner {

    @Autowired
    private MovieService movieService;

    private static final Logger logger = LoggerFactory.getLogger(BootElastic.class);

// add star wars and
// princess bride as a movie
// to elastic search
    private void addSomeMovies() {
        Movie starWars = getFirstMovie();
        movieService.addMovie(starWars);

        Movie princessBride = getSecondMovie();
        movieService.addMovie(princessBride);
    }

    private Movie getSecondMovie() {
        Movie secondMovie = new Movie();
        secondMovie.setId("2");
        secondMovie.setRating(8.4d);
        secondMovie.setName("The Princess Bride");

        List &lt Genre &gt  princessPrideGenre = new ArrayList &lt Genre &gt();
        princessPrideGenre.add(new Genre("ACTION"));
        princessPrideGenre.add(new Genre("ROMANCE"));
        secondMovie.setGenre(princessPrideGenre);

        return secondMovie;
    }


    private Movie getFirstMovie() {
        Movie firstMovie = new Movie();
        firstMovie.setId("1");
        firstMovie.setRating(9.6d);
        firstMovie.setName("Star Wars");

        List &lt Genre &gt  starWarsGenre = new ArrayList &lt Genre &gt();
        starWarsGenre.add(new Genre("ACTION"));
        starWarsGenre.add(new Genre("SCI_FI"));
        firstMovie.setGenre(starWarsGenre);

        return firstMovie;
    }

    public void run(String... args) throws Exception {
        addSomeMovies();
        // We indexed star wars and pricess bride to our movie
        // listing in elastic search

        //Lets query if we have a movie with Star Wars as name
        List &lt Movie &gt starWarsNameQuery = movieService.getByName("Star Wars");
        logger.info("Content of star wars name query is {}", starWarsNameQuery);

        //Lets query if we have a movie with The Princess Bride as name
        List &lt Movie &gt  brideQuery = movieService.getByName("The Princess Bride");
        logger.info("Content of princess bride name query is {}", brideQuery);


        //Lets query if we have a movie with rating between 6 and 9
        List &lt Movie &gt  byRatingInterval = movieService.getByRatingInterval(6d, 9d);
        logger.info("Content of Rating Interval query is {}", byRatingInterval);
    }

    public static void main(String[] args) throws Exception {
        SpringApplication.run(BootElastic.class, args);
    }
}

If we run it the result is:

015-02-28 18:26:12.368  INFO 3616 --- [           main] main.BootElastic: Content of star wars name query is [Movie{id=1, name='Star Wars', genre=[Genre{name='ACTION'}, Genre{name='SCI_FI'}], rating=9.6}]
2015-02-28 18:26:12.373  INFO 3616 --- [           main] main.BootElastic: Content of princess bride name query is [Movie{id=2, name='The Princess Bride', genre=[Genre{name='ACTION'}, Genre{name='ROMANCE'}], rating=8.4}]
2015-02-28 18:26:12.384  INFO 3616 --- [           main] main.BootElastic: Content of Rating Interval query is [Movie{id=2, name='The Princess Bride', genre=[Genre{name='ACTION'}, Genre{name='ROMANCE'}], rating=8.4}]

As you can see the interval query only retrieved Princess Bride. We did not do any configuration right? It is unusual. I have to share the huge configuration file with you:

spring.data.elasticsearch.cluster-nodes=localhost:9300
 # if spring data repository support is enabled
spring.data.elasticsearch.repositories.enabled=true

Normally you would use port 9200 when you query your elastic server. But when we programmatically reach it we are using 9300. If you have more than one node you would separate them with a comma and use 9301, 9302 etc as port numbers. Our pom file is no surprise either. Just elastic starter pom and we are set to go.

    4.0.0

    caught.co.nr
    boot-elastic-sample
    1.0-SNAPSHOT
    war

    
    
        org.springframework.boot
        spring-boot-starter-parent
        1.2.2.RELEASE
    

    
        
            org.springframework.boot
            spring-boot-starter-data-elasticsearch
        

    

    
    
        
            
                org.springframework.boot
                spring-boot-maven-plugin
            
        
    

As you can see thanks to Spring Boot and Data it is quite easy to work with elastic search. Lets check what we indexed from the server api as well. I'll use Sense -a chrome plug-in for elastic commands-.

Here's the result json:

{
   "took": 2,
   "timed_out": false,
   "_shards": {
      "total": 1,
      "successful": 1,
      "failed": 0
   },
   "hits": {
      "total": 2,
      "max_score": 1,
      "hits": [
         {
            "_index": "elastic_sample",
            "_type": "movie",
            "_id": "1",
            "_score": 1,
            "_source": {
               "id": 1,
               "name": "Star Wars",
               "genre": [
                  {
                     "name": "ACTION"
                  },
                  {
                     "name": "SCI_FI"
                  }
               ]
            }
         },
         {
            "_index": "elastic_sample",
            "_type": "movie",
            "_id": "2",
            "_score": 1,
            "_source": {
               "id": 2,
               "name": "The Princess Bride",
               "genre": [
                  {
                     "name": "ACTION"
                  },
                  {
                     "name": "ROMANCE"
                  }
               ]
            }
         }
      ]
   }
}

You can check out the whole project in the github.