Saturday, October 24, 2020

docker tutorial part 2 - the good the bad and the ip

In this second part of docker tutorial, I'll explain basics of networking in docker with stand-alone containers. Remember from the previous part of the tutorial where we mapped port of our host machine and the container? In this part, we will setup our containers so they can communicate with each other and for that we don't need to do an explicit mapping like we previously did with -p subcommand.

Containers have their own virtual network when created. Let's begin by checking the ip of our host machine and then container ip.

I'm on a mac so I'll do ifconfig:
ifconfig |grep "inet "
gives me
inet 192.168.178.164 netmask 0xffffff00 broadcast 192.168.178.255
Ok so my host machine ip is 192.168.178.164. Now let's see the docker container one.
docker container run -t -d --name alpine1 alpine
Notice that I used --name parameter so I can call my container with this name instead of its alphanumeric id.
docker container ls
will return us our container. Yep it's running. Now let's check container's ip.
docker container inspect alpine1
will return a huge list of properties related to container. If you feel on the adventurous side, you can also try format and get less data.
docker container inspect --format "{{.NetworkSettings.IPAddress}}" alpine1
In my case I got "172.17.0.2" as ip. As you can see it's different than my host machine's ip. The reason is that containers use their own virtual network. Can we check all these virtual networks? We are talking about them but I'm not really convinced if I don't see them.
docker network ls
This command will return virtual networks of docker containers. "bridge" is the default one. Which containers is using this virtual network then?
docker network inspect bridge
Under containers part you'll see our alpine1.
 "Containers": {
            "2b8b07d5062743fc4ffe6dc8557adb25948ae3385d66052b9f254fa60ba52c6d": {
                "Name": "alpine1",
                "EndpointID": "5ce2ea615f9022ac6f1cae1af4fa44d1058f79dd73994a96a2d15305b3539f7f",
                "MacAddress": "02:42:ac:11:00:02",
                "IPv4Address": "172.17.0.2/16",
                "IPv6Address": ""
            }
        }
What about the promise of containers that can communicate with each other? Let's create a new custom virtual network and connect several alpine containers together.
docker network create my_custom_network
Let's check if my_customer_network is there:
NETWORK ID          NAME                     DRIVER              SCOPE
3ad04560bc37        bridge                   bridge              local
905cd858e5a6        host                     host                local
cba2a990c957        my_custom_network        bridge              local
Yep now let's attach the existing container to this virtual network.
docker network connect my_custom_network alpine1
If you want to check what's bound to this network:
docker network inspect my_custom_network
You'll see that it's successful. 
Now let's get crazy and run a container with a customer virtual network.
docker container run -t -d --name alpine2 --network my_custom_network alpine
If you do network inspect you'll see alpine1 and alpine2 as containers. 
Now it's time for communication between containers. Let's execute bash on them (or rather ash) and make them ping each other.
docker container exec -it alpine1 ash
/ # ping -c 3 alpine2
PING alpine2 (172.22.0.3): 56 data bytes
64 bytes from 172.22.0.3: seq=0 ttl=64 time=0.117 ms
64 bytes from 172.22.0.3: seq=1 ttl=64 time=0.153 ms
64 bytes from 172.22.0.3: seq=2 ttl=64 time=0.155 ms
Nice! alpine1 was able to ping alpine2! But how? Yes same virtual network and stuff but how do they know each others ip? How the container name gets resolved to an ip? It's thanks to automatic dns resolution. Containers can resolve each other based on their container name regardless of their ip address on the same virtual network. 

To recap, in this part of the course we learnt how to create a virtual network and attach our containers to it so they can communicate with each other. Also we learnt how to check container ip and why it is different than host ip.

Tuesday, October 13, 2020

docker tutorial part 1 - fistfull of container

In this docker tutorial series, my aim is to explain basics of docker and to do that with min number of words but maximum number of examples. Before doing anything I want you to install docker on your machine and I hope you have an idea what docker does. If not please check docker.com.

Now let me explain two basic things related to docker:

image: File to be used in docker container. You may have a docker image of mysql for instance. It will contain an operating system, and whatever is needed to make mysql run.

container: A running instance of an image. So if you have an image of mysql, you can use it and run mysql on docker.

Now let's see why we need these two.

First open a command prompt or terminal and write

docker version


This should give you details of your docker server and client similar to


Client: Docker Engine - Community

....

Server: Docker Engine - Community

...


This means we installed docker successfully and good to go.

Now let's download an image and start a container using it. I'll use nginx which's a web server + load balancer and perfect for my simple examples.

docker container run nginx

which gives me this

/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration

/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/

/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh

10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf

10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf

/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh

/docker-entrypoint.sh: Configuration complete; ready for start up


Seems to be working right? Your docker first checks your local repo for images, if you don't have this image it checks docker hub and download the latest version of this image because you did not specify it.

Let's open our browser and type localhost. It's a web server so there should be something in localhost:80 right?

Hmm that does not seem right. Maybe we did something wrong. Did we bind container's port to our machine? I think we missed that.

docker container run --publish 1234:80 nginx


But which one's which? Left is your host machine, right is the container port.

Now let's check our browser by going to http://localhost:1234/.

Nice! I don't want to have a terminal window for every container I run so I hope engineers in docker find a solution to that.

docker container run --publish 1234:80 --detach nginx

66f1accc69854a254da58c009aa7cdbd3808036533847ece598190683a677035


Cool! I even have a huge alphanumeric thing. Also nginx is working on port 1234!

Let's check which containers are running right now:

docker container ls


Wait a minute that container id is similar to the one I got in the previous step! Also I can see my port binding under ports. Awesome!

What if I want to check some logs? 

docker container logs 66f

will return nginx logs of my container. I did not write the whole id but docker's super-smart as you can see. I have logs but this does not work like tail -f. Then try

docker container logs -f 66f


Then go to localhost:1234 for a few times and see the access log scrolling.

Let's stop the container before saying goodbye.

docker container stop 66f


Ok. It's stopped I guess? Should we check it?

docker container ls


Nothing's there, that's good news. You can also see your container cemetery:

docker container ls -a


In this part of my docker tutorial, you learnt two basics concepts related to docker: container and image.

Also you learnt how to run containers, to check running container logs, to stop containers. 


Tuesday, September 8, 2020

best intellij idea plug-ins

I've been using Intellij Idea as IDE for some time and wanted to share my favorite plug-ins as I think they can be quite useful for every developer out there. Without a specific order here they are:

Key Promoter X: Although its name's like cheap midi controller, it is a really useful plug-in for those who want to master intellij idea shortcuts. It shows notifications for almost each click you do and tells you its shortcuts. Also it holds statistics for you, so you have a list of most popular clicks and their corresponding keyboard shortcuts.

Lombok: This is the only plug-in in my selection that's specific to a programming language. Lombok is a java library that lets you boilerplate code with annotations. For instance, you can add @getter and have a getter for every field in your pojo. For a flawless experience with this java library, you will need the lombok plug-in. If you don't have a special need for upgrading to latest jdk with every version, I'll highly recommend lombok. If you always update to the latest, then lombok can be problematic as they don't support the new versions right away and there's no way to upgrade lombok if its version does not support your java version.

Sonarlint: You may already know sonarqube. It's a statistical analysis tool which finds code smells, potential bugs and so on in your codebase. Sonarlint is a plug-in that acts as a sonarqube server that runs inside your IDE. It's great for finding potential problems in your code without any additional step.

Rainbow Brackets: The idea is quite straightforward. This plug-in colors your parentheses for better readability.


Extra Icons: This plug-in provides additional icons for your project files. After this plug-in is installed, your gitignore file gets a git icon. Your mvnw and mvnw.cmd files have maven run icons and so on.

Sunday, July 19, 2020

how to publish jar to maven central

You created your brand new project and you want to release it to maven central so people can use in their maven based project. It is not a really straightforward thing to accomplish so I wanted to write a step by step guide on it (based on this stackoverflow post). 

-Login to your jira account
-Create a ticket for your project: For this step you will need a group id, a project website and a link to your source control. 

I used "com.sezinkarli" for group id because I own this domain. If you don't have a domain and you use github, you can easily use "io.github.yourusername"

Project website can be your github link for the project and you can link your github .git link as source control. So as you can see, github will be really useful here.

-Create a PGP key. You can download it here. Then open a command prompt and do

gpg --gen-key


It will generate our public and secret keys and sign them.
In the meantime it will print something like this:

gpg: key [Your key] marked as ultimately trusted

We will use this key for the following command:

gpg --keyserver hkp://pool.sks-keyservers.net --send-keys  [Your key] 

-Now we will update our user's maven settings. Go to your .m2 folder and edit/add settings.xml .

<settings>
  <servers>
    <server>
      <id>ossrh</id>
      <username>your jira username for sonatype</username>
      <password>your jira passwordfor sonatype</password>
    </server>
  </servers>
  <profiles>
    <profile>
      <id>ossrh</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <properties>
        <gpg.executable>gpg</gpg.executable>
        <gpg.passphrase>passphrase you used for gpg key</gpg.passphrase>
      </properties>
    </profile>
  </profiles>
</settings>
-Now we will update our project's pom.xml .

Add parent:

<parent>
    <groupId>org.sonatype.oss</groupId>
    <artifactId>oss-parent</artifactId>
    <version>9</version>
</parent>

Add distribution management:

<distributionManagement>
        <snapshotRepository>
            <id>ossrh</id>
            <url>https://oss.sonatype.org/content/repositories/snapshots</url>
        </snapshotRepository>
        <repository>
            <id>ossrh</id>
            <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
        </repository>
</distributionManagement> 
 
Add build plugins:

<build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-source-plugin</artifactId>
                <version>3.2.1</version>
                <executions>
                    <execution>
                        <id>attach-sources</id>
                        <goals>
                            <goal>jar-no-fork</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-javadoc-plugin</artifactId>
                <version>3.2.0</version>
                <configuration>
                    <source>11</source>
                </configuration>
                <executions>
                    <execution>
                        <id>attach-javadocs</id>
                        <goals>
                            <goal>jar</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-gpg-plugin</artifactId>
                <version>1.6</version>
                <executions>
                    <execution>
                        <id>sign-artifacts</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>sign</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.sonatype.plugins</groupId>
                <artifactId>nexus-staging-maven-plugin</artifactId>
                <version>1.6.8</version>
                <extensions>true</extensions>
                <configuration>
                    <serverId>ossrh</serverId>
                    <nexusUrl>https://oss.sonatype.org/</nexusUrl>
                    <autoReleaseAfterClose>true</autoReleaseAfterClose>
                </configuration>
            </plugin>
        </plugins>
    </build>

-Now we have everything in place. After we made sure from sonatype jira ticket that we are good to go (they do a check for group id), we can deploy our project to maven central:

mvn clean deploy

And that's it!
I had a problem while deploying because I did not have any javadoc. After I added it, everything worked like a charm.

Tuesday, May 12, 2020

java puzzlers from oca part 7


In this part of the Java Puzzlers from OCA series, I will show multiple ways of defining Strings and potential surprises related to that. Two basic types of creating Strings are creation with new keyword, and by just using the string literal.
String strWithNew = new String("hey");
String strWithLiteral = "ho";
As Strings are frequently used JVM uses a string pool and use the values in it so it won't have to create new objects for same values again and again. So seeing that the object address of same string literals are same should not be a surprise.
public class Puzzler {

    public static void main(String[] args) {

        String s1 = "myString";
        String s2 = "myString";

        System.out.println(s1 == s2); // true
    }
}
Ok then, this should be the same also right?
public class Puzzler {

    public static void main(String[] args) {


        String s1 = new String("myString");
        String s2 = new String("myString");

        System.out.println(s1 == s2);
    }
}
Not really. This will print "false". So if I create a new string with literal "myString" it is placed in the string pool. If I create it with new keyword then it's not searched in the pool, and when it's created it's also not placed in the string pool.
public class Puzzler {

    public static void main(String[] args) {


        String s1 = new String("myString");
        String s2 = new String("myString");
        String s3 = "myString";
        String s4 = "myString";

        System.out.println(s1 == s2);
        System.out.println(s2 == s3);
        System.out.println(s3 == s4);
        System.out.println(s1 == s4);
    }
}
I hope you can guess what happens above. s1 creates a new string and does not put it in the pool, s2 does the same thing. s3 takes a look to string pool does not see myString and creates it and places in the pool. s4 says "ah ok it is in the pool". So if we count how many strings are created, it is 3 and if we count what's placed in the pool it's 1 (myString). false, false, true, false are what's printed to the console.

Saturday, May 9, 2020

java puzzlers from oca part 6


Even for new Java developers, constructors are probably no big mystery. In essence, when you create an instance of a class, the constructor of this class is started. In the 6th part of Java Puzzlers series, we will see a case related to constructors.
public class Puzzler {

    public Puzzler(){
        System.out.println("Puzzler no arg constructor");
    }

    public static void main(String[] args){
        Puzzler puzzler = new Puzzler();
    }
}
In the example above Puzzler() constructor will start and "Puzzler no arg constructor" will be printed to the screen. Now lets see a new example.
public class Puzzler {

    public void Puzzler(){
        System.out.println("Puzzler no arg constructor?");
    }

    public static void main(String[] args){
        Puzzler puzzler = new Puzzler();
    }
}
As you can see we added a return value to the constructor of Puzzler and you may expect that "Puzzler no arg constructor?" will get printed but this is not right. When we add a return value to the constuctor, it stops being a constructor. So it won't get started when a new instance is created.

java puzzlers from oca part 5

In the fifth part of the Java Puzzlers series, we will see something related to X.parseX(String s) methods.


You can see what we expect from X.parseX() methods.
public class Puzzler {

    public static void main(String[] args){
        int i = Integer.parseInt("2"); 
        System.out.println(i); // prints 2
    }
}

We give the methods a String that can be converted to the primitive representation and hope for the best. Now lets check another example which will give us a NumberFormatException.
public class Puzzler {

    public static void main(String[] args){
        int i = Integer.parseInt("integer"); // java.lang.NumberFormatException: For input string: "integer"
    }
}

As the input is a word and not something that can be parsed to an integer, we get NumberFormatException. What happens above is consistent for each number type. So Integer, Byte, Short, Long, Double, Float won't surprise you when you call their parse methods with some random String. You'll get a NumberFormatException.
Now lets check what happens with boolean.
public class Puzzler {

    public static void main(String[] args){
        final boolean b1 = Boolean.parseBoolean("boolean?");
        System.out.println(b1);
    }
}
Can you guess what happens? The parse call will probably throw java.lang.BooleanFormatException, right? Not really. If you run that, it'll print "false" to the screen. The reason is Boolean.parseBoolean() just accepts anything and if it can't parse it, it just returns "false" value. Now lets see the other example.
public class Puzzler {

    public static void main(String[] args){
        final boolean b2 = Boolean.parseBoolean("TrUe");
        System.out.println(b2);
    }
}
You probably expect false again? That's not the case because parseBoolean is case insensitive and will return "true" in this case.

Saturday, May 2, 2020

java puzzlers from oca part 4

In the fourth part of Java Puzzlers, we have something related to char type.


public class Puzzler {

    public static void main(String[] args){
        char myChar = 'a';
        myChar++;

        System.out.println(myChar);
    }
}

You may have guessed it. It will print "b" and the reason for it is that char type is unsigned numeric primitive in the disguise of a character. So if I add one then I'll get the next character in unicode representation.

Then let's take a look at that one


public class Puzzler {

    public static void main(String[] args){
        char myChar = 'a';

        System.out.println(myChar + myChar);
    }
}
Will this print "aa"? Or  which's 97 + 97 = 194 (where 97 is value of 'a'). I don't know if you guessed it right but the result is neither. It's "194". When Java sees plus it tells "hmm that's an addition not a concat" and adds myChars up and returns the int value for it.

java puzzlers from oca part 3

In this third part of Java puzzlers, we will see a surprise in variable naming restrictions.


If I show you this, I'm sure you won't be surprised that this does not compile. static is one of the reserved keywords so why should it work?
public class Puzzler {

    public static void main(String[] args){

        int static = 2;
    }

}
Now I'll ask you a more difficult one. What you think about the below code. Will this compile?
public class Puzzler {

    public static void main(String[] args){
        int bool = 0;
        int integer = 1;
        int const = 2;
        int goto = 3;
    }
}

None of these should be reserved keyword. This is not C right? If you thought that it will compile, you're wrong. const and goto are reserved keywords, but bool and integer are fine.

Sunday, April 19, 2020

java puzzlers from oca part 2

Welcome to the second part of Java Puzzlers from OCA. In this part we will see an interesting case about the underscore separator in numeric literals which came with Java 7.


In the below class you can see the separator underscore in the decimal literal. Also notice the class compiles now without a problem. Octal is the octal representation, binary is the binary and I'm sure you can't guess hex.

public class Puzzler {

    public static void main(String[] args){

        int decimal = 12_345;
        int octal = 04321;
        int binary = 0B1010;
        int hex = 0X4321A;
    }
} 

Octal literal is defined with 0, binary with 0b/0B and hex with 0x/0X. Ok then, let's begin putting _ for a better readability after them.
public class Puzzler {

    public static void main(String[] args){

        int decimal = 12_345;
        int octal = 0_4321;
        int binary = 0B1010;
        int hex = 0X4321A;
    }
} 
Neat. It compiles without a problem. Lets move to binary and hex.
public class Puzzler {

    public static void main(String[] args){

        int decimal = 12_345;
        int octal = 0_4321;
        int binary = 0B_1010;
        int hex = 0X_4321A;
    }
} 
Nope. You'll get "Illegal Underscore" there. I'm sure this is designed that way with something in mind, but it sure is a surprising behavior.

Saturday, April 18, 2020

java puzzlers from oca part 1

I'm reading Oracle Certified Associate Java SE Programmer book from Mala Gupta in my spare time and I'm surprised with some of the new things I learn. Some of the time they really don't make sense, some of the time they make sense but really surprising to see. So in this article series, I wanted to share them as "Java Puzzlers" which sounded much cooler than "Java Surprises".


Lets check the below code and see what happens when we call an empty object reference's static method or field.

public class Puzzler {

    public static int field = 1;

    public static void printField() {
        System.out.println(field);
    }

    public static void main(String[] args){
        /*
        * Lets see what happens when the
        * reference is null.
        * */

        Puzzler puzzler = null;
        puzzler.printField(); // prints 1
        System.out.println(puzzler.field); // prints 1
    }

}

When you try to guess what will happen, you can think that we will get NullPointerException while doing the method and field calls as the reference does not have an object attached to it. But remember that static methods and fields belong to the class itself and not to the instance. So without the need of an associated object you can use them and won't get an exception for doing that. An also the way we call the static method are usually in Puzzler.printField() form which tells more.

Thursday, March 5, 2020

scripting with painless language

Recently I've been working on a complex scoring function in Elasticsearch and stumbled upon this really cool article for painless language. Very well prepared and easy to understand.

Saturday, January 18, 2020

lambdas and streams master class

If you want to master lambdas and streams from Java 8 & 9, you can check these java koans. That's one of the best koans I could find for a deeper learning. Also there are two videos on these koans: part1 and part2.

Tuesday, January 7, 2020

send your data async on kafka

For a project, I'm trying to log the basic transactions of the user such as addition and removal of an item and for multiple types of items and sending a message to kafka for each transaction. The accuracy of the log mechanism is not crucial and I don't want it to block my business code in the case of kafka server downtime. In this case an async approach for sending data to kafka is a better way to go.

My kafka producer code is in its boot project. For making it async, I just have to add two annotations: @EnableAsync and @Async.

@EnableAsync will be used in your configuration class (also remember that your class with @SpringBootApplication is also a config class) and will try to find a TaskExecutor bean. If not, it creates a SimpleAsyncTaskExecutor. SimpleAsyncTaskExecutor is ok for toy projects but for anything larger than that it's a bit risky since it does not limit concurrent threads and does not reuse threads. So to be safe, we will also add a task executor bean.

So,
@SpringBootApplication
public class KafkaUtilsApplication {
    public static void main(String[] args) {
        SpringApplication.run(KafkaUtilsApplication.class, args);
    }
}
will become
@EnableAsync
@SpringBootApplication
public class KafkaUtilsApplication {
    public static void main(String[] args) {
        SpringApplication.run(KafkaUtilsApplication.class, args);
    }

    @Bean
    public Executor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(2);
        executor.setMaxPoolSize(2);
        executor.setQueueCapacity(500);
        executor.setThreadNamePrefix("KafkaMsgExecutor-");
        executor.initialize();
        return executor;
    }
}
As you can see there's not much change here. The default values I set should be tweaked based on your app's needs.

The second thing we need is addition of @Async.


My old code was:
@Service
public class KafkaProducerServiceImpl implements KafkaProducerService {

    private static final String TOPIC = "logs";

    @Autowired
    private KafkaTemplate<String, KafkaInfo> kafkaTemplate;

    @Override
    public void sendMessage(String id, KafkaType kafkaType, KafkaStatus kafkaStatus) {
        kafkaTemplate.send(TOPIC, new KafkaInfo(id, kafkaType, kafkaStatus);
    }
}
As you can see the sync code is quite straightforward. It just takes the kafkaTemplate and sends a message object to the "logs" topic. My new code is a bit longer than that.
@Service
public class KafkaProducerServiceImpl implements KafkaProducerService {

    private static final String TOPIC = "logs";

    @Autowired
    private KafkaTemplate kafkaTemplate;

    @Async
    @Override
    public void sendMessage(String id, KafkaType kafkaType, KafkaStatus kafkaStatus) {
        ListenableFuture<SendResult<String, KafkaInfo>> future = kafkaTemplate.send(TOPIC, new KafkaInfo(id, kafkaType, kafkaStatus));
        future.addCallback(new ListenableFutureCallback<>() {
            @Override
            public void onSuccess(final SendResult<String, KafkaInfo> message) {
                // left empty intentionally
            }

            @Override
            public void onFailure(final Throwable throwable) {
                // left empty intentionally

            }
        });
    }
}
Here onSuccess() is not really meaningful for me. But onFailure() I can log the exception so I'm informed if there's a problem with my kafka server.

There's another thing I have to share with you. For sending an object through kafkatemplate, I have to equip it with the serializer file I have.


public class KafkaInfoSerializer implements Serializer<kafkainfo> {

    @Override
    public void configure(Map map, boolean b) {
    }

    @Override
    public byte[] serialize(String arg0, KafkaInfo info) {
        byte[] retVal = null;
        ObjectMapper objectMapper = new ObjectMapper();
        try {
            retVal = objectMapper.writeValueAsString(info).getBytes();
        } catch (Exception e) {
            // log the exception
        }
        return retVal;
    }

    @Override
    public void close() {
    }
}
Also, don't forget to add the configuration for it. There are several ways of defining serializers for kafka. One of the easiest ways is adding it to application.properties. 

spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer spring.kafka.producer.value-serializer=com.sezinkarli.kafkautils.serializer.KafkaInfoSerializer

Now you have a boot project that can send async objects to the desired topic.