Recently we have encountered a failure, on one of our Redis nodes due to reaching the maximum number of clients connected. As an immediate action for resolving the incident, we wanted to modify the existing connection timeout property of the node and do a failover master node into a slave, since the master node is not responsive anymore.
The approach we followed for the first time.We modify the redis.conf file on both master and the slave nodes. Then do a Redis service restart on the master node. Due to the service restart on the master, slave promoted itself into a master node.
The disadvantage of this approach is, once the master is restarting, it is losing any of the ongoing operations in the master node. This is not the appropriate approach recommended by the Redis.
The approach recommended by Redis. Redis has inbuild command to failover master node into a slave node. CLUSTER FAILOVER [FORCE|TAKEOVER]If we look at those two, you may think it's really simple to solve those issues by sleeping the thread. For the first scenario, you can define a maximum number of retries and sleep in between each retry. For the second scenario, imposing explicit thread sleep will reduce the rate of your API access.
Let's assume we are checking connectivity each second and assume we got the connectivity by 6.5 seconds. If we use a sequential approach such as thread sleep, we are attempting 7 times to establish a connection.
When re connecting we have to consider two key aspects
With spring retry we can try out few different approaches, it can be a linear or Fibonacci or a custom approach.With the latest spring versions you may find this as a feature of the frame work but if you are not using spring or using an older version of spring framework, you can use this library as a dependency.So we are no longer required to use thread sleep as our default wait mechanism as well as we are not required to re invent the wheel when we wanted to try out some other re trying sequence.
<plugin>
<groupId>org.owasp</groupId>
<artifactId>dependency-check-maven</artifactId>
<version>2.0.0</version>
<executions>
<execution>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
Once plugin is configured we can invoke the plugin by executing following maven goal
mvn clean install
It will cross check your dependencies against the vulnerability data base and generate a report if anything suspicious available. But keep in mind there can be false positive results as well.
Example report:
[INFO] Analysis Complete (5 seconds) [WARNING] One or more dependencies were identified with known vulnerabilities in prject name: jar-file-name-1.3.1.jar (jar-file-name:jar-file-name:x.x.x, cpe:/a:groupId:artifact_id:x.x.x) : CVE-2020-9999, CVE-2020-4444444 See the dependency-check report for more details. [INFO]
By now we know what is Terraform, why it is there and how to create a resource with it. In this post, I'm going to modify resource which I created and lastly destroy the infrastructure we create. Updating an infrastructure Let's modify the instance type. Previously we create an instance with t2.micro size so now I need something bigger than that, so I modify my previous script.
resource "aws_instance" "example" {
ami = "ami-13be557e"
instance_type = "t2.medium"
}
First thing first, before do any thing to your infrastructure, execute a dry run.
It clearly specifies there will be an update in resources ,( -/+ ) symbol stands that.
instance_type: "t2.micro" => "t2.medium" (forces new resource)This is what we expected, whenever an update happens on ec2 instance type AWS destroy existing ec2 and gives us a new instance. Let me apply it and check the status.
As we expected our infrastructure is changed and new resource with t2.medium is available.
Destroying an infrastructure.
Before moving to destroy I' wanted to do an experiment. Usually, If you need to modify or delete an AWS resource you need an ID but in our previous example we didn't specify any identification, so where they come from. Definitely, it should be retrieved from the status file. So I'm going to delete the status file and apply another instance size change.
Then I execute terraform show It gives me there is no status. Then I execute terraform applySo keep in mind, never ever delete the status file, that why terraform also keeping a backup file whenever a change happens.So the best place to keep this is your repository. Make sure to commit this status file along with other source code.
Removing an infrastructure also soo easy with terraform. Same steps as before, first we change the code, go for a dry run then apply the changes. This time dry run is little bit different terraform plan --destroy to actual deletion terraform destroyDuring my past blog post on Terraform we discuss what is Terraform and why it is so popular among platform engineers.In this post, we will have a look on how to install Terraform on ubuntu machine and we will discuss instantiate an ec2 instance with Terraform.
Installing Terraform.cd /home/amith/Documents/Software unzip terraform_0.7.13_linux_amd64.zip
PATH=/usr/local/terraform/bin:/home/amith/Documents/Software:$PATH
terraform
As we discussed previously it's all about writing a source code. As all other source codes terraform also associated with a file extension and usually it is .tf. vi ec2_create.tf
Before going forward we need few data.
Here is the sample code.
provider "aws" {
access_key = "ACCESS_KEY_HERE"
secret_key = "SECRET_KEY_HERE"
region = "us-east-1"
}
resource "aws_instance" "example" {
ami = "ami-0d729a60"
instance_type = "t2.micro"
}
Before move further try to understand the source.There are two ways of defining a resource with Terraform.
JSON is more machine friendly language so if you plane to generate terraform script in problematically make sure to use JSON format. But if you need a more human-friendly way of defining infrastructure then go with terraform format. Actually, terraform format is a wrapper for JSON, so there is no harm of using any one of them. It's up to you decide the appropriate format for the scenario.
If you remember the goals we discussed in previous blog post, Terraform support dry runs, by using that feature we can plan the final outcome before actually doing the change.To run a plan you have to navigate into the file location where you define the terraform code.
cd /home/amith/WorkSpace/sandbox/terraform terraform plan
Explain plan.
Plan: 1 to add, 0 to change, 0 to destroy.
This section summarizes the final outcome. According to this, it says one resource has to create and no any update or delete.
+ aws_instance.example This section describes which resource going to create, + symbol denote a new resource creation. If it is - that means item is about to remove and if it is +- that means resource about to update. For more simplicity in this report they use color codes
availability_zone: ""
ebs_block_device.#: ""
ephemeral_block_device.#: ""
instance_state: ""
Those values will be generated by the provider since this is a dry run those data are not available at the moment.
Now we have a fair understanding of what will be the out come so let's create the resource.
terraform apptly
this process takes some time to complete. Every 10 seconds it update the report.Once this is completed Terraform will create a new file which contains status(metadata) about the infrastructure and saved on the same location - terraform.tfstate.If you plan to share the code make sure to share this file as well.Without this file terraform will note able to do an update or show a status report next time so it's really important to keep this file safe.
To inspect the status.
terraform show
Well, this briefly explains what it capable of. So let's look at some of the key goals of Terraform as per the author of this nice technology.
It's all about writing some code to model your infrastructure. You specify the resource on code snippet and describe it and hand over to Terraform to create it. The beauty of modeling an infrastructure as a code is, it brings all other advantages we got from a source code.Simply we can keep them on a repository, version them, review them, integrate with a CI/CD pipeline, automate the tests on infrastructure.
It's capable of handling any of the above, as an example
With Terraform you don't need to go and create the infrastructure. You can predict the infrastructure by dry running or here we called it plan. It gives you a report on your infrastructure and how it will be once you execute the script.Then you can review the changes and safely build or upgrade the infrastructure without affecting to any of existing.
You don't have to bound to any specific provider. You can instantiate an ec2 which is AWS and you can use some other platform to create a database.
It's not Chef or Puppet, both of them are cool technologies where we use to install and manage software on a hosts in other words they are managing configurations but Terraform is not.But you can use any of those configuration management technologies along with Terraform to configure the infrastructure which created with it.
CloudFormation, yes it has some similarities. Terraform inspired by the problem they solved the problem of modeling infrastructure as a code. But it is limited to a specific provider, you can't create a hybrid infrastructure with CloudFormation.
SDK like python boto empowered developers to access cloud providers in a programmatic way but terraform is not used for pragmatic access to cloud its simple infrastructure modeling in a human friendly way.
I think this is enough for a single blog post will meet you soon with another blog post with my hands on experience with terraform.
How you tackle a defect in your code, specially a server side code? Are you remotely debugging the code at first step? Yes it is convenient to do so and most of the time it save you from the defect. The real question is, can it save you from defects all the time? Imagine you have written a code which is highly dependable on external factors such as data base connection, server level configurations and fortunately all goes well in development environment as well as QA environment. Assume you suddenly got a complaint from the customers that your functionality is not working on production as they expected. How you tackle where it went wrong, are you still willing to perform a remote debug procedure on the production where we need to maintain high availability? Who’s gone to save you now? Don’t worry logger is here. Most of the time it save us from incidents (not all incidents are bugs)
My ABC of defect tackling on a server side code.So it is very important to maintain a good server log. My opinion is we should be able to tackle any issue without remotely debug the code.
Impact on application performance by java logging.Logging has big influence on application performance since logger has to perform IO operations, string concatenations each time we ask logger to log a message so most of the developers avoid them but it is not advisable to avoid all of them. What we can do is to identify what are needed to be log and what are not. Keep in mind both less and excess logging is not preferable.
Determine the correct type of before log.I’m using log4j as my logging provider but SL4j is grooming as the standard logging API. In log4j there are four types. It is really important to place log statement on correct category.
DEBUG : The lowest restricted logging type , ideally we should include all the information we need to debug the application but keep in mind DO NOT enable this level on production because it heavily impact on application performance.String concatenation is a costly operation we should minimize concatenations, even we are using INFO as our debug level so logger is never log anything mark as DEBUG but in JVM it is executed . As I mentioned above DEBUG should contain all the details we should use to tackle the issue so most probably there will be many string concatenations. Example : Solution 1 : Check debug level when there are too many string concatenations . Solution 2 : Use parametrized logger framework such as SL4J.
Example : There are two possibility of getting NPE in above log statement. I’ve seen scenarios where actual exceptions are hidden by the exceptions thrown by the logger statements so try to avoid them because loggers are here to help us not to worst the situation.
Is there any use of follow log statement?
As every one can see sample one class does not generalize its map but in sample two it is generalized.Now we compile both classes and we get two byte codes , Lets name them as byte code 1 and byte code 2 .
Are those two byte codes are same or not ?
If you look blindly you may answer that question with no , because every one can see that the content of two classes are different, so there can not be same byte code for both sample one and sample two classes . But truth is they are same.
How they are same and why it is same ?
Its better to know why it is same before we hands on how it is same . Simple answer for why it is same is , to ensure the backward compatibility of the code.Java introduce Generics with java 5 version , if we change infrastructure of java byte code with version 5 those code may not be compatible with previous versions . In simple your pre java 5 classes can not communicate with your java 5 object if their byte code infrastructure is different.
Now we can hands on how it is done.When java compile the source code it ensure the type safety by using generics after that compiler erase all the generics using eraser utility and then compile the source code now there is no generics in the byte code but code is type safe to more information on how this work go to follow this java documentation .
/**
* @author amith jayasekara
*
* simple multi value annotation
*/
@interface MultivalueAnotaion {
String message();
//this will set mata data print number to 23 if there is
// no explicitly deceleration
int printNumber() default 23;
}
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/**
* @author amith jayasekara
*
* simple multi value annotation that retain in runtime and that
* can be used in class level
*/
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
@interface MultivalueAnotaion {
String message();
//this will set mata data print number to 23 if there is
// no explicitly deceleration
int printNumber() default 23;
}
import java.lang.annotation.Annotation;
/**
* @author amith jayasekara
* sample code for testing annotation in runtime of the programme
*/
@MultivalueAnotaion(message = "sample message")
public class MultivalueAnnotationTest {
public static void main(String[] args) {
//we create a object of the class
MultivalueAnnotationTest test = new MultivalueAnnotationTest();
//check is annotation is available
if (test.getClass().isAnnotationPresent(MultivalueAnotaion.class)) {
MultivalueAnotaion ma = test.getClass().getAnnotation(MultivalueAnotaion.class);
System.out.println("annotation message " + ma.message());
System.out.println("annotation print number " + ma.printNumber());
}
}
}
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/**
* @author amith jayasekara
* single value annotation
* retain till runtime and only be able to apply for method level
*/
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
@interface SingleValueAnnotation {
boolean isPrintMessage() default true;
}
import java.lang.reflect.Method;
/**
* @author amith jayasekara
*/
public class SingleValueAnnotationTest {
@SingleValueAnnotation
public static void annotatedMethod() {
SingleValueAnnotationTest st = new SingleValueAnnotationTest();
Class cls = st.getClass();
try {
Method method = cls.getMethod("annotatedMethod");
SingleValueAnnotation ano = method.getAnnotation(SingleValueAnnotation.class);
System.out.println("is print message " + ano.isPrintMessage());
} catch (NoSuchMethodException e) {
System.out.println("no method found");
}
}
public static void main(String[] args) {
annotatedMethod();
}
}
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/**
* marker annotations
* this annotation is available in runtime and the target is class level
*/
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
@interface MyMarker {
}
/**
* @author amith jayasekara
*/
public class MarkerAnoTest {
public static void main(String[] args) {
MarkerClass mark = new MarkerClass();
if (mark.getClass().isAnnotationPresent(MyMarker.class)) {
System.out.println("Mark is marked with marker annotation");
}
}
} HashMap() HashMap(int initialCapacity) HashMap(int initialCapacity, float loadFactor) <link href='http://alexgorbatchev.com/pub/sh/current/styles/shCore.css' rel='stylesheet' type='text/css'/> <link href='http://alexgorbatchev.com/pub/sh/current/styles/shThemeDefault.css' rel='stylesheet' type='text/css'/> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shCore.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushCpp.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushCSharp.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushCss.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushJava.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushJScript.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPhp.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPython.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushRuby.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushSql.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushVb.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushXml.js' type='text/javascript'></script> <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPerl.js' type='text/javascript'></script> <script language='javascript'> SyntaxHighlighter.config.bloggerMode = true; SyntaxHighlighter.config.clipboardSwf = 'http://alexgorbatchev.com/pub/sh/current/scripts/clipboard.swf'; SyntaxHighlighter.all(); </script>(Note: current version of syntax highlighter does not support text wrapping if you need that replace current with 2.1.364 in above code)
<script type="syntaxhighlighter" class="brush: html"><![CDATA[ <html> <head> <title>Title of the HTML page</title> <meta name="title" content="Help Give to the Carter Tomorrow Fund" /> ]]></script>Note : you can replace 'html' to any of selected language (e.g. java, php, csharp, js, sql)
 |
| sample Employee immutable class |
