AWS AutoScaling EBS Tagging

When an EC2 instance launched using Launch configuration of AutoScaling Group(ASG). It will tag only EC2 instances but not EBS volumes attached to instances. Look like launch configuration is not supporting tagging EBS volumes and ASG propagate tags to only EC2 instance.

If you want to tag both EC2 instance and attached EBS volume then use Launch Template resource and link it to ASG.

Below given sample terraform code:

## Launch template
resource "aws_launch_template" "app_launch_template" {
  name_prefix     = "app-template-prod"
  image_id        = "<ami-id>"
  instance_type   = "<instance-type>"
  vpc_security_group_ids = ["<Securitygroups-ids>"]
  iam_instance_profile {
    name ="<instance-profile>"
  }
  user_data            = "${base64encode(data.template_file.app-user_data.rendered)}"

  tag_specifications {
    resource_type = "instance"
    tags = "${merge(
      module.global.resource_tag,
        map(
          "Name", "app-${module.global.env_workspace}"
      )
    )}"
  }

  tag_specifications {
    resource_type = "volume"
    tags = "${merge(
      module.global.resource_tag,
        map(
          "Name", "app-${module.global.env_workspace}-ebs"
      )
    )}"
  }

  tags = "${merge(
    module.global.resource_tag,
      map(
        "Name", "app-${module.global.env_workspace}"
    )
  )}"
}

## ASG
resource "aws_autoscaling_group" "app_asg" {
  name = "app-${module.global.env_workspace}-asg"
  launch_template = {
    id = "${aws_launch_template.app_launch_template.id}"
  }
 }

Unix root app log file permission

We may need some log files read access for other user process for tailing logs or monitoring logs or collecting logs using collectd, telegraf, splunk, etc. So When an application started by root user, all the log files or runtime files are created with only root user access permission. Changing the file permission will not helps since log files are rotated after sometime with only root user access(600). Here sharing one way of making only log file readable by all the users.

# Before starting the application create directories and set permission using setfacl. From below example application should be creating the log files in /opt/application/logs path.

mkdir -p /opt/application/logs

chmod 755 /opt/application
chmod 755 /opt/application/
chmod 755 /opt/application/logs

setfacl -d -m u:telegraf:r /opt/application/logs

touch /opt/application/audit.log
Then start your application

java: Method not found exception

Even after adding right dependencies to classpath, we may get method not found exception. It is hard to find out the conflicting class file in jars. Use below unix command to find out the jar files which has same class file.

Below command finds Hello.class in jar files.

 find . -type f -name '*.jar' -print0 |  xargs -0 -I '{}' sh -c 'jar tf {} | grep Hello.class &&  echo {}'

Hive: Create table like with Storage format

In Hive, You can specify the storage format while creating a like table.

Example
create table target_table like source_table stored as ORC;

Apache Hive JIRA: https://issues.apache.org/jira/browse/HIVE-7122

Unix: 'echo' remove new line character

For removing new line character in Unix echo command. Then using option "-n" do not print the trailing newline character.

echo -n "choicest word in the world is deed" > result.txt