Avoid JVM DNS Caching Problems in AWS

rstaatsDevops0 Comment

Certain older versions of JDK have forever caching by default. This poses a problem if you have a Java application that calls ELB/ALB or ElasticSearch Clusters. When any action is made to modify a setting in AWS ElasticSearchService or an update happens the cluster doubles in size, the shards are replicated to the new nodes and the old nodes are removed. Queue up the dramatic music and a Java application in trouble.

To avoid this issue simply set the java.security ttl as per the following AWS link: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-jvm-ttl.html

By setting $JAVA_HOME/jre/lib/security/java.security file to networkaddress.cache.ttl=60 your application will now play nice with ELB/ALB IP changes and ElasticSearch cluster node replacements.

Extending Partition Without Rebooting (EBS Amazon Linux)

rstaatsUncategorized0 Comment

This guide assumes that you are using a Linux OS in AWS EC2 and are not using LVM. In the example below we will walk through the steps of growing an EBS volume.

First we need to go to the “Volumes” page of EC2 and right click the volume we wish to expand and click modify. Increase the size to the desired amount and press modify. Once the storage amount changes in the EBS volume screen we are ready to proceed.

SSH into the machine we extended the drive for and run the following command to identify the drive. This may vary on systems with multiple disks, however for most single volume EC2 machines this will be /dev/xvda1

sudo lsblk

NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT

xvda    202:0    0  40G  0 disk

└─xvda1 202:1    0  40G  0 part /

Next once you’ve identified the disk (in my case above it’s /dev/xvda1, you’ll want to run the following commands:

To grow the partition run(make note of the space between the device and partition number): sudo growpart /dev/xvda 1

Finally to grow our partition to be used by the OS:
sudo resize2fs /dev/xvda1

Note: if you are running an XFS file system you may need to run xfs_growfs instead of resize2fs

4 Years of Blogging!

rstaatsIT General0 Comment

Well it’s that time of year again when the domain renewal happens the holidays are about to wrap up and the year is about to come to a close. This week marks 4 years since I started this blog. When I set out to create this blog I was about a year into my first real systems administrator job in a mostly windows environment for about 100 users and managing a 4 node VMware ESXi cluster and a decent number of VMs as well as lending a hand at another site here and there. During this year I had also made my first foray into Linux and AWS and jumped into the world of DevOps. Fast forward 4 years and I’ve become almost exclusively a DevOps engineer working primarily with Linux. My career has taken me places I never thought it would, I’ve migrated large sites from on premise datacenters to AWS, on site at a large annual award show to work closely with content teams, and worked closely with development teams on the ground to help redesign and rebuild a video streaming service platform.

 

The purpose of this blog 4 years in remains the same as it started. Simply to help other admins/engineers with fixing odd and poorly documented issues, and to share cool tips and tricks that have been helpful to me in hopes of helping others. As this year wraps up I have a series blog planned for the new year around Ansible that should be a good 3-4 part blog. Drop me a comment if there’s something you’d like to see me cover in the new year!