Testing Open Ports With NetCat or Powershell

rstaatsLeave a comment

Ok so it’s 2021. We all know telnet is a dumpster fire no self respecting system engineer wants on their boxes. So what’s the alternative when troubleshooting whether you can connection a port or not. So lets take a look at alternatives below:

Linux with NetCat

 nc -zv <hostname or ip> <port number>

If your connection was successful you’ll get a message back like this

Connection to example.com port 80 [tcp/*] succeeded!

If your connection was not successful it will hang until it times out, at which point you’ll get a message indicating it failed with operation timed out.

Windows with Powershell

Test-NetConnection <hostname or ip> -Port <port number> -InformationLevel Quiet

This will return simply a true of false. If you want more detailed information you can use Detailed instead of quiet or simply omit the -InformationLevel flag and everything after it.

Replicating MySQL To MySQL With DMS Without Losing Secondary Indexes and Foreign Key Constraints

rstaatsLeave a comment
Easy Right?

We wouldn’t be in spring time without spring cleaning right? As part of this spring cleaning I decided to do some DB consolidation to make better economies of scale out of my RDS instances (especially since Multi-AZ instances aren’t exactly cheap). Having used AWS DMS (Database Migration Service) to replicate MySQL to Redshift data warehouse I figured it’d be a trivial left to stand up a new RDS instance and make it the replication target of several source MySQL instances I wanted to combine/condense and since there no conflicting table or db names it should be easy.

OK Maybe Not…

If this were the case this would be the end of this post…but wait there’s more! So data loaded and replicated just fine…comparing table max ids, row counts, and spot checking data everything looked great. Until I started checking indexes and foreign keys. It was only then that I realized DMS with target table preparation mode set to ‘drop and create’ will not create secondary indexes or FK constraints. Furthermore it doesn’t set auto-increment by default on any columns which would have spelled a major disaster in production.

A Solution at Last!

Exploring other options I attempted to do a mysqldump of just the table create DDL and applied this to my target successfully. Then in DMS I chose the target table preparation mode option ‘do nothing’..and well it didn’t do what I wanted…so I tried ‘truncate’, this just produced hundreds of table errors due to FK integrity violations and the fact that the table loads were not necessarily in the same order as their constraints and dependancies. It was at this point I finally arrived on a successful solution. In DMS there is an option on the target to disable FK checks and just load the data. This in tandem with truncate gave us exactly what we wanted.

TL:DR Instructions
  1. Dump your table DDL from your sources
mysqldump --no-data --skip-set-charset --single-transaction=TRUE -h{{db host}} -u{{dbuser}} -p {{database name}} > myddldump.sql

2. Import your table DDL to your target (the db you’re migrating to obviously use caution here if the target has existing tables or dbs that share a name)

mysql -h{{db host}} -u{{dbuser}} -p {{database name}} < myddldump.sql

3. In your DMS Endpoints, set the following “extra connection attributes” on your target

initstmt=SET FOREIGN_KEY_CHECKS=0

4. Modify your DMS task target table preparation mode to ‘truncate’

5. Start your task and profit!

Apple Silicon(M1) 3 Months Later

rstaatsLeave a comment

Well here we are 3 months into 2021 and 3 months into having M1 unleashed on the world. As noted in a previous post back in December most everything I run on a day to day basis works, however I’ve now had some time use it as my daily driver for several months and developers have begun to release more Apple Silicon builds, which means better performance and less dependance on Rosetta 2.

More M1 Native Software Builds

Generally speaking everything has run equally as fast or faster than it did on my old late 2016 Macbook Pro 15″. The one really big exception to this were JetBrains products. I’m a daily user of DataGrip, IntelliJ, and PyCharm. On December 30, 2020 JetBrains announced via their blog that they had Apple Silicon versions of the 2020.03 release of all their main products. Performance with these has been astronomically better on M1 and it now handily beats my Intel Mac hands down for IDE performance.

Docker Coming Along

Onto the biggest gotcha yet…Docker. While not having Docker support out the gate for M1 was a huge bummer and made many of us who use Docker in our jobs either as developers, engineers, or devops users, they were quick to get a preview build out the door that is available for use. The reasons for this are widely documented including here however the TL;DR version is that the are some differences between the M1 ARM silicon and x86 processors in terms of virtualization options and supported container architectures.

Enter the Docker Desktop for Apple Silicon Preview build. At the time of this writing the current preview build is 7. I was quickly disappointed when I ran the docker-compose for my local dev env and was immediately greeted by warnings and containers dying with this message

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

However manually running the containers in my environment with the following flag once and then stopping them made them play nice with Docker-Compose

--platform linux/amd64
#Example Running a container
docker run --platform linux/amd64 -it mysql:5.7

Performance with Docker using the compatibility flag has been fine. The only major problem that I’ve had is that disabling Use gRPC FUSE for file sharing in the experimental features requires a restart of Docker Desktop and it gets hung restarting until I reset all settings to default. All in all it’s usable enough for my purposes and the performance has been quite acceptable. This should continue to improve with time.

Audio and Video Production

Logic Pro X works like a charm of course (it better, it’s first party). All the plugins I use from ToonTrack, Neural DSP, EastWest, and Izotope all seem to work rather well, maybe with the exception of Ozone 9 occasionally causing crashes of Logic (only on startup, but I’ve managed to work around this by disabling the plugin before saving and closing the project and then toggling it back on once the project loads). iMovie has of course been fine, rendering times have dramatically improved for me over my MacBook Pro, and the heat is non-existent during rendering.

Best Battery Life….Ever….Seriously…

Part of the fun of #DevOpsLyfe when working with retail customers is being on call and high alert during the black Friday/cyber Monday holiday shopping traffic increases. I decided to keep an eye on monitoring dashboards on my M1 Air this year with Caffeine toggled on to prevent screen sleep…..13 hours later I still had a 34% batter remaining. Never have I been able to keep a dynamically updating monitoring page going for that long on battery. In fact the next day I did the same thing with my Intel MacBook Pro from work (it’s a 2018), and the battery lasted a whole 3.5 hours. The lack of heat produced by these machines, the efficiency of SoC ARM, and the low power draw has really made these the absolute best devices on the market for desktop computing power on the go.

What’s Next?

It remains to be seen what will come next in the Apple Silicon world. One thing is for sure though, it has changed the game and it is here to stay. I’m interested to see how this SoC architecture scales beyond 16GB of shared memory, and how a greater number of I/O ports is handled. One thing is for sure though, when the more powerful high end pro devices rock Apple Silicon it’s going to be truly impressive.