naftuli.wtf An urban mystic, pining for conifers in a jungle of concrete and steel.

A New Year in 2019

January 02, 2019

So 2019 is totally happening.

Year in Review: 2018

2018 was an interesting year.

Pros

A lot happened. Here’s a list of the positives this year.

Rust

I got over the hill in Rust this year. I now feel comfortable enough to write general purpose applications in Rust now, which will hopefully be my main go-to over Python.

Why? Well, a lot of reasons, largely centered around deployment and performance. At Grindr, the team and I built a suite of system tools in Python that we needed to ship installed with the system Python, and this was, to be honest, a huge embarrassing failure. System Python is a dangerous tightrope, with very specific versions of libraries duct-taped together.

Rust answers this problem easily. The flexibility of Rust means that it’s easy to link to common system libraries like OpenSSL and libsodium or to just statically compile them into the binary like Go. I can create a Rust utility that can be deployed to Linux, macOS, and Windows, even supporting multiple architectures like x86_64 and ARM.

Performance is likewise incredible and straightforward in Rust, with libraries like crossbeam, parking_lot, rayon, and tokio, with async support landing soon, making it easy and possible to challenge and exceed performance of NGINX and other notoriously fast services. The cool thing about Rust is that you can achieve all of this without your code being a nasty ocean of C and living in constant fear of dereferencing a NULL pointer, double frees, use-after-free, etc.

rayon makes it possible to take single-threaded code and easily make it parallel. Consider the following synchronous code:


let digests: Vec<[u8; 32]> = my_collection
    .iter()
    .map(|x| sha256::digest(x.read()))
    .collect();

and transform this to use all CPU cores on a system:

let digests: Vec<[u8; 32]> = my_collection
    .par_iter()
    .map(|x| sha256::digest(x.read()))
    .collect();

I’m stoked to be at this level of competency with Rust and I can’t wait to tackle more problems in 2019 in Rust.

Golang

I got over the hill in Go this year as well. I had a project at work wherein I built an ETL job in Golang to transform a large amount of data in a bucket and insert that data into a hosted analytics database. Where Rust isn’t possible, I’ll be using Go to replace Python.

It may sound like I’m this huge Python opponent, but I assure you, this is NOT the case.

Also, though I’m truly head-over-heels for Rust in a way that I haven’t been for any technology in ages, Go is a good general-purpose language to replace the JVM. It puts parallelism in the hands of those that otherwise wouldn’t have it. The JVM is still a thing, and Python/Ruby/NodeJS can’t compete with it for performance, as they’re all single-threaded with green-threads pretending to be real threads.

My main gripes with Go are the fact that I need to change my filesystem layout to get things working and that Go feels very boring to use. I can one-liner things in Scala, Ruby, Python, and Rust that just boil down to for loops and if statements in Go.

Go is great, and the performance alone makes it far more attractive than Python/Ruby/NodeJS, so I’m going to try to use it more whenever Rust isn’t possible.

Open Source Maintainership

I’ve built a lot of tools over the years, but more so in 2017 and 2018 than ever before. There are too many to list and to link to, but probably the most significant stuff I’ve done is my continuously integrated Ansible role ecosystem.

This system is composed of a number of components. One is that I maintain six different Docker “VMs” which use a real init system to mimic actual VMs as closely as possible with very little overhead. Here are the VMs:

These VMs give me an execution environment to run Ansible against to test my roles continuously.

I pair these with my degoss role, which allows me to run Goss tests against one or more hosts to validate that a role enforced its contract.

There are a lot of roles which I maintain, but by-and-large they are around development environments for various programming languages so I can build Vagrant VMs with little impedance. I’m also working on my nfty tool which will help conform projects to a given standard, making it easy to update the ~10 roles.

Here are the Ansible roles for development environments:

There are many other roles which I’m not going to link to which form common pieces like managing system time via NTP and timesyncd, but they’re there if you look :trollface:

All of these development environment roles can be overlaid to provide multiple language support (e.g. Rust + NodeJS + Docker).

Titan is also notable as a network framework for AWS and more recently, GCP. GCP still hasn’t hit a 1.0 because GCP up until recently has lacked NAT as a service and private DNS zones. Some of the architecture realities in GCP force strange adaptations to the design. I’m hoping in 2019 to provide documentation for these repositories.

Cons

I’m not going to dwell a lot on the cons, but largely they are composed of the following:

  • I didn’t get to do enough Rust.
  • I didn’t get to do more Go.
  • I didn’t maintain a consistent flow of code shipped to GitHub for open source maintainership.
  • I didn’t spend enough time playing music and working on non-engineering projects.
  • I spent around 8 weeks in San Francisco, and I felt like I was really missing my life.
  • Because of operational details on the ground, I wasn’t able to provide the leadership I wanted to provide in my roles this year. I hired a lot of awesome engineers that I really respect, but didn’t get have the time or authority to work with them to build rad things.

My goal in going into management was to facilitate engineering, both as an engineer and as a manager. I accomplished a lot, but there’s still a lot that I didn’t feel that I accomplished very well. In retrospect, the choices made were the right ones based on the information at the time, but I really value working with other engineers and I wanted to do more to create a healthy and exciting engineering environment.

One notable thing which was interesting was that I prepared for around six weeks for an interview that didn’t work out. It was kind of a bummer, but I left it all on the field: I took something like 150 pages of notes on 8.5x11” paper, did something like 50 practice interview problems, and spent an inordinate of time outside of normal work hours preparing. On interview day, I hit two out of the five interviews out of the park, but I bombed two of them and only did okay on the fifth. I have no regrets and I can’t really say that I would do something different because I really challenged myself and pushed myself harder than I ever have. It could easily be that it was for the best, but it was still somewhat disappointing.

Travel was not something I expected to be part of my new role, but it happened that way. I’m proud that I was able to weather it pretty well, because travel is extremely hard on my psyche.

2019

I don’t really make New Years Resolutions™, but I still would like to take this opportunity to lay out some general goals for the new year.

  • More Rust.
  • More Go.
  • Embedded: I really want to start playing around with RTFM on Arduinos.
  • Electrical Engineering: I want to get comfortable enough that I can build embedded systems from the ground up with a bunch of wires, microcontrollers, capacitors, transistors, resistors, and will-power.
  • Brand: I have a lot of ideas for branding and apparel that I hope to set up this year and start passively selling. I’m hoping to work with a graphics designer to actually translate the logo to SVG and then from there make some awesome shirts, stickers, and more all relevant to my music and writing. I’m not looking to make a lot of money, rather I want to get a better handle on more undefined and hard-to-quantify creative work. Maybe this will manifest in taking some design courses for vector/raster graphics and limited video production, but the other things here are more important toward this end.
  • Music: I have a ridiclous amount of music that I’ve composed over the last fifteen(:interrobang:) years, and it’s high time to start actually producing it and making it available.
  • Writing: I also have a lot of writing that I’ve done, and I’d like to continue writing and actually start forming things into stuff I can release to the public and to friends.
  • Language: I’d like to actually learn a new verbal language this year.
  • Friends: I want to start more actively spending time with my friends outide of work. One of my friends and I talk every day over Signal, but as useful as this is, it’s important to do more to actively spend time with my friends IRL and outside of work.

More Go and Rust are pretty self-evident, and I hope they’ll naturally occur as part of my work.

For embedded and electrical engineering, it’s part of my race to the bottom to be able to actually understand things all the way up the stack. Presently, I have a working knowledge from massive autonomous and distributed systems all the way down to registers and cache coherence. I understand the general principles of transistors and logic gates, but I want to keep going. I hope by the end of the year to feel like I can just choose a project and build it. Specifically, I’ve done a lot of work to figure out how my car’s head-unit speaks to the CD changer in the back of the car and I want to build a Raspberry Pi/Arduino system to be able to speak the protocol to build a complicated media playback system from scratch. I have the pin-outs and the protocol definition as implemented in C, and I’d like to actually build a system which will allow me to play back my full ~60GiB music library similar to the PhatNoise system I’ve been using for the last decade.

For branding, my goals are pretty simple. I have a logo that I need to get digitized and then start combining this with my music and writing. If I’m able to get the logo digitized, then it’s just a matter of slowly but surely creating content and items that the world can follow and engage with.

As for music, a friend and I spent time on Christmas Eve producing a song which turned out incredibly well. I’m extremely grateful to this friend, he gave me back my music. I’m starting music lessons this year to work on my voice, piano, guitar, drums, music theory, and hopefully acquire the ability to produce my own music effectively and feel empowered to fearlessly create things and publish them.

My writing goals are of a similar vein. I have a lot of content that I want to start combining and organizing so that it can be released, perhaps privately at first.

As for languages, I can understand a lot of different languages if I listen and focus, but I want to be able to hold basic conversations in another language by the end of the year.

Friends are really important, and I regretfully didn’t see my friends as much as I should have this year. Signal and Slack are great to keep a heartbeat in a friendship, but I want to make an effort to see people more regularly.

Conclusion

2018 felt like a very busy but not very productive year, though in writing the above and reading it, it’s clear that I actually did accomplish a lot. I feel like 2018 came and went and I can hardly believe that it’s now 2019.

All-in-all, I did a lot in 2018 despite not feeling like it. When it comes down to it, from an engineering perspective there is nothing outside of my reach. Operating systems, device drivers, memory allocators, web services, CLI tools, automation… I can sit down and do all of it. I want to extend this to areas where I don’t have the same confidence. I want to translate my engineering experience into non-engineering projects where I can have that same level of confidence.

Let’s do it :muscle:

Related Posts