Ever since I’ve taken on the role as a full-time devops in my new team, the amount of tasks that I have to juggle in a workday has exploded exponentially. I’ve written/complained about it nearly 2 months ago on organizing chaotic information.
It is now time to share what I’ve tried, what worked and what didn’t.
At this point I’m not sure whether to call this weekly any more cause I’m just haphazardly writing roughly on a weekly basis but damn it I’m just going to keep this going.
I am pleased to say that I have finally passed my AWS DevOps Engineer – Professional certification! It was quite a lot of hard work, like it was honestly harder than I expected it to be cause most of the questions were situational and very AWS specific in-terms of CICD. Honestly, I took this because I thought it would be easier compared to the Solution Architect Professional. But man I was wrong.
This also means that I would probably be looking to pick up the CSAP cert when I have the time for it, perhaps at the end of the year.
It has been a long time since I’ve studied so hard for something, and it was really helpful not just for the exam, but I realized that there were a lot of tools/services I could’ve used for my current team that we weren’t using yet. I think we are very capable in designing functional services, but there’s still a gap between change management and having full visibility over everything. I’m planning to apply some of the things I’ve learnt in my team, cause it helps to bring us one step closer to having DevOps as culture.
Food for thought used to be a series in my old blogs for something that I just dump my random musings, I’m reviving it now because I’m in a new phase of life.
Up until recently, I felt that people are jaded/spoilt when they spend “unnecessarily” on things they want instead of things they need, or getting the more premium option when the normal option would’ve worked fine.
But after reviewing my spending trends across the past year, I feel like I’m starting to become that kind of person. For example, taking a cab used to be: “I’ll take a cab only if I’m really tired or it’s really just way too troublesome to get there”. Nowadays it’s more like: “I’ll take the public transport only if I’m feel energetic and I feel like it”.
I’m attributing this change to having a job, and not having many obligations (yet). “Adult money” as me and my partner would like to call it, is dangerous to us. Given the means, there is an increasing number of things where it does not makes sense to spend the extra/time and effort when money can solve it.
When I think about money as a function of my life energy that I’m exchanging for, I want to spend the remaining life energy (and time), on the things that I truly care about. When I’m meeting friends, the part that matters is meeting the people, not the travelling to the destination. If I can find a tool that gives me a better working experience, it’s worth the extra because it reduces the friction of doing “something”, which has a multiplicative effect on the consumption of life energy.
function of life energy: spending X amount of life to get Y money.
I feel that the relationship between income and the amount someone is willing to spend on conveniences is approximately proportional. The (rough) graph drawn below shows the relationship, and I feel like I’m breaching into the conveniences territory which prompted this FFT post.
This also reminded me of The Last Bit in the Container post I’ve read may years ago. TLDR; squeeze your toothpaste if it only takes a few seconds, else it’s not worth it.
Light bulb moment
Have I been misunderstanding those people who have been spending for convenience and quality of life? Is this because that I have not reached the same phase of life as them?
Moments like this really make me hit the pause button and think really hard about my life.
An interesting conversation I had sometime ago was an advice to “be kind to the older folks in IT” even though sometimes their ideas/suggestion may seem very outdated. It’s because their reality has been shaped by the experience of working with the older systems and they have learnt many lessons the hard way; instead of dismissing it as “irrelevant in current context”, it could just be an old solution to a still plausible problem (that we may have missed).
I feel like I’m experiencing the “You’ll understand when you’re a parent yourself” moment before becoming a parent.
Getting back on track, I don’t think I would change my spending drastically even with this realization because I’m still able to justify for them. What I would do instead, is question myself: “would this money be better spent invested in something?”.
(Ending off this post as I sip coffee on my new $20 thermos mug)
In preparation for my upcoming AWS Exams, I’ve gotten some PDF materials to go through and study. However, the formatting of the document is terrible, and it made it really hard for me to study.
So I moved to my iPad, hoping that some Apple magic might help with making the text more readable instead of sprawling across the entire width of the screen. That didn’t help.
“There has got to be a way to reflow the text”, I thought. I ended up downloading 3 different PDF reader apps looking for that magical bullet that would solve all my problems.
One didn’t have the functionality
One worked but it made the format worse
One works beautifully, but I cannot annotate or highlight in the “Reflow” mode which made it basically useless for studying
I gave up on the iPad and I thought, there has to be a way on the Desktop that would help me to reflow the text. My default go-to PDF reader: SumatraPDF didn’t have that option. After Googling for way too many minutes, there was basically no obvious option that could solve my problem of having reflowable text and still annotatable (and free).
It was when I came across (rediscovered) that yes, you could convert a PDF into a Word document. So I quickly searched for “Word” in my start menu and guess what, I don’t have it; because I recently formatted my computer. The version I had in the past was my education version that I shouldn’t have access to anymore. But I still tried, logging into my old school email to dig for that option that allows me to install Office.
That led me to searching online for what’s the cheapest way to get Office legitimately, preferably something that is a one-time license and not a subscription fee for a product that I only use infrequently. It was during this search that I saw someone mention that “education” should be free. I thought, okay, why not give it another shot.
Bam! Logging in with my student email through the official Microsoft Office site gave me an option to download a genuine version of Office that is properly licensed. The best part? I apparently performed some kind of voodoo in the past for claiming office, and the license actually belongs to my personal account and it all ties in nicely with my existing documents.
I finally have Word now.
Yes, it was able to convert PDFs into a Word document, no problem. This solves my reflow and annotation problems.
Then I thought, hey, doesn’t this mean that I could now study on my iPad?
Pushed the document over to my iPad then I thought, “wouldn’t it be nice to have Word here too”. I remembered that I was able to use it freely on my 8″ Xiaomi Tablet. Downloaded it, fired it up and it asked for my account, I logged in, only for it to tell me that
Confused, I did a quick check online, so…
Apparently, any devices >10 inches are considered professional use, which basically rules out all iPad Pros out there. Which means that I would need to purchase a subscription in order to use it. Nevertheless, this caused me to open the Pages app for the first time ever, and it managed to open up the document flawlessly.
It was at this point when my girlfriend asked me, “why not just print this out?”
In my relentless pursue for a digital solution to read a damn PDF comfortably, it totally slipped my mind that sometimes having a physical copy is much simpler and elegant.
As I slowly sink into the rabbit hole of the world of mechanical keyboards, I finally got my hands on my first set of SA Profile Key Caps from Domikeys [AliExpress].
For those that are not familiar with key cap profiles, here’s a quick primer on them. Key caps comes in various shapes and heights, most common of them are OEM and Cherry Profile which are on most of the pre-built mechanical keyboards.
SA profile as you can see is much higher and has a retro feel/look to it. I have been curious about it cause I think it has a type writer look which tickles my fancy.
I bought black and white because I realize that I didn’t have any basic sets like these. Coincidentally, this goes really well with my Black Drop CTRL keyboard that I’m currently using.
It feels distinctively different
I have tried OEM, Cherry, DSA, XDA keycaps before, but SA is a whole different beast altogether. The main difference for me is that the gaps between the keys are much wider than I’m used to and it feels like my fingers will always fall into the crevices when I’m not too careful with the way that I type.
The other difference is its weight. Because it’s so much taller than the other keys, there is obviously much more material to it, which makes it heavier. But because the keys is heavier, it makes typing feel lighter. It also feels sturdier because these are some thiccc bois.
I like it. The keyboard definitely has a nicer thock sound with SA key caps.
I have two GMK key caps set shipping in Oct and Dec. And I am extremely excited for them.
GMK Blue Samurai
GMK Mito Laser
My main motivation for getting this set is because I didn’t like how cheap the PBT key caps that came with the CTRL keyboard feels. I’ve swapped them out with another cheap set that I have but it only improved the feel ever so slightly.
Have also placed an order for the Keychron K6 (hot-swap version) which should be arriving this week as well. This would be purely for experimenting different switches and possibly be my portable keyboard when I feel like working outside.
Took an online introductory course (Udemy) on Microsoft Azure AZ-900 because lo-and-behold, my team has chosen the Azure platform for our translation services (will write more about this next time).
As someone who has been 99.99% working on the AWS platform and Linux systems in general, Azure feels pretty foreign because most of the concepts seem to tie into the Windows systems more so than anything else.
Access control? Active Directory
RBAC? Active Directory
Networking? Virtual networks
Compliance? Almost everything under the roof
The main difference I find between AWS and Azure is that: AWS is a loose collection of services that are “grouped” through networking, Azure is a logical collection of services that are “grouped” by “folders” of resources.
The title sounds grander than this really is. It was one of those work days where I felt like I didn’t get much done. I checked my calendar and there wasn’t many meetings, only the one in the morning. It felt like a really busy day but I couldn’t think of a concrete task that I have accomplished that day.
As I lay in my bed, tossing and turning, being unable to sleep, I figured out why I couldn’t get my tasks done for the day, and came up with a simple workflow that would solve this.
Why I wasn’t able to work on my tasks
A day in the life of a software/devops engineer is pretty chaotic. You have various information requiring different context streaming in from multiple sources throughout the day. For example, I was working on updating some configuration mapping on Kubernetes for our new SES SMTP Relay credentials. Then I get a message clarifying about a story that I completed yesterday, about a backend API written in GO. Then I had to join a meeting about decoupling our entire platform from an external service that many of our logic is intertwined with.
This came about because of something I discovered recently about building a second brain. The prospect of it is extremely enticing for me.
Idea is that over time you build a second brain that is like a digital collection of all the knowledge that you’ve gained over your lifetime.
As someone working in the digital field, the amount of information that I go through on a daily basis is pretty huge. I’ve been taking notes for a million and one things, but I realize that I’ve almost never really gone through my notes and make something out of it. Which I felt has been really wasteful because, why would I even write them in the first place if I’m not going to use it? How many % of the things I’ve written can I actually remember in my dumb human brain?
Armed with the motivation to build a digital brain that I can tap into for creating new ideas and products, I embarked on part 1 of the journey.
Finding the right tool
The “original” tool (that I know of) is known as Roam Research, however, it’s a web only tool currently, and it’s a paid service of $15/month. This makes it slightly undesirable as I would prefer if it’s something that I could potentially migrate/export out of the system. I also wish that there exist a free option that I can try out to see if this second brain business is something that I really want.
I checked out 8 different not taking tools to see what works for me and compare across them.
I’ve recently write a simple merge request “bot” for my team. To be honest it’s more of a glorified reminder but hey, it works! I’ve done a slightly more technical write up at dev.to.
Summary: It’s a bot that will collate all the open Merge Requests we have on our private and public Gitlab repositories, that have the Review Me label, but don’t have at least 2 reviewer labels yet, and send it to our Slack channel.
It’s been running since 3rd August, and in this post, I just want to note down my observations for the past 20 days.
More people are actively taking on the role of reviewing the MRs. Before the bot our scrum master has to manually collect them and bring awareness to the team that there are MRs that needs people to take ownership of. But for the past 3 weeks, it seemed to have improved. Most of the times when someone ask for reviews on their MR, we usually have 2 people who take them up before the next reminder*.
*the reminders are set to run at 11am and 3pm (4 hours apart)
The team seemed to be quite receptive of the bot reminding them about the open MRs, the little easter eggs of encouragement seems to help with the team morale from time to time as well.
Hard to quantify, but the time taken for an MR to get approved seems to have shortened, which may indicate that people are more eager to approve and may not review as thoroughly. Case in point, I approved one MR that contains relatively inefficient code, that was pointed out my my colleague when she was working on that section.
People are taking on the role of reviewing but sometimes it slips under their radar and forget to approve even when all the issues have been resolved. I guess this is something that the bot can improve on but don’t really have a clear idea of what can be done about it yet.
I’m glad that I wrote it. It didn’t really take a long time, just some inspiration and inconvenience that prompted it. I am carefully considering adding metrics for how long an MR stays open for and all that. But I feel like this might create unwanted attention into our work so it’s something that I haven’t explore deeply into yet. But I think it would be pretty interesting!
The past week has been extremely exciting and nerve-wrecking. My team has finally completed the migration from on-premise to the cloud. It’s the first time that I’ve done anything like this and I’m blessed to have someone senior to lead us through the migration period.
ps: I wrote but forgot to post so this was actually 2-3 weeks ago
I’m a part of the MyCareersFutureSG team, so our users are the working population of Singapore, and we host hundreds of thousands of job postings, so there are definitely some challenge in migrating the data.
It’s the first time that I’ve handled such huge amounts of data when migrating across platform and the validation and verification process is really scary, especially when we couldn’t get the two checksum to match. It’s also the first time that I’ve done multiple Kubernetes cluster base image upgrade rollover. There were multiple occasions where we were scared that the cluster will completely crash but it managed to survive the transition.
Let me sum up the things I’ve learnt over the migration.
When faced with large amount of data, divide and conquer. Split data into smaller subsets so that you have enough resource to compute.
When rolling nodes, having two separate auto scaling groups will allow you to test the new image before rolling every single node.
If you want to tweak the ASG itself, detach all the nodes first so that you will have an “unmanaged” cluster, then no matter what you do to the existing ASG, at least your cluster will still stay up.
When your database tells you that the checksum doesn’t match, make sure that when you dump the data, it’s in the right collation, or right encoding format
Point your error pages at a static provider like S3, because if you point it at some live resource, there’s a chance that a mis-configuration will show an ugly 503 message. (something that happened briefly for us)
Data less than 100GB is somewhat reasonable to migrate over the internet these days
Running checksum hash on thousands and thousands files is quite computationally and memory intensive, provision enough resources for it.
Overall, the migration actually went over quite well and we completed ahead of time. Of course, the testing afterwards is where we find bugs that we have never found before because it’s the first time in years that so many eyes are on the system at the same time.
The smoothness is also thanks to the team who has carefully planned the steps required to migrate the data over, as well as setup streaming backups to the new infrastructure so that half of the data is already in place and we just need to verify that the streamed data is bit perfect.
Since it’s been a couple of weeks since this happened, I realize that I am lucky to be blessed with the opportunity to do something like this. Cause I’ve just caught up with my friends and most of the times, their job scopes don’t really allow them to do something that far out of scope. Which… depending on your stage of life it could be viewed as a pro/con. I’m definitely viewing this 4 day migration effort over a public holiday weekend as positive cause it’s something not everyone can experience so early on in their career!