A Dependency Upgrade

The views expressed here in this post are not representative of my employer, 1Password except where explicitly called out.

Last week, I made a PR to an open-source repository. It was a seemingly innocuous change. One that only upgraded a submodule in the repository to ensure the underlying C code was kept up-to-date. Fast forward to today, I woke up to multiple Slack, Discord messages and emails. "[your work] is getting mentioned in a thread about an SSH backdoor", "sorry about all the [...] drama". Implications that I was involved in a larger plot. My PR updated xz inside of a Go bindings library.

For those who haven't heard, a malicious backdoor was found in xz. What's most interesting about this is that it comes from a maintainer who had been working on the project for years. Without going to into the original timeline, it seems as-if this was a concerted effort by an individual over years to gain a position of power of xz and subsequently rollout a backdoor. Thankfully, this was (seemingly) caught well before it could have a larger impact surface.

I created my PR to the Go bindings for this impacted C code as a way to help chip in to a project that hadn't seen love in 3 years. I've always been a fan of xz as a compression algorithm as I find myself constantly in bad bandwidth scenarios (even as I write this now, I'm suffering with my flaky connection in Vietnam). So, when I was faced with using xz in another one of my projects, I opted to try to find a library that wasn't plagued with speed issues and had a better maintenance track record. That's when I stumbled upon https://github.com/jamespfennell/xz. My only concern with using this library was that it hadn't been updated in 3 years, hence the motivator for seeing if I could get it to a better place. James wrote an amazing vendorc tool that automatically generated everything required from the underlying C submodule which enabled me to easily do this work. I created the PR, used it in my project, and then swiftly forgot about it until I noticed a ping to update the LICENSE files to reflect the public domain -> 0BSD transition. That was the last I heard anything, until this morning when I woke up to plethora of messages asking me about my PR. What my motivators were, if I had any involvement with the upstream, etc. For a simple dependency upgrade.

Examining the upstream situation more closely, it's easy to see why people immediately lept to conclusions about my work.

The maintainer behind the vulnerable code, Jia Tan, appears to have been behind many pushes to various distros to include the vulnerable code as soon as possible. Which, from the perspective of PR, could appear as if I was behind the same thing. However, my PR didn't include the files required to make the exploit possible to begin with and the usage of this library is basically non-existent (so it'd be a questionable target...). As it tends to be during high stress situations, people continued to speculate none-the-less.

With the help of my employer, given that they had been notified of my association with this vulnerability at this point, I put out a statement detailing the context behind my changes mentioned above.

Thankfully, it appears this is the end for the intense period of scrutiny I faced. However, it leads me to reconsider the possible impact open source changes can have. What if I had decided to mess with the generator tool present in that repository and it accidentally added more files? What if those files had been among the ones required to make this exploit happen? What if my employer didn't want to deal with any potential negative association? These are all among the questions that will be present in my head anytime I make a contribution in the future. I worry these types of attacks will shatter the culture of trust we have in the OSS world currently, which could lead to increased barriers to contributions that allowed me to get to where I am today.

I want to give a shoutout to my employer 1Password for being with me from the beginning, no questions asked, to the end of this experience. ❤️ you all. Special thanks to Evan Boehs for his detailed notes that helped create this post.