What we know about the SolarWinds hack that allowed a breach of multiple government agencies, and what we can do about it
In this episode, we about federal and state antitrust lawsuits against Facebook, and a new DNS technique backed by Apple, Cloudflare, and Fastly called Oblivious DNS. Then we speak with Hector Monsegur, security researcher and former blackhat hacker, about a major hack against multiple government agencies. Then we chat with Penelope Phippen, tech lead at Stripe, and a Director at Ruby Central, about the release of Ruby 3.0.
Saron Yitbarek is the founder of Disco, host of the CodeNewbie podcast, and co-host of the base.cs podcast.
Josh Puetz is Principal Software Engineer at Forem.
Hector X. Monsegur is director of research at Alacrinet and a former black hat hacker.
Penelope Phippen (she/her) is a multifaceted Rubyist who works as a Director at Ruby Central, is the creator of Rubyfmt, and was formerly a lead maintainer of the RSpec project. She frequently writes and speaks about about complex aspects of the Ruby grammar, and issues of social justice for trans people in computer science. She's sad that she can't hug every cat.
[00:00:10] SY: Welcome to DevNews, the news show for developers by developers, where we cover the latest in the world of tech. I’m Saron Yitbarek, Founder of Disco.
[00:00:19] JP: And I’m Josh Puetz, Principal Engineer at Forem.
[00:00:22] SY: This week, we’re talking about federal antitrust lawsuits against Facebook and a new DNS technique backed by Apple, Cloudflare, and Fastly.
[00:00:31] JP: Then we speak with Hector Monsegur, Security Researcher and Former Black Hat Hacker, about a major hack against multiple government agencies.
[00:00:39] HM: This goes beyond just, “Oh, yeah, just another company you got hacked today.” No, no, no, no. This is a major deal. In this case, it is definitely a national security level issue.
[00:00:48] SY: Then we chat with Penelope Phippen, Tech Lead at Stripe and a director at Ruby Central, about the release of Ruby 3.0.
[00:00:55] PP: None of them are that huge of a change to the previous review release. But if you sum them up over many years, there has been massive incremental progress.
[00:01:07] SY: So Cloudflare announced in a blog post last week that in conjunction with Apple and Fastly, that they have created a new DNS standard, they’re calling Oblivious DNS in an effort to make… I love that name. Isn’t it great? In an effort to make your internet browsing more private. For those of you who might not know, DNS stands for “Domain Name System” and among other things, it translates domain names to numerical IP addresses, mapping, and connecting those names to internet resources. Now the thing about DNS is that there are possible privacy and security risks that can allow service providers and other people to see what sites you’re visiting. Traditional DNS queries are done unencrypted so an attacker can see what site you’re requesting and even return a different IP address. So Cloudflare says that Oblivious DNS prevents potential snoops from seeing the addresses you’re going to by encrypting it with clients using HPKE, Hybrid Public Key Encryption, and transmitting these requests to a proxy over an HTTPS connection. They’re testing performance to see if the addition of the proxy and encryption slows things down too much, which obviously is not good. But according to one study, the early results are looking pretty good. Josh, what do you think?
[00:02:26] JP: I kind of totally forgot that DNS queries are usually not encrypted.
[00:02:31] SY: Surprise!
[00:02:32] JP: Surprise! This is wild, right? Like I think this is an excellent step, obviously, but it makes you wonder, like, “What’s been taken so long?”
[00:02:42] SY: Yeah. Now that we’re looking at just the facts and the history, it feels like an obvious thing that we should have had for a while now. It really does kind of pose the question of like, “Why have we waited? And also what has prompted it to happen now?” Is it some breach? Is it just people being more concerned about privacy and security? Did something happen in the last, I don’t know, when this project started? But you know what I mean? Why now? What is the thing that got them interested in this now?
[00:03:12] JP: I mean, I think we’ll be talking about this with our guests today, but I think we’re hearing about more and more state-sponsored cyberattacks. And I think the focus on security and encryption and just protecting internet traffic is moving from a personal level to a government nation level. And I think that’s kind of why we’re seeing some of the push behind this. I mean, it takes these large agencies that are, Cloudflare and Apple and approving regulatory bodies over the internet, it takes them years to come up with these proposals and test them and then get them adopted.
[00:03:46] SY: Right. Right. Right. Right. That’s what I’m really curious about is the adoption of the Oblivious DNS technology. Is that something that everyone is going to do? Is it going to be implemented? On our behalf, how’s it actually going to be used in real life? It’s something I’m really interested in.
[00:04:02] JP: Yeah. And reading about this, I didn’t realize there’s a kind of intermediate technology called DNS over HTTPS, which just basically does kind of like when you log into your bank or any other site nowadays, you get an HTTPS connection that is encrypted instead of a plain text HTTP connection. And there’s a proposal to add just that layer of protection to DNS. And it’s actually been around for a number of years and hasn’t really gotten any traction. You mentioned testing about the latency and I wonder if that’s a big part of it. Anytime you’re encrypting and decrypting, you’re adding time. And I don’t know about you, but I’ve had a situation where a DNS server has been down or responding slowly. It’s maddening.
[00:04:41] SY: Obviously. I’m really curious to see what the performance results are. It seems promising for now, but we’ll see where it goes.
[00:04:48] JP: Well, now turning to Facebook. We might be starting to see some of the first major outcomes from that congressional hearing where tech giants Facebook, Amazon, Google, and Apple were all grilled about their practices in front of Congress. Now after an over 18-month investigation into Facebook, the Federal Trade Commission and more than 40 States and a huge antitrust lawsuit say that the company illegally squash competition by buying up its challengers. The two major acquisitions that are the basis of the lawsuit are the $1 billion purchase of Instagram and the $19 billion purchase of WhatsApp. So this will of course be a very long legal battle. But if Facebook is found guilty of anti-competitive practices and monopolization, prosecutors are hoping to break off Instagram and WhatsApp into separate companies once again and create new restrictions and regulations on who Facebook would be able to purchase in the future. Saron, what do you think about this? Is this what you thought might happen?
[00:05:49] SY: No, definitely not. I did not. I feel like there are always congressional hearings about this. Aren’t there?
[00:05:55] JP: Right. Yes.
[00:05:56] SY: I feel like this has been a very predictable thing that’s been happening for years. I can’t remember the number of times we’ve seen all the big giants in the courtroom and I just kind of got over it. I was like, “Nothing’s really going to happen. These are just the formalities and they’ve got too much money. They’ve got too much influence.” They’re just too powerful, like nothing’s actually going to happen to them. So hearing about this advancement, it’s like, “Oh, wow!” Something kind of happened.
[00:06:24] JP: The last I remember something like this happening in tech was I think it was the mid-90s, there was the threat of breaking up Microsoft.
[00:06:31] SY: Right.
[00:06:32] JP: It didn’t actually go through. I don’t remember the exact particulars of the case, but ultimately Microsoft of course was not broken up. But looking back at the history, there was damage definitely done to Microsoft’s brand.
[00:06:48] SY: Competition. Yeah.
[00:06:50] JP: Yeah. And you could argue that being involved in a legal case kind of blunted their corporate strategy for a number of years. And I don’t know if that’s why Microsoft isn’t the behemoth it was back in the ’90s, but it definitely took a toll. So maybe even if prosecutors don’t think they actually can break up Facebook, maybe they’re thinking like just the lawsuit being in the public consciousness can slow them down somewhat.
[00:07:16] SY: Interesting. Interesting. Yeah. I mean, the pessimist in me is kind of like, “Nothing’s going to happen with the lawsuit.” I still think that Facebook is fairly safe and being able to do what they’ve always been doing, but it definitely makes me feel like we have a little bit more power in the situation and it definitely makes the whole congressional hearing thing feel a little bit more valuable than it did before the lawsuit happened. But ultimately, I don’t really think anything’s going to come of it.
[00:07:42] JP: Do you think we’ll see other lawsuits? I could imagine Amazon being prime for monopoly lawsuit.
[00:07:47] SY: Possibly. Yeah. Yeah. Possibly. I mean, Facebook, I feel like has been the target for longer than Amazon, if I’m remembering my history correctly. So it makes sense that Facebook will be the first one to be sued and maybe Amazon is next. Yeah.
[00:08:02] JP: Well, we’ll have to see, we’ll definitely be looking at this for many, many months to come. This lawsuit is not going anywhere quickly.
[00:08:10] SY: Speaking of the US government, coming up next, we are joined by Hector Monsegur, Security Researcher and Former Black Hat Hacker, to shed some light on a major attack that breach several government agencies, including the US Treasury and Commerce Departments after this.
[00:08:47] JL: Triplebyte is a job search platform that allows you to take a coding quiz for a variety of tracks to identify your strengths, improve your skills, and help you find your dream job. The service is free for engineers and Triplebyte will even cover your flights and hotels for final interviews.
[00:09:02] SY: Vonage is a cloud communications platform that allows developers to integrate voice, video, and messaging into their applications using their communication APIs. Whether you’re wanting to build video calls into your app, create a Facebook bot or build applications on top of programmable phone numbers, you’ll have all the tools you need. Formally known as Nexmo, Vonage has you covered for all API communications projects. Sign up for an account at nexmo.dev/DEVNEWS2 and use promo code DEVNEWS2 for 10 euros of free credit. That’s D-E-V-N-E-W-S, in all caps, and the number 2, for 10 euros of free credit.
[00:09:48] SY: Here with us is Security Researcher and Former Black Hat Hacker, Hector Monsegur. Thank you so much for joining us.
[00:09:54] HM: Well, thanks for having me.
[00:09:56] SY: So tell us a bit about your hacking background and the kind of work you do now.
[00:10:00] HM: Well, I’m not too proud of the past here, but it’s a good lesson. I’m a former black hat. Starting back in ’95 when I was 12 years old, I really got deep into the whole hacking scene and the concepts. One of my first compromises was a Japanese university. That was very early on. And it’s how I learned Unix. It’s how I learned about local Unix exploits when I was 16, three years later. One of the first major hacking operations I ever executed was against the United States Navy for the abuses that were taking place in the Island of Vieques. They were using depleted uranium shells on an inhabited island. Right after that, it was a conflict with the Chinese government, 2001. And then you fast forward, Indonesia and Pakistan and India. You fast forward even further, I just kind of went crazy during the Arab Spring, starting with attacking the Government of Tunisia during the uprising, shutting down their self-service and internet service right at the height of it, focusing my efforts on the Government of Egypt during the revolution. Algeria, Syria, Libya, Iran, you name it. I was there, compromising elements with the governments, taking down elements of their communications, attacking company, I was about to say organizations, but governments like the Russian government and even looking into North Korea. I’ve kind of been around. I’m not proud of it. I’m not here to glorify that past, but it gave me a lot of perspective. And as for what I’m doing now, I’m essentially what they would call a white hat where I do the same kind of work, except that it’s legal this time. I have defined scopes with clients and I help them figure it out, meaning I break into their systems, their networks, do some social engineering, whatever it is I have to do to compromise the environments, and then I provide them a report on how I did it.
[00:11:56] JP: So this week, it was revealed that various federal government agencies, including things like the US Treasury and Commerce Departments, they were compromised as part of a global cyber espionage campaign. Can you paint us a picture of what we know about these breaches right now?
[00:12:10] HM: So what you’ve read in the news really goes into the conversation that we all should be having regarding supply chain attacks. And that’s the core attack vector, in regards to this big story. One provider, SolarWinds, was compromised at some point, and its applications were also compromised to the degree that it allowed attackers to distribute malware across various clients. We’ve seen these kinds of attacks before then, but they’re much more rare because it requires that the attacker focus on a provider as opposed to attacking the clients directly. So imagine this. Imagine you are a state actor and you want to compromise, let’s say, Department of Homeland Security, but you really don’t want to deal with the DHS’s security policies and mechanisms and all that good stuff. So you have to think outside the box, “Well, what do DHS system administrators run on their networks?” And it just so happens that SolarWinds is one such application. It’s quite broad, it’s distributed across various industries from governments to corporates, even cybersecurity in the case of FireEye. So it allowed attackers a wide spectrum of potential attacks. And that’s essentially what happened there.
[00:13:40] JP: Do we know what kind of information was stolen or compromised from this latest SolarWinds hack?
[00:13:46] JP: What we do know is that the compromise of SolarWinds and the Orion products led to the compromise of several clients or mostly government security companies and probably those that hold a lot of intellectual property. We know the compromises take place, but unfortunately, because of ongoing investigations, we’re probably not going to see the impact of those compromises until either a research has come out and kind of like dump it or some sort of exclusives comes out or maybe there’s even a court case five years from now. The truth of the matter is we’re not going to know the full impact until very, very later on.
[00:14:28] SY: What is so novel about this attack? It feels very different from, I guess, what we’re used to reading about with phishing campaigns and bug exploitation. It just feels like a different thing. What’s so new about it?
[00:14:39] HM: Well, it’s not necessarily new. It’s just that it could be considered sophisticated. What you’re seeing in the news day-to-day are attacks that are specific to the human element, which is social engineering via email. In some cases, like in the case of Equifax, you had a situation where there was a vulnerability and the attackers were able to directly attack Equifax. In the case of social engineering and phishing, this is a situation where attackers could focus on the users, right? So the best way I can put this here is that a supply chain attack is when you just completely bypass all of that. And your focus is more targeting the provider that provides you services. A good example would be this. What if someone were to attack Microsoft, compromise Microsoft, and they were able to distribute malware directly from Microsoft update servers? As you could imagine, that scale was so mind blowing that you would say, “Yeah.” I mean, at that point, we’re all compromised, right? It’s kind of the same thing with SolarWinds. They were a provider for a large user base and one major point is that when you’re dealing with a supply chain attack, you, the customer or the agency or whatever it is, you trust SolarWinds’ products to distribute updates securely and properly. Because of how embedded SolarWinds as a product was or in this case, SolarWinds’ Orion products, it required a bunch of permissions, administrative permissions that usually are not permitted to, like, let’s say a day-to-day malware. Now with that being said, it was much easier for the attackers to leverage their access of the SolarWinds’ servers to move laterally across clients, simply because there were a lot of permissions given to that software.
[00:16:33] JP: How common is it for attackers to target an IT provider like this? And it seems like this would be very powerful. Why don’t we see more hacks of this technique?
[00:16:47] HM: That is a great question. These kinds of attacks are much more rare and we’ll try to get into that in a moment, but they’re also very sophisticated. And what I mean by that is that it requires, one, a lot of time. It requires that you know your target environment very well, more than likely the attack on SolarWinds probably consisted of a lot of information gathering, research, infrastructure, just kind of like stakeouts, right? I mean, just think about when you watch old films, where like this FBI agent following somebody and they’re like sitting in the car for like two days, it’s kind of like that. They’re casing the joint out. We’ve seen this attack happen several times. OPM was a great example, the Office of Personnel Management. We know for a fact that the attack was involved in that and I say “attackers”, plural, because it seems like there was at least two actors involved. We know that it was many years long engagements. It wasn’t necessarily a supply chain attack, but it’s kind of the same modus operandi. A more recent example that you guys may remember is the one with CCleaner. That was a major compromise because it required that the attacker compromised not all the developers of CCleaner, and then go beyond that, like understand the infrastructure, understand the code base to a degree that the attackers were able to code updates into the source code.
[00:18:15] SY: Wow!
[00:18:16] HM: And that’s pretty amazing. And then going beyond that, there are similarities, at least for me, between the CCleaner attack and this one with SolarWinds because the attacker was very specific. Once they got the access that they needed, they were able to then kind of choose their targets. Right? In the case of CCleaner, the software was installed on millions of computers around the world. Okay? But the attacker only chose to compromise, I don’t know, maybe two dozen machines out of millions. This was a very specific attack. In the case of SolarWinds, it’s almost the same. So yeah, it’s definitely sophisticated. We don’t see it as often as you would see like a phishing attack and I think it’s because of time consumption and cost.
[00:19:04] SY: So I think what’s most alarming about this attack to the rest of us, what makes it just so wild is that it was a cybersecurity firm that was hacked like this. And it kind of feels like if a cybersecurity firm that’s doing business with the government isn’t safe from this type of attack, then who is? Do the rest of us have a chance against a hack like this?
[00:19:27] HM: Well, the truth of the matter is, is that, and I’m not here to spread fear, uncertainty, and doubts, but no, we don’t have a chance. I mean, that’s just the way it is. It’s like if someone were to compromise your ISP, whether it’s Optimum or AT&T or whatever it is you’re using, if someone compromises your service provider, there’s nothing you can do about that. And if the attack involves compromising you along the way, there’s not much you can do about that either. So it becomes a difficult position for all of us. This goes beyond just, “Oh yeah, just another company got hacked today.” No, no, no. This is a major deal. In this case, it is definitely a national security level issue. So expect a lot of investigations and questions and consequences as a result of this. In fact, I believe a couple of nights ago, these SolarWinds’ headquarters were raided by the FBI. So this is a major.
[00:20:25] JP: So you’ve mentioned that there’s, as end consumers, if our service provider were to be compromised in this way, there’s very little we could do. Is there anything that the companies and agencies involved in this attack downstream from SolarWinds could have done to protect themselves?
[00:20:43] HM: When you started looking at the reports that are coming out, FireEye has been doing a fantastic job ads disclosing indicators of compromise in their own research because they are victims of this, right? So they are publishing their research online for free. Go to FireEye.com and look at the blog. And they’ve done a fantastic job explaining what it is that the malware actually does once it affects your machines. Unfortunately, there was a lot of research being done on the side of the attackers. It would have been difficult for a random Joe or a Jane to secure themselves from that infection. The malware itself is distributed through SolarWinds’ Orion’s update feature or plugin feature. And that software itself has administrative privileges on your machine. So in theory, in order for you to defend yourself, you would have had to disabled the SolarWinds’ software prior to the disclosure of the attack. And in that case, you don’t really need the software anymore, you’re going to have it disabled, right?
[00:21:48] SY: Right. So it seems that whenever there’s some type of attack that’s believed to be carried out by a nation state, all fingers tend to point to Russia. That’s kind of where everyone goes. But how do you really know? What are the different ways you can determine where a hack is coming from?
[00:22:05] HM: Well, we have to look at the level of sophistication. So as you guys know, my story’s a little bit weird because I was just like this random guy, I lived in the projects, like I was nobody, I was not a nation state, but I was doing nation state attacks. And what does that mean? That means that I was doing highly sophisticated, fast paced or long-term engagements against government agencies or security companies. So if hadn’t I been arrested and hadn’t you and I have just been on the podcast right now, someone somewhere would suspect me as doing this. Right? And that’s the scary thing. Attribution becomes a problem because there’s nobody in the world that could positively point a finger at whoever and say, “Yeah, it was that guy.” Right? So it requires a lot of investigations. FireEye has been doing a great job at researching. This is all the companies too that have been publishing their content. I would suggest you start at the FireEye blog and then you just follow the rabbit hole. You’re going to find a lot of great research out there, especially on InfoSec Twitter. And there’s a lot of fantastic folks, kind of like looking into the attack, vector, the methodology, the malware being used. So the key takeaway here, the biggest takeaway here is that yes, I would say most more than likely it was a nation state. It wasn’t a regular Joe blow. It wasn’t a Hector, right? It was likely a government and at the very least it was subcontracted to a bunch of guys like my former self.
[00:23:34] JP: You mentioned that the level of sophistication of the attack is one thing that could indicate it’s from a state actor. Is there anything else that researchers look for to determine if it’s really a state actor versus another company or private individuals?
[00:23:51] HM: The one cool thing is that every attack has a signature. That signature is basically comprised of what they call IOCs or Indicators of Compromise. And the Indicator of Compromise can mean anything. It can mean an IP address. it could be a certain strain of malware or even a methodology. It could be as complex as, “Hey, these three different attacks from these three different sectors use the same exact commands in the same exact sequence. This must be one group.” Right? Or it’s one automated tool. So they’ll attach a name or pseudonym to that, like Cozy Bear in this case and say, “Yes, we’ve seen similar attacks using several malwares, similar capabilities. So we’re connecting that to Cozy Bear and we know from previous research and previous compromises that some of the Indicators of Compromise have gone back to Russia.” Now is that a hundred percent positive sure? No, but is it as close as you can get? Right? So if you’re a regular person out there listening to this, you can start looking at patterns and things and make your own list of Indicators of Compromise. A good example would be like, I’m sure you guys have noticed all the bots on Twitter, right?
[00:25:04] JP: Yeah.
[00:25:05] HM: Twitter knows where a lot of these bots are coming from because they’re using the same IPs or the bot runners are just lazy to wanting it off their own networks. So Twitter, for example, would be able to tell you, “Okay, so we think these set of bots are probably related to the Russian government because 800 of the 900 accounts are using Russian IP addresses.” So you start to do estimations, right? You start to like at worst make assumptions that, “Okay, this is might be related. It’s the same thing with what we’ve been talking about so far. And I’m hoping that the research continues “ands”, so this is one of the points I want to get across, is that we do more intelligence sharing in the community. I think that there’s a lot of organizations that have been compromised. They probably do have Indicators of Compromise as I discussed, and they just haven’t shared it with the community yet. The more data we have, the better it is for us as a society to be able to pinpoint these attackers.
[00:26:00] JP: When the average person or even your average developer reads these stories in the news, they’re really concerning. They’re really scary. Should the average citizen be concerned about their own personal data being hacked by a nation state?
[00:26:15] HM: It’s difficult for me, right? So the short answer is yes. You should be concerned with a nation state actor getting access to your information. But the reality of it is that your information is already compromised. We all just have to accept it that each of us, everyone on this podcast and your neighbors and your family members and your local politicians, we’ve all been compromised from Equifax to OPM, you name it. Every other company has been compromised at some point, essentially leads to a data breach, that leads to information that’s exposed and we’re pretty much all compromised at this point. What’s important about a supply chain attack is knowing that a lot of the issues that we’ve seen, post compromise, leads me to believe that developers probably need to start focusing on the development pipeline. Organizations need to hire dev ops security folks as well. That’s the new trend. I’m sure you guys have seen it on LinkedIn jobs. Those people are getting paid a lot of money. I’m not sure if you’ve seen a salary for dev ops sec person, but I think that because of all of these compromises we’ve seen over the last decade, organizations have said, “Okay, yeah, we spend a lot of money and time trying to get our developers security minded, but we’re still having oversight issues. Vulnerabilities is still coming out. Regardless of our testing and QA and pre-prod and all that good stuff.” So now they’re bringing in security professionals that also are developers like myself to wean out vulnerabilities before, hopefully before deployment.
[00:27:49] SY: Is there anything else that we haven’t already talked about that you want to cover?
[00:27:53] HM: So we have to understand that supply chain attacks are not new and they’re not going to go away. And unfortunately, it’s just the tip of iceberg. So you would have to expect kind of the worst. You don’t have to become cynical, but you have to be aware that these compromises are happening every day. Not every one of them is being operated or executed by a nation state. And if you really honestly care about your security and privacy and all that good stuff, you should be the unpopular person or become unpopular and tell the government, “Hey, I think it’s time that we strengthen cybersecurity laws here and internationally.” Cyberwar for example, it’s a word now, but it’s not really defined. You know this, is like nuclear treaties, right? And this is all these different treaties. You can’t use a specific biochemical weapons during war, or in general, you can’t use white phosphorus. You can’t randomly just drop a nuclear bomb on Siberia. Cyberwar, cybersecurity is not there yet. So because the need for our own government to kind of engage themselves in these kinds of attacks, we kind of have an open door. And I’m going to leave you with this. It’s an interesting food for thought. So as you guys know, I’m a former black hat. I was a former bad guy. I know I was a bad guy. I apologized to my victims in the past, from the past. First, I was able to turn over a new leaf, but I was never arrested for hacking foreign governments, which that was most of my career. I compromised elements of the Chinese government or the Indonesian government or Pakistan and India and Russia and Ukraine. I could give you an entire list of the countries that I have compromised, but it was only after I started compromising elements of the United States is when the FBI were interested in arresting me. You can take it however you want to take it. Well, the truth of the matter is it was not illegal for me to compromise a foreign government. Since we have that gray area open, you could expect these attacks will continue.
[00:30:01] SY: Well, thank you so much, Hector, for being on the show.
[00:30:02] HM: Oh, thank you. Appreciate it.
[00:30:12] SY: Coming up next, we talk about the release of Ruby 3.0 with Penelope Phippen, Tech Lead at Stripe and a Director at Ruby Central, after this.
[00:30:35] JL: Join over 200,000 top engineers who have used Triplebyte to find their dream job. Triplebyte shows your potential based on proven technical skills by having you take a coding quiz from a variety of tracks and helping you identify high growth opportunities and getting your foot in the door with their recommendation. It’s also free for engineers, since companies pay Triplebyte to make their hiring process more efficient.
[00:30:57] SY: Vonage is a cloud communications platform that allows developers to integrate voice, video, and messaging into their applications using their communication APIs. Whether you’re wanting to build video calls into your app, create a Facebook bot or build applications on top of programmable phone numbers, you’ll have all the tools you need. Formally known as Nexmo, Vonage has you covered for all API communications projects. Sign up for an account at nexmo.dev/DEVNEWS2 and use promo code DEVNEWS2 for 10 euros of free credit. That’s D-E-V-N-E-W-S, in all caps, and the number 2, for 10 euros of free credit.
[00:31:44] SY: Joining us is Penelope Phippen, Tech Lead at Stripe and a Director at Ruby Central, about the release of Ruby 3.0. Thank you so much for being here.
[00:31:52] PP: I’m glad to be here.
[00:31:54] SY: Can you start by having you tell us about your developer background and your connection to Ruby?
[00:31:59] PP: I’ve been working in Ruby in one form or another for nearly a decade at this point. I was the lead maintainer of the RSpec testing framework for about seven of those years. Most recently I’ve transitioned out to being an RSpec maintainer, and I’m currently working on a Ruby format, Ruby auto-formatter. I’ve been deep in the source code and tunnels of the Ruby Interpreter. So yeah, really excited to be here today.
[00:32:25] JP: Okay. So I know you have some opinions. We are ready to receive them. So first off, what would you say is the current state of Ruby and where does it stand in the world of tech?
[00:32:36] PP: Oh my God. That’s a big question. I guess perhaps my view of the Ruby programming language today is that it’s very multifaceted. It’s used in all sorts of places you would and wouldn’t expect. It powers company is like Shopify, Stripe, where I work, GitHub, as well as things you’ve probably never heard of, like code inside the Japanese Space Agency used to control satellites. So Ruby is really interesting in the sense that it’s used in a lot of different applications and it powers things that we all use in our day-to-day lives, even though we may not realize we’re using it.
[00:33:17] SY: So I imagine that you’ve played around a lot with Ruby 3.0. What are some of the biggest new features and updates in this version?
[00:33:24] PP: Ruby 3 includes a bunch of stuff, but to me, sort of like the two headline things that are different between Ruby 3 and Ruby 2.7 is that Ruby now has a type profiler, so it can generate typing information about Ruby code by doing runtime analysis of the types of objects that are flowing through your system and that there is a new format that goes along with that called RBS, which is the format for sort of like declaring those types and that Ruby has a new feature called “Ractor”, which is basically like an actor concurrency-based system for Ruby 3.
[00:34:04] JP: What are the things in Ruby 3.0 are you most excited about?
[00:34:08] PP: I don’t actually know that any of this is like overwhelmingly exciting. I think despite the fact that there’s a large version number on this, actually what we’re getting here is really just like a lot of incremental progress. Like despite the new version number that isn’t a ton of breaking change and there isn’t a ton of like major new features that are like that different from previous Rubies. And so I guess like perhaps the best way I can phrase this is that one of the things that’s exciting to me about the Ruby language is that it remains overwhelmingly self-compatible. For the most part, you should be able to take code that you are writing against Ruby 2.0 and run it just fine on Ruby 3, even though like the language has undergone a lot of progress in the years between those two versions.
[00:35:04] SY: How does this release compare to other past releases? Is it drastically different? Is it kind of similar in different scopes? How would you compare it?
[00:35:13] PP: The Ruby team releases once a year, right? And one of the things that sort of enables them to do is have these sort of like continued incremental progress between versions. Right? So like in 2.7, we got garbage collector compaction. In 2.6, we got the JIT. In versions between 2.1 and 2.4, there were significant improvements to garbage collection in Ruby. And actually, the 2.7 had some garbage collection improvements as well. And so that like really what we find with all Ruby releases is none of them are that huge of a change to the previous Ruby release. But if you sum them up over many years, there has been massive incremental progress in the programming language. So for example, Ruby 3 is somewhere around three times faster than Ruby 2 on some benchmarks, which was a long established goal. But it’s nice to see that we’ve gotten that over the past, like, seven, eight years and I just think it’s sort of really impressive that the Ruby team continues to make this progress at such a regular cadence.
[00:36:26] JP: So you mentioned one of the big new features in Ruby 3 is a type system, RBS, and working at Stripe, Stripe had a Ruby type system called “Sorbet”. And I was wondering if you could kind of talk about how they’re different and maybe do you see a future for Sorbet? Do you see them coexisting? Do you see one taking over? I’m just kind of curious what you’re thinking about types.
[00:36:49] PP: So one of the sort of like fundamental pieces here is that Ruby 3 actually isn’t introducing type checking into the Ruby language. What Ruby 3 is introducing is a Ruby core team approved format for expressing the types of clauses and functions and what’s called a “type profiler”. So what the type profiler does is it can analyze a running Ruby program and output this new format, RBS. And what RBS is, is this type declaration format. So sort of contrast that with Sorbet, right? Sorbet doesn’t exactly have a dynamic analysis component to my understanding. Sorbet instead does type inference based on statically typing the standard library and then effectively, gradually typing all of the code in your code base. Sorbet has a different format for type declarations called “RBI”, which is actually RBI files are syntactically valid Ruby programs, whereas RBs is not. It’s a different format. So today, Sorbet is a Ruby type checker as well. It can statically analyze type declarations to validate that your program is well typed in Sorbet’s type system and it has a runtime typing component as well or it can do runtime type assertions. So these things are actually different. So to sort of like start answering your question, like today, the Ruby core team has not expressed an opinion on what type checking in Ruby should look like. It has published the tooling that makes building type checking in Ruby more possible than it previously was. And again, I should be very clear here that I’m not speaking for Stripe or the Sorbet team, but like one evolution that I see here is that Sorbet could adopt RBS as its type declaration format and continue to work in exactly the same way as it currently works. So that if anything, the introduction of RBS and the type profiler is actually helpful to Sorbet rather than competitive with it, because it gives Sorbet more information to do type checking on top of.
[00:39:10] SY: Was there anything in this release that you were hoping would be there, but didn’t quite make it?
[00:39:14] PP: The all-access system is interesting to me because it signals that the Ruby team, instead of rearchitecting how virtual machine works in terms of the virtual machine look is effectively moving to a model where instead of exposing threads to Ruby programmers, they are inventing new obstructions and then asking you to program on top of those obstructions. So my view on that is that I would have loved to see a real threading story. The reason Matz has stated he doesn’t want to do this is that like real threads are like a very sharp tool and that inexperienced programmers can cut themselves by using them. That’s totally fair. Right? I agree with that position. It’s just that my view is sort of like a systems engineer is that having that would be a really useful tool in my arsenal. So I understand why the actor obstruction was introduced, but I would have loved to see like real threads in Ruby 3 as well.
[00:40:25] JP: Do you think there are any interesting new ways people will use Ruby now that we have some of these features like type hinting and a threading model?
[00:40:35] PP: Yeah. So it’s funny, just this morning I was having a conversation with somebody about things that we can do in Ruby 3 that we can’t do in Ruby today. That are sort of like entire clauses of system that are really hard to develop in Ruby today because it doesn’t have a real concurrency obstruction that will become easier to implement on top of actors. So a really good example of this is any kind of server where instead of request response, you’re doing sort of like streaming. So if you imagine that over like periods of several minutes, you need to send like partial data to a user in a stream and maybe their streaming requests back at you as opposed to sort of the request response model we have with HTTP today, Ruby would really struggle to implement something like that. But with actors, it would be a lot easier. That’s a big area that those sort of things around like the idea of dispatching multiple HTTP requests simultaneously, you will become a lot more efficient now and we can build sort of like more interesting client software models than we currently have. I think this opens Ruby up potentially to like real-time programming domains that have traditionally been completely inaccessible to it. So really the answer here is that like for a long time there has been an argument against Ruby for like high concurrency workloads that is now like much harder to make because of the actor system.
[00:42:09] SY: So in GitHub’s State of the Octoverse, Ruby was sadly at the bottom of the most popular languages of 2020. It was even beaten out by Shell, which I found a little disappointing. How do you imagine Ruby might stay relevant in the future and is this release a part of that?
[00:42:26] PP: I mean, conversely, right? If you look at the RedMonk 2020 language rankings, Ruby ranked number seven, right? And so there is clearly some debate out there amongst analysts as to how relevant the Ruby programming language is. My view on this is sort of like there’s a really good compelling two-fold answer to this. The first is that Ruby on Rails, the framework, is probably the fastest way to go from like zero to a functional product on the internet, like in existence today. That has been true for a while and that like in the fact that Rails continues to improve, like Ruby will continue to be an extremely amenable programming language to. building and launching products very quickly for a long time. Right? If you’re bootstrapping, that’s extremely valuable. Honestly, even if you’re like a VC-backed company, you’re looking to go to Infinity, like being able to experiment quickly and use a language like Ruby to do so is like also incredibly valuable. I think also, if we look at the world of today, there are some companies with a billion plus dollar revenue figures that are all built on Ruby, it was Ruby in their very cores, right? There is this idea I think a little bit that people believe that Ruby can’t scale effectively, but some of that is sort of fudge that was published by Twitter way back in the day. Right? But the truth of the matter is that Ruby is scaling extremely well today and like highly commercially valuable applications. And so my view on this honestly is the kind of Ruby, I think, has this perception of no longer being like cool and trendy. And that’s where a lot of this comes from. But what we find instead is that Ruby is like stable and mature. That is also a really valuable thing for businesses to use. Right? And so to come back to something you asked me earlier, like, “Why is this Ruby release exciting to you?” And my answer, “Yeah, I kind of know a lot has changed.” That’s a really good thing in some senses because it means that it is a stable language for you to trust and build on top of that isn’t going to change out from under you every year.
[00:44:49] JP: Is there anything else we didn’t cover that you’d like to talk about?
[00:44:53] PP: One thing I kind of want to talk about in the Ruby 3 release a little bit, which is that we see a couple of changes to the syntax of the language right-hand assignment and endless functions and that to me, it’s really something we could have done without including into the language. I find that these days we are getting a lot of changes to the syntax of Ruby that aren’t adding new capabilities. They aren’t letting you as the programmer do something you weren’t able to do previously and they just increase the complexity of the language for not like a lot of benefit. I guess like the only thing I would add to this maybe is that my preference would be for us not to continue to expand the syntax of the Ruby language if we aren’t gaining new capabilities in doing so. But that is like purely my take. So that’s probably the only thing I would add.
[00:45:46] SY: Thank you so much for joining us.
[00:45:48] PP: Of course.
[00:45:59] SY: Thank you for listening to DevNews. This show is produced and mixed by Levi Sharpe. Editorial oversight by Peter Frank, Ben Halpern, and Jess Lee. Our theme music is by Dan Powell. If you have any questions or comments, dial into our Google Voice at +1 (929) 500-1513. Or email us at [email protected] Please rate and subscribe to this show on Apple Podcasts.