When is the right time to create your own language?
In this episode, we talk about creating languages with Edaqa Mortoray, puzzle designer at Edaqa’s Room and creator of the Leaf programming language, and Sam Scott, co-founder and CTO of Oso, which is powered by the language they created called Polar.
Ben Halpern is co-founder and webmaster of DEV/Forem.
Edaqa Mortoray is a creative programmer, writer, and cook. He's now primarily designing online escape games.
Sam Scott is Cofounder and CTO of Oso.
[00:00:01] What you build and where it takes you shouldn’t be limited by your database. CockroachDB, the most highly evolved distributed SQL database on the planet, helps developers build and scale apps with fewer obstacles, more freedom, and greater efficiency. So you can forget about the database and trust that it just works. Sign up for your forever free database and get a free T-shirt at cockroachlabs.com/devdiscuss.
[00:00:27] Cloudways is a leading edge Managed Cloud hosting platform built for your PHP projects. If you simply wish to focus on your business, Cloudways is the way to go. They take over server management and security and free uptime that you can dedicate to growing your business and acquiring new clients. If you want to give them a try, use promo code, DevDiscuss.
[00:01:16] EM: This syntax and these type features and all this stuff is utterly useless unless somebody has an easy way to say, “Well, let’s take this language and do something with it.”
[00:01:35] BH: Welcome to DevDiscuss, the show where we cover the burning topics that impact all of our lives as developers. I’m Ben Halpern, a Co-Founder of Forem. And today, we’re talking about creating languages with Edaqa Mortoray, Puzzle Designer at Edaqa’s Room and Creator of the Leaf Programming Language, and Sam Scott, co-founder and CTO of Oso, which is powered by the language they created called “Polar”. Thank you so much for joining us.
[00:02:05] EM: Thanks for having us.
[00:02:06] SS: Thank you. It’s nice to be here.
[00:02:07] BH: So before we get specifically into programming language creation, let’s get a bit into both of your backgrounds. Sam, can we start with you?
[00:02:15] SS: Yeah. So I started out as a mathematician. I did a PhD in cryptography in London. That didn’t scratch my itch for doing more applied stuff. So I ended up getting into the startup space and started my own company. And so that’s what I’m doing today at Oso.
[00:02:28] BH: And Edaqa, can you give us your backstory?
[00:02:31] EM: So I’ve been programming for quite a while now. I’ve been doing programming, a lot of different industries, so I’ve been fortunate to jump around, a lot of them, from like finance to video games from now and entertainment, back end, front end. I kind of enjoy all of it. And as a friend once quipped it, “Wouldn’t be an Edaqa project unless there was a language I designed for it.” So that’s kind of what I do on projects.
[00:02:52] BH: Awesome. And I am really excited to talk to both of you. Edaqa, you were blogging about Leaf on your own blog and on DEV, in the early days of DEV, when there just weren’t that many total people in the community. So like you were an early adopter and you kind of were like there to kind of see what it was like in the early days. And the Leaf Programming Language stands out because we were just excited someone was doing something so interesting and talking about it in our space. And it’s that out a lot and it’s really enjoyable to get into that right now. I’m really looking forward to that. And then reading about Polar really seems fascinating. And Sam, I’m really excited to get into some of the details of what it is. And we’re going to get to that in a little bit. But first, for the best possible outcomes for our listeners, we’re going to get into some basics of language creation and what it’s all about. Why don’t we start with the question of what is the actual difference between a programming language and a library or even a framework or something else like that? Which to me sounds like maybe that’s kind of obvious, but also maybe not like where does a programming language start and just a programming library end?
[00:04:16] EM: And I think it’s not obvious because if I see the evolution languages I’ve made, a lot of them start as basically a library that grows into more customization, and eventually they’d take that switch of becoming a language. But I think the defining characteristic of being a language is that it has its own sort of source code. Whether that’s a declarative source code, imperative search code, but it has source code. It has some sort of source. If it doesn’t have that, it’s probably not a programming language. That doesn’t necessarily mean the quality or anything, it just kind of that distinguishing factor where you’d say, “What is a programming language and what isn’t?”
[00:04:50] SS: Yeah, I think that’s exactly it. And this is probably the deepest of the questions, right? Like what is the language for really? Instead of taking things that you might normally represent in the library through methods and calls and stuff like that. You’re coming up with your own syntax and semantics to represent those same concepts, but you’re ultimately solving the same problems. It’s just deciding whether you feel like the capabilities of whatever, host language, maybe don’t match up with what you’re trying to do.
[00:05:16] BH: You mentioned the word syntax just now. Can you compare and contrast the word syntax to say an API?
[00:05:24] SS: Sure. So syntax in this case is the tokens, the characteristics list, whatever you’re using in the language to represent concepts effectively. In a regular library API, you have relatively few of them at a basic layer. A library API is going to be like the set of methods. Sometimes those two things, they do suck and overlapping. If you think about highly dynamic languages like Python, they might encourage you to do a lot of like operator overloading. So like the library API in something like an ORM for interacting with SQL objects, my overload things like addition and quality comparisons and things like that as part of the library API, but it’s assigned to be very syntactic at that point as well. You’re adding together two query objects and it’s doing some operation to them.
[00:06:10] BH: So we’re going to talk a little bit about types of programming languages as we go. And we’re going to get into what Polar is exactly, which is probably outside of the spectrum of what people can think of in terms of like different types of programming languages. And we’ll get into that a little later. But first, can we talk about just the difference between an object-oriented programming language and a functional programming language?
[00:06:37] EM: I don’t know if that’s the best one to start on, and it’s just Sam here going like, “Yeah, that’s a question.” Because the big thing with these types of languages, first off, there’s this big sort of battleground of which is the correct way. But aside from that, object-oriented is more about the way you represent data and access that data. And it’s not fundamentally incompatible with functional. It’s just traditionally in some of the languages like C and C++, that’s where it came from and so it tends to be mixed with the imperative programming as opposed to functional programming. And so functional programming, for example, comes more from like the sort of the math side is you basically have an expression and evaluates to something and you just keep combining expressions together in one big thing to accelerate one result. And the reason it’s hard to distinguish those that most modern languages tend to combine metaphors. They have a lot of functional aspects and they have a lot of imperative aspects, but I would consider object-oriented programming not necessarily one of the fundamental paradigms. It’s part of the imperative paradigm and it’s the way you group things. And so between paradigm and functional, imperative is you stepwise do this, then this, then this, then this, then this, whereas functional is this is where the expression looks, this is my equation, evaluate this.
[00:07:52] SS: Yeah, I think that’s a really good way to how you group things. Right? So typically in object-oriented data and these sorts of behaviors are coupled together. You can get very, very similar style, like looking things in a more functional language where you’re applying behaviors to the data structures, instead of it being coupled with it. The end result they can have very, very similar patterns. I’m a big fan of Rust. I Use rust quite a lot. I think it definitely sort of crosses those streams a bit. It has a lot of functional style, things you can use in it, there’s even guides on the ways that Rust is kind of your defining behavior kind of on your data, but you’re doing a very functional way. So those streams definitely kind of cross a bit.
[00:08:32] BH: So in my day-to-day, as a successful enough programmer, I certainly relate to the notion that I do whatever programming is needed that my language provides. And that’s going to have object-oriented patterns and functional patterns. And maybe if I was in a different paradigm, I might be more strict. But why would you say that this question, which I kind of asked, which I kind of agree, is like not really the question we want to be asking? But why am I going to find so many people with opinions and interests in this question in this debate online like if in fact we don’t end up really wanting to distinguish so much in how we actually do software development?
[00:09:17] SS: So what I’ve seen from this is that everybody has their own preference, for example, the kinds of things they want to think about while they’re programming, the kinds of things they don’t want to be thinking about while they’re programming. This can be at like at its simplest. Some people like very, very explicit, a little bit worthy code that just says, “Everything’s happening. You can kind of read through and you can see everything that’s happening.” Some people would rather this was like pushed elsewhere that they didn’t want to think about it. It had very concise code and that all these details were like hidden under the surface. They could just focus on like the core logic. You can talk to two people at like either end of this and they’re definitely going to have different preferences around the libraries they’re using, the languages they’re using, the way they write their code, the conciseness of variables they use. This is also your personal preference that it can just like make everything on that sort of vary.
[00:10:06] EM: I think a lot of it has to do with experience as well is that as I find people that have played with more languages and done more things, they tend to go in the direction of, “Yeah, just going to use whatever’s there to get it done.” And as you know more, the paradigm is just like, “This one seems to work here. This one works here. There’s kind of no point in arguing about them because it’s like, “Well, some of them work better in different places.” And as we see, all languages tend to migrate to supporting a little bit of all the different paradigms because they all prove useful in various situations. So you ended up using all of them. And so I think a lot of the people, they want to sit in the one paradigm, I think, because they’re experienced with it or they have a particular domain they’re in, and that’s the other one too is domain. In certain domains of programming, some paradigms work a lot better than other ones. And as you program in more domains, then you can see more and more of how these other paradigms fit better in these other domains.
[00:11:02] SS: Yeah. I think like speaking about everything converging to having the same set of features, I think about types a lot. As I said before, I’m a fan of Rust. Rust uses lots of types. The interesting thing is as you’re seeing them, more and more are being brought into the other languages. Python’s now bringing in a little type hints and type check lists like Mypy. Ruby has been doing a lot with bringing in more and more type systems as well. The reason is everyone wants to get these kind of like same sets of benefits, which is like reasoning about your code and knowing how it’s going to behave and checking and things like that. You can get them from any of these programming paradigms. It’s just a case of like how do you kind of get it into the language.
[00:11:36] BH: So let’s talk about the Polar language.
[00:11:39] SS: Sure.
[00:11:39] BH: What is Polar and where does it fit in the spectrum of paradigms? Like it’s not a functional language. Let’s get into that.
[00:11:49] SS: Yeah. So let’s start at the top right. So at Oso, the main problem that we’re solving is the problem of building authorization into your application, you know, designing who can do what in your application using things like roles, relationships, hierarchies, that kind of thing. When we started out on this problem, we were looking for the best way to allow people to express that kind of logic. You can access this resource because you’re a member of the organization the resource belongs to. Things like that. We ultimately decided that a declarative logic-based language would be the best approach for that. Some saw like a parallel paramount would be the best way to solve this kind of problem. And so we built Polar for this. So it’s declarative and logic-based, I’d say are kind of the two main pieces of this. The main thing that kind of differs from something like imperative is instead of the code kind of executing like line by line and accumulating the data on their funds, you basically just had it like, “Here’s what I sort of think to be true of the world.” In authorization language, like, “Here’s who should be able to access what and why logically.” You give it as logical rules and like the evaluation of the language is you sort of ask questions like, “Okay, well, it’s not a lot to read this resource.” And it effectively searches through the things you told its knowledge base. It searches through that to answer that question. So it doesn’t like execute then it’ll read through it. It’ll effectively like execute a searching algorithm through the code that you wrote.
[00:13:08] EM: To jump in there and sort of give the definition and a declaration as well to help understand it is the biggest difference here is in a declarative language, you’re essentially describing what you want, whereas in the imperative functional world, you’re describing what you want to do. But the declarative one says, “This is basically my answer.” And then you let the language figure out how to do that, but you’re saying, “I’m just going to start with my answer,” whereas the functional imperative ones you’re like, “I’m starting with nothing and I’m going to build my answer through executing statements.” And declarative is very good for domains where you can say, “I know what I want and I’ll let somebody else figure out how to get that for me.”
[00:13:43] BH: Did you perceive a lack of declarative programming language choice in choosing to actually build your own programming language? Or was it another interest? Or how did this come to be?
[00:13:57] SS: A bit of both, I would say. We knew that we wanted to use a logic-based language and so we looked at like the existing prologue implementations, which is like probably the most widely known, widely used logic programming languages as sort of Polar variance. And so I think some of them could have done the job. That’s sort of the key difference, which is sort of like a novel part of the authorization story is we wanted to have the language itself be embeddable in another host language, so in a Python or Ruby or Java or whatever, because we felt like the cool thing of authorization is that it should be able to evaluate logic over data from your application. Or like if I build an app, I built GitHub, it has all bugs and repositories and uses, that’s my data. That’s the stuff that I’m going to do authorization over. Now we think is the right choice to do a logic-based language to write the logic for that, the rules that dictate who can do what, but you don’t like not trying to move your Postgres database from Python into like a language, configure it there separately. So we wanted that to be this like interaction between the two so that you could write your logic in one language, but it was like execute over your Python objects, for example. That’s just something that we didn’t really find that existed out there.
[00:15:10] BH: What’s this state of its use right now in your production project?
[00:15:16] SS: So we open source the Oso library for authorization sort of like last summer. Since then, we’ve been growing like the Oso community around there. I think we have several hundred people in our Slack community, and yeah, a good chunk of those are using it today from POC implementations and we’ve got a number of people in production with it as well. So it’s out there in the wild now.
[00:15:37] BH: And to clarify the uses of the language beyond what you’re doing with Oso, any thoughts about how other people may or may not be adopting it and running with it?
[00:15:48] SS: Yeah. So we’ve designed the part of the language to be generally a logic-based language. It’s not specifically for authorization. We just think it’s good for the use cases. We’ve internally have done some hackathons and stuff like that to try and find other new and fun use cases for Polar. Some people in the team, she wrote a text-based adventure game using the language as like the instruction set. So you can like look north. You write that in Polar syntax and it kind of executes that way. We’ve had people asking us around more general like “rules engine” style questions where these lines, again, are very blurry between authorization, business logic, rules engines, and that kind of stuff. But any situation where you’d normally have like a ton of nested conditional logic and like confusing flows tends to map very well to like a logic-based language where you can kind of write it as like a relatively like flat, concise set of rules and it can execute through these like across the structures and flow around. So we’ve had some people like playing around with some of those ideas.
[00:17:08] RudderStack is the CDP for developers. Its 150 integrations make it easy to connect your data sources to all of the tools used by your marketing, product, and data science teams. With RudderStack, you can say goodbye to the headache of managing custom built integrations and easily manage all of your customer data pipelines in one place. Sign up at rudderstack.com to give it a try.
[00:17:32] Cloudways platform offers a choice of IAAS partners, AWS, Google Cloud, Digital Ocean, Linode, and Vultr. In addition, you get a performance optimized stack, managed backups and staging environment where you can test your code before pushing it to live servers. Best of all, composer and kit come pre-installed so you can get your projects up and running quickly. All this power, simplicity, and peace of mind falls right with their brand slogan, moving dreams forward. If you want to give them a try, use promo code, DevDiscuss.
[00:18:07] BH: Edaqa, can you tell us what the Leaf Programming Language was and what inspired you to take it on?
[00:18:16] EM: So I think back a little bit now, so what was the reason I did it? Part of it was I was using a lot of these different languages and there are a lot of these features from all these languages, all the different languages out there, but one of the big problems I had with it was that there was a syntax mismatch or they’re missing some other features. I had done a lot of C++ programming too. I liked the features of the language, but the syntax left a little bit to be desired. And I was also using Python and was like, “Well, I kind of like the Python syntax. It’s simple. There’s no types in here.” But I’m like, “I want to do a language that kind of looks like this without types, but it’s actually strictly typed.” And so that was one of the cores of my goals was a lot of type inferencing to get a language that was basically super friendly, like the modern dynamic languages are, yet actually had the safety of the strongly-typed languages. And so that was a major goal, was an imperative language that basically take a bunch of these features and languages, make a nice, clear, basic syntax that sums up all these new features and provides a nice way to house this altogether to be friendly and yet still safe.
[00:19:21] BH: And as you began building it, how did your opinions change on what it needed as you learned more, as you went from like, “Oh, why I might want to take this on,” versus like adoption working with people, showing it off, like testing it in the world? What was that journey like?
[00:19:44] EM: So I think the journey was when I started, it was a lot like, “This looks like a fun thing to do.” It’s a challenge for me. I’ve done a lot of other small languages before and this one lets me try a lot of things. And it was one of the first languages I did that was sort of fully end-to-end compiled from my source code parsing down to machine code. And so it was this challenge with it. And as I went forward, I got to see a lot of the hard, like the hard challenges and compilers in typing, how to solve them. I got to solve those. But as the project progressed and as I watched the other projects, I hit this sort of realization. It wasn’t foreign to me at the start, but I realized how important it was, the environment around the language. And particularly, how will somebody use this language? This syntax and these type features and all this stuff is utterly useless unless somebody has an easy way to say, “Well, let’s take this language and do something with it.” So I started focusing on doing that. That’s where the major uphill battle actually becomes because you have like this nice core language and it’s like, “All right, now give me a standard library with 8,000 functions everybody’s expecting to be there.” And you’re looking at it and going, “Okay.” And also give me a packaging system so I can just type install and give me this package, spec, information, make it all work together.” And for me, that’s sort of where my journey started to end partially because the challenge for me was disappearing because I knew how I do all that stuff and I did all this stuff I didn’t know how to do in the language. And so there’s this large body of work to get it useful. And I see that in a lot of other languages that what’s making the most popular languages nowadays is actually the environment about them, not the language itself, how easy it is to install, how easy it is to use these things. And I find that sort of frustrating and great at the same time. I mean, there’s a lot of times where I was like, “I would really like to sit down and write this thing in C++, but I just know it’s way too hard to get it to build. So I’m just going to do it in Python and hope it’s fast enough.” And so that’s sort of how my journey went through the language of how I did it. And then at the end of the project, it was more, like, I’ve exhausted the things I really want to work on and it’s not meeting my personal goals anymore. And so that’s why I moved on from the language at that point.
[00:21:47] BH: Do you think if you had sort of discovered a really killer use case or the environment around it had come together a little bit more outside of like your core development process you might still be working on it?
[00:22:05] EM: I don’t know if it came down to a use case, more in terms of like the time investment, because I think that’s the biggest issue with a general purpose language is finding unique use cases. So it’s like, “What would differentiate this?” It’s like, “It’s got this great syntax, it’s got this great stuff.” And it’s like, “Yeah, but what does it do new for me? Why is this going to help me?” And you’ll see that with a lot of the new languages, things like Swift for iOS or Kotlin for Android and stuff. They don’t really offer a lot new in terms of language design, but they’re highly specialized for a certain use case. And that’s what drives a lot of the adaptation is use cases. And this is why we have a lot of domain specific languages as well. And as Sam mentioned even with Polar, it looks like they’ve isolated like, “Here’s my use case. Let’s design our language around this.” So you have a clear reason to use it. Whereas the general purpose languages, it’s kind of hard to have a motivating use case. And that’s kind of my opinion of why despite how much problem it has, C++ is still going to be around 50 years from now because there’s not a strong motivation to leave it for some of the applications where it’s useful.
[00:23:14] SS: So you mentioned in that as well, you mentioned domain specific languages, right? DSLs.
[00:23:18] EM: Yeah.
[00:23:18] SS: I feel like that’s a term that gets thrown around quite a lot as like an insult with things like, “Oh, it’s a DSL.” It sounds like a thing to put something down. As someone who’s gone down the path of like writing a language, what do you think about that? Like the idea of using DSLs for like very specific use cases to scratching those itches as opposed to building something general purpose?
[00:23:38] EM: I’m actually very in favor of DSLs. So I don’t use the term in any derogatory sense. For me, this is where language has come from. And a lot of my languages came that ways that is you take any language and you program it and you start seeing a pattern repeating and you’re like, “Oh, this is syntax heavy. This is semantic heavy. And it’s like, “Well, if I just tweak some other language a little bit, I can start compacting this.” And so what you end up with essentially, a domain specific language. The idea of a domain specific language is that it’s very much tailored for one use case. It could do other things as well, but it’s been highly tailored for one use case and you’ll find it’s harder to use in other cases. But the great thing about them is that for the use cases it’s written for, you’re not likely to find a better solution because this has been made for that use case. So it’s like, “Hey, it’s the best syntax, it’s the simplest thing, and it’s great straightforward.” It’s just not very adaptable, but I really liked them. I think they’re great and I think eventually any mature project I think should have them on it. Otherwise, they have a lot of redundancy in their project if they don’t have one.
[00:24:43] SS: Yeah. I’ve been sort of trying to propose the idea that actually when you have these like incredibly large APIs with a ton of different stuff, you’ve created a language, you just have used the syntax of an existing one kind of thing because the amount of things you need to learn is so vast and the intricacies of how these two things combined you might as well be using a language at that point.
[00:25:03] EM: Yes. And I think that comes back to what Ben said earlier. Why it’s sometimes hard to distinguish between what’s the language and what’s the library, because as you get a highly specialized library, if it’s really only small syntactic things that change it between the language and why, is that a significant difference? I’m not sure.
[00:25:19] BH: Edaqa, you talked about the nature of languages to evolve, to have familiar features of other languages. Can we talk about any domain specific languages that brought in to become more generalist? Just from a historical perspective, I’m wondering if that’s something we can pluck out.
[00:25:41] EM: In terms of domain specific, I can’t think back to ones that I would say were strictly domain specific as opposed to being just restricted in the amount of functionality they provide. Like I say, something like Swift, I wouldn’t call that a domain specific language because it’s basically still a generic language, but it’s been tailored to a few of the libraries they use for their operating system. But domain specific goes a lot, sort of deeper than that.
[00:26:07] SS: HashiCorp has the HCL or these things Terraform through infrastructure’s code. Do you consider that as a DSL?
[00:26:15] EM: I’m not familiar with that one. Sorry.
[00:26:17] SS: Okay. So yeah. So HashiCrop has this product called Terraform. It allows you to deploy your infrastructure by writing code in this declarative language called HCL that like HashiCorp Config Language I think it stands for. So the idea is you basically say, “I want two servers on AWS and they should have public IP addresses.” You declare that in the language and it goes and deploys that. But I thought to Ben’s question, the reason that jumped into my mind is because early on it didn’t support things like conditionals, it didn’t support things like for loops, which I think like very common general public language in concepts, but it meant that you couldn’t like loop over a configuration and like create an incrementing IP address or something. And they had like all these horrible syntaxes kind of like shoved in there to do conditionals. You’d have to do like the count was zero if this condition was true, also count as one. It had a little stuff like that kind of like shoehorned into it. The reason I think that one’s interesting is because then a competitor sprung up called Pulumi, which allows you to basically use Terraform, but from your existing languages, from like Node.js to TypeScript, from Python, from other languages like that, partially for that reason where it was really pushed to the limit of what the DSL I’d say in this case could do. I think that’s an interesting one.
[00:27:34] EM: Yeah, that’s interesting. And you see that pattern I think on some things, things that slowly evolve. But I thought of an example I think applies perfectly, PHP. PHP was started as basically HTML templates and it evolved into what we call WordPress now.
[00:27:54] SS: A well-known language.
[00:27:56] EM: So I think that took the definite path of being very domain specific to becoming a general purpose, widely deployed programming language.
[00:28:03] BH: What about Erlang? My understanding is that that had a lot of special purpose in its origins, but maybe it was never actually something like a DSL and it was always kind of a general purpose programming language.
[00:28:17] EM: Well, I don’t know about Erlang particularly. Well, we talk about DSLs. It’s not like there’s some binary transition between what’s a DSL and what’s not. It’s kind of like how specialized it is. A lot of the early functional ones. They came into areas like computation. And so is that domain specific? It’s like, “Well, it’s highly focused on certain types of computation.” But it’s like, “Computation could be used in a wide range of fields. Is that really specific?” It’s kind of like what you’re doing and you can also take things like maybe like Mathematica and stuff. They have a highly specified language and they evolve that. Technically, it’s also domain specific, but it’s also widely available or widely used. And I don’t know. Everything moves along the spectrum. So we had a spectrum from domain specific to generic. They all kind of start somewhere on the specific side and slowly migrate towards the generic side.
[00:29:12] BH: When is it the right idea to create your own programming language?
[00:29:18] EM: I'm going to go with the basic thing here is that if you’re asking yourself the question, the answer is no. It should become fairly obvious that you need some sort of language before you start creating one. And I think a lot of this has to do with look at your project, look at your code. If you see the same pattern being applied over and over and over and over again, this would be the case where you want it. And I think that’s it. If you want to make a general purpose programming language, there’s not a lot of value in it. Honestly, I think like the big programming languages like Swift and Kotlin and these things, they had the bad motivation. A lot of these are being designed around vendor lock-in rather than providing people with actual new functionality. And I think that’s a bad way to go. And so general purpose language, I’d say you should probably never create one. Domain specific, basically when you’ve recognized you need it.
[00:30:09] SS: I think that’s a good way to put up. I think most people should not need to build the language. I do think of it coming back to like the very first question you asked as effectively an alternative to an API. And it’s just the question at that point just becomes, “What do you want the user experience to be? Who are you building for and what you want the experience for them to be?” Because they’re trade-offs. In the case of Polar, you do need to learn a language. You may even need to learn a new paradigm, if it’s like logic base instead of the imperative you’re used to. We do what we can to decrease that. But ultimately, we made the decision that we thought that was easier than trying to ask people to learn. I wish I could show you what the V1 of this looked like in Python because I guarantee that one was no easy to learn a new language. But in this case, it’s because the domain is very conditional and logic driven, which isn’t a good fit for something like Python.
[00:30:58] EM: Yeah, I think he hit on one of those good things. They had examples of what it looked like in Python. Somebody looked at the screen and they got scared and they wrote something else that basically sums up a hundred lines of code and like three simple expressions or something. This is often the birth of these little languages like this.
[00:31:13] SS: That is the one.
[00:31:14] EM: Yeah. I do encourage people to work on like miniature things. I mean, there’s an area of domain specific languages that essentially configuration files that a lot of people forget about. You don’t have to have like a syntax heavy language. You can use something like YAML, create a structured document, a JSON, and basically have your program load that interpreted. In a way, that’s also a language. And I think that’s very valuable in a lot of projects. It’s extremely specific just to your project. And I think that’s fine. And I think that also blurs the lines of what is configuration versus what is the language, but I don’t think there’s a clear definition there either.
[00:31:46] SS: Yeah, that’s a really good one. I think a lot about like you have actions. You specify a lot of the configuration with YAML, but it has an expression syntax inside it. So you can say, “This job runs if the last one failed,” or something like that. It’s got that tiny little language basically embedded, but the language doesn’t have a spec. The language doesn’t have a syntax highlighting. It didn’t have that kind of thing. So at that point, it’s definitely blurry.
[00:32:31] Scaling an SQL cluster has historically been a painful task. CockroachDB makes scaling your relational database much easier. The distributed SQL database makes it simple to build resilient, scalable applications quickly. CockroachDB is Postgres compatible, giving the same familiar SQL interface database developers have used for years. But unlike older databases, scaling with CockroachDB is handled within the database itself. So you don’t need to manage charts from your client application. And because the data is distributed, you won’t lose data if a machine or data center goes down. CockroachDB is resilient, adaptable to any environment and Kubernetes native. Hosted on-prem, run it in a hybrid cloud and even deploy it across multiple clouds. The world’s largest banks, massive online retailers, popular gaming platforms and developers from companies of all sizes trust CockroachDB with their most critical data. Sign up for your forever free database and get a free T-shirt at cockroachlabs.com/devdiscuss.
[00:33:33] BH: We’re now going to move to the segment where we ask the question to the audience and a bunch of folks weighed in with a comment on DEV. And the question was pretty much what we just talked about. So we’re basically going to expand this for comments right now. So we have a bunch of written in comments here. Anna writes, “If you’re a developer working on a project, I’d say never. Why reinvent the wheel when there are 16 or so other wheels to choose from and they each do their job and have their own team of masochists maintaining them?”
[00:34:06] EM: Well, I think we basically answered it, but I think the answer to that is that essentially the wheel is sometimes broken and there’s no way to get around that. The wheel can sometimes be holding you back to a great degree. And so this is why we have new languages because we have an idea. When you basically look at the current ones, it’s just not possible. There’s something mentally wrong with this one. And so we have to evolve that. If we don’t evolve these things, we’re going to be stuck with the old system for very long, but we know the old languages have a lot of problems. We know the old paradigms have problems. They have to evolve. So although I recommend to an individual, “Hey, you shouldn’t do a new language,” we definitely need new languages to be developed and evolved.
[00:34:49] SS: Yeah, definitely agree with that. If I was working at a company and we were like, “Oh, we need to solve this authorization thing,” and someone was like, “Hey, let’s build a language.” I think that I’d have a strong conversation with them.
[00:35:03] BH: Julian writes that the right time to build a language is that when you’re skilled enough to create something elegant and easy to use.
[00:35:13] EM: Well, I can answer first then and say, I’ve looked at a lot of new languages and clearly, most people have not waited for that point.
[00:35:22] SS: Yeah. I would encourage anybody to try building a language. I think it’s not as terrifying and intimidating as it sounds. I think you can noodle around this. There’s some fantastic books out there and tutorials out there for building your own language, which is a lot of fun. It’s really not that terrifying a thing. You can implement a prologue varying in several hundred lines of code. It’s not crazy, but that person’s comment, it does hit on something, which is like doing language design well is very, very hard. We’ve been working on Polar for over a year now and we’ve still constantly evolving it. We’ve had to sit down with various people and ask them to try using it. And so when they get confused or see if they understand what’s happening and so on and so on, it is hard and it does take a lot of work, and like the good languages, they’ve got to where they are over like years if not decades. So don’t expect you’re going to recreate that like overnight somehow.
[00:36:18] EM: I agree with that. If you’re a programmer and you want to understand languages more, you should just attempt to create one. Start with something simple. Start with anything because the process of building one lets you appreciate a lot about what the other languages do and a lot about where their problems are from and why they can’t do certain things certain ways as well. I think that’s an excellent sort of learning opportunity, but go into it. This is basically a hobby project. And languages do have to evolve. And the other problem with trying to design a perfect language from the start is kind of the problem and kind of when I experienced that with Leaf as well is that you have a startup sort of problem that if you wait until you have the perfect language before people start using it, it’ll be antiquated by the time it’s used and nobody will ever use it. So you have no choice but to have sort of like start with something rough and let it evolve. And the reason why most languages end up being great clunky is because they had to evolve and nobody’s solved this problem yet of how do you evolve a language when you keep it looking nice and simple and friendly.
[00:37:25] BH: Yeah. To follow up on that, is there a point where a language needs to define what it’s not supposed to be good at? Are there programming languages which were once maybe elegant and tried to accept solutions to new problems maybe they should never have tried to and maybe it became bloated and less usable?
[00:40:23] EM: Yes, it absolutely does. And I think you hit on something there about the language evolution as well. You often have those types of concerns about performance driving the evolution of a language, because you’re going to hit something that I need to do this, this language isn’t doing it very well. All right, so we have no choice but to find another way, then you find some other way, then the other language eventually finds, “Well, we found a way around that performance bottleneck in a slightly different way,” and then somebody else says, “But you have this problem now and so you have to evolve this other one here.” So it’s kind of this continuous give and take about what do you actually need. And I think this also fuels a lot of the arguments in line about what languages should you use to do something because they change quickly. And so a lot of times like, “Well, I need to use C++ for system programming because it’s the only one that’s going to be fast enough for what I need here.” And it’s probably true at the time then you know eventually some of the other languages whether they’re fast enough, right? So things can be fast enough or they provide different features. And I think there’s a sort of competition pulling them back and forth the entire time. But as you say, there’s this ecosystem around them that shapes them often a lot more than the language itself. There’s the need to do certain things in the ecosystem of how they’re used and that shapes how those languages evolve.
[00:41:42] SS: Yeah, definitely. And the browser is such an interesting ecosystem where certain things that like Google doing Chromium, automating their own use case, and they’ll like push that status forward, and the others will maybe CAPTCHA or there will be different support in different parts of it. Maybe then they’re trying to get like some set of agreed APIs in common that they all can agree to support. There’s like almost two different moving pieces here. There’s like the language moving piece and there’s the browsers, which are like a pseudo operating system at this point, moving underneath them as well. Right? You don’t quite see the same in like Linux or Windows. They’re not like changing up. Well, they support all the time. Different languages do different things.
[00:42:19] BH: Let’s get back to one more listener response, which I think is a good one to end on. Lorenzo wrote that it needs to be a moment in your life when you have time or at least you can find some time in your busy dev life and then you can read and learn a lot about the subject and then go do it. So this probably speaks to a component of things. It doesn’t necessarily always matter whether you have the skills, the interest, or the need. Sometimes it’s about being realistic about the commitment, right?
[00:42:54] EM: Yes. Even for the smallest language, there’s commitment there. I guess it depends how much it does, but yeah, anything beyond super isolated domain specific requires a significant commitment. But if you’re just doing it for learning, I mean, you don’t have to worry about that too much. If you want it to be useful, the moment you get into it, you’re going to see a lot of the things that you’re like, “Oh, I forgot I’m going to have to do that. Oh, and that. Oh, and that, and that and that and that.” So it’s like the simplest thing kind of grows fairly fast, but I don’t think you have to have a lot of time. I mean, if you’re doing it for your own project, you should go within it with the assumption that it’s not going to be used in your language. I know that sounds like a horrible assumption, but you should go in with that and make your journey more important than the outcome here. Because, realistically, that’s what’s going to happen and you don’t want to set yourself up for disappointment when nobody wants to use the language.
[00:43:45] SS: Yeah. Build it for yourself. What are you trying to get out of it? And that’s what you should be building it for you. You’re learning. You're having a bit of fun and you're scratching that itch. Build it yourself.
[00:43:55] BH: Sam, Edaqa, thank you both for joining us.
[00:43:58] EM: Thank you.
[00:43:58] SS: Thank you.
[00:44:08] BH: This show is produced and mixed by Levi Sharpe. Editorial oversight by Jess Lee, Peter Frank, and Saron Yitbarek. Our theme song is by Slow Biz. If you have any questions or comments, email [email protected] and make sure to join us for our DevDiscuss Twitter chats every Tuesday at 9:00 PM US Eastern Time, or if you want to start your own discussion, write a post on DEV using the #discuss. Please rate and subscribe to this show on Apple Podcasts.