Rust, PostgresML and the Art of Simplicity: Silas Marvin on Building Accessible Machine Learning Tools | Techstrong Unplugged EP28

Hey everybody and welcome back to Techstrong Unplugged. I'm your host, Atan Solomon, and this is episode 28 of our series. In this episode, we're taking you to AI Dev Summit 2024.

There our co-host Cassandra Chen met up with Silas Marvin, and they discussed Rust Postgres ml and the Art of Simplicity. Silas is a software engineer at Postgres ml and he shares his journey into programming as well as the power of open source communities. Without further ado, let's head over to AI Dev Summit.

Welcome back to Text on Unplugged. My name is Cassandra Chin, and today we're here with Silas. Marvin.

Great to meet everyone. Great to meet you. Can you do a short introduction about yourself?

Yeah, for sure, for sure. My name is Silas Marvin. Um, I work as a software engineer at Postgres ml, where we build up Postgres extensions.

Yeah, I've been programming now for, I think about seven years. Started like right when I was getting outta high school. Um, went to Portland State and got my degree in computer science.

I mostly work in Rust, uh, and I'm really passionate about machine learning. Um, yeah. How did you get into Rust?

Rust? Oh man, this is actually funny. I had just finished college and, and like for everyone who's like done computer science or hasn't done a computer science class, you kind of, uh, they make you take like a, a couple classes on a specific language like CBOs plus to teach you programming.

And then after that, like year you kind of get to choose whatever you wanna do. And I mostly worked in Python and JavaScript, which was really fun, but I'd just gotten outta college and I was like, man, you know what? I really need like a specific language that, uh, that I'm just like passionate about and could really use.

So I bounced between like Goling, I was thinking at Golan a law. I was looking around at some other languages and I kind of just decided to jump into Rust and I feel like I chose correctly. I'm a big Rust fan.

Um, but there are a lot of other awesome languages out there too. What like impacted your decision with Rust? Do you just like the code syntax or the community or?

That's A really great question. I think that when I first started with Rust, I didn't understand why everyone loved it so much. 'cause it's like a very confusing and complicated language.

I would say it's one of the harder languages to learn. Um, but the more that I got into it, the more I was like, wow, this is incredible. Because even like working through the Rust book or something, and I've had other friends who have gotten under arrest tell me this too.

Like, it really feels, um, it feels like a language that people were very passionate about when they made it right. But it also feels like a language that was constructed very intentionally, right? Like there isn't a lot of, you know, random things that don't really make sense in the language.

Everything feels very well thought out. Um, and they have like some really good examples in their RFCs of why they're doing things the way that they're doing. I think one of the reasons why I love Rust so much right now is not only do I love the features of the language, but I feel like it has probably the best community that I've ever seen.

And I'm sure there's other communities, but probably the best like open source community that I've ever seen. io, um, all of the popular packages are done so phenomenally well and the documentation is incredible. It's really easy once you get into Rust to really build whatever you want because the tools that you need are pretty much there for anything.

And like, that's really cool how you're so passionate about Rust. I think it's an incredible language. There are a lot of other like amazing other languages out there too.

Um, JavaScript is incredible to work in. Well, it can be incredible to work in. TypeScript is, you know, extension of JavaScript is awesome too.

And the, I just love groups that have great open source communities and you see that with Python too. Python is also, um, amazing, especially for like machine learning applications of working in machine learning. Um, I think the groups like TensorFlow and Pie Torch and, and even like Google Jacks are, are doing Incredible.

When you say community, do you mainly engage with people online like GitHub or do you go to like in-person meetups? That's great. I guess when I'm talking about community right now, I'm not even talking about like necessarily like a Discord community or, or, or like a Macedon community or an in-person community.

Um, I more mean like the ecosystem of the language, right? Like when you look at like the Rust ecosystem and you, it's like I would, and I would say that a big part of the ecosystem are all the crates in the ecosystem, right? Like when you talk about the Python ecosystem, I'm talking about like the pi PI packages out there, right?

Or or whatever package manager you want to use. Um, yeah, I guess that's what I mean. So the people Yeah, the people, yeah.

But a lot of times just the open source developers that are building these things out. Have you built any applications with Rust or like things which you're proud of? Yeah, great question.

Great question. All of the stuff that I do for work, most everything that I do for work is in Rust and yeah. Oh, it's, it's a lot of what we do is open source.

So, um, at Poster some we have a lot of open source stuff we push out. It's all rust written. I'm pretty proud of it.

It's, it's, I think it's cool stuff. It's really fun stuff to work in. Um, yeah, and I mostly work, it's actually interesting because I write, uh, our Python and JavaScript SDKs, but I write them using Rust.

So it's all Python and JavaScript wrappers around our actual rust. SDK. So it's fun 'cause I do get to work in a lot of cool, like dynamically typed languages, but I also get to work in rust, which is really fun.

What's the reason for robbing the rappers in a different programming language? This is great question. Um, there's a couple reasons I think that I think there are a lot of benefits for using a strongly typed, statically compiled language, right?

When you're building complex systems, um, especially a language like Russ, where you have some guarantees of memory safety, right? So by using this underlining system, not only are we fast, we have guarantees of memory safety, we wanna translate those to other languages. The other reason is, you know, there's a lot that goes into building a complex system like the, the actual, um, like what I've worked on, it's like thousands of lines of code.

It's a lot of work. It takes a lot of effort to build this out. You have to write a bunch of tests for it, right?

Like there's is months of developer work to get this. If we were to rewrite it in Python, it'd be the same thing. Yeah, we already have the code and rust, right?

But if I'm gonna translate it to Python, that's a ton of work. I've gotta rewrite all my classes in Python. I've gotta write a bunch of tests in Python, right?

I still gotta make sure it's all working the way that I want it to, even if I already know what I want it to end up looking like. It's still a ton of work. Same for JavaScript, right?

If I wanted to end up writing this for JavaScript, it's gonna be a ton of work, right? Um, I'm gonna have to rewrite all my tests, we've reread all my classes, but if I'm just writing some wrappers around it, that's a lot easier. So that's, that's the main reason for, for working in Rust.

Um, it also lets us bind the languages like see really easily. Um, so yeah, that's one of the primary reasons, Like that's an interesting take on rappers. You can kind of save the amount of effort of completely rewriting it.

Yeah, absolutely. And, and you see, like, you see this in most languages, right? You see a lot of, like, there are ways to call down and to see functions with Python.

There's way to call down and to see with JavaScript you get that with Rust too. There's a lot of rust interaction with seed. So it's like, it's really fun to stay pure within a language and just like work within Rust or just work within Python or just work within JavaScript.

But if you wanna have this, like if you wanna be able to reuse code that other people have written in, you'd usually end up dipping into foreign function interfaces, um, which isn't always that bad. It's a little bit of a learning curve sometimes, and sometimes errors cannot be the most fun to deal with. Um, but yeah, it's cool.

It's really Cool. Do you wanna talk a little bit about like maybe more of what you're doing in your current job? Yeah, for sure.

I could talk, I could talk a lot more about what I'm doing on my current job. So, um, I work for Postgres, MLSI mentioned we build, um, machine learning extension for the Postgres database. If you're not familiar with Postgres database, it's a relational database system.

It's very similar to MySQL. Um, it's open source. It actually got like the number one most liked database or most to use database on Stack Overflow this year, which was really cool.

That's huge for the open source community world. Um, and yet we build PGML, which is an extension for it. And really what PGML does is it takes machine learning models and it brings them to your data.

So if you've used anything like hugging face before and you're familiar with some of their APIs, you can do pretty much everything that hugging face does now with SQL functions in Postgres. Um, so I do work a little bit on our actual extension, but primarily what I work on is making it accessible to application developers, right? Not everyone is a SQL developer.

Not everyone knows how to write complex SQL queries and take advantage of in database machine learning. So I work primarily on making that accessible for application developers who work in Python, who work in JavaScript. We're building out an elixir, um, extension right now, which is a really cool language.

Uh, we also have a c um, s STK now too. So that's primarily what I work on. I also do some integrations with third party libraries.

So if anyone's familiar with LAMA Index, um, they're like a, they're, um, production abstraction. It's a Python library, a jobs strip library. So kind of an abstraction for developers building rag applications.

Um, I built a integration for them this last week, which was really fun. Earlier you were saying that you make this more accessible to developers. Yeah.

Like how exactly do you do that? Yes. Yeah, this is great, right?

So, so when you're building any kind of library, right, there's a trade off, right? Let's, you know, like, like this is a great example. So, um, there's a lot that you can do really anything, right?

With our extension, right? You have a bunch of SQL functions that give you machine learning capabilities for text generation. Splitting, embedding, that's awesome.

If you wanna build an application from that, that means that you need to manage uploading documents, splitting documents, embedding documents, storing these sectors. That can be a lot, right? So you have, you would have to write that yourself now to make that more accessible.

'cause we know these are common patterns that people want to be able to do consistently. We can write a library, another library on top of our libraries. Now we're two libraries deep here.

Um, that does that for you, right? So we know the common tasks that developers want. We know what they like doing.

We can write another library that makes that easier, that does a lot of that functionality for you, right? So instead of you having to write all these functions, you can call a function that we wrote, you know, create a class, construct a class on Python, pass 'em the name of the collection that you want, pass 'em some documents and we'll do that all for you. Um, yeah.

Yeah, I guess that's what I mean when I talk about making it more accessible. I mean, abstracting away some of the complexity, right? So you don't have to be SQL developer anymore and you don't necessarily have to write all these pipelines out yourself.

I think it's nice you can make it a little more simple. Yeah. It's, it's, it's awesome to make things more simple and it's really a trade off, right?

Like when you're building libraries like this, there's a really fine line between giving users complete control over everything they want, um, and making it really complicated, right? You don't want to give people too many knobs to turn or you're not abstracting anything. Um, so that's a interesting trade off to kind of figure out and learn and build out.

I can think a lot of like desktop applications where you're balancing between more features or simpler UIs. Absolutely. Think of, um, you know, GitHub has like a popular GUI interface right?

That a lot of people use for, um, managing their, um, their git repositories, right? And, and that's great and it's an abstraction behind, it's a library, right? It's an abstraction behind the actual git command, which you can do everything.

It's just using the GI command underneath, right? It's just an abstraction over that GI command because the GI command can be really complicated and you can do incredible, crazy complex things in the GI command, right? Um, yeah, I think it's, I think it's, I think it's exactly like that.

So today we're at AI Dev Summit. Have you been to conferences like this before? I have been to, um, uh, I've been to the Postgres conference in San Jose.

That was interesting. That was a lot more Postgres developers, um, and a lot more Postgres focused. This conference is cool because I think it, uh, focuses a lot more on machine learning and artificial intelligence, which is really fun.

Um, I haven't been dual a conference this big before. Is there any like talks or sessions you're excited to learn about? Um, Or are you giving your own session?

I am giving my own session. Yeah. So to be honest, I haven't, uh, I haven't looked around a ton at other sessions.

I'm mostly probably gonna be managing our booth and then I have a talk tomorrow, uh, 11 on our integration with Llama Index. Um, and why we think Postgres ML makes rag applications easier, which will be fun. Um, yeah, that'll be really fun.

I think that's definitely interesting. Yeah, it should be cool, you know, um, yeah, it should be cool. It should Be really fun.

A llama seems really popular nowadays. I hear it everywhere. Yeah.

Llama. Yeah, I feel like, um, I don't know, it might've been meta that started it with their llama models. I'm not a hundred percent certain, but Llama as I phrased Yeah.

Is, is very popular. It's funny 'cause you get, you get things like llama file, llama CPP, which are both backends for running machine learning models and you get LAMA index, which is, you know, a, a library for building out rag applications and then actual meta, uh, meta lama models themselves. So yeah, it's, it's really popular.

And look, we've had a really good chat today, Silas. Yeah. Yeah.

Thank you. Uh, thank you. Uh, I really appreciate AB So thank you for joining us today.

Yeah, Of course. Of course.