Welcome to this episode of The Rabbit Hole where we get into the pros and cons of importing libraries versus rolling them yourself! Opting to import a library is a common choice due to its pre-written functionality, but sometimes we import huge ones, just so they can perform a few functions. We then get left with bloated code containing a bunch of unnecessary script. In those instances, it can be very helpful to roll your own, but that can also turn nasty quickly as problem piles on problem, most of which have probably already been solved by a pre-existing library somewhere in any case! One of the other plus sides of using off the shelf libraries is the community support that they offer.
Today we get into tradeoffs such as these, and cover additional themes of finding a middle ground between writing and importing, traumatic memories of deciphering ancient base codes, why companies should update their base code but how challenging it can be, how library support can just die suddenly, and how to check whether to use a library or not. In the end, all code is slowly rotting, and you’re screwed whichever option you take! Just kidding, it’s not all doom and gloom (or is it?). Press play and find out!
Key Points From This Episode:
Transcript for Episode 137. When to roll your own
[INTRODUCTION]
[0:00:01.9] MN: Hello and welcome to The Rabbit Hole, the definitive developer’s podcast in fantabulous Chelsea, Manhattan. I’m your host, Michael Nunez. Our co-host today.
[0:00:09.8] DA: Dave Anderson.
[0:00:10.8] MN: Our producer.
[0:00:12.0] WJ: William Jeffries.
[0:00:11.8] MN: Today we’ll be talking about when to roll your own library.
[0:00:14.4] WJ: And when not to.
[0:00:16.6] MN: And when not to, that’s important. I think, figure out if the library exists and think whether you want to roll that bad boy up into your own application or just take it off the shelf. Let’s talk about some of the tradeoffs when using another library versus rolling your own.
[0:00:36.0] DA: I think that’s the corner stone of it. Sometimes people look at what’s out there and they consider like, “My use case is very specific,” or you know, “I could definitely do a better job so I’ll just do this myself,” but they may not consider the pragmatic aspects of rediscovering all of the chases that someone else has already gotten through.
[0:01:00.2] MN: Right, I mean, one example that comes into mind is like, Lodash, right? You shouldn’t re – doing certain methods that Lodash can help you in JavaScript. One of the problems of bringing in Lodash is that it could be very bloated. If you use Lodash just for like map for example or reduce and you have to bring the entire Lodash library into your application, it’s going to have a lot of bloat just for those two methods that you really want to use in this particular part of the code base.
[0:01:30.5] WJ: Yeah, I remember people recommending actually going in and copying the code for the one method in Lodash that you wanted and pasting that into your code base. Although now, they allow you to export just the one.
[0:01:43.4] MN: Yeah, I mean, that was – that’s been around for some time but I think before, you had to import all of Lodash and I think the solution as what William said, which is like good old copy pasta right over that method. Name it Bobby Map and then you’d be able to bobby map all across your objects which is great. Name it whatever you want.
[0:02:03.4] DA: There you go, roll your own Lodash. Bobby dash.
[0:02:08.8] MN: This is the best kind of dash.
[0:02:10.7] WJ: Yeah, it seems like those are the two primary ways that you can get yourself in trouble is one, bringing in a library that is massive and has a lot of complexity to do a very simple thing because you didn’t want to write a method.
[0:02:25.4] MN: Right.
[0:02:26.5] WJ: And then the other one is, trying to come up with something that some other library has already figured out how to do and thinking that you know better than the person who wrote that library and doing just the one thing and you know, tweaking it to fit your specific use case and then not realizing that pretty soon after you’re going to need yet another part of that library and then another part of that library and then you realize that you could have saved yourself a ton of time by just relying on someone who is already thought about this problem a lot, come up with a whole lot.
Come up with a whole library that’s generic enough to handle most use cases which means probably your use case.
[0:03:12.5] DA: Yeah. Although, sometimes when it is something that’s written to be so generic that it handles all use cases in a loose way – I’m thinking of like serialization frameworks like change the rest framework. If you’re going with the base case where you’re just like serializing a model, sending it over the wire, then it’s like pretty good, it’s like, it’s kind of magic but it does its job pretty well but then when you start considering like cases of like nested objects and how that gets sent back and forth then you start to question like what Tom Christie was thinking when he made it and like when you could just write the code yourself to like you know, update a field or something like that and it’s like, five lines of code instead of one line that you have to look at the docks for three hours in order to figure out.
[0:04:08.3] WJ: I remember, I had to make a proxy server and it was going to be extremely simple, like the only thing that the proxy server needed to do was forward a request with a header, that was it and I was like, “This is so simple, I shouldn’t bring in Express, like it’s got to be written in Node but I can do this with the native node http server.” And so I wrote my own.
[0:04:38.1] MN: Okay.
[0:04:40.0] WJ: Then you know, pretty soon, somebody came along and they were like, “Hey, we need to add caching,” and I was like, “Okay, well, if I were doing Express, that would be really easy but now I have to go and look at the docks trying to do this with Node and nobody maintains these docks because nobody does this, they use express.”
[0:04:58.7] MN: Yeah.
[0:05:00.0] DA: The solution is just like some of the shelf middleware that you can just plug in.
[0:05:04.0] WJ: Right. Then somebody came along and was like, “We need to add tracing and you know, we have – you should just use the library that’s already been made that adds it automatically as middleware for express servers.”
[0:05:20.7] MN: Yup.
[0:05:21.4] WJ: We even have a version for Restify if you don’t want to use Express and was like, easier than those. I ended up rewriting the whole thing in Express, just like sometimes it’s better to just use the damn library.
[0:05:36.1] MN: Yeah, you mentioned one thing where you had hand rolled the header aspect in native node and you having to read up on the docks for how to cache something. One of the alternatives to using the off the shelf is the idea that other people will also – especially if it’s open source, other people are also making the off the shelf library better so that you can read the docks because they are updated because other people are using and updating those documents as well.
Most probably like, the alternative as to why you would want to use an open source library rather than hand rolling your own because who is writing the docs, who was writing your docs when you were hand rolling the header wrapper in node?
[0:06:22.3] WJ: Definitely not me, I didn’t write any docks. You know, even if it’s not formal documentation, just being able to Google it and get a stack overflow post is extremely valuable.
[0:06:33.2] MN: Right. You’re less likely to get a stack overflow post of something that you created unless you were to post it and answer it yourself. Although every once in a while, you know, you come across your own stack overflow question and you’re like, this is like exactly what I needed – somebody else had this exact same problem, who was it? I didn’t learn last time.
[0:06:56.8] WJ: There you go, it’s me again, your old friend.
[0:07:00.5] DA: Yeah, sometimes people don’t want to add dependancies because they’re afraid of getting like locked in to a vendor or a particular like provider of a functionality too, if you're using like a SaaS provider for you know, logging issues or obsessed writer for a feature, toggling, you’d be like, “Well you know, I’m paying these people and maybe one day I’ll write my own thing or I’ll go to a different provider so in that case, maybe it may make sense to put an abstraction layer to wrap that third party library so that you know, you define the API, but then, the heavy lifting is just done by calling out to the third party library.
[0:07:49.7] MN: Yeah, I mean, I think designing your application comes into to play so that you’re able to swap out that third party library or like, you know, as you mentioned like the logging library that you’re using, if you find like a cheaper end alternative, you would have like an interface that just says like ‘log message’ and regardless of whatever this logging mechanism is, whether it was hand rolled or you’re replacing with something else allows you to then swap it out effortlessly.
I mean, I find that it might be more difficult like if you are using like a database or what not but for services you want to make sure that you’re – you design your application in a way that allows you to swap those third party or hand rolled relay things as effortlessly as possible.
I think another reason why one may want to use a library that’s off the shelf is because other people might know that library because it’s pretty popular on the internet to use. I mean, the idea that like, you were about to hand roll a express server would have made it very difficult for someone who is new to the organization to read your code but maybe well versed.
[0:09:03.2] WJ: The time it seemed like such a good idea, I swear. It was like you know, I don’t know, like six lines of code, it was so good.
[0:09:11.6] MN: Yeah, I mean, as it would have exponentially gotten bigger than it would have been a little more –
[0:09:17.0] WJ: It started growing, it was like a tumor.
[0:09:20.0] MN: Slowly but surely, yeah. But like, someone will be more well versed to being able to extend the feature without having to pick your brain if you were up and gone and they would look at your git commit message and be like, “Ah well yeah,” with their first in the air being extremely pissed at you. But then it was Express, you’re fine, don’t worry about it. I mean, I think I have experience with dealing with a hand rolled app like third party tool.
[0:09:50.0] WJ: What is it they say? Always code as though the next person to work on your app is an axe murderer who knows where you live?
[0:09:56.4] MN: I was ready to shank Bobby with what he left the code base bro, man. So upset. Bobby hand rolled his own Redux in regular vanilla, like in JavaScript react and it was just so hard to follow.
[0:10:13.8] DA: That was cool of him.
[0:10:14.4] MN: No, it’s not. I knew Redux, they’re like yeah, you know, come in, help extend the code base and use this library that Bobby built and was gone and left people to their own devices and the only people who knew how to use it –
[0:10:31.6] DA: That’s interesting. I feel like I’ve heard this story before. This is our version of this library for whatever.
[0:10:40.8] MN: I mean, I was at then helped the organization just like, “All right, let’s slowly but surely decouple this and use Redux when necessary and ensure we add features to refactor this old janky version of Redux and actually do the successful version.” But it was hard.
[0:10:59.2] WJ: That’s always so hard, yeah. Because you got to get the company to buy in to actually try and migrate away from the old thing which is going to be time consuming, expensive and it’s probably never going to be done.
[0:11:10.4] DA: Right, and also like, but even if you did undertake it like – it’s kind of worrisome, it’s like trends change quickly and you know, our kids – do kids still love Redux.
[0:11:21.9] MN: I haven’t been in JavaScript world in a minute so I actually don’t know.
[0:11:25.2] WJ: I don’t think the cool kids are into Redux anymore.
[0:11:26.8] DA: Yeah, you know, Dan [inaudable 0:11:28], the Facebook people are like, putting all that stuff in React and trying to take over the world in GraphQL and – keep your Redux.
[0:11:40.5] WJ: So when you make that decision it’s like, “We’re going to cut out the tumor.” We are going to extract this bad pattern, we are going to replace it with something more standard that is easier to hire for, easier to maintain. You are probably going to end up in a terminal state that is still includes a little bit of the old crappy pattern because there is going to be some part of the code base that is just not worth refactoring.
[0:12:05.4] MN: Oh that really legacy code base filled with spider webs and dust.
[0:12:10.1] WJ: And then you know, you got to get buy in from people that it’s actually worth it to get 80% of their refactored done like that is going to get you enough additional productivity to be worth, not just the investment of time but also then having two data stores or two of whatever the thing is.
[0:12:29.7] MN: Oh yeah, it was hard like we just got through it and slowly – I didn’t see the sunset of the old application but it was nice to know that I helped pave that way a little bit like get the engineers riled up like, “Come on guys, we don’t need to live like this anymore. We can do it. Well let’s crush it and let’s import Redux,” like you’d probably hate me right now if they’re listening.
[0:12:59.8] DA: “This guy convinced us to use Redux and then he left.
[0:13:02.8] MN: Yeah, don’t forget the other Bobby.
[0:13:05.6] DA: It is the legacy of people moving on. Yeah, I was feeling that too with that wrapper pattern that I was talking about like sometimes it works out fine but you know sometimes you have a Bobby that writes a wrapper pattern then moves on and it is just leaky. It is just leaking the abstraction all over the place where like it was an ORM, a custom ORM but it was based on SQL all command under the hood and in order to use it, you have to understand this pattern for brightening database actions but then also under SQL alchemy as well.
[0:13:52.3] MN: Oh man.
[0:13:53.6] DA: But you can’t use all the cool features that SQL alchemy had. You had to deal with whatever limited send to go.
[0:13:59.8] MN: Would it have been easier to just use SQL alchemy without the wrapper in general or?
[0:14:06.0] DA: I think so because I would have been able to Stack Overflow something. It was just like software archeology. It was like, what happened here? What great people are arose and then fell.
[0:14:17.9] MN: Oh man just like the Aztecs in Mexico.
[0:14:23.0] DA: No trace.
[0:14:24.8] WJ: These area is a crime scene.
[0:14:26.3] DA: All they left is the ORM. It is like the calendar with the wheels.
[0:14:32.8] MN: I mean yeah, I think like they’re just – it should be noted that like even the things that are heavily supported off the shelf will also become deprecated kind of. Like it will become legacy too. It’s just like you have more support of the internet helping you but there maybe times where you know, high rolling was the way to go and that’s the feature that you needed at the time.
[0:15:01.1] WJ: Right, I mean it seems like the message that we are giving people is if you roll your own, you’re screwed. If you use an off the shelf library, you’re screwed and if you try and wrap it off the shelf library then you are going to be screwed two different ways.
[0:15:14.5] DA: Oh yeah and what we’re saying is it’s software engineering guys.
[0:15:18.0] MN: No but it is different though.
[0:15:19.4] WJ: Maybe we need a better more uplifting message.
[0:15:21.6] MN: But it is different though because when you are using an off the library tool, you’re screwed with the rest of the internet and you get to find a way to move together as a unit as you migrate from one library to another. I think we pan roll it yourself, you’re by yourself bro. You just got to deal with it.
[0:15:41.7] DA: Well it is sad like if you use a library that – the support just dies off entirely and then –
[0:15:48.3] WJ: Oh my god that is the worst especially when it is a really good library and then they just stop supporting it and then it just starts to rust and everything else upgrades around it.
[0:15:57.9] DA: Yeah right. You are pinned on the earliest version of Django or you know, two version of Django behind because your sleeve’s in tasty pie and you know, they haven’t updated that in a year or two.
[0:16:11.6] MN: Yeah, I just thought of CanCan, the Ruby CanCan, oh man.
[0:16:16.5] WJ: Oh yeah, I remember CanCan.
[0:16:18.0] MN: Cancan, ah man, RIP. I just went and Googled it. The last commit change was seven years ago.
[0:16:24.2] WJ: Didn’t somebody forked it and made a CanCanCan?
[0:16:27.1] MN: Yeah, I think there is a CanCanCan.
[0:16:29.2] WJ: How is that one doing? Is it still alive?
[0:16:31.1] MN: Let’s find out.
[0:16:33.5] DA: Yeah, it is actually when you go to Stack Overflow where you’re like I need to answer this question with this library and the answer is, “Please don’t use this anymore.”
[0:16:42.8] MN: All right, well the last commit for CanCanCan was nine days ago, that doesn’t look so bad but imagine if you are using CanCan at that time, seven years ago and then suddenly, Bobby just disappears. What do you do?
[0:16:57.3] DA: You got to make CanCanCan.
[0:16:58.7] MN: You got to add that on the cans, you CanCan, you know what I mean?
[0:17:02.8] WJ: Yeah, well this CanCan can’t but this CanCanCan.
[0:17:06.5] MN: This got CanCan cans but I think like yeah, you got to move with the community if you work on an off the shelf library but there are instances where you – just it rolls off and yeah, you just got to be on your toes and have your pulse. Yeah, have your – know that where your library is headed. You have to be mindful just as if you were to hand roll your own thing. Do you have guidelines as to what kind of off-the-shelf libraries you would use personally? Because I think GitHub stars does it. It is a lot of stars. I think I might rock with that one over another.
[0:17:44.5] WJ: I think the number of stars is an indicator of popularity. I think like how recent the most recent commit is can also be a helpful indicator. I think reading a little bit of the code in the library just to see if it is well factored and checking to see if they have any tests. That is helpful. Looking at the documentation, making sure that the documentation is easy to follow.
[0:18:05.7] DA: Making sure it exists, yeah.
[0:18:07.3] WJ: Yeah exist at all.
[0:18:08.8] MN: I mean like I would have gotten burned with the star thing actually because CanCan has 6.3 case stars and CanCanCan has 4.5 but the commits as we mentioned earlier, a more later than the CanCanCan supported by the CanCan community.
[0:18:24.1] WJ: I think also asking other people who have faced similar problems what libraries they worked with and what their experiences were like. That can be helpful.
[0:18:33.0] DA: Sometimes it is driven by what you can hire in your market. Like if there are a lot of people graduating from boot camps and they all know Ruby and Rails then you know that’s – and you need two new developers then that can inform your decision to be like, “Okay, I don’t need to roll my own custom thing. I am going to go with something with more framework-y that is a common tool in the community.”
[0:19:04.5] WJ: Right or if there is one really critical tool that you absolutely have to use that could also be a factor. I think Twitter switched from Ruby to Java specifically so that they could use Lucene.
[0:19:16.9] MN: Lucene?
[0:19:17.3] WJ: Yeah for search.
[0:19:18.3] MN: Oh I see and I mean there is a lot of fail wheels back in the day too though so that might have done it.
[0:19:24.5] WJ: Yeah.
[0:19:25.5] DA: Yeah I think the instance of just Scala.
[0:19:27.7] MN: Yeah, Scala, Scala, Scala. Yeah, so I think as you mentioned before, looking at the latest commit and test is probably more helpful, asking the community what application they would use, that kind of thing is good.
[0:19:39.4] DA: Seeing how many outstanding issues are there.
[0:19:41.9] MN: Oh yeah because if they keep piling up and no one is answering them, ooh boy that may be a problem.
[0:19:47.4] WJ: I think that code is sort of like an asset that you’re investing in and it depreciates overtime, you know? All assets and all of your code is on an inexorable march to the grave. It is all just slowly rotting. I was trying to get a more uplifting interpretation here but I seem to be going in the wrong direction.
[0:20:12.2] MN: No but then you have to slowly replace that.
[0:20:14.8] WJ: That is what I was getting at, yeah is that you should just accept that whether you roll your own or whether you use a third party library, that it is eventually going to stop being maintained like code rots. The code that is important will get rewritten many times and it will stay relevant because you are constantly rewriting it because it is important.
[0:20:35.9] MN: Right and stay healthy.
[0:20:37.3] DA: Yeah, momento moire, just gotta live today.
[0:20:42.2] WJ: Just because your green field app eventually becomes a brown field app and that tasty prior library that you used to love becomes an albatross around your neck doesn’t mean that you are necessarily made the wrong call. Like that was going to happen even if you rolled your own eventually that code will have rotted like I don’t know, you are going to have to invest in order to fix it and that is going to happen no matter what you choose.
So just be okay with it. I would say like use the [inaudable 0:21:09] judiciously. If you are only going to use a small piece of a library maybe don’t bring in the full library but if you start finding yourself using more and more of the library, stop rolling your own and adopt something while you are early on enough in the process that doing the rewrite is relatively painless.
[0:21:29.6] DA: Right, I think there is something to be said for like, you know, that, balance of doing your own, like breaking out of the shell of the – what a framework provides you or what a library provides you and sculpting your own domain around that because if you do it well then that shell, that domain will still be applicable as long as your business is applicable. So you know, check out those design patterns.
[END OF DISCUSSION]
[0:21:59.8] MN: Follow us now on Twitter @radiofreerabbit so we can keep the conversation going. Like what you hear? Give us a five star review and help developers like you find their way into The Rabbit Hole and never miss an episode. Subscribe now however you listen to your favorite podcast. On behalf of our producer extraordinaire, William Jeffries and my amazing co-host, Dave Anderson and me, your host, Michael Nunez, thanks for listening to The Rabbit Hole.
[END]
Links and Resources:
