Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

As a beginning Python programmer, is it a good idea to build and understand my own libraries before jumping to advanced 3rd-party libraries that contains the functionality I need?

Some projects (e.g. web frameworks like Django) are probably too large for this approach. But other projects (e.g. Web Crawlers, graph libraries, HTML parser) seem to be feasible.

I worry that early reliance on 3rd-party libraries would stunt my growth.

Note: this question and this question seem to focus more an experienced programmers, who are probably more focused on the efficiency of reuse than the learning benefit. My question, I think, is focused on beginners.

share|improve this question
5  
"nothing more than a library integrator rather than a programmer": customers want you to be productive. Being able to evaluate and integrate existing libraries is an important skill that you need to be productive. If there isn't an available library that suits your needs... otherwise focus on solving the business problem that your application is addressing –  Hamish Smith Apr 28 '11 at 23:24
2  
What distinction are you making between "library integrator" and "programmer". In 30 years -- even making embedded chip-level device controllers -- I've never worked without libraries. Can you explain this distinction? –  S.Lott Apr 28 '11 at 23:50
    
Poorly phrased. Have edited. –  MikeRand Apr 29 '11 at 1:17
    
I would rather hire a library integrator. Why concentrate on the how while you could concentrate on features? After seeing a few dozen badly reinvented bicycles I cringe at the thought of seeing another one. Library design is hard work and a balancing act. A typical software shop does not specialize in library design does not have the right skills, resources, and patience to do it right. Beware of rewrites. It is better to use code that has been tested by others than start from scratch. Only when there is a vacuum should one invent, IMO. –  Job Apr 29 '11 at 2:24

6 Answers 6

up vote 11 down vote accepted

This is always a trade-off.

As a beginning programmer, you should ask yourself two questions when considering reusing code vs. reinventing the square wheel:

  1. Will I learn more about the problem I want to solve by writing everything from scratch, or by focusing on the problem domain and putting aside complexity not critical to the problem that I am interested in?
  2. Is it more important to me to solve the problem at hand or is it more important that I understand some fundamental concepts?

If you don't have to finish your project, it's fine to spin your wheels on complex problems that other people have already solved, because you'll learn something. But you'll probably move on to something else before you "finish," which may or may not matter to you. Other projects will start to look shiny fast when you get in over your head on a complex domain that looks simple until you start trying to solve it yourself.

Don't obsess about giving up control because you're deferring to someone else's way of thinking; focus more on what you're trying to accomplish.

If your goal is to write an HTML parser because you want to understand how parsers work, go for it. If your goal is to write an HTML parser because you want to sanitize user input or transform some random bits of HTML, you're probably focused on the wrong thing, because you probably are more interested in the application of parsing rather than the parsing itself. If you feel like writing an HTML parser because you don't want to take time to understand someone else's library, you're probably wasting your time, because, at least in this case, I guarantee someone else has spent more time figuring out how to solve this problem effectively than you'll have. In really trivial cases, you may save time by not reusing code, but in complex ones, unless the library you use sucks or your ability to read documentation and code samples sucks, you'll just waste time.

On the other hand, I would say that it's worth writing your own graph library, since you'll be more focused on transferable, fundamental algorithms and data structures that you'll be able to apply to other domains, even if you end up using someone else's library when you work on those problems.

share|improve this answer
    
Fantastic framework for thinking about the problem. Much appreciated. –  MikeRand Apr 29 '11 at 10:24

Most beginning programmers vastly underestimate the complexity of a problem. This leads to many unpleasant situations, but I'll focus on just one: they dismiss a lot of third-party libraries for being "bloated" or "too complicated". Then they try to write their own code to do the same job, and make a complete mess of it, because they didn't anticipate all the stuff that they'd have to handle.

Take web crawling and parsing HTML. You seem to have a pretty casual attitude towards those tasks - which leads me to believe that you never tried to do them on anything but a very small scale. Here's just a short, non-comprehensive list of the potential problems that a real web crawler needs to deal with:

  • Malformed HTML.
  • Multiple character encodings (the whole world doesn't run on ASCII).
  • Links that are generated or modified by JavaScript.
  • Links to multi-gigabyte binary files.
  • Incorrect mime-types.
  • Crashing in the middle of a crawl session.
  • Tar pits.
  • SEO consultants.
  • Microsoft.
share|improve this answer

I worry that early reliance on 3rd-party libraries would stunt my growth and make me nothing more than a library integrator rather than a programmer.

That's false.

Early reliance on 3rd party libraries will teach how good libraries work.

Premature development is always a waste of time. Writing your own libraries without carefully studying (i.e. using) existing libraries dooms you to reinventing the wheel -- badly -- for the rest of your career.

I've spent many billable hours cleaning up those messes.

share|improve this answer
    
Not sure I entirely agree with that.. Sometimes 3rd party libraries are poorly written and can actually teach bad habits. If the underlying algorithms are well understood, then re-inventing the wheel may be a good idea, and then comparing them to other libraries to see how well or poorly yours stacks up to the others. In either case, obviously some form of mentorship is hugely beneficial :) –  Demian Brecht Apr 28 '11 at 23:47
    
Having said that, there is benefit to studying existing API :) –  Demian Brecht Apr 28 '11 at 23:47
2  
@Demian Brecht: "Sometimes 3rd party libraries are poorly written and can actually teach bad habits" While this could be true, most open source projects are heavily vetted by a community of users and bad ideas don't withstand the wisdom of the crowd. Further, "poorly written" is a judgment no n00b can make effectively. –  S.Lott Apr 28 '11 at 23:49
    
@S.Lott: Well said (re: n00b) :) And I guess you'd just have to be careful about which 3rd party libraries you look at, as some have very small (if any) communities. Poking around random projects on github for example, may start leading you down the wrong path. –  Demian Brecht Apr 28 '11 at 23:52
2  
@Demian Brecht: You can't develop any judgement until you've seen a variety of libraries. There's no possible way to "pre-filter" the libraries for a n00b to use. It's all libraries or no libraries. –  S.Lott Apr 28 '11 at 23:54

This sentence, which is the first one is very logically problematic for me:

As a beginning Python programmer, is it a good idea to build and understand my own libraries before jumping to advanced 3rd-party libraries that contains the functionality I need?

How can you believe you can effectively write advanced (your word) functionality libraries as a beginner that will be as correct, feature complete and tested as existing libraries?

This doesn't even mention the massive amount of time you would waste re-inventing a wheel that you don't understand as a beginner, will eventually abandon when if you are lucky you realized you are over your head, and have to learn the exiting library anyway.

Take HTML parsing for example, you mention this as feasible, there is no way you can do a better job than Beautiful Soup as a beginner. Plain and simple you will fail. The things you think are feasible have huge complexity issues that you are not taking into account, you don't know the problem domain in enough detail to understand the complexity, you are doomed to failure just from that.

My opinion is learn what other people have done and already solved your problem for you, until you can't find something that addresses your problem.

share|improve this answer

This is just my opinion, but I'd suggest building your own libraries for pretty simple, re-usable code. For anything more complex, you can use 3rd-party libraries, which have been built and tested thoroughly (hopefully!). Creating a robust library can be very time-consuming, depending on what it is. I'm sure you'll get plenty of programming practice just by building your application!

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.