Ways to describe or measure the learning curve for a new technology?

Question

As I begin to learn some major new things beginning from zero, I wonder: is there any way to describe, perhaps even quantify, the height of a learning curve?

Having worked at many places on many projects, and half the times not being a good fit, I see we all need better ways to describe skill levels and skills needed for a project. I'm interested, for the purposes of this question, in describing the amount of increase in skills/knowledge/savvy for someone to be effective on a given project, staying with the same employer or in the same situation generally.

Example: The owner comes to me and says "Hey, this new thing, it's vital technology for our next major project. No one here knows much about it. How long will it take you to get up to speed on it?"

(BTW, I want to consider only competent people. No PHBs or fanboys of glitzy new things. Those confounding factors are important in real life, but secondary to the main point here.)

Loose relative words like "steep", "easy" etc don't mean much. The person learning the new thing, and their manager, may have different expectations. I'd like to find something more objective, even if not perfect.

In some magically ideal world, we might say "it takes (n) hours to learn C++, to skill level (x), including all of OO, STL and (blah-blah) set of design patterns". Hours could be used to mark calendars, assign team members and generally plan well. But of course, we all come with different thinking styles, different past experience, different IQs, different levels of passion to know or use the new thing. And then, do you want to design it, fix it, or just use it? Hours might not be useful, even if there was some formula to account for all those factors.

Personally, I learn new ideas in electronics or photograpy about 3 to 10 times faster than new ideas in software, but don't know if I'm comparing new ideas of the same "size". (And I'm not sure in what sense I mean "faster"; not necessarily hours or days.)

In part, I'm wondering about the "size" of knowledge it takes to become competent at, or to master, a new technology. There is the rough rule that it takes 10,000 hours to master something. Maybe for becoming a superb flute player in the symphony, or to be a top-notch helicopter pilot, but obviously one can master tying their shows in far less time. Shoe-tying is a "small" skill, the others are worthy of full time careers.

Maybe someday, with future discoveries in psychology, neurscience and education theory, one could say something like: "it takes (n) million mental thought quanta to learn (new thing) given you already know x, y and z and have a natural talent for grokking (subject)." Then there's be a rule of thumb that a million thought quanta on average correspond to so many minutes or hours, with some of us faster or slower, affected by stress, health and other work. Yeah. Nice science fiction, but not useful today for deciding whether or not to accept a project, which candidate to hire, or making career changes.

What can we do today to describe the steepness of the learning curve, or rather, the height of the hill we're climbing, metaphorically speaking? What's better than vague subjective generalizations?

Answer 1

My gut says that this is actually a question for an educator, and people who specialize in the acquisition of skills (professional trainers, etc.). There are "Education" and "Education Technology" forums out there, a couple of things in Area 51 waiting for graduation, and tons of people asking questions about this very topic, within the discipline of education.

I don't think learning a programming skill is any different than learning other skills. The question you ask is pretty broad, but as a starting point, I'd say:

1. Break the "skill" into its component parts - foundational concepts required to get from "zero to proficient", "zero" being some assumed prior knowledge (e.g. the person has completed certain courses/has certain experience/can solve certain fundamental programming problems)
2. Start working with people that you know.
3. Measure their progress through the foundation concepts
4. Test their resulting proficiency by having them build real things, facing real problems, and see how good they are.

Now, these steps are really vague, leaving implementation up to you, but as others have noted, while your goal is very specific, the road to you knowing enough to pull this off is its own learning curve.

Maybe by studying your own skill acquisition, what works for you, and applying what you learn to trying to measure/predict your own skill acquisition would be insightful into the kinds of problems you'll need to solve in order to apply this more broadly.

Answer 2

Fund a study which will take 100 or 1000 programmers who do not know the skill in question and measure how long it takes them to master said skill. That is the only way to quantize the curve. Or, if you hire a lot of people, take careful measure of how long it takes them to come up to speed and use that as your data set.

When you talk of measuring how long it takes to master a skill, you're not talking about the steepness/slope of the learning curve, rather, you're talking about the integral of the curve -- that is, what is the sum of all hours taken to reach mastery.

When we speak of a learning curve being steep, what we're saying is that it takes a lot of initial effort in order to get something usable. Human beings are way too variable to say with conviction that with 7 hours of study, you will be able to create reasonable object classes in language X that are usable in our project. That's why we use loose terms like steep, shallow, long, and short.

Just because it's descriptive doesn't mean that it isn't useful. We use another concept in programming (which I'm sure you're familiar with) called Big-O notation. Using it, we can categorize algorithms into broad categories which give an idea of how long they will take to run even though we don't know specifically (in term of hours or days, whatever) how long any particular algorithm takes. We know that an O(N²) is worse than a O(1) method, but far better than a O(2^N) one.

An interesting example might be Perl vs. Python. So many people deride Perl as being read-only while the very-similar analog of Python as being much more maintainable.

I would instead argue that this is a good example of how learning curves work. Perl is extraordinarily flexible in allowing code which DWIM (do what I mean) applies. Perl has a very long, but shallow, learning curve -- meaning that you can very quickly get useful and meaningful results from it very quickly, but it may take you a long time to produce elegant, readable, and maintainable code.

Python on the other hand has a slightly steeper learning curve because it is much less flexible in what it accepts in order to produce equivalent results. In Python, there is a "right" way to do things, so it takes more effort and study to fit their language model. As a consequence, useful and meaningful code is produced in a way that seems more elegant/readable/maintainable because the form is followed, but it will take longer to do so because more has to be learned before that stage is reached.

I suppose (now that we're at the actual bottom line) the effectiveness of a the descriptive rather than quantitative measure of the learning curve is useful because we work by analogy, comparison, and history/experience. We've all learned things with shallow and steep curves, and to hear that something has a curve which is very (steep|shallow|long|short), means that we can compare it to other skills which were similarly described and apply it to ourselves within our own context.

Answer 3

Wikipedia has a good overview of Learning Curves, and Experience Curve Effects.

The approach to measuring learning rates takes the approach of defining units of production, then measuring the time taken to create each unit. As a person (or team) learns, the time taken per unit produced will decrease.

From this, you can define "up to speed" as either the number of units you need to produce to be able to produce new units in under some set time limit, or the number of units you need to produce to get to the point where the delta time between your last two units is less than some percentage.

Note that this model does not account for the "fits and starts" that occur during the learning process. Basically, you learn quickly for a while, then reach a plateau (that is, you show little or no improvement for a while), then you start learning again.

Another factor that complicates measuring learning curves is the previous experience of the learners. A learner that has experience in a related task will learn more quickly. For instance, it is easier to learn C# if you have experience with Java than it is to learn C# if your previous experience was in C. A person who has mastered several programming languages will learn a new one more quickly than someone who has only mastered on language. One way to measure this is to define novelty factors that is a measures of the number of new skills you need to learn, and how different they are from your previous experiences.

Now to your question -- How do you provide an estimate to the amount of knowledge you need to acquire and the amount of time it will take to come "up to speed" on a new technology?

Here is one way to approach the estimate:

1. Gather data. For the times you have had to learn new technologies, how long did it take, how many new skills were involved, and how new were these skills to you at the time?

2. Estimate the # skills and their novelty to you. Then find the most similar previous learning effort.

3. Be conservative. It is easy to overestimate our abilities and underestimate the time it will take us to perform. See the well traveled road effect.

Answer 4

It is impossible to quantify such things. Its like a fingerprint...different for everybody. There's too many factors that go into that equation (passion, ambition, general intellect, concentration, etc.).

I think what's more important is knowing YOUR learning curve, and being able to judge the task at hand as such for yourself. Say you have quite a bit of experience with Javascript, and you want to pick up jQuery. Its a lot of new stuff, but you should know relatively easy how difficult it'll be for you personally.

Answer 5

It depends on:

Whether you have other tasks to do while you learn X If your are expected to be an expert or merely competent It is comparable to anything you currently know (e.g. it's C# and I know Java) You will have access to the best tool set for X There a local expert/user group/resource to whom you can ask questions

I would bring in a contractor to work with you/your team until you are comfortable with the new technology.