How are algorithms developed?
Generally, here's how algorithms are developed.
- Find a problem (or a problem with an existing solution)
- Find a solution to the problem
- Repeat steps 1 & 2 until you either can't find a problem or you can't fix it.
The resulting solution is the algorithm to resolve the problem.
"Really Useful" algorithms come about when you have an existing single-purpose algorithm and realize that it could be used in a different context with a few modifications. That realization is what re-triggers the iterative cycle.
This is where a lot of the generalization behind an algorithm comes from. And, as Robert Harvey stated, this extended iterative cycle is where software patterns come from.
It's equivalent to saying "I solved
foo with this, and if I tweak that solution a bit then I could solve
bar as well." And then that potentially extends into solving
baz and so on as well. Solve enough challenges with the approach and it becomes generalizable as an algorithm or a pattern.
How long does that take?
It all depends upon the nature of the problem, the available solutions, and the tooling used to tackle the problem.
To help clarify Step #2 in the general approach: This is where a lot of the hard work and inspiration goes into creating an algorithm. Your first attempts at solving the problem will likely either fail or won't be all that elegant.
But you start with "well, how would I break that problem apart and solve it?" Remember that an algorithm is a step-by-step approach to solving a problem. So you break the problem down into steps and see how well that approach solves the problem. Also keep in mind that Problem Solving is a broad field of research. Some methods are more successful than others, and sometimes it's the unorthodox approach that develops the elegant solution.