To somewhat formalize what you have already said about looking for loops that can be parallelized, the key concept being exploited is flat data parallelism. A majority of the parallel libraries (that I know of) seem to exhibit this (e.g. Microsoft Task Parallel library for .NET, Microsoft Parallel Patterns Library for C++)
Flat data parallelism happens when you have a chunk of (flat) data (e.g. an array) that you want to perform work on, you then divide up the data into chunks of how many processors you have, does independent work in parallel for each of those chunk, and finally potentially aggregate/combine the result of each chunk to get your final result.
A sample article on this: