I think Structure and Interpretation of Computer Programs is a great book to read. However, there are are a couple of problems with it.
The biggest problem in my experience is that, while the book requires absolutely no programming knowledge, and is geared to complete newbies, it is geared to complete newbies who study at MIT. And so, while it does not assume any programming knowledge, it does assume quite a bit of domain knowledge, e.g. in the fields of electrical engineering, physics and math. Note: these have nothing to do with the concepts being taught, they are just needed to understand the exercises and examples.
Greater people than me have explained it much better, in a cleverly titled paper The Structure and Interpretation of the Computer Science Curriculum. Thankfully, they aren't just whiners, they wrote their own book instead: How to Design Programs. If you think about reading SICP, you might want to read HtDP first: it covers much the same ground as SICP, but it only assumes average high-school level domain knowledge. (In fact, it is written to be taught to high school kids.)
After you have finished HtDP, you can still go back and read SICP, since now some missing bit of domain knowledge won't be as harmful anymore because you have seen the concept before.
Note: there is a draft for a second edition of HtDP, which you might want to read instead of the first edition. The material about imperative programming has been removed, and is going to be covered in the as-of-yet unwritten second volume How to Design Components, but you can take those either from the first edition or from SICP or both.