As you state in your question, the RFCs are either de facto standards for internet operations or they provide additional guidance for those operations. Their appendix regarding RFCs does a better job at illuminating the various sources of RFCs and their overall meaning, including considerations of obsolescence.
Do I read them in their entirety and make notes or do I use them more as a reference for when I don't understand something or have a problem?
Treat them like you would treat any other standards reference.
- Some folk like to read a reference all the way through; it gives them a very solid understanding of what's required.
- Others will skim through a standard and refer back to it when they run into problems that the RFC should address.
- Others still will never peruse the contents of an RFC.
In general, I think they are much more useful as a reference than as explanatory text. But I fall into the second category of people that I described. On the other hand, if you're really wanting to understand the nuts and bolts of a particular technology then you may want to dive deeply into the standard. As with so many things, it depends upon what your aims are.
For what you're doing with the REST tutorial, option #2 is probably your best bet. Skim, glean something new, refer back as necessary.