My approach to agile development is to build "vertical slices". I take a story from the UI to storage and implement it. I have some lightweight tools that I use to help this process (like a simple IRepository/UnitOfWork framework that has adapters for In Memory(usually for testing), Entity Framework, and NHibernate. At this point I don't think there's much of an argument of whether one should use an O/RM but rather which one to use. And for me it depends on the environment.
I've found that this approach wins over naysayers regarding agile development because they get to see working software more quickly than if I spend a lot of up front time creating a core. Combined with Domain Driven Design and some other techniques I use, I'm usually able to get a lot of working functionality in front of the users very rapidly.
TDD or even post-hoc unit testing is important because part of maintaining velocity as your application grows is having that safety net that a comprehensive unit test suite provides. "I need to make a change to this class, how do I make sure I didn't break anything?" With a good unit test suite, it's as simple as running that test suite. Without, it becomes a matter of manually running through your app to verify. Not fun at all.