Fragmentation comes from memory that is unusable. Dynamic allocation is similar to playing Tetris - if you play fast you end up with holes (fragmentation), and you can't predict what type block is going to fall down next. With dynamic allocation, you can't predict when and what memory is going to be freed - imagine playing Tetris where blocks disappear randomly! Also keep in mind that dynamic allocation may require allocating variable-sized blocks - Tetris would involve polynominos instead of tetrominos!
A bit on dynamic allocation techniques:
The way you described is fixed-size block allocation. It is usually implemented as a free list. The problem with this method is that it can't allocate different size blocks and it can lead to bad caching behavior.
One of the simplest ways to implement dynamic allocation is buddy memory allocation - implemented as a binary tree that satisfies the heap property. The problem with this method is that it has internal fragmentation. Internal fragmentation comes from using techniques that use predetermined blocks, usually powers of 2. This means that allocating 150 bytes would actually allocate a block of 256, wasting 106 bytes due to internal fragmentation.
Other methods try to minimize fragmentation, such as slab allocation. Slab allocation is primarily used for kernels, as it was designed to allocate small objects - it's not ideal for general purpose malloc.
Anyways, the point I'm getting to is that it all depends on the type of allocations you're doing. Operating systems don't know which allocation method would be best for a program, and that is why they choose one that balances speed with fragmentation. It's nearly impossible to have 0 fragmentation without constantly pushing data around, that's just the nature of dynamic allocation.