There really is no difference. Aluminum will expand and contract more quickly than stainless, but I seriously doubt you'd ever notice a difference in tone between the two. The internal construction of the muffler is where the unique sound comes from.
In my opinion, different style mufflers sound better on different engines than others. Ever hear a GM LS powered vehicle with a chambered muffler (Flowmaster)? I think they sound terrible. But put a packed muffler (like a Magnaflow) on it, and it sounds pretty good. The old 5.0 Mustangs sounded killer with a chambered muffler, but not quite as good with a packed muffler. It's really a pretty subjective argument. What I like, you may not, and vice-verse. A particular engine's firing order, exhaust pulse, and pipe routing will also have a lot to do with the sound. To use the 5.0 Mustang again, have you ever noticed that regardless of the muffler used (not including blown out, straight pipes, or junk exhaust), they ALL sound pretty good? To get even more into it, I've never heard a bad sounding Pontiac GTO, regardless of exhaust, even though it has the same LS engines as many other GM vehicles.
Heaven for me is a big cam, long tube headers, 3 inch dual exhaust, and dumped 40 series Flowmasters. That's the setup on my 55. It shakes your guts rolling by. Really deep snarl, and it sounds like a Pro Stock car at full tilt. I love that.
Sorry for rambling, I love talking performance stuff.