I did a bit of searching around, and a majority does say that the Aspire m5100 supports 8GB of memory. I think the Acer site failed to note that 4GB would be the max for a 32bit OS (although really 3.25GB), and 8GB is the max it supports for 64bit.
When you say "it won't boot", it just hangs after the POST screen? Have you tried testing the memory sticks to make sure they're not faulty?
Also, do you really need 8GB of RAM? It would just be a waste if you're using the computer for general purposes, even gaming.