Essentially most game data files are compressed with a proprietary format based on huffman encoding or some other tree based compression scheme. I did this a while back for Ragnarok Online 2 - the game had custom files "GRF" files which needed to be unpacked. I remember examining the hex with a hex editor and figuring out some info from the file header. After that, it's pretty much a reverse engineering problem to solve the compressing scheme.
Post more info maybe? Encryption and compression are completely different BTW.
