For a shorcut to the actual target file:
Right click > Properties > Shortcut > Change Icon
I'm not sure if you can change original file icons, wait for someone elses response. There must be a way, but it might be complicated. I'm assuming you're talking about shortcuts on your desktop, etc... so that should work fine.