Most routers have a built-in firewall, but yes, you will want AV on every computer at least. Both Avast! and AVG offer free AV solutions that are highly regarded. I use Avast! myself.
It doesn't hurt to have a personal firewall on each computer as well. ZoneAlarm and Sygate are the top two software firewalls.
Don't forget to change the password on the router away from default...