You can do this two ways with home router.
Method 1) Competely disable or turn off the DHCP service on the router. Then assign all your computer it's IP address, subnet, Default gateway, and DNS. Here is an example, if the ip address you use to connect to your router is 192.168.1.1 that means you are in the 192.168.1.0 network with a subnet mask of 255.255.255.0 so your computer would have to have an IP address of 192.168.1.2 to 192.168.1.254 with subnet mask of 255.255.255.0 and default gateway of 192.168.1.1 and DNS address that is givent to you by the ISP. You can find the DNS address in the router WAN log or do ipconfig /all on a computer that was setup to get an IP automaticly.
Method 2) Some Linksys router come configured with DHCP to give out only a certain amount of address, lets say 192.168.1.100 to .150, this is the DHCP pool, it supports 51 clients. You can use static IP address if only this situation is true by assigning the computer an address out of this range and of course giving it the same subnet mask, default G, and DNS.
Method 3) Run DHCP services on a server with full features and control. Not feasible for home user and much more advanced. Used in small to large companies with a dedicated server running.
There are advantages and disadvantages to both method. Method 1 insures that there will be no conflict if you manually enter address and you get to control which client gets what address but is alot of work if you have alot of computers. Method 2, if all is true about the pool, you can still manually assign some client staticly and use DHCP for other. Method 2 is much easier and less work, but might conflict if you enter the wrong address or something changes, you don't have full control of the IP address.