Then yes, plug both systems into the router. If they are both Windows XP system, go to Control Panel, open up Network Connections, and click on "Create a new connection on the left side of the window.
From the new connection wizard click Next, select "Connect to the Internet", click Next, select "Setup my connection Manually", click Next, select "Connect using a broadband connection that is always on", click Next, click Finish.
Now both systems should be able to see each other. Share the folders/drives you want to be able to access from the other system, and you should be able to browse to it using My Network Places.
