One of my databases is on a server where my config file says the server is localhost. How do i connect to it from a different server.

Depends on the database. In most cases, it's the external IP of the Internet connection. In some cases, it's the server IP. In some cases, it's the path to the database (Access is like this, for example.)

You will also want to check your Access Hosts for the database (may have another name if you are not using MySQL) - if you are accessing the database from an outside IP, you will need to add the IP of the site your script is being hosted on (hosts not present in the Access Hosts list are blocked by default).

and the port on the remote server used for communication will need to be open for TCP and/or UDP traffic through the server firewall

you can use IP e.g. 192.168.0.1
or
http://somedomain.com

instead of localhost

Try going to the Cpanel or admin section of the database that you are trying to connect with. A lot of the time you will be able to determine the connection IP clicking on the database icons.

Try there first.