Bear with me since I'm a noob, but what's the correct way to setup a domain to make it display the site you have at your web host? Am I supposed to:
1. Change the nameservers to the ones given by my web host? or..
2. Change the www A Record to the one given by my web host?

So far I've only changed the A Record and the domain does go through, but it seems like changing the nameserver would be the logical thing to do.

1. Change the nameservers to the ones given by my web host? or..
2. Change the www A Record to the one given by my web host?

Yes and Yes.

Can you explain the purpose of doing both when just changing the A Record works?

And also why is it that some sites like the blogging service Tumblr emphasize that you only need to change your A Record? I realize these services are different from full web hosting, but I'm just wondering.

Please note that you need to change your domain's or sub-domain's A Record, not its nameservers. And, you don't need to forward or redirect anything, you only need to change the A Record.

Most hosting will setup the subdomains that your hosting requires (or provides) eg:

www.hostname
mail.hostname
cpanel.hostname
webmail.hostname
etc

Simply pointing the 'A' record will not set these up and you will need to do them manually, repointing the nameservers will deal with it.

Blogging services only use the hostname so the 'A' record is sufficient.

Originally Posted by AndrewNoNumbers

Can you explain the purpose of doing both when just changing the A Record works?

That will be more then enough. Just do not try to use both. Because if you change A record you must leave your nameservers as default ( the ones which your domain name registrar gives you)

Nameservers hole the A records that people query for when they look at your website.

Your A records point to the IP address that your website is on.