Is there any way to see the traffic sources for a specific day? Lets say I have 5 visitors on a certain day, is there a way to see where they came from?

As the only way I seem to be able to do it is by sifting through each traffic source and seeing which visitors came on which days.

If that all makes sense?

Yes there is.
On the dashboard, top right corner, you should have a time frame (usually last month to date). Change that to the specific day you want to check your data for.
And there is it, you will see how many visitors you got, where did they come from, what keywords they used, which browser, color of their eyes etc etc....

I hope this helps. Let me know if it was unclear.

Ahhh simple! Thanks.