What do you want to know? I'm not much of a collector, I've only got about 700 albums after selling off over 300 over the last couple years. Other than the ones I bought new back in the day, I rarely pay over $1 for an album. I hit garage sales and get the word out. I just bought 3 Ventures albums in Near Mint for $.25 each last weekend!
If you are going to "collect" them with an eye to resale, then only buy Near Mint records. If you just want them to listen them, then you have more flexibility because then you don't have to worry so much about what the cover looks like. The hot albums now are Northern Soul and late '60s Psych and Garage so good luck finding those for $1. Eventually though, things usually swing around and something else like New Wave or blues will become hot and Garage will be cheap.
ALWAYS check that the correct album is in the cover! I've been burned a few times buy looking at a great cover and sliding the album out just enough to get a decent look at the how worn it is, then getting home and finding someone put the wrong album in the sleeve.
Buy a price guide if you are planning on spending more that I do. Keep your eye on eBay and Gemm.com.
Buy a decent turntable and keep a good stylus/needle on it.
Most of all, HAVE FUN!