As you can tell, I've been bitten by the running bug. Mattman posted a cautionary story that I think deserves further comment.
First, I'd recommend "Galloway's Book on Running" by Jeff Galloway to anyone who is starting or even someone is already a runner. The book is very easy to read and is aimed at normal people, not track stars or elite athletes. This book covers all of the basics such and describes how to get started. Most importantly, it tells you how to run without injuring yourself.
One thing I found very interesting was where he describes the stages of running. I forget the different stages, but one was where you get very gung ho and goal oriented. This happens after you start getting in shape and start feeling good and it is a very dangerous phase because it can have two very bad consequences. The first is that it take the fun out of your running. When you are running and expecting improvement, it is easy to become disappointed if you don't set a personal record in a race or if you run slower than your goal pace. Unless you are shooting for the Olympics, what's the point of that? The second is that this goal orientation will lead you to push harder, to run faster, and to increase the amount of training you are doing and often this leads you to do too much, too fast and injure yourself. Nothing will end your enjoyment of running more than to have to stop and heal. Pain and injury suck. The last stage is one where you run just for the enjoyment. I think that I am there. Once I realized I was in that goal driven stage I consciously stepped past it because it wasn't fun. Now I try to do it just because I like it. I still have goals for motivation, but they aren't the main focus of my running.
Most people push too hard when they are starting out. If you are uncomfortable while you are running, then you are pushing too hard. If you are out of breath and tired the rest of the day, you did too much. Running should invigorate you and make you feel good. One way to keep from overdoing it is to get a heart rate monitor (HRM). When I was starting out I read about them and thought it was a good idea. When I strapped it on I had one of those "Ahah!" moments. My heart was racing at 170. This was very close to my max heart rate. I set a goal heart rate of 145 and anytime my heart was going faster I walked until it slowed down to 130. This felt really easy and comfortable and I walked more than I ran, but the heart rate monitor assured me that that was enough to improve my fitness. Over time I was able to run continuously without exceeding my limit and over more time I was able to increase the limit. But this helped me understand what a proper training intensity felt like. And it isn't as hard as you would think. If you are starting out or haven't run for a while I'd suggest getting an inexpensive HRM from Polar or Timex or Nike.
One important rule to follow to prevent injury is to not increase your mileage and your intensity at the same time and to not increase your weekly mileage by more than 10% a week. If you disobey this rule, like Mattman did above, then you WILL get injured. Your body needs a chance to adjust and adapt and it simply can't handle more than a 10% increase per week. Your bones, muscles, tendons, and ligaments all need a chance to strengthen and adapt to the increased stresses and if you don't give them time then they'll start breaking down faster than they can rebuild and you'll break.
So, if you want to start exercising then don't jump into it. Ease in. Be consistent but moderate. Gradually ramp up. Listen to your body. Don't exercise until you are exhausted. If you aren't in good shape, then it may take several years before your body is ready for more intensive training. YEARS. Not days, weeks, or months. The improvements will be gradual, so I find it helpful to keep a running log. I use a free one on nikerunning.com. That way, when you feel like you aren't making any progress you can look back and see what you were doing a year ago and see that you have made improvements. It also allows you to track your mileage so you don't increase it too much.
Anyway. If you feel inspired to start running, have fun but be safe.