Soreness is going to happen especially in the beginning as you get going. Eventually your body will begin to recover faster between workouts. You will probably never be free of soreness entirely but it's duration and intensity may subside for you over time.
Bear in mind that soreness is not necessarily an indication of a good workout. In the begining it is most important for you to develop proper exercise form and to avoid injury while your body gets used to the new demands you are placing on it. You should work out with lighter weights and concentrate on learning the exercises properly. After you have proper form you can increase weight and work on gaining strength.
If you feel real pain while doing something you should stop. Don't tear a muscle or ligament just to finish a set.
Scheetive posted a great beginner workout in this section and I highly recommend you read it. Should be a sticky post. He has links to photos and instrcutions for the exercises. If you really need help you can join a gym for a month and work with a trainer to learn your form before returning to workout at home.
Remember when working out at home to be mindful that you may not have a spotter there so you should think before you lift and do not perform and exercise that is too big for your development or one that unreasonably may cause you an injury.
The longer you lift the more you will come to understand your limits. That is why it is important to start properly, get good form and progress at a reasonable pace. I can tell you are motivated which is great but remember that Rome wasn't built in a day. No use causing yourself an injury that prevents you from following your program.