The guitar should be held mostly with your leg and by cradling it in your body. Your left hand is used to stabilize the neck and fret the strings. Hold the neck in the V created by your thumb and forefinger. You should be able to smoothly move your left hand up and down the neck without having to hold it up.
Even if you hold the guitar correctly, you may experience some discomfort while getting used to playing. Do not become discouraged if your shoulder hurts in addition to your neck, arms and hands. You will eventually get used to it.