It's kind of like his way to flirt with you. It's just an expression as in saying he's going to be gone and don't kill yourself with not being with him. It's not a bad thing, it's good. It's like he wants to be with you, but he has to go, so kind of like saying don't miss me while I'm gone. Pretty much flirting with you so yeah.