That sure is a tough problem! The only time my babies slept in a different place, I was there too, and they had no problem. Is your baby usually kinda clingy to you?