In the sutras there is a story of wealthy parents who became separated from their child. The boy grew up as a homeless, frightened beggar. When he was a young man he was reunited with his family, which took him in, loved him, fed, clothed, and cared for him. Yet, he never felt secure or happy because he did not trust his turn of fortune. He thought it must be a horrible trick to be treated like a king for a while and then, when he became comfortable, to be turned out again, or sacrificed in some barbaric ritual. Hence he suffered, even while living lavishly.
If you are fearful and doubtful, you will suffer no matter what. It is not due to the environment; it is due to your state of mind. For some, New York City is paradise, for others it is hell. Your karma may determine whether you find yourself in fortunate or unfortunate conditions, but your mind chooses to suffer.
Suffering Caused by the Body