Yes, you can say it that way. Your physical body is also called the body of retribution. The environment is also called the environment of retribution. They exist because of the effects of sentient beings' past karma. A fundamental principle of Buddhadharma, called "interdependent origination, " or "conditioned arising, " explains that all phenomena are interconnected and arise and perish because of karma. The environment and our bodies are manifestations of both individual and collective karma.

Everything that we are, experience and encounter is the consequence of our previous actions. The previous actions generate the karmic force which is stored in our consciousness (fifth skandha). Eventually the karmic seeds within our consciousness manifest, becoming what we encounter and experience. And as we experience and act upon the effects from previous karma, we create more karma, which in turn gets stored in this consciousness.

Now let's go back and compare the framework of the five skandhas to that of the Yogacara school. Sensation, perception and volition (second, third and fourth skandha) roughly correspond to the first six consciousnesses (sight, sound, smell, taste, touch and thought) of the Yogacara school. The consciousness of the five skandhas refers to the seventh and eighth consciousnesses of the Yogacara school.

The fifth skandha is also known as the karmic retribution consciousness. When the five skandhas were originally taught, there were no fine distinctions made between a sixth, seventh and eighth consciousness. Eventually, Yogacara expanded upon the five skandhas and made finer distinctions between the functions of this karmic retribution consciousness, and so they speak of a sixth, seventh and eighth consciousness.