The verses we are considering should be understood in the context of the section in which they are found. Genesis 9:18 begins a new division which continues to chapter 11, verse 10. Moses wrote of the repopulation of the earth through the sons of Noah. 

Genesis 9:20-27explains the three-fold division of the race for its spiritual dimensions. While the Canaanites are under God’s curse, Shem will be the line through whom the Messiah will come and Japheth will find blessing in union with the line (and the seed, ultimately the Messiah) of Shem.

Many believe, as do I, that Shem is actually Melchizedeck as described in Genesis 14. 

When Noah became drunk, Ham made fun of it. Shem and Japhth refused to participate, and actually covered up Noah. But, obviously, Ham blabbed it all over town.

When Noah woke up, he was embarrassed and ended up CURSING Ham and Ham's decedents, especially his son Canaan.

Whether it was "fair" or not is besides the point. He did it. And we are seeing the effects of it to this day...