Genesis 11:1 "And the whole earth was of one language, and of one speech."

“One language, and of one speech”: God, who made man as the one creature with whom He could speak (1:28), was to take the gift of language and use it to divide the race, for the apostate worship at Babel indicated that man had turned against God in pride (11:8-9).

“One language”: is literally “one lip,” meaning language or dialect. There is a single family in one place speaking one language.

We know that when Noah, Shem, Ham, and Japheth were on the ark, all the people of the earth, eight, truly did speak the same language. As we have said before, the Bible was not written in chronological order.

Verses 3-4: “Let us make brick … let us build us a city and a tower”: While dispersing, a portion of the post-Flood group, under the leading of the powerful Nimrod (10:8-10), they decided to stop and establish a city as a monument to their pride and for their reputation. The tower, even though it was a part of the plan, was not the singular act of rebellion.

Human pride was which led these people to defy God. 

Genesis 11:5 "And the LORD came down to see the city and the tower, which the children of men builded."

“And the Lord came down”: The “coming down” is significant (18:2, 21; 19:1). God was already aware of the program, but being the righteous judge, He wished to examine it closely. No matter how high their tower, He still comes down.

You notice in the Scripture above, that God called them children of men. They were followers of the flesh, and not the spirit. Whatever felt good, they did, following their fleshly lust.