Each colonist unit is actually a "clan", a unit of 100 people plus all their luggage. Coincidentally, they take up precisely as much space in a starship's cargo hold as a torpedo, a fighter craft, or one unit of minerals or supplies.
In order for a player to control a planet, they must have at least one clan on the planet. The number of colonists determines such things as the maximum number of structures the planet can build and support, the amount of taxes that can be collected (whether from the colonists themselves or from friendly natives), and that sort of thing.
If a planet has more colonists than its climate can support, it is considered overpopulated, and a percentage of the colonists may die each turn. Likewise, if Amorphous natives are present and the colony isn't growing enough to compensate for the regular population loss, the net population will decrease each turn. When there are not enough colonists on the planet to maintain structures on the planets surface the structures will decay at a rate of 3 structures per turn. This applies to mines, factories, and defense posts. All these facilities are automated once built and will continue to function as long as there's at least one colonist remaining on the planet.
WARNING:
If a planet is reduced to 0 colonists at any time, any starbase in orbit will be lost.