Factions, in Madison's view, are to be avoided--and to be stymied from taking the unchallenged control they desire.
In fact, Madison argues that protection against domination of the whole by a faction "is the first object of Government."
He went on to write that, in a well-constituted republic:
The filibuster has been one part of what has been, to some extent at least, a republic whose checks and balances keep it from falling away from the vision of its founders. That is, we've enough protection against the tyranny of factionalism to keep us from descending into fascism or totalitarianism. Or, we have had.
Let the filibuster go, and we are taking a step down a very dangerous path--one our founding fathers warned us against.