I wouldn't say that. The electoral process is critical to democracy; having it function properly is certainly not a "privilege."It seems to me that appearing on a ballot is being treated like a privilege like driving, which can be denied or revoked at the pleasure of the appropriate governing authority according to its own laws and procedures.
That being said, the American electoral process is more than a little wacky, and IMO the reason the 14th Amendment provides very little detail on how the ineligibility should be carried out is that most of the details of elections are left to the states.
When the 14th Amendment was written, most states weren't using a popular vote to determine which presidential candidate to support; that decision was made by state legislatures. Same thing for the selection of senators.
It seems to me that the 14th Amendment was written to leave most of the details of implementation (at least for elected positions) to the states, recognizing that under the decentralized American system, this was the proper place for these details to be decided.