So if the Tea Party/Libertarian/Ayn Randians secede the union and start their own completely capitalist paradise I wonder what they'd miss first?

This is not a diss on them. Whenever you leave the culture you are raised in you miss something even if you like your new culture better. For example, in France I miss people who can find a bathroom in a bar on their own.

I'm trying to figure out what would trouble them:

1. Public schools. Honestly, I think this would not be a big deal. A lot of them want to home school anyway. It would suck for families where both parents want to work, but it would be an agrarian community anyway, and farmers are notoriously good at taking care of their kids.

2. Well maintained roads. This one is a puzzle. I think they would be annoyed at the potholes at first, but they would run out of money for gasoline pretty quickly and as long as the land is flat horse and buggies will move over most things.

3. A functional military. This is a crap shoot. Nobody is going to invade the new Randbraska. But, they will never be able to invade Kansas.

4. Fire departments. I think this might be it. How would a fire department work in pure capitalism? Either you'd have to pay people in advance to help you should your house catch on fire and then trust them. Or you'd have to go running out of your house and hand people money as soon as it caught fire. Neither seem particularly practical to me. That said, if any Tea Party person wants to PayPal me now, I'll promise and pinky swear to throw water on your house should it burn 10 years from now.

5. Police. Everyone will have to hire mercenaries. The problem is that your neighbor may want your shit. And, say your neighbor is willing to pay the mercenaries to turn on you. This could cause problems.

6. Medical care. The upside - most of these people don't want vaccinations, so they'll save money there. The downside - plague. It will be interesting to see if a society based on trading livestock and crops can convince doctors to give them antibiotics and heart transplants.