One thing I make sure to tell my students (and this is something I told myself before accepting to go to Texas for a Ph.D.) is that it is also completely OK to change your mind. Deciding to pursue a Ph.D. is not a prison sentence, and if they realize that they do not enjoy math as much as they thought, or that they don’t really want to spend so much time learning more mathematics, then there is no reason to stay.
Of course, other things besides not enjoying learning math can happen, like not passing quals or prelims or whatever they’re called these days. It is my opinion that people who truly want to stay in graduate school find a way to do so. I have also had friends who have left for a few years, come back, and are very successful. Other friends have switched from one institution to another that was a better fit. This reminds me of a talk Kathryn Leonard gave at MathFest this past August, “I failed, and no one died”. In it, she explains how we need to teach our students (and learn for ourselves) to distinguish between failure and Failure. A Failure is when an airplane pilot fails to land his airplane or a surgeon botches a procedure. These are bad places to fail at what you’re doing. In mathematics, we are constantly faced with little-f-failure. We are working on a research problem that we can’t prove, or we don’t pass our qual, or we get a bad grade on our Real Analysis midterm. These are not huge problems, and no one is going to die. They are also indicative that you need to change something that you’re doing. Either try a new approach to your research problem, figure out how to study for your qual, find a study group for your next midterm. These failures could also indicate other issues, like maybe your problem is much more difficult than you thought (maybe you need to assume GRH!) , and in the other two situations, maybe you realize that you don’t care enough to struggle. This is important, because math is difficult, and if you don’t enjoy it then it is very hard to get through some of these obstacles. This goes for many difficult occupations, by the way.
Take a look!