Check out this guy's blog for a post about how his students make his life much more difficult. In this case, 2 of them were late for their midterm exam and threw off his whole plan.
As a teacher, of course you have to expect and hope for the best out of your students. I start the semester off assuming that they want to learn English and that they'll come to class each day ready to participate and learn. Most of my students do live up to this expectation for the most part but there are always a few who don't. It's human nature. There are just those people who don't care for whatever reason, or have some bad stuff happening outside the class, or don't have any friends in the class, or just don't like me, or don't like the style of class that I run.
But, I'm also a realist. Which is why I don't run my midterm exam on such a tight schedule that 2 late students will throw off my whole day. I get all the students to come at the class start time, and then bring them into my office in groups of 4 for their test. I use the attendance sheet and go from top to bottom. If someone is not there at the beginning, I'll put them in the last group. There's always there by that point and their lateness is really no problem. And the last students actually don't mind waiting because they have an hour or so of extra cramming. And the first students don't get angry at having to go first because I've done it purely on the order of my attendance paper and haven't played favorites.
Anyway, it really is possible to design your class (and testing) to take the unknown, variable kind of stuff into account and not have so much stress over something that's not a big deal.