From Baumslag’s “Topics in Combinatorial Group Theory,” chapter V, Exercises 2(3)(iv):
is of order e > 2 if, and only if, for some primitive e-th root of unity .
There’s really nothing to this statement — just put the matrix in Jordan Canonical Form and draw the obvious conclusion — but it really surprised me when I saw it used in an argument. Another way to put this would be that, for a 2×2 matrix, the trace and determinant determine the eigenvalues (which is equally obvious).
(The problem in the cases e = 1 and e = 2 is that then the eigenvalues are identical, which means that the matrix isn’t necessarily diagonal when it’s in Jordan Canonical Form — it could be a 2×2 Jordan block.)
From this and one other fact it follows that the elements a, b, and ab in the group actually have the desired orders.