Since H and 
are equivalent, they have the same discriminant
and represent the same integers. To say that H is reduced implies that
P is the minimum of H on 
and that R is the second
minimum, hence 
and 
. Equality of discriminants implies 
,
hence 
since they are both nonnegative.
Let as usual 
and 
be the generators of the modular group 
, let 
be the
usual compact fundamental domain for the modular group in the upper half plane
.
Let 
be a definite binary quadratic form, 
and 
the roots of 
. Since H is reduced in the usual
sense, we have 
.
Now let 
be an automorphism of H.
If 
, then it fixes 
, hence by the usual theory,
is either 
, 
(if 
), 
or 
(if 
).
If 
, then it swaps 
and 
, so that
Taking imaginary parts, we get a=-d. Taking real parts, then using
and 
, we obtain
bP=aQ+cR. Finally, the determinant condition gives 
.
One easily checks that the three conditions a=-d, 
and
bP=aQ+cR are necessary and sufficient conditions for M to be an
automorphism of H.
Now we clearly have 
and 
, and since
H is reduced we have 
. Thus we obtain the following:
If 
, then 
and P=Q=R, so b=0 hence c=-a,
d=-a and 
.
If a=0, then bc=1 and 
so R=P and 
, hence
b=c and d=0.
If c=0, then 
and bP=aQ. This implies that either b=0
and Q=0, or b=a and P=Q.
This finishes the proof of the lemma. Note that it follows from this result
that the group G of automorphisms of H is always isomorphic to a group
of the form 
, with m=1, m=2 (when Q=0 or P=R
or P=Q are the only equalities), m=4 (when P=R and Q=0), or m=6
(when P=Q=R).
