Subgroups of order up to 6 in $PSL(2,p)$

The permutation group $PSL(2,p)$ is represented as the factor group $SL(2,p) Z/ Z)$, where $Z = Z(GL(2,p))$ consists of all non-zero multiples of the identity matrix. It is much easier to handle a $2\times 2$-matrix than a permutation of length $p+1$, since we can apply linear algebra to matrix problems. We therefore represent subgroups of $PSL(2,p)$ by the matrices which generate them modulo $Z = Z(GL(2,p))$.

Theorem 7 Up to conjugation in $PSL(2,p)$ there are the following stabilizers and the corresponding numbers of orbits of 6-sets. Here $a,b,c \in GF(p)$ are chosen such that $a^2 + b^2 + 1 = ab$ and $c$ has order 5 if $5 \vert p-1$.

$$\begin{array}{\vert l\vert c\vert c\vert} \hline Group & Ge... ...}{} \end{minipage} & 2 \ if\ 5\vert p-1 \\ \hline \end{array}$$

Proof

A subgroup $C_6$ of $PSL(2,p)$ is generated by

$$\left( \begin{array}{cc} 2 & 1\\ -1 & 1 \end{array} \right)$$

modulo $Z(GL(2,p))$. It is easily veryfied that this matrix induces a permutation of order 6 on the projective line. The determinant is 3 which is a quadratic residue mod $p$ exactly if 12 divides $(p-1)$ or $(p+1)$, see p. 92 of [17].

There is just one orbit with this stabilizer on 6-sets. A representative is the set $\{0,1,2,(p+1)/2,p-1,p\}$.

The 5-orbit covered by these 6-sets has as a stabilizer in $PGL(2,p)$ the subgroup of order 2 generated by

$$\left( \begin{array}{cc} -1 & 0\\ 1 & 1 \end{array} \right)$$

modulo $Z(GL(2,p))$. This has the two fixed points $0,2$ and the 2-cycles $(p,1),(i,i/(i-1))$ for $i \ne 1$. In particular, $(p-1,(p+1)/2)$ is one such 2-cycle.

The matrices

$$\left( \begin{array}{cc} -1 & 0\\ 1 & 1 \end{array} \right), \left( \begin{array}{cc} 1 & 1\\ -1 & 0 \end{array} \right)$$

modulo $Z(GL(2,p))$ generate an $S_3$ in $PGL(2,p)$.

The matrices

$$\left( \begin{array}{cc} 1 & 1\\ -1 & 0 \end{array} \right), \left( \begin{array}{cc} -a & b-a\\ b & a \end{array} \right)$$

where $a^2 + b^2 + 1 = ab$, modulo $Z(GL(2,p))$ generate one $S_3$ in $PSL(2,p)$. There are $(p+1)/12$ orbits with this stabilizer on 6-sets.

By conjugating the matrix

$$\left( \begin{array}{cc} -a & b-a\\ b & a \end{array} \right)$$

with the matrix

$$\left( \begin{array}{cc} -1 & 0\\ 1 & 1 \end{array} \right)$$

which is not in $PSL(2,p)$ we obtain another such subgroup:

The matrices

$$\left( \begin{array}{cc} 1 & 1\\ -1 & 0 \end{array} \right), \left( \begin{array}{cc} -b & a-b\\ a & b \end{array} \right)$$

where $a^2 + b^2 + 1 = ab$, modulo $Z(GL(2,p))$ generate another $S_3$ in $PSL(2,p)$. There are $(p+1)/12$ orbits with this stabilizer on 6-sets.

The last two subgroups isomorphic to $S_3$ are not conjugate in $PSL(2,p)$. So, they are stabilizers of 6-sets from different orbits. The normalizer in $PSL(2,p)$ of each of the two subgroups is generated by $C_6$ and the chosen subgroup. They have index 2 in their normalizer such that the 6-sets invariant under one such $S_3$ fall into pairs that are mapped onto each other under the action of the normalizer. By the homomorphism prinziple, these pairs represent the orbits of $PSL(2,p)$ on 6-sets with stabilizer conjugate to the chosen $S_3$.

All the listed elements modulo the center normalize the subgroup $C_6$ of $PSL(2,p)$ generated by

$$\left( \begin{array}{cc} 2 & 1\\ -1 & 1 \end{array} \right).$$

It is easily veryfied that this matrix induces a permutation of order 6 on the projective line. The determinant is 3 which is a quadratic residue mod $p$ exactly if 12 divides $(p-1)$ or $(p+1)$, see p. 92 of [17].

The subgroup $C_3$ of order 3 of $C_6$ is generated by

$$\left( \begin{array}{cc} 1 & 1\\ -1 & 0 \end{array} \right)$$

modulo $Z(GL(2,p))$. This group has $c = (p+1)/3$ orbits on 3-sets. So, these yield ${c\choose 2}$ combinations for a 6-set invariant under $C_3$.

We have to subtract those 6-sets first that are invariant under a subgroup of order 6 containing our $C_3$. We have to subtract those $(p+1)/6$ which are invariant under $C_6$. Each of the two $S_3$ has $(p+1)/12$ subgroups conjugate to it in the normalizer of our $C_3$, which is a dihedral group of order $p+1$. So, we for these groups have to subtract 2 times $(p+1)/12$ times $(p+1)/6$ 6-sets. We are left with

$${{(p+1)/3}\choose 2} - (p+1)/6 - 2(p+1)^2/(12\cdot 6)$$

$$= (p+1)/3 \cdot (p-11)/12$$

6-sets which are invariant under $C_3$ but not under any proper overgroup. The orbits of $N_{PSL(2,p)}(C_3)$ on this set are the intersections of the $PSL(2,p)$-orbits with this set. This normalizer has order $(p+1)$ and its orbits have length $(p+1)/3$, since $C_3$ is the stabilizer of each of theses 6-sets. So, there remain $(p-11)/12$ orbits of 6-sets with $C_3$ as a stabilizer.

The subgroup of order 2 of $C_6$ is generated by

$$\left( \begin{array}{cc} 1 & 2\\ -2 & -1 \end{array} \right)$$

modulo $Z(GL(2,p))$.

This element also represents the only conjugacy class of elements of order 2 in $PSL(2,p)$. Let us call the subgroup it generates $U$. Then $U$ has $(p+1)/2$ orbits of length 2. Any 6-set invariant under $U$ is formed by selecting 3 of these orbits. We apply Lemma 4 to determine the number of orbits of $PSL(2,p)$ on 6-sets where $U$ is a stabilizer of some representative. We first have to subtract the 6-sets that are invariant under any larger group. So, we have to subtract the $\frac{p+1}{6}$ 6-sets invariant under $C_6$. Since all conjugates of $C_6$ have trivial intersection, our $C_2$ lies in only one $C_6$. There are 6-sets with a stabilizer of type $S_3$ and $C_2$ is contained in some of these, because any subgroup of order 2 of an $S_3$ is conjugate to $U$. As a group of order 6 each such $S_3$ has $\frac{p+1}{6}$ invariant 6-sets, namely its orbits. We now have to find out the number $\lambda(U,S_3^{(i)PSL(2,p)})$ of conjugates of $S_3^{(i)}$ for $i = 1,2$ that contain $U$. This can be done by doubly counting the set

$$\{(C,S)\vert C \in U^{PSL(2,p)} \ and\ S \in S_3^{(i)PSL(2,p)} \ and\ C < S \}.$$

We obtain

$$\vert U^{PSL(2,p)}\vert\lambda(U,S_3^{(i)PSL(2,p)}) = \vert S^{PSL(2,p)}\vert\cdot 3,$$

and therefore

$$\lambda(U,S_3^{(i)PSL(2,p)}) = 3\cdot \frac{\vert N_{PSL(2,p)}(U)\vert}{\vert N_{PSL(2,p)}(S_3)\vert}.$$

The normalizer of $U$ is a dihedral group $D$ of order $(p+1)$ and the normalizer of $S_3^{PSL(2,p)}$ has order 12. So, $\lambda(U,S_3^{(i)PSL(2,p)}) = \frac{p+1}{4}$ for each $i$. By this we get the number of 6-sets with stabilizer $U$. The number of orbits of $D$ on them is simply obtained by dividing by the length of each orbit which is $\frac{\vert D\vert}{\vert U\vert} = \frac{p+1}{2}$.

So, we get

$$\frac{2}{p+1}\{ {\frac{p+1}{2}\choose {3}} - \frac{p+1}{6} - \frac{p+1}{2}\frac{p+1}{6}\} = \frac{p+1}{6}\frac{p-9}{4}$$

orbits. $\Diamond$

We illustrate some part of the subgroup lattice by the following picture.

While the counting of orbits is completely solved in this way the construction of orbit representatives still needs some work. We have to find out whether a 6-set $K$ invariant under a subgroup $U$ has $U$ as its full set stabilizer. In the case of Steiner systems this can be done by the computation of

$$\lambda(T,K^{PSL(2,p)}) = \frac{\vert N_{PSL(2,p)}(T)\vert}{\vert U\vert} \mu(T^{PSL(2,p)},K)$$

and discarding $K$ if this value is larger than 1.

There are two possible reasons for such a value. At first, $T$ may be contained in more than 1 block of the orbit $K^{PSL(2,p)}$. Then this orbit cannot be in a Steiner system. The second reason is that $U$ may not be the full stabilizer of $K$. So, in a consideration of all representatives of subgroups as possible stabilizers of some block a conjugate of $N_{PSL(2,p)}(K)$ will occur stabilizing some block in $K^{PSL(2,p)}$.