I am trying to prove that a group of order $p^2q^2$ where $p$ and $q$ are primes is solvable, without using Burnside’s theorem. Here’s what I have for the moment:
If $p \neq q$, we shall look at the Sylow $p$-subgroups of $G$. We know from Sylow’s theorems that $n_p \equiv 1 \pmod p$ and $n_p \mid q^2$, therefore $n_p \in \{1, q, q^2\}$.
If $n_p = 1$, it is over, because the Sylow $p$-Subgroup $P$ is normal in $G$ of order $p^2$, and $G/P$ has order $q^2$. Thus both are solvable and $G$ is solvable.
Thanks in advance for any help!
Laurent
You argument works just as well with $p$ and $q$ switched, so the only time you have trouble is if both $n_p=q$ and $n_q = p$. Since $1\equiv n_p \mod p$ and $1\equiv n_q \mod q$ this puts very strong requirements on $p$ and $q$.
Hint 1:
Unless $n_p=1$, $n_p > p$.
Hint 2:
If $n_p=q$, then $q>p$. If $n_q =p$, then $p>q$. Oops.
The OP’s argument is currently flawed in the case $n_p=q^2$, so this answer is only truly helpful after that flaw is fixed.
A very similar argument to the one given in this answer works. First part of your argument works, and the $p-q$ symmetry helps:
If $n_p=1$ or $n_q=1$, then the group is solvable.
Now we use the Sylow counting again to get some severe restrictions:
If $n_p \neq 1$, then $n_p \in \{q,q^2\}$ and in both cases we have $1 \equiv q^2 \mod p$. Similarly, if $n_q \neq 1$, then $1 \equiv p^2 \mod q$.
Unfortunately now we don’t get an easy contradiction, but at least we only get one possibility:
Since $p$ divides $q^2-1 = (q-1)(q+1)$, we must also have $p$ divides $q-1$ or $q+1$, so $p \leq q+1$ and $q \leq p+1$, so $p-1 \leq q \leq p+1$. If $p=2$ is even, then $q$ is trapped between 1 and 3, so $q=3$. If $p$ is odd, then $p-1$ and $p+1$ are both even, so the only possibility for $q \neq p$ is $q=p-1=2$ (so $p=3$) or $q=p+1=2$ (so $p=1$, nope). Hence the only possibility is $p=2$ and $q=3$ (or vice versa).
In this case, we get:
If $p=2$ and $q=3$, then $n_q \in \{2,4\}$. Considering the permutation action of $G$ on its Sylow $q$-subgroups, we know that $n_q=2$ is impossible (Sylow normalizers are never normal) and $n_q=4$ means $G$ has a normal subgroup $K$ so that $G/K$ is isomorphic to a transitive subgroup of $S_4$ containing a non-normal Sylow 3-subgroup and having order a divisor of 36. The only such subgroup is $A_4$, so $K$ has order 3. Hence $G/K\cong A_4$ and $K \cong A_3$ are solvable, so $G$ is solvable.
I will give a different series of hints/steps to show the following: if $p >q,$ then $G$ has a non-identity normal $p$-subgroup. Let $P \in {\rm Syl}_{p}(G),$ and suppose that no non-identity subgroup of $P$ is normal in $G$ (that includes $P$ itself, of course). Notice that $q \not \equiv 1$ (mod $p$), since $1 \neq q < p.$ Hence $G$ must have $q^{2}$ Sylow $p$-subgroups, and we must have $q^{2} \equiv 1$ (mod $p$). Hence $p | q+1$ ( we can’t have $p|q-1$ as $q <p$).
But $q <p,$ so $q+1 \leq p,$ so we must have $q = p-1$. Now $p \neq 2$ as $p>q,$ so $q$ is even. Hence $q = 2$ and $p=3,$ as $p$ is a prime. Hence $|G| = 36$. Now $P = N_{G}(P)$ by Sylow’s Theorem. Furthermore, there is non proper subgroup $M$ of $G$ which strictly contains $P.$ For otherwise we would have $[M:P] \equiv 1$ (mod $3$) and $[G:M] \equiv 1$ (mod 3), forcing $[G:P] \geq 16,$ which is not the case. Now let $g^{-1}Pg$ be another Sylow $3$-subgroup of $G.$ Then $P \cap g^{-1}Pg \neq 1$ as $|P||g^{-1}Pg| > |G|$.However $P$ and $g^{-1}Pg$ are both Abelian,
so $P \cap g^{-1}Pg \lhd \langle P,g^{-1}Pg \rangle >P.$ But there is no subgroup of $G$ strictly between $P$ and $G,$ so $P \cap g^{-1}Pg \lhd G.$ (actually your (Laurent’s) argument works in $G/(P \cap g^{-1}Pg)$ to show that a Sylow $2$-subgroup is normal in that quotient group.
Assuming that you know that groups of order $p^2q$, $pq$ and $p^k$ are solvable, it is enough to prove that a group of order $p^2q^2$ is not simple.
Suppose that $G$ is a simple group of order $p^2q^2$. By symmetry (and since $p$-groups are solvable) we may assume $p > q$. Steps to reach a contradiction:
Prove the following: if $G$ is a finite group with a subgroup $H$ of index $r$, where $r$ is the smallest prime divisor of $|G|$, then $H$ is a normal subgroup.
By 1. $n_p = q$ cannot happen since $G$ is simple. Therefore $n_p = q^2$.
If there exist distinct Sylow $p$-subgroups $P_1$ and $P_2$ such that their intersection $D = P_1 \cap P_2$ is nontrivial, then $D$ has order $p$. Now $D$ is normal in both $P_1$ and $P_2$, but not normal in all of $G$, so $N_G(D)$ has order $p^2q$. This is a contradiction by 1.
Therefore distinct Sylow $p$-subgroups of $G$ have pairwise trivial intersection. By 2. this means that there are $q^2(p^2-1)$ elements of order $p$ or $p^2$. But then $G$ has a normal Sylow $q$-subgroup.