A few people remember a commutator formula of the form $[a,b]^n = (a^{-1} b^{-1})^n (ab)^n c$ where $c$ is a product of only a few commutators (say $n-1$) of them. Here $a,b$ are in a (free) group and $[a,b] := a^{-1} b^{-1} a b$.
Does anyone remember such a formula with proof?
Some such formula must exist where $c$ is in the commutator subgroup of $\langle a,b\rangle$, but my recollection is that $c$ is a product of something more like $n^2$ commutators.
Answers that only work for $n=2$ are less interesting to me. There should be a radical difference for $n \geq 3$.
Positive results are fairly nice. I explain them in this note for commutators of powers (see the last pages for pretty pictures and crazy formulas). I didn’t get around to the special case of powers of commutators, but Culler and Bavard give definitive results.
Negative results: These are just well known true formulas that have “$c$” being way too long, even allowing ourselves to omit longer commutators. If you’ve never done it, try to write $(ab)^3 = a^3 b^3 c$ and actually get a formula for $c$ only involving commutators (of commutators) of $a$ and $b$.
Leedham-Green–McKay, Corollary 1.1.7
$$[x,y]^n = (x^{-1} y^{-1})^n (xy)^n [y,x]^{\binom{n}{2}} [[y,x],x]^{\binom{n}{3}} \cdots$$
$$[y,x]^n = [y^n,x] [[x,y],y]^{\binom{n}{2}} [[[x,y],y],y]^{\binom{n}{3}} \cdots$$
modulo the subgroup generated by commutators containing at least 2 $x$s. Again quadratic (cubic, quartic, etc. if don’t mod out by $\gamma_3$) not linear.
Culler’s formula is $$[a,b]^3 = [b^a,a^{-1} b^a b^{-1}][bab^{-1},b^2]$$ express a product of three commutators as a product of two commutators, which is disturbing. In fact Culler showed that the commutator length of $[a,b]^n$ is less than or equal to $\tfrac{n}{2} + 1$, and Bavard showed one has equality (with the greatest integer less than or equal to $\tfrac{n}{2}+1$). Danny Calegari and Alden Walker have improved the algorithms used in these papers while using the same basic topological idea. I would also mention the diagrams used in these works are on the inside covers of Rotman’s group theory textbook.
$n=2$ is special and has a finite formula with everything sorted:
$$(ab)^2 = abab = aab[b,a]b = a^2 b^2 [b,a] [[b,a],b]$$
$n=3$ is more like the rest, and has no finite formula if you try sort the commutators by weight. Even the unsorted formula is pretty long:
$$\begin{array}{ll}
(ab)^3 &= ababab \\
&= aab[b,a]bab \\
&= aab[b,a]ab[b,a]b \\
&= aaba[b,a][[b,a],a]b[b,a]b \\
&= aaab[b,a]^2[[b,a],a]b[b,a]b \\
&= a^3b[b,a]^2 b [[b,a],a] [[[b,a],a],b] [b,a] b \\
&= a^3b^2 [b,a][b,a,b][b,a][b,a,b][[b,a],a] [[[b,a],a],b] [b,a] b \\
&= a^3 b^3 [b,a][b,a,b][b,a,b][b,a,b,b][b,a][b,a,b][b,a,b] \\
&\quad [b,a,b,b][b,a,a][b,a,a,b][b,a,a,b][b,a,a,b,b][b,a][b,a,b]
\end{array}$$
Of course if we go mod $\gamma_4$ then we lose all that $[b,a,b,b]$ nonsense and are left with:
$$(ab)^3 = a^3 b^3 [b,a]^3 [b,a,b]^5 [b,a,a]^1 \mod \gamma_4$$
The powers on those commutators are called Hall polynomials if you let $n$ vary.
$$(ab)^n = a^n b^n [b,a]^{\binom{n}{2}} [b,a,a]^{\binom{n}{3}} [b,a,b]^{2\binom{n}{3}+\binom{n}{2}} \mod \gamma_4$$
has cubic growth. The Hall polynomial for $[b,a,a,\ldots,a]$ is always $\binom{n}{k}$. There are also three variable and higher versions.
Actually, the commutator length may be much shorter than the formulas indicate.
$$(ab)^3 = a^3 b^3 [ {(ab)}^{-1} b^2, b^{-1} (ab)^2 ]$$
$$(ab)^3 = (ba)^3 [aba,bab]$$
The first can be applied with $a=x^{-1} y^{-1}$ and $b=xy$ to answer the main question.
Guided by scallop by Alden Walker and Danny Calegari, I found $$(ab)^4 = a^4 b^4
[a^{-1}b^2, a^{-2}ba][ ba^{-2}ba,(ba)^2b ]$$
I think these shorter formulas lose some of the theoretical importance that the “nicer” formulas had, but I worry this sort thing will give a positive answer to the question. I continue to be interested in a positive answer.
To prove Schur’s theorem that if $[G:Z(G)]$ is finite then so is $G’$, Ornstein showed this in a way very similar to Cullen’s formula and the commutator length ideas. The first step was the remembered claim $[a,b]^n = (ba)^{-n} (ab)^n u$ where $u$ is a product of $n-1$ commutators. This follows from induction on $n$ with $n=1$ being clear. $$\begin{array}{ll}
[a,b]^n
&= [a,b] [a,b]^{n-1} \\
&= [a,b] (ba)^{1-n} (ab)^{n-1} u_{n-2} \\
&= (ba)^{-1}(ab) (ba)^{1-n} (ab)^{n-1} u_{n-2} \\
&= (ba)^{-1}(ba)^{1-n} (ab)^{n-1} (ab) [ (ab), (ba)^{1-n} (ab)^{n-1} ] u_{n-2} \\
&= (ba)^{-n} (ab)^n u_{n-1}
\end{array}$$
Thanks to Babak for finding this simple proof. This is used with $n=[G:Z(G)]$ since then $(ba)^{-n} (ab)^n = 1$ because $(ab)^n \in Z(G)$ and $(ab)^n = ((ba)^b)^n = ((ba)^n)^b = (ba)^n$. This gives that $[a,b]^n$ is a product of $n-1$ commutators, rather than $n$. In any product of commutators of minimal length, no commutator appears to a power higher than $n$. Since $xyx = x^2 y^x$ and the commutator length of a conjugate is the same as the original, we can sort any such expression to bring all copies of a commutator into a power. Hence no commutator appears anywhere in a minimal expression $n$ or more times. Since there are only at most $n^2$ commutators, that is a total of less than $n^3$ expressions, so $|G’| \leq [G:Z(G)]^3$. I suspect the other formulas we have give that $|G’| \leq 3[G:Z(G)]^2$.