How to prove $\int_0^\infty J_\nu(x)^3dx\stackrel?=\frac{\Gamma(1/6)\ \Gamma(1/6+\nu/2)}{2^{5/3}\ 3^{1/2}\ \pi^{3/2}\ \Gamma(5/6+\nu/2)}$?

I am interested in finding a general formula for the following integral:
$$\int_0^\infty J_\nu(x)^3dx,\tag1$$
where $J_\nu(x)$ is the Bessel function of the first kind:
$$J_\nu(x)=\sum _{n=0}^\infty\frac{(-1)^n}{\Gamma(n+1)\Gamma(n+\nu+1)}\left(\frac x2\right)^{2n+\nu}.\tag2$$
Mathematica gives the following result:
$$\int_0^\infty J_\nu(x)^3dx=\frac1{\pi\,\nu}{_3F_2}\left(\frac12,\frac12-\nu,\frac12+\nu;1-\frac\nu2,1+\frac\nu2;\frac14\right)+\frac{\Gamma\left(-\frac\nu2\right)\Gamma\left(\frac{1+3\nu}2\right)\cos\frac{\pi\,\nu}2}{2^{\nu+1}\ \pi^{3/2}\ \Gamma(\nu+1)}{_3F_2}\left(\frac{1-\nu}2,\frac{1+\nu}2,\frac{1+3\nu}2;1+\frac\nu2,1+\nu;\frac14\right),\tag3$$
which can be significantly simplified for odd and half-integer values of $\nu$. The results at those points allow to conjecture another, simpler general formula:
$$\int_0^\infty J_\nu(x)^3dx\stackrel?=\frac{\Gamma\left(\frac16\right)\ \Gamma\left(\frac16+\frac\nu2\right)}{2^{5/3}\ 3^{1/2}\ \pi^{3/2}\ \Gamma\left(\frac56+\frac\nu2\right)},\tag4$$
which agrees with $(3)$ to a very high precision for many different values of $\nu$. It also has an advantage that it is defined for all $\nu>-\frac13$ whereas $(3)$ is undefined at every even $\nu$ and requires calculating a limit at those points.

Is it possible to prove the formula $(4)$?


Mathematica is able to evaluate $(1)$ for even values of $\nu$ in terms of the Meijer G-function. Plugging those expressions into $(4)$ we get another form of the conjecture:
$$G_{3,3}^{2,1}\left(\frac14\left|\begin{array}{c}2a,1,2-2a\\\frac12,1-a,a\\\end{array}
\right.\right)\stackrel?=\frac{\Gamma\left(\frac16\right)\ \Gamma\left(\frac23-a\right)}{2^{5/3}\ 3^{1/2}\ \pi\ \Gamma\left(\frac43-a\right)}.\tag5$$
Incidentally, the case $a=\frac12$ would positively resolve my another question.

Solutions Collecting From Web of "How to prove $\int_0^\infty J_\nu(x)^3dx\stackrel?=\frac{\Gamma(1/6)\ \Gamma(1/6+\nu/2)}{2^{5/3}\ 3^{1/2}\ \pi^{3/2}\ \Gamma(5/6+\nu/2)}$?"

Thank you for posting this question, I enjoyed trying to answer it.

Start with the expression that Mathematica gave you and replace each
argument $\frac14$ of a hypergeometric function with $\frac z4$,
because we will be taking limits. I will call the two hypergeometric
functions $Q_1(z)$ and $Q_2(z)$. Each term can be brought to a closed
form by using identity 16.6.2 from the DLMF.

Setting $a=\frac12$, $b=1-\frac\nu2$, we get
$$ Q_1(z) = (1-z)^{-\frac12} F\left(\frac16,
\frac36,\frac 56; 1-\frac\nu2, 1+\frac\nu2; \frac{-27 z}{4(1-z)^3}
\right), $$
and setting $a=\frac{1+3\nu}{2}$, $b=1+\frac\nu2$, we get
$$ Q_2(z) = (1-z)^{-\frac{1+3\nu}{2}} F\left(\frac{1+3\nu}6,
\frac{3+3\nu}{6}, \frac{5+3\nu}{6}; 1+\frac\nu2, 1+\nu;
\frac{-27z}{4(1-z)^3} \right). $$
(Note that there are 6 possible identities to try per function, one for each possible choice of
$a$ and $b$ from the parameters, so it helps to do this on a
computer.)

The reason this works is that now the point $z=1$ is a singular point
of the hypergeometric functions on the right hand side, and Mathematica
will succeed in finding the limits as $z\to1$. The expression for the
whole integral that you have is
$$ Q =
\frac{2^{\frac43}\pi^{\frac12}}{\Gamma(-\frac16)\Gamma(\frac56+\frac\nu2)\Gamma(\frac56-\frac\nu2)\sin\frac{\nu\pi}2}
\left( -1 + 3^{-\frac{3\nu}2}\cos\left(\frac{\nu\pi}{2}\right)
\frac{\Gamma(\frac{1+3\nu}{2})
\Gamma(\frac56-\frac\nu2)}{\Gamma(\frac{1+\nu}2)\Gamma(\frac56+\frac\nu2)}
\right). $$
Call the large expression in brackets $A$, and then write
$$ A = -1 + B 3^{-\frac{3\nu}{2}}\cos\frac{\pi\nu}{2}, \qquad B = \frac{\Gamma(\frac{1+3\nu}{2})
\Gamma(\frac56-\frac\nu2)}{\Gamma(\frac{1+\nu}2)\Gamma(\frac56+\frac\nu2)}. $$

Now, Mathematica will not simplify $A$ or $B$ on its own, so it needs
help. Set $x=\frac16+\frac\nu2$, and use the multiplication formula to
get
$$
\frac{\Gamma(\frac{1+3\nu}2)}{\Gamma(\frac{1+\nu}{2})\Gamma(\frac56+\frac\nu2)}
= \frac{\Gamma(3x)}{\Gamma(x+\frac13)\Gamma(x+\frac23)} =
\frac{\Gamma(x)}{2\pi} 3^{3x-1/2}. $$
After this, $A$ simplifies to
$$ A = -1 +
\frac{\Gamma(\frac16+\frac\nu2)\Gamma(\frac56-\frac\nu2)}{2\pi}\cos\frac{\pi\nu}{2}
= -1 + \frac{\cos\frac{\pi\nu}{2}}{2\sin(\frac\pi6+\frac{\pi\nu}{2})},
$$
where I’ve also used the reflection formula for $\Gamma(z)\Gamma(1-z)$ to get rid of the
gamma functions. Some further amount of manual trigonometry yields
$$ A = -\frac{\sqrt{3}}{2}\frac{\sin\frac{\pi\nu}{2}}{\sin(\frac\pi6 +
\frac{\nu\pi}{2})}. $$

Finally, write
$$ \frac{1}{\sin(\frac\pi6+\frac{\pi\nu}{2})} =
\frac{\Gamma(\frac16+\frac\nu2)\Gamma(\frac56-\frac\nu2)}{\pi}, $$
and substitute back. Lots of things cancel, and the answer is
$$ Q = -\frac{3^{1/2}2^{1/3}}{\pi^{1/2}}
\frac{\Gamma(\frac16+\frac\nu2)}{\Gamma(-\frac16)\Gamma(\frac56+\frac\nu2)}. $$

This closed form is equivalent to the one you gave through the
use of $\Gamma(\frac16)\Gamma(-\frac16)=-12\pi$.

P.S.
I would also like to note that the integral
$$ I(\nu,c) = \int_0^\infty J_\nu(x)^2 J_\nu(c x)\,dx $$
and its general form
$$ \int_0^\infty x^{\rho-1}J_\nu(a x) J_\mu(b x) J_\lambda(c x)\,dx $$
appear in Gradshteyn and Ryzhik, and you can find a paper “Some
infinite integrals involving bessel functions, I and II” by
W. N. Bailey, which evaluates this integral in terms of Appell
functions, but only in the case $c>2$ ($|c|>|a|+|b|$), which is where the $F_4$ Appell
function converges. DLMF 16.16.6 actually gives a way to write this
integral as
$$
\frac{\Gamma(\frac{1+3\nu}{2})c^{-1-2\nu}}{\Gamma(1+\nu)^2\Gamma(\frac{1-\nu}{2})}
\,\,\,{}_2F_1\left( \frac{1+\nu}{2}, \frac{1+3\nu}{2}; 1+\nu; x
\right)^2, \qquad x = \frac{1-\sqrt{1-4/c^2}}{2}, $$
but the issue is that this is only correct for $c>2$, and the rhs is
complex for $c<2$. Appell function would only be defined by analytic
continuation in this case anyway, and I didn’t find anything useful about
non-principal branches of Appell or hypergeometric functions.

For $c>2$, Mathematica also gives the following:
$$
I(\nu,c) = \frac{\Gamma(\frac{1+3\nu}{2})c^{-1-2\nu}}{\Gamma(\frac{1-\nu}{2})\Gamma(1+\nu)^2}
\,\,\,{}_3F_2\left( \frac{1+\nu}{2}, \frac{1}{2}+\nu,
\frac{1+3\nu}{2}; 1+\nu, 1+2\nu; \frac{4}{c^2} \right), $$
but this is incorrect when $c<2$.

See section 6 in http://arxiv.org/abs/1007.0667 for that type of triple-product integrals.