Consider a two-sided coin. If I flip it $1000$ times and it lands heads up for each flip, what is the probability that the coin is unfair, and how do we quantify that if it is unfair?
Furthermore, would it still be considered unfair for $50$ straight heads? $20$? $7$?
First of all, you must understand that there is no such thing as a perfectly fair coin, because there is nothing in the real world that conforms perfectly to some theoretical model. So a useful definition of “fair coin” is one, that for practical purposes behaves like fair. In other words, no human flipping it for even a very long time, would be able to tell the difference. That means, one can assume, that the probability of heads or tails on that coin, is $1/2$.
Whether your particular coin is fair (according to above definition) or not, cannot be assigned a “probability”. Instead, statistical methods must be used.
Here, you make a so called “null-hypothesis”: “the coin is fair”. You then proceed to calculate the probability of the event you observed (to be precise: the event, or something at least as “strange”), assuming the null-hypothesis were true. In your case, the probability of your event, 1000 heads, or something at least as strange, is $2\times1/2^{1000}$ (that is because you also count 1000 tails).
Now, with statistics, you can never say anything for sure. You need to define, what you consider your “confidence level”. It’s like saying in court “beyond a reasonable doubt”. Let’s say you are willing to assume confidence level of 0.999 . That means, if something that had supposedly less than 0.001 chance of happening, actually happened, then you are going to say, “I am confident enough that my assumptions must be wrong”.
In your case, if you assume the confidence level of 0.999, and you have 1000 heads in 1000 throws, then you can say, “the assumption of the null hypothesis must be wrong, and the coin must be unfair”.
Same with 50 heads in 50 throws, or 20 heads in 20 throws. But not with 7, not at this confidence level. With 7 heads (or tails), the probability is $2 \times 1/2 ^ {7}$ , which is more than 0.001.
But if you assume confidence level at 95% (which is commonly done in less strict disciplines of science), then even 7 heads means “unfair”.
Notice that you can never actually “prove” the null hypothesis. You can only reject it, based on what you observe is happening, and your “standard of confidence”. This is in fact what most scientists do – they reject hypotheses based on evidence and the accepted standards of confidence.
If your events do not disprove your hypothesis, that does not necessarily mean, it must be true! It just means, it withstood the scrutiny so far. You can also say “the results are consistent with the hypothesis being true” (scientists frequently use this phrase). If a hypothesis is standing for a long time without anybody being able to produce results that disprove it, it becomes generally accepted. However, sometimes even after hundreds of years, some new results might come up which disprove it. Such was the case of General Relativity “disproving” Newton’s classical theory.
If you take a coin you have modified so that it always lands in heads and you get $1000$ heads then the probability of it being unfair is $100\%$.
If you take a coin you have crafted yourself and carefully made sure that it is a fair coin and then you get $1000$ heads then the probability of it being unfair is $0\%$.
Next, you fill a box with coins of both types, then take a random coin.
$P(U)$ : probability of having taken an unfair coin
$$P(U) = \frac{NU}{NF + NU}$$
$P(F)$ : probability of having taken a fair coin
$$ P(F) = \frac{NF}{NF + NU} = 1 – P(U) $$
\begin{align}
P(H) &= P(U \cap H) + P(F \cap H)\\
&= P(H \mid{U})P(U) + P(H \mid{F})P(F)\\
&= P(U) + P(H \mid{F})P(F)
\end{align}
By applying Bayes theorem :
$P(U \mid{H})$ : probability of the coin being unfair conditioned to getting 1000 heads
$$P(U\mid{H}) = \frac{P(H \mid{U})P(U)}{P(H)} = \frac{P(U)}{P(U) + P(H\mid{F})P(F)}$$
And that is your answer.
If $P(U)=1/(6 \cdot 10^{27})$ ($1$ out of every $6 \cdot 10^{27}$ coins are unfair) and you get 1000 heads then the probability of the coin being unfair is
\begin{align}
\mathbf{99}.&999999999999999999999999999999999999999999999999999999999999999999999\\
&999999999999999999999999999999999999999999999999999999999999999999999\\
&999999999999999999999999999999999999999999999999999999999999999999999\\
&999999999999999999999999999999999999999999999999999999999999999944\%
\end{align}
Very small coins like the USA cent have a weight of $2.5g$. We can safely assume that there are no coins with a weight less than 1 gram.
Earth has a weight of less than $6 \cdot 10^{27}$ grams. Thus we know that there are less than $6 \cdot 10^{27}$ coins. We know that there is at least one unfair coin ( I have seen coins with two heads and zero tails) thus we know that $P(U) \ge 1/(6 \cdot 10^{27})$.
And thus we can conclude that if you get 1000 heads then the probability of the coin being unfair is at least
\begin{align}
\mathbf{99}.&999999999999999999999999999999999999999999999999999999999999999999999\\
&999999999999999999999999999999999999999999999999999999999999999999999\\
&999999999999999999999999999999999999999999999999999999999999999999999\\
&999999999999999999999999999999999999999999999999999999999999999944\%
\end{align}
This analysis is only valid if you take a random coin and only if coins are either $100\%$ fair or $100\%$ unfair.
It is still a good indication that yes, with $1000$ heads you can be certain beyond any reasonable doubt that the coin is unfair.
In order to assign a probability to this event you need to start with a prior probability which can then be updated based on the data. Probabilities can’t really be derived from experience alone, you need to start with some sort of “inductive bias” in order to draw conclusions from evidence.
The heuristic approach of hypothesis testing gives a framework for making decisions in these situations, but it doesn’t pretend to assign probabilities to those hypotheses.
In my comment, posted this link to a more thorough treatment of the general question being asked. However, I will directly answer the question using the attained result on posterior density functions, as discussed in the linked article.
In particular, we note that if $r$ is the actual probability that the coin will land on “heads” rather than “tails”, then if we got $h$ heads and $t$ tails, the distribution of $r$ is described by the probability density function
$$
f(r\mid T=t; H=h) = \frac{(h+t+1)!}{h! \, t!} r^h(1-r)^t
$$
Now, let’s say that the coin is “fair” if $r$ (which should be exactly $1/2$) falls between $0.45$ and $0.55$. Then the probability that our coin is fair, given $h$ heads and $t$ tails, is given by
$$
P = \int_{.45}^{.55}f(r\mid T=t; H=h)\,dr =
\frac{(h+t+1)!}{h! \, t!}
\int_{0.45}^{0.55} r^h(1-r)^t dr
$$
Suppose we get $0$ tails and $h$ heads. Then this probability comes out to
$$
P = \int_{.45}^{.55}f(r\mid T=0; H=h)\,dr =
(h+1)
\int_{0.45}^{0.55} r^h dr = \\
\left. r^{h+1} \right|_{0.45}^{0.55} =
(0.55)^{h+1} – (0.45)^{h+1}
$$
As you can see, the more consecutive heads we get (assuming this is all the information we have), the less likely it becomes that the coin is fair. For $20$ straight heads, we compute
$$
P = (0.55)^{21} – (0.45)^{21} = 3.48 \times 10^{-6}
$$
Which is to say that it is extremely unlikely that the coin is fair. The probability that the coin is unfair is, correspondingly, very high.
For “almost all” priors, the answer is 100%: you have to specify an interval of “fairness” in order to get a nonzero value. Why? Because the bias $p$ can be any real number between $0$ and $1$, so the probability that it is a very specific real number is $1/\infty = 0$; hence the probability that it is $1/2$ is 0.
However, what I can tell you is that after your tosses, the heads probability is $$\frac{h + 1}{t + h + 1 + 1} = \frac{1000 + 1}{0 + 1000 + 1 + 1} = \frac{1001}{1002}$$
…assuming you initially believed the coin was fair (see @leonbloy’s comment under the question).
More generally, if the heads probability $p$ has prior distribution $\Pr(p)$ then the answer is:
$$\frac{\int_0^1 p\,p^{1000}(1-p)^{0}\Pr(p) \,dp}{\int_0^1 \phantom{p\,} p^{1000}(1-p)^{0}\Pr(p) \,dp}$$
Notice that for the uniform prior $\Pr(p) = 1$, this degenerates to what I gave above.
The derivation is longer and likely to be much more complicated than you might expect, so I’ll omit it to save typing… if you really want to see it, let me know.
(Basically, you need to apply Bayes’s rule and simplify.)
You asked
if this specific string of flips happens, what is the probability the coin is unfair?
And it seems to me that most answer here are answering the question
if the coin was fair, what is the probability of this specific string of flips happening?
If I try to literally answer your question, I get stuck unless we make additional assumptions. For your question, the sample space would have to be something like all instances ever of flipping a coin 1000 times. Not one specific coin mind you, but all instances ever, anywhere, of flipping one coin 1000 times. The next instance of 1000 flips can use a different coin. Some elements of that sample space would have 1000 heads. And some elements of that sample space would have involved a fair coin.
But we can’t do any calculating until we understand, among coins, what is the distribution of fairness? $p$ for a coin can range form $0$ to $1$, with only exactly $0.5$ being fair. If you assumed a specific distribution for the fairness of all coins, then we’d have the necessary information to calculate an answer for your literal question.
In a world where every coin is fair, then even with these 1000 heads, there is a 0% chance that your coin is unfair. In a world (more like the real world) where no coin is truly fair, then there is automatically a 100% chance that your coin is unfair. If you’d like to model with something in between, say 50% of coins are truly fair, and the other 50% is distributed some way among $p\in[0,0.5)\cup(0.5,1]$, then based on that information we could calculate a more interesting answer.
This is an issue not so much about mathematics as about the real-world background to the coin-tossing experiment. Convincing fake coins are fairly common: for example, about 5% of the £1 coins circulating in the UK are counterfeit. It would be just as easy to forge a double-headed (or double-tailed) coin as a normal one. And probably (!) forgers have made a few, either as a joke or to cheat people. On this basis, your coin (by a Bayesian analysis) is almost certainly double-headed.
Well, the next step is to inspect the coin to verify that it has a head on only one side. Then you have to ensure that this is actually the coin that is thrown each time; that the way it is thrown is fair; that the people checking the coin and recording the outcomes are honest and scrupulous; and so on. The more boxes are ticked, the less plausible is the 1000-head outcome. But this outcome would be near impossible even for a coin that is osmium–iridium on the tail side and magnesium on the head side. So doubts would still arise about the checking and recording procedure. For example, is a conjuror or skilled hoaxer involved at some stage?
The discussion of this experiment can obviously go hyperbolic. To cut things short, if the conditions of the experiment are guaranteed, in some absolute cosmic sense, to be fair, and only the balance of the coin is in question, then supposing the 1000-head outcome would be close to postulating a miracle; and that does not usually admit rational discussion.
As the supposed number of heads reduces, so the above argument weakens and needs to become more hedged, more complex, more numerical, and more uncertain. At some point, the only supportable response is to shrug one’s shoulders and say “who knows?”.
The Bayesian approach others are taking requires guessing at a prior distribution of the coin’s probability of heads, introducing another assumption into the question. We can simply take another approach, which is to ask “if the coin was perfectly fair, what are the chances I’d observe a string of heads of length $k$ or longer?”
It is well known that this follows the binomial distribution, which says that the probability of getting exactly $k$ heads in $n$ independent flips, each with probability $p$ is given by
$$
f(k;n,p) = \binom{n}{k}p^k(1-p)^{n-k}
$$
If I want to know the probability of length $k$ or longer, in general, I have to sum up the probability for length $k$, $k+1$, $k+2$…up to $n$ flips. However, in this case, $k=n$ because we are calculating what happens when all the flips are heads, so there is only one term in the sum. If the coin were perfectly fair, then $p=1/2$. All this gives
$$
f(k;k,1/2) = \frac{1}{2^k}
$$
For $k=1000$, this gives a probability less than $1$ in $10^{301}$. For $k=50$ the probability is less than $1$ in a quadrillion. For $k=20$ the probability is less than $1$ in a million. For $k=7$ the probability is $1$ in $128$. It’s up to you to decide if these outcomes are unlikely enough to warrant the conclusion of unfairness; that number is the probability that you got a string of heads by chance from a truly fair coin. If that probability seems unreasonable to you, you can instead conclude that the coin is not truly fair.
Given the chance for a head-up is 0.5:
$(1/2)^{1000} = 9.332636 \times 10^{-302}$
50 times?
$(1/2)^ {50} = 8.8817842 \times 10^{-16}$
Depends on your standard of unfairness, I would say it is pretty unfair for cases 50 and 1000 of straight heads.