I would gladly use a notation for “A is a finite subset of B”, like
$$A\sqsubset B \text{ or } A\underset{fin}{\subset} B,$$
but I have never seen a notation for that. Are there any?
While waiting for a future standard, I will use Joffan’s $\ddot{\subset}$ coded as
$\newcommand{\finsub}[0]{\mathrel{\ddot{\subset}}}$
$A\finsub B$
$\newcommand{\finsub}[0]{\mathrel{\ddot{\subset}}}$
I will paste the new command in the first row, and then use \finsub, resulting in
$A\finsub B$ which I will explain after first use in each text. I guess that is satisfying enough.
And really, as you define sets and functions in a text, you could as well define relations without standard notations.
The usual way is to use two different notations, one of which means that $A$ is finite, and the other means it’s a subset of $B$.$$A\subset B,\quad|A|<\infty.$$
There are two options:
You use the notation often. Then define it properly at the beginning (or when you first need it) and use whatever you think is reasonable. I’d suggest, as others:
$$ A \subset_{\mathrm{fin}} B, \quad A \sqsubset B, \quad A \mathrel{\ddot{\subset}} B, \quad A \subset\!\!\!\!\!\cdot\!\!\cdot\, B \quad \ldots $$
You use it one or twice. Then just spell it out:
Of course, you can use $|A|<\infty$ or $|A|<|\mathbb N|$ or $|A|<\omega$ or whatever, just try to imagine being a reader of your text and think what is the least confusing thing.
In set theory there are two standard notations for the set of finite subsets of $X$:
In naive set theory, you can also find $\operatorname{Fin}(X)$ quite often.
So if you’d want to write that $A$ is a finite subset of $B$, you could say that $A\in[B]^{<\omega}$ or $A\in\mathcal P_\omega(B)$ or $A\in\operatorname{Fin}(B)$.
Outside of set theory, I believe writing $|A|<\infty$ is probably one of the most accepted ways of writing that $A$ is finite. But I’m sure that in some fields of mathematics there are more or less common notations, and you should probably align yourself to the crowd which will read your work.
If you are writing for yourself, then it really doesn’t matter what you’re using, right?
In any case, do remember the standard advice about notation:
Notation is used to reduce clutter, not to abbreviate. Define your notations and be consistent with them. Don’t force the reader to keep track of your symbols, if there’s no need to do so.
If I had to invent a notation for this, the most suggestive I can think of is
$$
A\in \mathcal P_{<\omega}(B).
$$
One could define the RHS using another notation that in fact is more or less standard in some areas:
$$
\mathcal P_{<\omega}(B) = \bigcup_{n \in \Bbb N}\binom Bn
$$
which notation is derived from the that for the binomial coefficient similarly to the way the Cartesian product notation $A\times B$ is derived from that for multiplication, or $Y^X=\{\,f:X\to Y\,\}$ from exponentiation.
Probably use ordinal/cardinal numbers?
$$|A| < \omega$$
$$|A| < \aleph_0$$
$A\in[B]^{\lt\omega}$ where $[B]^{\lt\omega}$ denotes the set of all finite subsets of $B$.