So, let’s say you have 5 friends, and went on a trip together and over different occasions, different people paid for different things, with a plan of combining all the bills together at the end and splitting them evenly across all.
What is the best way of achieving this mathematically?
If you add up the total and divide among the number of people you get the amount each person should have paid. If you add up the bills each person has paid and subtract from the amount they should have paid you have the amount each person owes. These amounts should total to 0.
In terms of optimising, I understand that you want to split this up into person-to-person debts rather than person-to-group and group-to-person debts in such a way as to minimise the number of person-to-person debts. This looks rather like a bin-packing problem, so it is probably NP-complete to optimise.
(From a social point of view, the best solution is probably to have one trusted person collect from those who owe to the group and pay to those who are owed).
If you are interested in broader questions about mathematical insights into fairness questions that arise in fair division, cost allocation, apportionment, etc. take a look at this book for a good introduction: Equity in Theory and Practice, H. Peyton Young, Princeton U. Press, 1994.