The point of a certificate is that one endpoint can be sure that the other endpoint is who they say they are, and not somebody else.
In the most common case, it is the client which checks that the server presents a valid certificate matching the domain name they connected to, for which the server has the associated private key, and which the client can trace back to someone they trust.
So if you connect to example.com, the server should have a certificate for example.com, with the matching private key, and this certificate needs to be signed by someone they trust.
The default "trust" of browsers and most other clients is a list of root certificates which belong the certificate authorities (CAs) which have been vetted against criteria chosen by the CA/Browser forum. It involves a lot of rules on how they check domain ownership (and possibly other criteria for OV or EV certificates), security, audits, revocation, etc.
CAs which match these criteria are usually included in root certificate stores, either in the OS (Windows, macOS, various Unix distributions...) or the browser (Firefox, most others rely on the OS).
If the certificate of example.com is signed by one of those CAs, then it is trusted. If not (as in the case of a self-signed cert), then it isn't.
If you want your web site to be available to everyone, they you don't have much choice than getting a certificate from one of those CAs.
If on the other hand you control all the client devices, then you can either add the certificate to the trusted certificate store of all those clients.
The alternative is to allow insecure connections (not check if the certificate is trusted), but the end result is that... it's insecure!