How does email get delivered and why is it rejected sometimes?

it does not make sense why email for one domain goes through and another is rejected.

Differing destinations may have different policies and/or criteria for acceptance/rejection of email. For example, your domain or IP address may be blacklisted (see the two threads about UCEPROTECTL3 here and here for an on-going saga). There are several of these:

• zen.spamhaus.org,
• cbl.abuseat.org,
• bl.spamcop.net,
• xbl.spamhaus.org,
• sbl.spamhaus.org,
• bl.spamcop.net,
• psbl.surriel.com,
• dbl.spamhaus.org,
• dbl.spamhaus.org

are the ones my server consults for making accept/reject decisions.

Your server may be configured as an open relay (big email providers REALLY hate these…they're a huge source of spam).

Email domain operators may have local defenses -- ProofPoint, etc. Recipients may have defenses (Gmail rules for example) that prevent delivery. My domain, even though it's private, has at least three layers of these…

Your mail server/system may not adhere to email standards/best practices (e.g., SPF, DKIM, DMARC)…which almost guarantees rejection at most of the large email providers.

It's hard to answer your question without more information. In any case, this is a question you need to pose to your recipients and/or correspondent domain operators.

-- sw

Thanks for the awesome, rapid and concise answer stevewi! My intent was to try to provide a detailed general article for this question. I posted the question and started to write up the answer and a very happy to see I was beaten to the answer.
My response which will appear here is still a work in progress

At it's most basic level, email is just sending messages from one user in one domain to another user's mailbox in another domain. It seems like is should just work.

Your Grandparents' Email

Originally, if Alice alice@yodel.com wanted to send a message to Bob bob@geemail.com the exchange would go something like this…

(Any Sending Computer) "Hey DNS Server, What server handles mail for geemail.com?"

(DNS Server) "The MX (Mail Exchange Server) for geemail.com is mail.geemail.com"

(Sending Computer) "Hey DNS Server, I don't know who mail.geemail.com is, what is the network address of mail.geemail.com?"

(DNS Server) "mail.geemail.com is 203.0.113.24"

(Sending Computer) "Hey 203.0.113.24, I have a message from alice@yodel.com for bob@geemail.com: Hey Bob, The code for the lock box is 346576."

Spoofing

There was a problem with this way of sending email. Anyone could send mail pretending to be anyone else (called spoofing). You would not like someone to be able to fake an email from you. They can do bad things like offering a false link to a fake version of your website to steal your customers' passwords.

Preventing Spoofing

Some receiving domains will accept mail from anybody, but many commercial mail providers will to try to verify the authenticity of the mail sender.

Reverse DNS

An easy check that can be done to see if an email message is at least being sent by the domain specified in the email is to look at the domain of the network address. If the message (supposedly from someone at yodel.com) comes from 198.51.100.54 and we ask DNS about the owner of this network address, called a reverse DNS lookup, and we get a response like smtpout.yodel.com, and we verify that the domain owner says that smtpout.yodel.com points back to 198.51.100.54 this is a good indication that sender is at least coming from yodel.com.

SMTP Banner Check

Software that provides the functionality of the SMTP Protocol responds with an SMTP and usually includes the DNS name of the server. We can connect to the smtpout.yodel.com server and check that it does indeed announce itself as rDNS name of the server that originally sent the mail.

SPF (Sender Policy Framework) Records

Some organizations have many servers in different locations that need to send mail. Some organizations hire agencies to send mail on their behalf.

The commonly accepted way of specifying which servers are allowed to send messages for a domain is to put a TXT record for your domain in SPF format. This looks like:

"v=spf1 a:smtpout.yodel.com -all"

This example specifies that mail from the IP address matching the DNS name of smtpout.yodel.com is allowed to send mail for yodel.com. Mail from any other address should be strictly rejected as not coming from yodel.com. The details of SPF can be found here: https://datatracker.ietf.org/doc/html/rfc7208

Note: Not all recipients respect SPF records and those domains that do not check may still pass a spoofed email to their users. This is on them, there is nothing you can do about them not checking.

Return Path Spoofing

Savvy hackers are sometimes able to spoof where replies or bounce messages to an email are sent. SPF records alone are not able to combat this. A combination of standards called DMARC (Domain-based Message Authentication, Reporting & Conformance) and DKIM (Domain Keys Identified Mail) can prevent this. More will be explained about this later.

TLS Encryption

In our example above, Alice has sent Bob the code for a lockbox. Originally email was sent in plain text over the network so any network in between could read the email being sent. A network that is able to read the email would also in theory be able to modify its contents as well. Ensuring that your email server has TLS encryption enabled can greatly help to prevent in-transit eavesdropping and spoofing of messages. Some recipient domains and blocklists will check that your domain supports this feature.

DKIM

DMARC

How did I end up on that blocklist?

Getting off of blocklists

Honey Pots

Spam Heuristics

Spam Folders vs Junk Folders

Sending Mail from a Linode

CloudFlare DNS and rDNS