FileMaker 18 added SMB, LDAP(S), and SMTP(S) to the list of protocols supported by the Insert From URL script step. The full list of supported protocols now is HTTP(S), FTP(S), FILE, SMB, LDAP(S), and SMTP(S). This blog post will focus on making use of cURL and the SMTP protocol to send HTML-formatted emails.
Prior to FileMaker 18, plain text emails could be sent using the Send Mail script step. FileMaker 17 introduced the ability to include multiple attachments. Now, fully stylized rich-text emails with attachments can be sent, but the process is not as simple as it was with the Send Mail step. This blog post breaks down much of what you need to know and supplements that with a demo file for some additional hands-on learning.
Insert From URL
To send an HTML email, the Insert From URL script step needs to have the following items specified:
- Target – stores the result of the cURL command
- URL – specifies the SMTP server URL where the cURL command will be directed to
- Verify SSL Certificates – determines whether the SMTP server SSL certificate should be verified
- cURL options – specifies the options used to fine tune the cURL command
The URL specifies the protocol (SMTP or SMTPS), the fully qualified domain name of the SMTP server, and the port.
Here are some examples of SMTP URLs:
The cURL options specify additional instructions for delivery of the email. Here are some of the options that you may use, along with some comments for each one:
|–mail-from <address>||This is the email address of the person sending the email. Specify the address only; e.g. firstname.lastname@example.org. Do not also include the name like this: Liam Lopez <email@example.com>.|
|–mail-rcpt <address>||Use this option for the “To”, “Cc”, and “Bcc” addresses. Don’t place multiple recipients on a single line. Instead, repeat this option once for each recipient. For example:
--mail-rcpt firstname.lastname@example.org --mail-rcpt email@example.com
|–user <account>:<password>||Not needed if the SMTP server does not require authentication.|
|–upload-file <file>||This option tells cURL where to find the message that is to be sent. In FileMaker, we can use a variable name, like $mailFile, in place of <file>. We’ll cover the message format of the mail file in more detail a bit later.|
|–trace <trace>||This option causes all information from the command execution to be saved, which can be very helpful for debugging. In FileMaker, we can specify a variable, like $trace, for <trace>.|
|–show-error||This option causes the Get(LastExternalErrorDetail) function to return useful troubleshooting information.|
|–ssl-reqd||Require SSL/TLS. The SMTP URL should use the “smtps” protocol designation in this case.|
|–ssl||Try to use (but do not require) SSL/TLS.|
The SMTP URL and cURL options tell cURL where to send the email, to whom, and how, but they do not directly specify what to send. Instead, the cURL options reference a mail file which includes the content of what is to be sent. This mail file, referred to as the message, has to be constructed following particular formatting rules.
The rules can get quite complex, and a big driver of the complexity is the fact that each new email standard aims to be backward-compatible with the older standard that it supersedes. Some examples of this are the rules that require content to use 7-bit US-ASCII characters with each line of text limited to 78 characters or less (more on that later).
In the simplest case, where the email message consists of only ASCII characters, none of the lines in the message is longer than 998 characters, and the email does not have any attachments, the message can be constructed using just one “part.” In more complex cases, the email message will use Multipurpose Internet Mail Extensions (MIME) formatting to handle different character encodings, to allow for attachments, or to include alternate (e.g. plain text) versions of the email message.
The message will always consist of a header and a body. In the case of MIME multipart messages, the body itself will be split up into multiple parts, each of which will have its own header and body.
Each part in a multipart message will specify a MIME type to indicate the type of content included in that part. Some examples of MIME types include:
- Plain text: text/plain
- HTML text: text/html
- PNG image: image/png
- PDF: application/pdf
- Zip: application/zip
The full list of defined MIME types can be found here.
So that email clients can tell where one part ends and another begins, each part in a multipart message is separated by a boundary which has to be a string of printable 7-bit ASCII characters (codes 32 through 126), with a length of 70 characters or less, and it must not appear in any of the encapsulated parts. Here is an example of a valid boundary: gc0p4Jq0M2Yt08j34c0p.
Each time a boundary is used, it is prefixed with two hyphens. The last boundary is also suffixed with an additional two hyphens.
There are three kinds of MIME parts, and therefore three kinds of part boundaries.
Alternative: The “alternative” boundary separates alternate versions of the message. For example, there could be a plain text version of the email to accompany the HTML version, so that old-school plain-text-only email clients can display the message correctly.
The other two types of MIME parts have to do with attachments. Attachments can be embedded within the email message, or they can be attached to the email message in such a way so that they have to be saved out of the email before they can be accessed.
Related: The “related” boundary separates embedded attachments from each other and from the email message.
Mixed: The “mixed” boundary separates non-embedded attachments from each other and from the rest of the message.
The various different parts are nested with the mixed parts forming the outer-most portion of the structure and the alternative parts forming the inner-most portion.
At this point, your head is probably spinning with all of the details, so hopefully, this visual explanation will help clarify your understanding:
The main header section can contain fields such as From, To, Cc, Subject, and Date, but note that the To and Cc fields are not used to determine where the message should be delivered. That information is determined from the cURL options (covered above). The analogy is a paper letter that contains the destination address both on the envelope and on the letter itself. The address on the envelope is the one that is used for delivery. The address shown at the top of the letter is included for informational purposes only.
MIME Format Example
Here is an example of a simple, non-MIME formatted message:
From: Liam Lopez <firstname.lastname@example.org> To: Cameron Turner <email@example.com>, April Wright <firstname.lastname@example.org>, Theodore Hall <email@example.com> Cc: Zara Allen <firstname.lastname@example.org> Subject: Test Date: Wed, 14 May 2019 19:34:21 Content-Type: text/html; charset="utf8" <html><body>Hello, here is my <b>bold</b> email message.</body></html>
A couple of things to point out:
- The message body is separated from the header section with an empty line.
- Each line should be separated by CRLF (carriage return and line feed) characters.
- Each line should be kept to a length of no more than 78 characters. The technical limit is actually 998 characters, but the recommended limit is 78. Longer lines should be broken out into multiple lines with a whitespace character beginning each new line. This process is called “folding.”
- The message (both the header and the body) must be comprised of 7-bit ASCII characters.
- The Bcc addresses, if there are any, are typically not included in the message header.
For MIME-formatted messages, non-ASCII data can be converted to 7-bit ASCII using an encoding scheme such as “quoted-printable” or base64 (RFC 2045). When encoding content that consists of mostly Latin-based text, the “quoted-printable” encoding is preferable, since it is human-readable. This makes troubleshooting easier. Both of these encoding schemes automatically fold lines at lengths of 76 characters.
Quoted-printable Encoded Text
Here’s an example of what quoted-printable encoded text looks like:
Congratulations to Luka Modrić 🇭🇷⚽ for winning the Ballon d'Or!
Congratulations to Luka Modri=C4=87 =F0=9F=87=AD=F0=9F=87=B7=E2=9A=BD for w= inning the Ballon d'Or!
Unfortunately, FileMaker doesn’t provide a native quoted-printable encoding function, so I used base64 encoding in the demo that accompanies this blog post.
Additional MIME Examples
Here are some additional visual guides that show how email messages (both MIME and non-MIME) are put together.
The pilcrow character (¶) is displayed to denote places where an empty line should be inserted.
To simplify things, some of the text is represented using placeholders. This is denoted with angled brackets; e.g. <<EMAIL_MESSAGE_HTML_TEXT>>.
HTML Email Only
The first example is of a simple non-MIME HTML email.
When I tested sending out this kind of email, I was able to include non-ASCII characters, but the standard specifies that only printable 7-bit ASCII characters should be used. This is another example of the point made earlier, which is that different SMTP servers and email clients implement the standards in different ways, including building in support that allows for stretching the rules. However, if you want to ensure that your email implementation will continue to work regardless of what server or client is used, better adhere to the rules defined in the standards.
HTML Email + Attached Files
This second example shows two attachments included along with the HTML email. In this case, the attachments are not embedded within the email message.
The preamble message is there so that non-MIME compliant email readers can have some intelligible message to show to the user. This message can be anything you want. For example:
This is a message in MIME format. If you see this, your mail reader does not support this format.
HTML Email + Embedded Files + Attached Files
The following example adds two additional attachments, but unlike the previous two, these new ones are embedded within the email. This is done by setting the content disposition to “inline”.
Embedding attachments is commonly done with images. When the HTML for the email is constructed, these attachments can be referred to by their content ID, which is defined in the part header. For example, suppose that the content ID for one of the attachments is defined as follows:
Then the image can be referenced inside the HTML email like this:
Plain Text Email + HTML Email + Embedded Files + Attached Files
The last example uses “alternative” boundaries to add a plain text version of the email message.
Using Encoded-Words in Message Headers
The preceding examples describe how a message body can be encoded to account for non-ASCII characters, but what if the message header also contains these unsupported characters?
For instance, the name of a recipient or the subject of the email could contain characters that fall outside of the 7-bit ASCII range. In that case, you can use encoded-words to convert sequences of unsupported text using the following syntax:
For example, suppose the email subject consists simply of the following smiley face emoji: 😋
As a base64 encoded-word, the subject message header would appear as:
In the example above, the character set is specified as “UTF-8”, the encoding is specified as “B” (for base64), and the encoded text is specified as “8J+Yiw==”, which is the base64 equivalent of 😋
The two encoding options are “B” for base64 and “Q” for quoted-printable.
Similarly, the following recipient message header…
To: 老子 <email@example.com>
…could be encoded as follows:
To: =?UTF-8?B?6ICB5a2Q?= <firstname.lastname@example.org>
You can read more about encoded-word syntax in RFC 2047.
Conclusions and Demo File
We now have the ability to send HTML emails with attachments, but the step up in complexity is significant compared to the Send Mail script step.
The accompanying demo file includes a script which does a lot of the heavy lifting for you. The script was written to be portable so that you can copy it into your solution without having to set up any tables or fields. (The script does rely on the “GetMIMEType” custom function though, so you will need to copy that before copying the script.)
The demo file supports fully stylized emails which can be achieved through HTML or through FileMaker field rich text formatting (using the GetAsCSS function). Attachments can be designated as embedded (“Inline”) or as regular, non-embedded (“Attachment”).
The demo file does not, however, support alternative MIME parts, which are used to provide alternate (e.g. plain text) versions of the email message. And it does not do any folding or encoding of message headers. So if your message headers (e.g. subject or recipient email addresses) use non-ASCII characters or are longer than 998 characters in length, the email might not get sent out.
If you do not have an SMTP server readily available for testing, you can create a Gmail account and use the Gmail SMTP server. If you have two-factor authentication enabled (and you should), you will need to create an application-specific password. You can follow these instructions to do so.
- Message format for non-MIME emails: https://en.wikipedia.org/wiki/Email#Message_format
- RFC 5322 standard for non-MIME emails: https://tools.ietf.org/html/rfc5322
- Wikipedia article on MIME messages (includes links to the relevant RFC standards): https://en.wikipedia.org/wiki/MIME
- Message header fields: https://www.iana.org/assignments/message-headers/message-headers.xhtml
- Encoded-word syntax (RFP 2047): https://tools.ietf.org/html/rfc2047
- Complete list of MIME types: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Complete_list_of_MIME_types
- An excellent guide to multi-part MIME messages written by Daniel Clark: http://qcode.co.uk/post/70