BlogFileMaker

FileMaker 16: cURL and Container Fields

By May 15, 2017 May 22nd, 2019 11 Comments

FileMaker 16 introduces newly available options to the “Insert from URL” script step that utilize the populate cURL library. If you are not familiar with cURL, it has long been a standard for command line interaction with all manner of internet protocols and applications. The introduction of these options heralds a host of new options for FileMaker developers looking to interact and integrate with all kinds of web services.

In this post, we will focus on one particularly useful feature: the ability to upload a file stored in a container field directly from the field. That means you do not need to write the data out to the file system before referencing it with your script.

It makes good sense. The location of the data, the contents of the container field, is known, so you should be able to reference it directly without having to save it somewhere locally first. Now we have a way to do just that, even if it is not apparent at first. Admittedly, the syntax was not intuitive for me when I first tried to get this working.

The Need For This Functionality

There are times when you need to upload a file to a web page, and in fact, most web scripting languages have built-in functions to handle file uploads. After all, the HTML form input type “file” has been around forever to allow exactly this sort of functionality. We know we can handle file uploaded on the web server side of things. We will assume that the file is being transferred as binary data and not Base64 encoded. Some web services may require it and there may be limits on the amount of Base64 text you can transfer.

What we do with the file once it is uploaded is irrelevant to this blog post. Our focus is on how to make that request from a FileMaker 16 script. We will assume that the web scripting side of things works as expected.

Enter cURL

Luckily for us, the cURL functionality needed for this is included with FileMaker 16. External scripting or third party plugins are no longer required. Since “file” is a valid input type for HTML forms, the cURL library includes support for submitting this data as part of an HTTP request, most commonly transferred with the POST method. This means that the payload of the request is not sent as part of the URL string like you would have with a GET request. Consequently, an arbitrarily large request can be made without running into URL string limits.

cURL syntax with FileMaker Variables

Let’s review the relevant steps. First, we set a variable to the value of our container field.

Set Var: ["$myFile" ; Table::UploadField_r ]

How can we use this variable in our cURL expression? The “-F” flag, which is short for “form”, allows cURL to emulate a filled in form in which the user has pressed the submit button.

We can use this flag with the new “Insert from URL” cURL options like so:

-F "file=@$myFile"

Note that when we include the variable in the cURL options, it appears inside the quotation marks. This is the correct syntax; placing it outside the quotes will not work.

This is analogous to how a file name is specified inside of quotation marks in an HTML form element:

<input type="file" name="myFile" />

Similarly, on the server you can access the data the same way any uploaded files are received. For example, with PHP, you can use the $_FILE variable to handle and save the file on the server.

Side Effects

A side effect of this syntax is that, while it allows you to pass the binary data directly from a container, the file name that appears on the receiving end will be the name of the variable you use. In the above example, the file name would be “myFile” which may not be what you want.

A possible solution will be to pass the file name as a separate parameter and then rename the file on the server. However, if you are not in control of the web scripting, that won’t be an option.

To make it easier for web services to consume, we can set the variable name to the file name of the container data. The trick here is to do this dynamically and to avoid illegal characters and reserved words. Note that variable names are subject to the same rules as field names.

A simplified version of this can be done with a few variables and using the Evaluate and Let functions to dynamically set a variable.

Set Variable [$myFileName ;
Substitute ( GetAsText ( Detail::Upload_File_r ) ; [" " ; "_"] ; ["(";""] ; [")";""] ;  ["+";""] ; ["-";""] ; ["=";""] ; ["≠";""] ; ["&";""] ; ["<";""] ; [">";""] ; [",";""] ; [";";""] ; ["[";""] ; ["]";""] ; [":";""] ; [""";""] ; ["$";""] ; ["/";""] ; ["*";""] ; ["^";""] ; ["}";""] ; ["{";""] )  ]
Set Variable [ $null ; Evaluate ( "Let ( $" & $myFileName & " = " & "Detail::Upload_File_r" & " ; "" )" ) ] 
Insert From URL [ Select ; With dialog: Off ; $your_result ; $your_url ; cURL options: Evaluate ( ""-F file=@$" & $myFileName & """ ) ]

You may also note that the target of the result of the insert can now be a variable. This is a new change in FileMaker 16. You can now call this function without requiring a field to be placed on a layout to hold the result, as was the case before.

Testing

As mentioned above, you can write your own web script and host it on your web server. You can also use a service like Request Bin to inspect HTTP requests that are made. If we do that, we can see that the file gets submitted as part of the HTTP request and that it is sent as binary data.

Feel free to try it out yourself from the sample file.

The sample file also includes a sample script to download a file directly to a container field. Although the functionality shown is not an entirely new feature to FileMaker, the part that is new in 16 is the ability to verify SSL before downloading data. The sample file has this new feature enabled.

Of course, this just scratches the surface of what you can do with FileMaker 16 and the newly introduced cURL options. I look forward to seeing what can be done with it myself and seeing what others can come up with.

References

Mike Duncan

Mike Duncan

Mike is an AWS Certified Solutions Architect as well as a certified FileMaker Developer. In addition to his work, Mike also enjoys pursuing his art, freelance writing, traveling, and spending time with his family.

11 Comments

  • Avatar Martin G says:

    Hi Mr. Duncan, thank you for this post; I found your solution really useful. Quick question though – what about multiple files, is that achievable in a single “InsertFromURL” command?

  • Hi Mike,

    Thank you very, very much for this post. Specifically the fact that the file variable has to be in between the quotes. For an old fashioned FileMaker developer who knows little of http Post and cURL this was not imaginable without your help.

    Some remarks or questions.
    – Should the -F in: -F “file=@$myFile” not be within the quotes, as in: “-F file=@$myFIle” ?
    – In my case I had to post data in csv format. The data are gathered in a script in a loop into the variable $data, just before the Insert from URL. As all the help texts and explanations talk about uploading ‘a file’, I thought that I should give the file path in the Insert from URL. After a lot of trial and error I understood that in my case the ‘file’ is the data itself. So, in the script the variable $data is used in the Insert from URL as in:
    “-F file=@$data”.
    Do I misunderstand the use of the word ‘file’ here? Could it be a path to a file, or should it always be the content of a file? (If you understand what I mean.)

    • Avatar Mike Duncan says:

      In this example, “file” is just the name of the parameter that gets passed. Similar to “name=Martin” where “name” is the key and “Martin” is the value. Of course you would need to adjust according to the requirements of the web service or script you were submitting to.

  • Nice article! I’ve been playing with this as well, with a Dropbox integration. In this case the curl options include a line like this:

    –data-binary @$file

    And I set the $file variable to the contents of a container. In this case I don’t need quotes around the variable.

    You can play with this file here:

    http://www.filemakerforever.com/articles/133

  • Avatar mcarrgomm says:

    Hi Mike,

    I’m looking to hopefully take this solution one step further. Is it possible to use FileMaker’s new cURL functionality to upload say a jpg file in the container field up to a bucket on my Amazon S3 account? and if so, do you have any pointers on the calculation formula to use when specifying the cURL options?

    • Avatar Mike Duncan says:

      Possible? I think so, but AWS uses a header signature that is non-trivial to create, and is different for each service. I am actually working on that now, and can share when done. Usually you might use a SDK to do the heavy lifting for this part, but with cURL options, have to build it all yourself.

  • Avatar akim says:

    Mike,
    Thank you for your great insights into avoiding the issues of uploading files, via FMP’s Insert From URL command.
    Your solution was quite creative!
    I however found that your coding of literal terms might cause a problem, should the container file name or its table name change.

    I added the following lines to keep the names as references rather than as strings.
    …
    Go to Field [Table::UploadField_r]
    Set Variable [$TableName_DoubleColon_ContainerFieldName ; Value:Get ( ActiveFieldTableName ) & “::” & Get ( ActiveFieldName )]
    Set Variable [$null; Evaluate ( “Let ( $” & $myFileName & ” = ” & $TableName_DoubleColon_ContainerFieldName & ” ; \”\” )” )]
    …

    After that the command to Insert From URL will have an active reference to post, and less likely to error on not finding a document or other data to upload, due to a static name.
    I’d appreciate your thoughts on this, considering that your creativity has brought us all this far.

    • Avatar Mike Duncan says:

      Sure, and I have seen others use custom functions to abstract table and field names. In presenting the technique, I hope it is easy to understand, and if this abstraction is needed it can be added. Thanks for you input!

  • Avatar Johan Sundell says:

    Think you can use the filename to tell the remote server what you named the file, from the curl manual

    “You can also explicitly change the name field of a file upload part by setting filename=, like this:
    curl -F “file=@localfile;filename=nameinpost” url.com”

    • Avatar Mike Duncan says:

      This is right, but may be dependent on the server, and what is handling the file on that end. You mileage may vary 🙂

Leave a Reply