FileMaker Server 18’s New Data Restoration in Action

By June 13, 2019 September 10th, 2019 4 Comments

A major new feature that rolled out with FileMaker Server 18 is its Data Restoration. In a nutshell: it will keep a transaction log of all the changes (data and schema) that are being made to the databases; if FileMaker Server or the server machine were to crash, it will use these logs to make sure that all of the logged transactions are applied to the database at the next startup.

FileMaker’s white paper explains the basic workings of Data Restoration. I recommend the white paper as a very worthwhile read and won’t repeat much of what it says. However, I do want to highlight some important practical aspects included in the content. Then, I’ll walk you through a crash event, show how the data restoration works, and list details to monitor.

It’s On By Default

First off: the feature is on by default when you install FileMaker Server 18. You can turn it off only by making an Admin API or CLI call to your server.

If you have not used the Admin API yet, the FileMaker community has released a good set of tools that use the Admin API. You can also use an API testing tool like the Postman to execute the necessary calls.  The data restoration setting is part of the “General Configuration,” and you can check its current state with a GET call to proper endpoint:


The two new configuration settings are highlighted below in the JSON returned by the API call:

Screenshot with the two new configuration settings highlight in the JSON returned by the API call

New configuration settings

A PATCH call to the same endpoint is all you need to toggle it on or off. Or to change the path to where the logs are stored.

Performance Impact

Speaking of the path where the logs are stored, we see two fairly immediate performance impacts with this feature:

  1. The required disk space: FileMaker Server will need 16GB of disk space for this feature. 8GB of which it will claim immediately and pre-allocate for its logs. It will use an additional 8GB will if you need a data restoration, but Server cannot perform it immediately. In that case, it will save the old logs and start a new set. In a typical scenario, the encryption key for the hosted files has not been saved on FileMaker Server and requires manual key entry by an administrator.

    16GB of disk space is sizeable. It is not uncommon for server’s to be configured with less than the recommended disk space, especially in virtualized environments. You must consider the extra required disk space for this feature iwhen upgrading. The extra disk space consumption may severely affect the server’s performance if it eats up much of the currently available disk space.
  2. The required disk activity: if your solution creates, deletes, or edits many records, it will expend more resources on writing transactions to its log before applying them to the database. If your machine has marginal specs (slow disk i/o, slow processors, low free disk space, etc.), you’ll certainly feel this effect. Consider your current server’s specs when deciding to upgrade, and favor high-speed disks when deciding to spend money on upgraded hardware.


The question has already come up in the community: “Can we use those transaction logs?”  The answer is: no.  These logs are strictly for FileMaker Server’s data restoration.  If you need transactions in your scripted workflow, you still need to code to that using the existing techniques.  You cannot use these transaction logs for this.

Backup Strategy / Disaster Recovery Strategy / Business Continuity

Where does the new feature fit in with the other tools that help us with a backup and disaster recovery strategy?

It complements the existing tools; it does not replace any of them, but they all fit together like this:

  • Data Restoration gives you just one restore point: the database as it was at the time of the crash. The restore time is fast and pretty much automatic at the moment FileMaker Server starts again
  • Progressive backups give you two restore points that you can rely on in case the Data Restoration fails. Their age depends on the interval you use, and that interval is typically relatively short (5-15 minutes). The restore time is usually fast but does involve copying the backup set over.
  • Backup schedules give you as many restore points as you have configured (hourly, daily, weekly, etc.) Restore time is the same as with progressive backups unless you need to retrieve a backup from an off-machine / off-site location.

Just like the Progressive backups, Data Restoration does not give you an off-machine or off-site backup. For that, you need to work with the regular backup schedules combined with some OS-level scripting.


If you are going to rely on this feature as part of your overall backup or disaster recovery strategy, you must test it thoroughly and understand how it works.

Probably the first thing to test for is whether the data restoration is actively running.  It may turn itself off at startup when it detects that there is not enough free disk space.  You can use the Admin API to perform scheduled testing to verify that the feature is on, and you can also monitor the FileMaker Server event log for messages that the server failed to start the data restoration at startup.  Our upcoming DevCon session on server monitoring with Zabbix will include this kind of functionality.

A second test to perform from time to time would be to crash the server and become familiar with the normal course of operation that a Data Restoration goes through.  That way, you can verify that the data restoration took its normal course and spot anomalies when it does not.

Obviously, you would do this on a test machine with a backup of your solution.  One of the challenges in testing features like this is generating enough relevant load so that the test is somewhat representative of a typical situation on your server.  After all, you want to ensure you’ve engaged the transaction logging part of the server so that you have something in the logs to restore.

Our Beta Test Process

During beta testing, we completed the following:

  1. An AWS EC2 windows instance with FileMaker Server 18 installed
  2. A hosted file that creates records through PSoS, in a continuous loop, with the “wait for result” toggled off. This allows us to spawn as many of these as we want, generate a good load, and pretty much guarantee that FileMaker Server is writing to its transaction log when we induce a crash.
  3. The “Not My Fault” Microsoft utility that allows you to crash Windows

With the server busy creating many records through the PSoS sessions:

Screenshot of records created through the PSoS session

Records created through the PSoS sessions

I induce a crash with the Microsoft tool:

Screenshot showing the Microsoft Tool to induce a crash

Using the Microsoft Tool to induce a crash.


After the server recovers from the crash and reboots, let’s look for what happened.  There is the obvious evidence of the crash with Windows prompting us to acknowledge the unexpected shutdown:

Screenshot of the Shutdown Event Tracker that appears when Windows prompts user to acknowledge the unexpected shutdown

Windows Shutdown Event Tracker popup

And an entry in the Windows System log, at the time Windows rebooted itself, confirming that the crash did indeed happen:

Screeshot of the entry added to the Windows System log

Entry added to the Windows System log

In the Application event log, where FileMaker Server also logs its events, we see FMS starting up at 10:21 am, at the time Windows restarted:

Screenshot of the Application event log that shows when the FileMaker Server started

Application event log

As per the best practices, I have FileMaker Server configured to start up but not auto-host any files.  This allows administrators to inspect the sequence of events that brought down the machine before deciding to continue with the files as they are.

Screenshot of FileMaker Server configuration

FileMaker Server configured to start up but not auto-host any files

At this point in time, my FileMaker Server is operational, but no files are being hosted.

A little further in the event log, we have confirmation that the Data Restoration feature is enabled:

Screenshot of the Event Log showing that Data Restoration is enabled

Application Event Log showing Data Restoration is enabled

This is an important event to confirm because, without it, we could try to open the files, but there will be no attempt to restore them to their pre-crash state!


The next event that we want to see is that the restoration has been applied and completed without error:

Screenshot of the Application Event Log showing that the restoration has been applied and completed without errors

Event log showing restoration has been applied.

Without this event, we cannot continue to use this file, and we would need to revert to a progressive backup or a regular backup.

Note that FileMaker Server performed the data restoration when its processes started, not when the file was opened to be hosted.  Given that we have auto-hosting disabled, at this point in time, the FileMaker file is not yet hosted and available to the users.

The only exception is if the file has Encryption-at-Rest (EAR) enabled and the encryption key is not saved. In this scenario, FileMaker Server will cannot do the data restoration at this point but will postpone it until the files are opened through the admin console and the administrator inputs the encryption key.

In our scenario, the data restoration has been done successfully, and we can proceed to open the file in the admin console:

Screenshot of opening the file from the Admin Console

Opening the file from the Admin Console

When we go back to the event log, we see FileMaker Server logged an “improperly closed” event when we opened the file:

Screenshot of the Event Log shows that FileMaker Server logged an "improperly closed" event

Event log showing that FileMaker Server logged an “improperly closed” event

This is normal, albeit somewhat unexpected and a bit confusing. Given that the data restoration was successful, we did not really expect FileMaker Server to log this warning. You must confirm the event ID 1070 (data restoration completed) preceding this warning event.

If unsuccessful, you’d see this warning about the file improperly closing. However, the file would not have been restored to its pre-crash state. Per best practice, you cannot safely re-use a file in that state.

In Conclusion

The new Data Restoration feature can play an important role in how we deal with crashes and outages.
Understanding its behavior is crucial; however, in making sure, we can rely on the outcome of a FileMaker Server restart:

  1. Is the data restoration feature enabled?
  2. Did the data restoration complete successfully?

As always feel free to leave questions and comments.

Wim Decorte

Wim Decorte

Wim is a Senior Technical Solution Architect at Soliant. He is a FileMaker 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17 and 18 Certified FileMaker Developer and the author of numerous Tech Briefs and articles on FileMaker Server. Wim is one of the very few multiple FileMaker Excellence Award winners and was most recently awarded the FileMaker Community Leader of the Year award at the 2015 FileMaker Developer Conference. He is also a frequent speaker at the FileMaker Developer Conference and at FileMaker Developer groups throughout the world. In addition to being a renowned expert on FileMaker Server, Wim also specializes in integrating FileMaker with other applications and systems. His pet project is the open source fmDotNet connector class that he created.


  • Excellent walk-through Wim, thanks for providing it.

  • Avatar Daniel Wood says:

    hi Wim,

    Great article thanks for sharing. I’m curious to understand why there’s no mention in the release of FileMaker Server 18 about multi-core utilisation and the improved performance that was mentioned during the ETS programme. I am under the assumption startup restoration is a feature added as a necessity due to the increased number of threads FileMaker Server can utilise, leading to more potential for corruption?

    Has the introduction of the transaction log and restoration option essentially negated any performance benefit seen by utilising more cores, and as a result is this why there was no mention on release of better performance (because there actually is none?) Our own tests with restoration on suggest 18 is slower than 17 when there’s over 3/4 users doing something at once – not what we wanted to see

    • Wim Decorte Wim Decorte says:

      Hi Daniel,
      From my testing, the data restoration has not negated the potential performance benefits from better parallel processing. Not sure if you saw the test results from our “Punisher” tool that was discussed during ETS; it demonstrates that some things are faster and some are slower and much of that depends on the nature, design, concurrent load and hardware specs… in other words: there’s an infinite number of variables in this equation and the outcome on any given deployment can be hard to predict. It’s equally clear that the larger the load and the better the hardware, the bigger the positive impact of the better parallel processing. Conversely, a big load on sub-par hardware may see an overall performance hit depending on what the solution does and how it is designed. Long and short: given how many things affect the performance, I’m not surprised that FMI is showing restraint. Things will become a lot cleared as everyone gets more experience with the features. We’re hoping to release that “Punisher” at some point and discuss its results.

  • Avatar Ninad Dixit says:

    Very good write up, Wim. The information on the new feature in FileMaker 18 – data restoration is explained in detail. The data restoration process after a crash is clear. This new feature will be helpful and it may help to restore data successfully. Thanks for sharing this information with all.

Leave a Reply

Need to adjust your business processes quickly? We're helping clients use technology to keep their teams productive and running smoothly in these times of uncertainty. Our team can guide yours if you need help in these areas.

Talk to a Consultant