an honest guide to canvas app offline (part 2)

Before you continue— make sure you read through an honest guide to canvas app offline (part 1) first! There you will find a basic guide to the offline expression pattern, advice on the proper development approach, a step-by-step guide to loading in all your data via Power Automate, and a first set of recommendations on whether or not to built out offline capability.

We’re now ready to dive into the practical application of offline architecture. How do we keep track of data modifications? What about synchronizing changes to the server? How do we efficiently test and debug offline code?

Look out for these indicators and tips:

🛑 High risk or blocking issues for offline architecture

🟢 Best practice tips

⚠ Gotchas and things to look out for

Buckle up! This second part has a lot of ground to cover!

tracking user-modified data

After following Part 1, our theoretical offline app has all of the reference data it needs for the user to enjoy a read-only experience. When the app initializes in an online state, we run PowerFx expressions that retrieve all the requisite data from the server and store them in local Collections, and then save the data for offline use.

But a read-only experience is not going to cut it! In almost all use cases, the requirements will dictate that the user both updates existing data and creates new data while they’re offline. We need to create the infrastructure within the app to do so– and this means saving more data offline!

creating the dirty collection

We start by setting up the Collection structure, and then creating screens that allow the user to edit or create some data.

🟢 Use input controls like Text and Dropdown to allow users to submit changes using Patch().

⚠ You can use the Form control, but not the SubmitForm() expression. SubmitForm() is a declarative-style expression which only synchronizes changes to the server— obviously not an option when creating an offline scenario. This isn’t a blocking issue for me, as I normally find the Form control to be too restrictive from a UI & layout perspective, and use input controls and Patch() as my default.

how to implement

1. Implement the offline data collection technique described in Part 1.
2. In App.OnStart, create a new Collection with Collect(). Your Collection needs to have the same schema as your target data, so an easy method is to collect the first record from your data source and then clear it. (Note: this code should only be executed when the app is online, so it’s best to wrap it in an If(Connection.Connected...) formula. Thanks Gil for pointing this out!)

Collect(
    colDirtyAccounts,
    First(Accounts)
);
Clear(colDirtyAccounts);

2. Create your edit screen. Link it to navigation from the beginning screen, and ensure you send the context of the row to update with a handoff variable (in my example, I used Navigate(scrnEditAccount, Cover, {selectedItem: ThisItem}) to pass the Account context).
3. Add your input controls to your screen. Set the default values to the data from your selectedItem.
4. Create a Button or similar trigger to save changes.

5. Add a Patch() to the OnSelect property of your Button and save all user changes to the dirty collection.
   • Don’t forget to include the GUID of the record you will update! This is absolutely required when working with updates to existing data to prevent accidental data duplication.
   • My code looks like this.

Patch(
    colDirtyAccounts,
    Defaults(Accounts),
    {
        Account: GUID(selectedItem.accountid),
        'Account Name': txtAccountName.Text,
        Website: txtWebsite.Text,
        Email: txtEmail.Text,
        'Primary Contact': First(cmbPrimaryContact.SelectedItems)
    }
);

6. Add an additional Patch() to the existing offline collection, too.
   • 🛑 This step may seem unimportant, but if you don’t update the working set of data, your users may falsely conclude that their changes weren’t saved. Updating both collections will prove to the user that their edits exist, and hold onto all new and updated records which still need synchronization to the server.
   • 🟢 Notice that I’ve wrapped everything into an error handling expression (which ensures the user is aware when changes are not saved) plus some Concurrent() expressions for performance. When there are no errors, the final step pushes the updated collections to local memory with SaveData() and navigates back to the gallery screen.
   • ⚠ When you test this code in the canvas app studio, you will get a runtime error because of the SaveData() step. This is normal and can be safely ignored. The browser doesn’t have the same device memory infrastructure as a tablet or mobile phone, so Power Apps throws an internal error.

IfError(
    Concurrent(
        Patch(
            colDirtyAccounts,
            Defaults(Accounts),
            {
                Account: GUID(selectedItem.accountid),
                'Account Name': txtAccountName.Text,
                Website: txtWebsite.Text,
                Email: txtEmail.Text,
                'Primary Contact': First(cmbPrimaryContact.SelectedItems)
            }
        ),
        Patch(
            colOfflineData,
            selectedItem,
            {
                name: txtAccountName.Text,
                websiteurl: txtWebsite.Text,
                emailaddress1: txtEmail.Text,
                _primarycontactid_value: First(cmbPrimaryContact.SelectedItems).contactid
            }
        )
    ),
    Notify(
        "Data could not be saved",
        NotificationType.Error
    ),
    Concurrent(
        SaveData(
            colDirtyAccounts,
            "Offline Accounts"
        ),
        SaveData(
            colOfflineData,
            "Offline Data"
        )
    );
    Navigate(
        scrnAccountGallery,
        Cover
    )
);

collecting new versus updating existing

Implementation of the dirty collection for new data entry is actually simpler, since you don’t need to maintain the context of the selected record or store the existing GUID. It’s up to you if you want to add new data into your existing offline collection (colOfflineData) as well as the dirty collection. Generally, I would recommend to keep it separate until the changes have been fully synchronized to the server, but your use case requirements should make the final determination here.

viewing pending changes

When we built out our real offline use case, we had a requirement to preview the offline changes to the user so they knew how many records they had created or modified, and how many records were pending synchronization to the server. We ended up using filtered Galleries to expose the dirty collection to the user, plus some helpful counters in the header to show the user how many records were sitting waiting to be synced.

While it will depend on what you’re building, I generally find this helpful to keep the end user informed and make sure they are alerted of pending changes before they close out of the app. If a counter doesn’t make sense, even just a text or icon indicator for pending changes is a good idea. Closing the app and dismissing it from active use can result in data loss since all saved offline data is stored in app memory. Keep your user informed to avoid tragic data loss situations!

managing a hybrid app

I imagine there’s a wide range of applications for offline functionality in canvas apps. Not all apps are going to be used in exclusively offline scenarios, or not all users may be affected by a patchy cell signal while out in the field. My first inclination when building an offline app was to attempt bifurcation of the code depending on the app connectivity status. For example, if the app is online, Patch directly to the data source, else if the app is offline, Patch to the dirty collection.

🛑 Don’t attempt this. Write your app architecture in such a way that all changes hit your dirty collection, and are subsequently integrated back to the cloud asynchronously. There are a number of reasons why, but here are the main ones:

• Any code which relies on an API call would attempt to evaluate, even if not in the If() branch that’s executing (see the section on option sets / choices below)
• Every data entry/update feature would require double the amount of PowerFx code to write and maintain
• You can write your synchronization code to achieve near real-time performance even when using the dirty collection data architecture, so there’s no real downside to keeping it simple

synchronization

There are two methods of synchronization to consider for your offline app: scheduled and on-demand. You can use both of them in the same app to provide a multitude of choices for your users.

scheduled synchronization

This method uses the Timer control to periodically ping the device’s connectivity status. Set the Duration to an appropriate interval for your use case (somewhere between 5-15 minutes seems reasonable), and then start your OnTimerEnd code with If(Connection.Connected…).

In my real-life scenario, users anticipated being offline the majority of the time. At first, we started with a hard-coded interval of 15 minutes, but quickly realized that with a Dataverse back-end, it was easy to create an even better experience by letting the managers set the default interval for their location on the table, so they could increase or decrease the synchronization interval based on what made sense for the size of their location.

how to implement

1. Add a Timer control to your app on the main screen.
2. Set the Duration value (start with 60000 for a 1 minute interval, 300000 for a 5 minute interval, or set this to a variable so it can be changed dynamically).
3. Set the OnTimerEnd property to check the connection status of the device: If(Connection.Connected, )
4. Write your synchronization code to Patch() the dirty collection back to the data source, leveraging ForAll() to perform the operation in bulk.
5. Remove all rows from the dirty collection with Clear().
6. Refresh your data set from the data source, re-collecting it into your offline collection. Make sure you terminate with a final SaveData() step so the user can take the refreshed data offline right away if their connection is spotty.

For other scenarios, where offline architecture is required just in case of poor connection, you might want to consider a fancier approach and use PowerFx code plus variables to set up a polling interval! By tuning the polling interval settings, you can achieve near real-time data synchronization when the device is continually connected, while preserving app performance by backing down the polling interval if the device is offline for an extended period of time.

Here’s what the below code snippet will do. If the device is connected, the dirty collection data is synchronized to the data source and the polling interval will remain at 1 minute. Otherwise if the device is offline, it will increase the polling interval by 5 minutes with each subsequent run up to a maximum of 30 minutes. If, after the extended polling interval, the device goes back online, the dirty collection will synchronize and reset to a 1 minute interval.

If(
    Connection.Connected,
    //code to synchronize dirty collection to cloud data source;
    Set(
        PollingInterval,
        60000
    ),
    If(
        PollingInterval < 1800000,
        Set(
            PollingInterval,
            PollingInterval + 300000
        ),
        Set(
            PollingInterval,
            1800000
        )
    )
)

Pair this code with setting the Timer.Duration to the variable PollingInterval, and make sure your App.OnStart sets the PollingInterval start value. I recommend to start with 1 minute (60000 ms), but theoretically you can set it as low as 50 ms. Chose intervals that make sense based on the general use case, and don’t be afraid to tweak it later based on user feedback.

manual synchronization

Just as it sounds, the manual synchronization option is an on-demand push of pending changes in the dirty collection up to the server. Simply tie the same code as described above into an interactive element like a Button, Icon or Image, and allow users to override the default interval to synchronize when they please.

Of course, you should always wrap that in an If(Connection.Connected, ) expression, because otherwise your users will get a nasty runtime error trying to synchronize if they’re actually still offline. Disabling the button while the app is offline is likewise a good protective measure.

If you implemented a polling interval, be sure to include some code to reset the interval to the baseline if a user successfully performs a manual override.

downsides

🟢 Will your app be used to capture net-new data only, with no feature created to allow users to modify existing records? This section will not apply to those scenarios, and there are no additional actions to take when implementing your canvas app offline architecture.

⚠ Will your app be used to capture new data and modify only data owned by the app user? This section will likely not apply, but proceed with caution.

🛑 Will your app be used to modify any existing data, regardless of ownership? Stop and carefully read this section. If you allow unfettered access to edit any existing data by anyone, you now have to implement a conflict resolution process for each editable data set you add to the app.

Without additional PowerFx code, none of the above synchronization methods will prevent data loss due to multi-user edit scenarios. Multi-user edit scenarios are common when users are working broadly across an application, especially if there is little security overhead keeping users from touching each other’s records (like a locked-down Dataverse with strict Business Unit Hierarchy and Security Roles).

When multiple users have access to edit data when either online or offline, every time your canvas app synchronizes the dirty collection to the cloud database, your PowerFx code must check for any difference in the last modified dates and make a decision whether to synchronize that row or not.

In my real-life scenario, we mitigated this risk of a multi-user data overwrite scenario by: limiting the editable data to just one set (everything else was just read-only reference data); filtering the data we brought into the canvas app to only rows assigned to the user (preventing them from accidentally updating someone else’s record); and not including any user scenarios where someone “back in the office” would be creating or updating new data while people were out in the field. I felt comfortable with directly synchronizing the dirty collection to the cloud database without a conflict resolution implementation because we had constrained the business process and security roles to avoid the situation altogether.

If your use case is more open-ended, proceed with caution, and if you have to, put in the extra work of a conflict management feature.

option sets / choices

Before we’re done talking about this hefty topic of offline, a quick note regarding everyone’s favorite Dataverse object type: the picklist slash option set slash choices. For the sake of sanity and grammar, I’ll continue to refer to them as Option Sets in this post.

If you’re not aware, Option Sets are a special data type in Dataverse and have been around at least since CRM 2011. Option Sets are two-pronged: first, the set of choices are defined as a solution-aware object; and second, the set of choices are bound to a column on a table. One Option Set can be bound to multiple columns across multiple tables, and the set of choices are centrally defined on the Option Set object. Cool.

🛑 How many Option Sets are in your data set(s), really?

⚠ Do you anticipate needing to change Option Set values on your data? Reference Option Set values in filters and conditions?

I’ll save you all the hair-tearing frustration we encountered and recommend you think very thoroughly about your data model before you decide to proceed with the offline functionality. When working with Dataverse Option Sets in canvas apps, each time you reference an Option Set value in a table, it will make an API call to Dataverse to check the global Option Set list.

When you are offline, this poses a huge problem. Our real-life use case burned way too many cycles trying to sort this one out. We had several boolean/Two Option fields on our table which were used in an If() condition during the synchronization Patch(). The app was working perfectly online and offline, but as soon as we tried to synchronize our dirty collection to Dataverse, the app would spin for 1 minute and ultimately time out with a runtime error.

Through some testing and troubleshooting, we saw the API calls attempting to contact Dataverse. Each call had a 20 second timeout and 2 retries… for a total of 60 seconds. 💡 Lightbulbs went off. This was happening even though the reference to the Option Set was not in the If() branch condition which was executing. It looked something like this:

If(Connection.Connected,
Patch( {OptionSetField: OptionSetObject.OptionSetValue}
),
Patch( {StringField: StringValue} )
)

The entire If() statement was being evaluated as a block, so the Option Sets needed to be evaluated, so the API calls were triggered. This changed our entire approach. In this case, our requirements were simple enough that we swapped out Option Set fields for Text strings and moved on with our lives, frustrated but wiser.

how to safely implement option sets in offline

There is a way to bring Option Sets into your offline data, but it is a workaround. Many thanks to my colleague James Battams for sharing his wisdom from his own real-life implementation of this method.

1. In App.OnStart, declare a global variable for each Option Set you will work with.
2. Create an array with each choice value.

3. Leverage the variable(s) wherever you would normally reference the Option Set. For example, set it to the values of a combobox or dropdown input; or reference it with Filter() or LookUp() conditions in your PowerFx code.

10/13/2021 Update: This paragraph has been updated to amend an error about saving your Option Set Variables offline. Thank you to John at Strategy365 for pointing this out! Since storing the Option Set values within a Variable does result in a getOptionSetItems API call, you do need to save the variables offline. Otherwise, when your app is started when offline, they will try to call Dataverse for the Option Set object definition and result in the timeouts I described above. You will need to implement this method for each and every Option Set you want to use throughout your offline app. Good luck!

testing best practices

Last but not least, testing the offline capabilities of your canvas app can prove to be challenging. I’ve got a few best practice tips to help you be successful.

• When writing the app, start with a placeholder variable instead of Connection.Connected. The browser does not recognize this expression and will always evaluate to true, so sub in a variable which you can control to test your offline code.
• Ignore SaveData() and LoadData() runtime errors. There’s nothing we can do here: it’s the same kind of error as trying to test the barcode scanner in the browser.
  • (Note: there is a currently-experimental feature as of 12/15/22 named “Enabled SaveData, LoadData, ClearData on web player” which will prevent these errors. Try it out!)
• Use Monitor to see what’s happening behind the scenes. Here you can see the response performance and size in kb of the data returned by API calls. (This is where we found the Option Set API calls!)
• Approximate an offline scenario by setting your network throttling to 0kb / Offline.
  • This varies by browser, but in Google Chrome you can find it under F12 > Network > Throttling.

• New! Try using Power Apps for Windows to test the app while your device is in Airplane Mode. This can be a bit faster than cycling through airplane mode on a mobile device.
• Test on a real mobile device with airplane mode. Nothing compares to the real thing– make sure you test like a real user on the device which will be used. If you plan to let users bring their own consumer devices, then test on as many different device types as you can.
  • It’s a good idea to issue minimum requirements to run your offline app to your users. You don’t want to be responsible for supporting very old devices!
• On a real mobile device, test to see if your offline architecture is impacted by actions like switching between apps, switching between different Power Apps, dismissing the app from memory and other likely actions your users could take during operation. Be sure to issue training guides and job aids if any of these actions could have a negative impact on their offline experience.

re-assess feasibility

Phew, that was a lot. Let’s revisit the feasibility checklist to ask yourself if an offline use case for your canvas app is a good idea. We covered the first four questions in Part 1:

• Is your app already built? Adding offline features after-the-fact could be a huge lift.
• Will your app be used on employee consumer devices? The end-user’s device can have a big impact on the user experience.
• Will your app require large amounts and/or variable amounts of data? Data has to be stored in app memory, giving us only 30-70MB of storage to work with.
• Will your app be complex, with many kinds of tasks or functionality included? The amount and complexity of components in your app can reduce the offline app memory available (and, of course, complex apps usually mean complex data sets!)
• Do your users require many canvas apps with offline capability? Running multiple canvas apps while offline may negatively impact the amount of available app memory.

We are left with a few final questions to understand:

• Will your app’s users be able to modify any data, regardless of ownership? Synchronization from offline to the server can introduce data loss when multiple users could be editing the same data at the same time. If you don’t take care with the data flow design, you’d need to build in conflict resolution capabilities for each data set a user could modify.
• Does your app’s Dataverse model have way too many Option Sets / Picklists / Choices? These data types are a real pain to work with when you go offline, and you’ll need to manually re-create them within global variables to ensure your app will work offline.
• Is the app’s offline capability a nice-to-have? Offline capability requires a lot of extra work in design, development and test. If offline isn’t core to your use case, then my recommendation would be to look at model-driven app for occasional offline use, and leave the pixel-perfect apps for your connected users.

closing thoughts

I had a lot of fun figuring out the architecture of an offline canvas app. In reflecting back throughout the writing process, it’s clear that today’s canvas app offline capability is for narrow use cases only. It leaves the architecture 100% up to the app maker, with little guardrails to prevent unwary makers from stumbling into difficulties and dead ends.

It is my hope that this blog provides insight and guidance on whether and how to move forward with your offline dreams. Microsoft’s canvas app offline roadmap looks extremely promising, and I expect we’ll see some marked improvements to the functionality in the next few release waves. Until then, take these lessons and avoid some of the pitfalls we ran into!

Happy app building!