Managing Case Files

One of our programmers details how we approached the issue of data retention in the Clinical Encounters platform.

Data Management

A lot of the work that has gone into Clinical Encounters has dealt with loading, saving, and managing data. This comes in the form of devising methods for parsing through XML files, handling the data during runtime, ensuring that all data is displayed correctly, saving the data to a file, and more. In particular, one recent area of concern is how users will access saved cases.

In early testing, files were saved locally through a local file system. This was fairly straightforward and simple, but in order to implement database functionality, the whole system had to change. A main menu interface was designed that would provide users with a way to preview and search through cases to select the ones they would like to view or edit. From this menu, users can also choose to download cases to save them for later if they didn’t want to use mobile data, wanted the benefit of faster load times from local files, or knew they would not have access to the internet for a while. Yet, part of the issues we faced dealt with creating a balance between the local files and the database that the main menu could still work with.

The Balancing Act (The Problem)

Part of the issue now is that there are two instances of cases. One will live in a database while another could live on a user’s local machine. Users are not guaranteed to have downloaded the case file before attempting to load it, and local case files should be accessible even when not connected to the internet.

As far as our case creator and viewer components are concerned, it’s fairly simple. Depending on circumstances, the software could load from an existing file, load from the database if no existing local files are found, or could be told to load from the database even if there are local files. This can all be decided from a few simple checks or flags. Once the data has been loaded, then it can be handled without having to talk with the database or access local files until it is time to save.

The main menu is a different story. When showing previews of many potentially large cases, it doesn’t make much sense to download entire cases, parse through the data to find what we wish to display, and then display it. This would be an unnecessary burden on both download time and computational time. So, a separate table in the database was created exclusively for the menu previews. This way download times are quick and many previews can be loaded easily. This, however, creates a bit of a problem for local files. Since the information is stored, formatted, and retrieved through the database, how should local files give this same information to the main menu?

The Solution

The current approach is to prepare the information the main menu will need and store it locally as a separate file for every case. These files are the only .txt files saved to the local directory, making it easy for the main menu to find which files it needs to parse so it can get the needed data. When we prepared a demo build for the East Coast Gaming Conference, before this was implemented, we used a similar tactic due to the uncertainty of any internet connection at the convention center. Our approach then was to have a single file with all menu preview entries in it. We could then parse through the file and find what we needed from it, but the amount of text parsing and swapping made it more prone to error and had the potential to interfere with other case previews. The new system makes sure to divide the cases so that any errors are contained while making it easier to replace old preview files as well.

Moving Forward

The system used to balance the local saves and the database’s saves is still relatively new, so some bugs are still present. Managing duplicate filenames, record numbers, not being able to check the database if offline, and prioritizing the database’s or the local files have been some of the areas of concern among others that are subject to change in the future. The biggest concern will be avoiding any overwrites or loss of data when overwriting the local or database cases, yet with a fully developed and tested systems, users should never have to think or worry about it.