The Image Problem in Data
In a systems implementation or upgrade project, image transfers from the legacy system are often a key component to ensuring that historical “proof” data does not get lost. Sometimes, the image data is implemented during the implementation of the new system or sometimes it is a stand-alone project. In either case, the data that is being handled requires the highest security as it is essentially just PII (Personally Identifiable Information) in physical form.
The MBS Image Problem Solution
MBS begins all data migration projects by studying the target system’s behavior and requirements and the source system’s extensive data profiling to understand critical components, usual business practices, and data structure, including corner case management. Then, MBS leads all partners through data profiling, cleansing, and migration via data mapping sessions and data issue resolution.
Information gathered includes the total file storage needed. i.e., total disk space used for both data and images.
The conversion or transfer of images involves two distinct parts:
- Physical Image files
- Metadata
- File path, folder, file name as part of the metadata in order to marry the file.
The specific steps involved in image conversion may vary depending on the requirements of the project and the landing place of the image data. However, I’ll outline a general overview of the process:
Step 1: Acquiring the Images
The first step is to acquire the images that need to be converted. Most of the time the data is already in its final format. That is, it has already been scanned and stored in a location accessible to the users. If this is not the case, there is an additional layer that may involve scanning physical documents and storing them in a central location. The converted image data can then be stored in a file system or transmitted over a network, depending on the project requirements. The file can be saved to a local disk, uploaded to a cloud storage service, or integrated into a database.
Typically, format conversion is done in the source system. If there is one agreed-upon format, such as PDF, then all the images will be converted from the original format to the desired PDF. In other cases, MBS will perform the conversion as a part of the process. Depending on the project requirements and the target format, compression techniques may be applied to reduce the file size so as to store and access the data more efficiently.
The physical images are then delivered via an agreed-upon protocol. They can be sent via Secure File Transfer Protocol (SFTP) or placed on a dedicated drive or folder. Some clients, to ensure the security of the data, deliver the data in person on a hard drive. MBS certainly understands the sensitivity and privacy that is being entrusted with the possession of this data and ensures that the decision on the delivery of the data is something with which both our client and the system vendor are comfortable.
It’s worth noting that image conversion projects can be more complex depending on the specific requirements. MBS has extensive experience with staging annotations, which is a tedious and difficult process. The ultimate goal of image conversion is to get the data to where it needs to go and for the user to be able to seamlessly use the data in their day-to-day processes within the applicable applications.
Step 2: Metadata
The second part of the image data conversion process is the Metadata.
Metadata conversion in a data conversion project involves converting the descriptive information and attributes associated with the files, such as file names, file sizes, creation dates, author information, keywords, tags, and other relevant details.
The process of metadata conversion typically involves the following steps:
The metadata is extracted from the source files or systems. This can be done through various methods depending on the file types and systems involved.
Once the metadata is extracted, it is mapped and transformed to fit the desired target system. This step involves following a mapping schema that identifies how the source metadata attributes correspond to the attributes in the target format or system.
During the conversion process, it is crucial to validate and clean the metadata to ensure its accuracy, consistency, and integrity. This may involve checking for errors, duplicate entries, missing values, or incompatible formats. Data validation rules can be applied to identify and rectify any inconsistencies or errors in the metadata.
In some cases, additional cleansing of metadata may be required, like adding new attributes or deriving values based on existing metadata.
Once the metadata is converted and validated, it can be loaded into the target system and integrated with the converted files.
Step 3: Reconciliation of Metadata to Physical
The reconciliation of metadata to physical files involves aligning the metadata information with the corresponding physical files to ensure consistency and accuracy. We identify the metadata attributes that link or describe the physical files. The actual reconciliation process can be tedious as it involves opening the files and cross-referencing the metadata attributes with the file itself. If any discrepancies are found during the comparison, corrective actions can be taken to resolve the inconsistencies.
It is key to maintain proper documentation of the reconciliation process, including any modifications or corrections made to the metadata. This documentation serves as a reference for future audits or inquiries regarding the accuracy and integrity of the metadata and file relationships.
Why the MBS Image Data Solution Works
This process of Image conversion is a proven method used during many successful projects. The method treats every step of the process as a checkpoint in the migration. The process promotes collaboration between all parties as it brings stakeholders from the client (and the vendor, if necessary) and MBS to the virtual table often for review of the results of each step; input from the client is encouraged and included in the decision-making. The process of reconciliation of metadata and the physical file is crucial to ensure the consistency and integrity of the migrated data. By aligning all aspects of the images and data, documenting the process used to correct discrepancies, and extensive quality assurance tests, the client can be assured that the overall results of the migration project are one of reliable and useful converted data and images.
