Why Data Integrity Take Place Nowadays?
In the digital era, small files I/O and requests have taken place for nowadays technologies, such as financial blockchains, artificial intelligence, internet of things and e-commerce. These happened everywhere in our daily lives, for example, paying grocery shopping via mobile payments, withdraw cash in ATM, activity sensors in offices, factories, and public transportation stations, and many more activities that you can’t imagine. The rise of mobile apps and innovative technology transformed nearly every aspect of our lives. The incredible convenience was created to serve people and hard to believe how it been working in the last decade.
However, the more convenience we have experienced, the more data we have created. However, have you ever imagined that once if the data has been incorrectly placed or recorded, what if the data was accidentally corrupted by itself, what if the wrong data has replaced the right one? With these ideas coming out, data integrity has become a major challenge for IT for the devastating threats and the fundamental of the digital era.
Why XCubeNAS?
In the principle of designing XCubeNAS, data integrity was at the top of the list making the development team to face multiple trade-offs between safety and performance. We have crafted a unique design for user-defined pools, which fulfills different application requirements; each of them serves a different purpose: Databases, General, and Media Streaming.
The database pool is designed to have the highest performance with smaller files. The general pool fulfills the requirements of 80% of applications, like backup and file sharing. The media streaming pool provides high performance for larger files and is optimized for 10GbE, Thunderbolt 3.
How XCubeNAS Ensure Data Integrity?
Data integrity and data safety are essential for databases. QSAN’s storage management software, QSM, offers two mechanisms to ensure an absence of data failures between servers and storage. First, we set the data to write through the cache to prevent data loss from accidental disasters, like flash damage or power outage. Second, we apply ZFS checksum. ZFS’ checksum was designed to ensure data integrity.
In other file systems, like EXT4, checksums are used for integrity checks and only protect against bit rots. ZFS’ checksum, however, prevents issues such as phantom writes, misdirected reads and writes, direct memory access parity errors and accidental overwrites. The checksum is accomplished by storing it in the parent block pointer instead of in the block itself which contains a self-validating SHA-256 checksum.
Additionally, ZFS also supports self-healing data by using a mirrored setup similar to RAID0. Apart from this, other file systems are not addressable if a block is corrupt unless it’s a metadata block, which means other file systems may pass bad blocks to the application and it might overwrite the accurate data in other mirrors. ZFS is a transaction-based file system, which means the operations are atomic, if an operation crash or power outage occurs, the file system can quickly recover from corruption.
Beyond Security, The Performance
Higher security requires higher resources. As mentioned above, ZFS is using the memory-based checksum, which claims higher memory capacity installed in the system. Also, ZFS has used ARC, the Adaptive Replacement Cache algorithm, which was implemented in ZFS to outperform the traditional RAM algorithm by providing a superior hit rate. Moreover, high performance is not the only goal for ZFS, to ensure each file and block are correct, ZFS also requires longer computing time and RAM capacity to complete the process, which may impact the latency and impact performance.
Conclusion
In this summary, ZFS has the capacity to prevent silent data corruption which secures the data during transmission and storage. In any application scenario, sending and receiving uncorrupt data is essential. That’s how the XCubeNAS was designed. Built on ZFS, it is a reliable and high-performance enterprise storage solution to achieve data integrity.
