IBM Storwize 18.104.22.168 Release has introduced support for MS-ODX feature for its entire Storwize family of products and for IBM Spectrum Virtualize (previously called as SVC-SAN Volume Controller) product.
MS-ODX stands for Microsoft Offload Data Transfer.
Primary objective of MS-ODX feature is to free up host CPU and network cycles and to offload all the copy operations to the Storwize/ISV controller itself.
In traditional/buffered Windows copy operation, each file copy triggers read request to go from host server to storage array which then fetches the data back to server memory via network and then a write request is sent from server to storage array with this same data again over the network. So every data goes through the wire twice, once in reading and once in writing. It causes huge latency in IO operations.
Incase of Microsoft clustering solutions (MSCS) environments this data gets copied from one host to another over the wire which causes further delay in IO copy operation.
Microsoft simplifies this with usage of 3 XCOPY commands introduced in SPC-4
1. Populate Token – PT
2. Receive ROD Token Information – RRTI
3. Write Using Token – WUT
For Windows host to start triggering ODX, storage array needs to be supporting it, for eg. Storwize family or products – V7000, V5000, V3700 and IBM Spectrum Virtualize (earlier name – SVC). I would be writing a detailed document for this in part 2.
PT – It creates token on storage array. It contains source vdisk (virtual disk) starting LBA, amount of data to be copied, list identifier along with other segment related parameters
RRTI – It fetches token created on storage array to server. It is server’s responsibility to share token with another host in MSCS environment.
WUT – It contains token received by RRTI and is sent to destination vdisk. It contains LBA to which data needs to be copied, no. of logical blocks etc.
Since now no data would gets copied over to Host, there is significant saving over infrastructure and network requirements. There is also huge copy performance boost that gets achieved due to this offloading.
This boost is even more visible when your server and network is busy doing other operations. In that case your CPU cycles will not be tied up in doing a read and write operations from storage array instead it will just send series of ODX commands(few KB’s) which takes care of offload.
Important thing to note is, Microsoft limits no. of offload copy requests (primarily write operations using Write Using Token – WUT’s) to 1 (yes one!) for most of its Hyper-V operations like VM Storage Migration, VHDX Creation, VM Cloning and few more operations. So in spite of storage array’s capability, all these operations gets triggered in the multiple of 1 copy operation at a time. There are other applications like Robocopy which utilizes ODX very efficiently by sending up to 128 copy operations.
Storwize V7000 does support up to 2048 copy operations per cluster(8 node) i.e it is capable of performing simultaneous 2048 copy operations (VM Cloning, Storage migration, any other copy operation combined) if it receives these many WUT commands from a single or a cluster of hosts.
MS-ODX has few caveats though:
From Microsoft side:
1. It works only with NTFS filesystem
2. File size to be more than > 256K to trigger ODX.
From Storwize side:
1. Only Intracluster support is present currently.
2. Blocksize of NTFS to be >=4K.
In my next series of this blog, I’ll share some performance no.’s I have been experimenting with Storwize.
Please note, these are my personal thoughts and not necessarily represent that of my employer.