Using isi_vol_copy_vnx for VNX to Isilon data migration

For most data migrations from VNX to Isilon EMC recommends that you use the OneFS migration tool isi_vol_copy_vnx. It can often be more efficient than host-based tools (such as EMCopy and Robocopy) because the performance of host-based tools performance is dependent on the network connectivity of the host, while isi_vol_copy_vnx depends only on the network connection between the source device and the Isilon cluster. Below is a basic outline of the syntax, the steps required, and a few troubleshooting tips.

You might consider migrating data with a host based tool if one or more of the following conditions apply to your migration:

  • The source device and Isilon cluster are on separate networks.
  • Security restrictions prevent the source device and Isilon cluster from communicating directly.

Command Syntax:

isi_vol_copy_vnx --help

The source must contain a source host, a colon, and then the absolute source path name.

 isi_vol_copy_vnx <src_filer>:<src_dir> <dest_dir>
                 [-sa user: | user:password]
                 [-sport ndmp_src_port]
                 [-dport ndmp_data_port]
                 [-full | -incr] [-level_based]
                 [-dhost dest_ip_addr]
                 [-no_acl]

 isi_vol_copy_vnx -list [migration-id] | [[-detail] [-state=<state>] [-destination=<pathname>]]
 isi_vol_copy_vnx -cleanup <migration-id> [-everything [-noprompt]]
 isi_vol_copy_vnx -get_config
 isi_vol_copy_vnx -set_config <name>=<value>
 isi_vol_copy_vnx -h | -help
 Defaults:
   src_auth_user       = root
   src_auth_password   =
   ndmp_src_port       = 0  (0 means NDMP default, usually 10000)
   ndmp_data_port      = any
   dest_ip_addr        = none

Note: This tool uses NDMP to transfer the data from the source VNX to the Isilon.

Migration Steps:

  1. Configure NDMP User

Create a new NDMP user on the source VNX. Log in to the control station and run the following command:

/nas/sbin/server_user -add <new_username> -ndmp_md5 -passwd

Select the defaults when prompted and be sure to make note of the password.

  1. Determine the absolute path of your filesystems and shares

If you’re using virtual data movers it changes the root path of your filesystem.  Issue the following command to review your file systems and mount paths:

server_mount ALL

Note the the specific path for the file system that is targed for migration. The path when using a VDM will be similar to this:

FILESYSTEM1 on /root_vdm_1/FILESYSTEM1 uxfs,perm,rw

In this case the path will be /root_vdm_1/FILESYSTEM1, which will be used for the source path in the isi_vol_copy_vnx command.

    3. Determine the target Isilon Data Location

Determine the destination location on the Isilon in the /ifs/data folder where the data will migrated.  If the destination folder doesn’t exist on the Isilon, it will create it with the exact same NTFS permissions as the source.  Create the command with the following syntax:

isi_vol_copy_vnx <datamoverIP>:<source_path> <target Isilon path> -sa : <-full or -incr>

isi_vol_copy_vnx 10.10.10.10:/root_vdm_1/FILESYSTEM1 /ifs/data/FILESYSTEM1 -sa ndmpuser1: -full
  1. Migrate the Data

The command outlined above will run a full copy using the ndmpuser1 account and will prompt for a password, it does not have to be shown in plain text. The password can be specified in the command by using the appropriate syntax (), however you will still be required to follow the username with a colon.

If successful, the message “msg SnapSure file system creation succeeds” will appear, which means the NDMP session created a checkpoint successfully and is starting to copy data from that checkpoint.

Note that this does not migrate shares, just data.  Sharedupe can be utilized for that or the CIFS shares/NFS exports can be manually re-created.  It is recommended that any other data migrations on the source VNX be disabled prior to the copy so that you don’t run into performance issues.

Caveats:

  • There is no bandwidth throttling option with this command, it will consume all available bandwidth.
  • Isilon does not  support Historical SIDs in versions 8.0.0 or earlier, which may result in permission issues due to being unable to resolve historical SIDs post migration from other platforms (see KB468562).  If SIDHistory is in use on the source, then this is not the proper tool.   From the comments section, please note that OneFS does support Security Identifier (SID) history beginning in 8.0.1 and later releases, and 8.0.0.3 and later releases (see latest docu44518_Isilon-Supportability-and-Compatibility-Guide).
  • If fs_dedupe is enabled on the Celerra or VNX, you will need to change the backup threshold to zero for each filesystem.  This means that when sending the data over NDMP send the full file, not the compressed or de-duped version.  Note that there is a risk here of inflating existing backup sets if they are being done over NDMP.
  • On the source the account performing the copy needs local administrator or backup operators permissions to the source CIFS Server, and full control over the source share.
  • Standard NDMP backups and isi_vol_copy_vnx can affect each other and the data backed up by the two NDMP clients.  See KB187008 for a workaround.

Best Practices:

  • isi_vol_copy target data use

Do not touch the data on the target Isilon until after the isi_vol_copy has completed.

Why:  This will create problems and you may have to re-do a full copy.

  • Simultaneous isi_vol_copy use

Do not execute multiple isi_vol_copy commands going to the same target, i.e. don’t have all your isi_vol_copy migrations going to the same target directory.

For example:
filer1:/vol/sourcedir -> isilon:/ifs/data
filer2:/vol/sourcedir2 -> isilon:/ifs/data

Why:  Creates problems for the copy process and may require remediation after migration.

Instead: Use an additional directory level:
filer1:/vol/sourcedir -> isilon:/ifs/data/filer1/sourcedir
filer2:/vol/sourcedir2-> isilon:/ifs/data/filer2/sourcedir

If consolidation is required, this can happen after the data is migrated and any potential merging of identically named subdirectories can be addressed.

  • isi_vol_copy use

isi_vol_copy is optimized to streams as much data as possible across a network, always monitor load on the source and target systems for potential impact.

Why:  Since isi_vol_copy is optimized to streams as much data possible, don’t overwhelm older source systems and create potential link saturation or disk problems especially if there are users connected and attempting to access files.

  • isi_vol_copy limits

Recommend less than 40 million files per volume transfer when using isi_vol_copy.

Why:  All programs have limits and this is the recommended maximum when using isi_vol_copy for each individual transfer.  Larger source volumes should be broken up into smaller chunks (i.e. use a separate isi_vol_copy stream for multiple subdirectories instead of one large transfer of an entire volume).

  • Optimize the network for the migration traffic

Optimize the migration network path; look to limit other production traffic from this network, limit network devices the traffic traverse (firewalls, IDS etc.). Ideally look to create a dedicated private migration network that can be optimized for only the migration traffic.

Why:  Separating the migration traffic from other network traffic will allow for maximum throughput and reduce potential impact to existing production traffic by limiting network saturation.

6. Use a specific migration account or account with group membership that has the required access to all source and target data, i.e. root.

Why:  Using a dedicated account will allow for oversight and management of the migration data access.  It will also allow for the separation of migration tasks and users from other production accounts

  • Watch out for root_squash

On the source cluster exports sometimes restrict access by using root_squash to prevent root users connecting remotely and having root privileges.  But this is something we need for migrating data.  Use the option, “no_root_squash” to turn off root squashing.

Why:  Must have root access (or equivalent) to migrate all files and directories.

  • NFS Exports

Create the new Isilon NFS exports and permissions prior to the data migration.

Why:  This will allow the creation and setup of the exports and export permissions prior to data migration and cutover for initial testing and access validation

Troubleshooting Tips:

  1. Checkpoint Creation on the VNX

The most common issue when running the isi_vol_copy_vnx command is with checkpoint creation on the source VNX.

If you are receiving a message that’s similar to msg SnapSure file system creation failed during a copy session, the command is failing to create a snapshot of the source file system. It could be due to many reasons including a lack of available disk space. Try manually creating a snapshot on the source VNX file system to see if it fails, below is the syntax to do so:

#fs_ckpt Test_FS -name Test_FS-ckpt01 –Create
  1. Permission or Connection Issues.

In general the error message itself will be self explanatory. Make sure you are using the correct credentials for the NDMP user in the migration command. The user should have sufficient rights on the source system and target systems. The logged in user should be able to create and modify directories and the files contained within.  As an example, in the case below, NDMP Port 53169 is blocked between the VNX and the Isilon.  Opening the port on the firewall resolved the issue.

ISILON568-1# isi_vol_copy_vnx 10.10.10.10:/Volcopytest /ifs/data/Volcopytest -sa Volcopytest:Volcopytest -sport 53169 -full -dhost 10.10.10.11
system call (connect): Connection refused
Could not open NDMP connection to host 10.10.10.10
isi_vol_copy_vnx did not run properly

3.  32bit Unix Application Issues.

If your app is 32bit, 32bit settings on the new NFS export must be enabled.

ISILON# isi nfs exports modify EXPID –zone=NFSZone –return-32bit-file-ids=yes

Replace the EXPID with the ID of the target export, verify the bit settings by viewing the export.

ISILON# isi nfs exports view EXPID –zone=NFSZone | grep -i return

ISILON# _

4. Snapshot creation on the target Isilon array.

Snapshots can fail for many reasons, but most often it’s due to lack of available space. In the example below the snapshot creation failed because there was an existing snapshot with the same name.

ISILON568# isi_vol_copy_vnx VNX-SERVER3:/Test_FS/NFS01 /ifs/data/Test_NFS01 -sa ndmp:NDMPpassword -incr
Snapshot with conflicting name ‘isi_vol_copy.011.1.snap’ found. Remove/Rename the conflicting snapshot to continue with further migration runs.
snapshot already exists
ISILON568-1#

Either delete or rename the existing snap to resolve the issue.  In the example below the snapshot was deleted.

ISILON568-1# isi snapshot snapshots list| grep isi_vol_copy.011
134 isi_vol_copy.011.0.snap /ifs/.ifsvar/modules/isi_vol_copy/011/persistent_store
136 isi_vol_copy.011.1.snap /ifs/.ifsvar/modules/isi_vol_copy/011/persistent_store
ISILON568-1#

ISILON568-1# isi snapshot snapshots delete –snapshot=134
Are you sure? (yes/[no]): y
ISILON568-1#
Advertisements

6 thoughts on “Using isi_vol_copy_vnx for VNX to Isilon data migration”

  1. Your remark about SIDhistory on Isilon is incorrect.

    OneFS supports Security Identifier (SID) history beginning in 8.0.1 and later releases, and 8.0.0.3 and later releases (see latest docu44518_Isilon-Supportability-and-Compatibility-Guide).

    😉

  2. Hello,

    I have a few questions regarding NAS migrationfrom Netapp/VNX to Isilon Cluster, Can you please help me on these one?

    Disable all SmartQuotas quotas. Do not re-enable quotas until the migration is completed. – When we are creating a SMB share in Isilon cluster it should same size and quota limit like Netapp/VNX share right? If we don’t disable the SmartQuotas for that share during migration what will happen?
    How is the Bandwidth will go over Source to Target ? Will it use management IP?
    How is the resource utilization will be in source and target systems and how can we record resource utilization during Full copy and incremental copy?
    During Cut-Over time, Restrict access to the source device or set the data on the source device to read-only – How can we set read only to CIFS and NFS in Netapp and VNX?

     

    1. When we are creating a SMB share in Isilon cluster it should same size and quota limit like Netapp/VNX share right?
      That is a design decision on your part. It would make sense to set the quotas on the Isilon to the same size or larger than you have them set on the NetApp/VNX file systems.

      If we don’t disable the SmartQuotas for that share during migration what will happen?
      This is a recommendation from Dell EMC in their “OneFS 8.0.0 Built-In Migration Tools Guide” document that is available for download on their support site. It does not specify the reason behind it, they simply recommend it in the “Cluster Preparation” section of the guide.

      How is the Bandwidth will go over Source to Target? Will it use management IP?
      It can use whatever interface (IP) you’d like it to use as long as both sides can communicate, but it’s unlikely you’d be using the management interface for data access. The command line syntax should include the source and destination IP addresses (The syntax I used when I ran it: isi_vol_copy_vnx source_IP:/source_path /ifs/destination_path -sa ndmp:password -full -dhost target_IP). I set up a direct 10GB FC connection between one node on the Isilon and the data mover I was migrating from on the VNX and used a private non-routable IP range (10.x.x.x). It can consume 100% of the bandwidth of the link you use and there is no throttling. That’s why I set up a private link, I then did not have to worry about bandwidth contention.

      How is the resource utilization will be in source and target systems and how can we record resource utilization during Full copy and incremental copy?
      From the CLI, you can use isi-statistics on the Isilon. server_stats on the VNX, and statistics on NetApp OnTAP. Going into more detail would be beyond the scope of this quick reply. You can of course also use whatever storage resource management tools you may have installed already, like ViPR SRM, Control Center, OnCommand Insight, Solar Winds SRM, etc.

      During Cut-Over time, Restrict access to the source device or set the data on the source device to read-only – How can we set read only to CIFS and NFS in Netapp and VNX?
      On the VNX, in Unisphere, go to Storage | File Systems, switch to “Mounts” tab, select the file system mount path, click on properties, and change the status from read/write to read only. That would cover SMB and NFS access, as it sets it to read only at the file system level, not the share or export level. I don’t have access to a NetApp array to check that process.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.