Sean Wallbridge
The following is a rehash of an internal blog post I did for the office. ESXi allows for different types of datastore connections and the following is the “quick and dirty” on those connections. Please do NOT consider this the be all and end all regarding ESXi and datastores, it is more like a “Coles Notes”.
ESXi allows for multiple methods to connect to a "datastore". The three methods we are most interested in for our customers are a LOCAL datastore, an NFS datastore or an iSCSI datastore.
The most obvious datastore is the LOCAL one created out of local disk on an ESXi server. The rules surrounding a local datastore are simple – the controller that manages the disks MUST be on the VMware HCL and the datastore will be created with default settings (1MB index blocks so a total max file size of 256GB with in the datastore) if ESXi is installed against the internal disks. This is something that you need to keep in mind IF there is a large amount of internal disk that will be used for a datastore and you have a requirement for VMDK’s that will be larger than 256GB. If you require settings other than the default ones for the local datastore then you will need to install ESXi to a USB stick and boot form the stick. Once booted from the stick you’ll be able to create a local datastore using whatever parameters that suit your purposes. Keep in mind that you will then need to always boot from the USB stick as ESXi is not installed against the local disk. Also keep in mind that if you break up the local disk into multiple RAIDsets (you DO use RAID, correct???) then you can install ESXi into one of the RAIDsets and still have the ability to apply your own configuration parameters to datastores created in the other RAIDsets.
An iSCSI datastore is usually created after ESXi has been installed on a server. The iSCSI data source (a SAN, a QNAP, a Windows server, whatever) has to be configured to allow access by the ESXi server. Once ESXi and the iSCSI source are connected, ESXi can format the datastore as per your required parameters. An iSCSI datastore is created within an iSCSI container which is really (as far as ESXi is converned) a pool of RAW storage. ESXi is capable of totally managing the internal "structure" of the datastore and parameters such as max file size are completely controlled by ESXi. Both an internal datastore and an iSCSI datastore can support multiple VM’s using ESXi’s built in file contention manager. The advantage of the individual datastores (as in individual iSCSI containers) is that resource contention is managed by the device but it does create an extra layer of required management (lots of iSCSI connections to track). FibreChannel SAN’s connectivity and datastores work in much the same fashion as iSCSI.
The last methodology for connection to a datastore is NFS. NFS is near and dear to an old Unix geek like me as it predated all of the other network connection protocols for data sharing. It is also the easiest network protocol for ESXi to use for a networked datastore. You simply set up the NFS share on your NFS box (a network filer, a QNAP, a Unix, Linux or OS X box), set the appropriate permissions and then connect ESXi to the NFS share (in much the same manner as you would connect a Windows client to a Windows Server share). ESXi cannot format the NFS share as it is not RAW storage, the NFS share is actually a connection into a live, formatted native filesystem on the NFS host machine. Therefore, ESXi cannot really manage anything within the NFS store; it just reads and writes files like any other NFS client. This means that a given NFS store cannot support multiple VM’s accessing it as the ESXi contention manager cannot manage the files. You can create multiple NFS shares on the host server and then map the NFS connections per individual VM and then the NFS host will manage access but it is not as efficient as ESXi’s contention manager or the contention management provided by a SAN.
The other big difference between NFS and iSCSI is one of access to the files outside of ESXi. As iSCSI is "container based" the host device (like a QNAP) can’t "see" inside the container, at least not directly. Using a QNAP as an example, the Web File Manager will display the contents of all the filesystems and directories on the box but will ONLY display an iSCSI container as one filesystem object; you can’t see the contents of the iSCSI container without using an iSCSI connection into the box. NFS, on the other hand, is just a protocol layered over top of an already existing filesystem. If you use the Web File Manager on a QNAP to look at a directory that is being presented as an NFS share you will be able to see the files within the folder without a problem. Obviously, NFS is less "secure" than iSCSI from that point of view so access to the actual file folders on the NFS host has to be monitored and restricted.
I hope this helps understand the whole datastore connectivity thing.