Current NexentaStor Disk Layout

Here is a current view of how my NexentaStor NAS is configured from a disk and controller perspective. I will be modifying this some here soon. Go here to take a look at what parts were used to put this all together.

What you see here is that starting on the top left going down and then again in the next row of disks and so on is what makes up our zdevs (mirror groups) within each zpool.

NexentaStor-NAS-Drive_Layout

There are 3 dual port sas controllers and each row alternates across controller ports. Which means that each disk that is part of a disk group is connected to a different controller so if a controller is lost the zpool does not fail. Each different color represents a controller path which you can reference the screenshot above to see the disk layout again.

NexentaStor-NAS-Controller_Layout

This is what the front of the actual enclosure looks like to put the above layouts in perspective.

09-30-10

 

Enjoy!

3 thoughts on “Current NexentaStor Disk Layout

  1. Nexenta 1x IBM Host 24Gb of Ram 8x Processors V 3.1.3.5

    2x 68GB 15k SAS disks Nexenta OS

    5x450Gb 15K SAS Datastore1 Striped Mirror 1 Hotspare

    1x60gb SSD as L2arc for Datastore1 Compression and Dedupe on

    12x146gb 10k as Datastore2 Striped `mirror Compression on

    dataset1 is using an onboard Raid card so all drives are passed through using a Raid0

    Dataset2 are SCSI drives and are correctly passed through

    these are presented as 3 ZVOL's

    2x300Gb on Dataset1

    1x800Gb on dataset2

    Networking on the Nexenta side

    4x1gb Nics

    e1000 VLAN 10 MTU 9000 ISCSI1

    e1000 VLAN 20 MTU 9000 ISCSI2

    AGGR1 LAN1 MTU 1500 SMB/MGMT

    Vmware side

    IBM x3455

    QLE 406xc ISCSI Hardware adapters 1 in each ISCSI vlan MTU set to 9000

    Path set to Round Robin

    Switch is a Linksys SRW2048 with Jumbo frames enabled

    • Sorry didn't explain with my config.

      I am seeing fairly poor iscsi performance and am seeing during clone operations etc that the storage gets dropped by VMware. It appears to be some sort of lock but all of the datastores go offline. So far all I can see to fix is a Nexenta restart

    • @James Let me ask a few questions that standout initially.
      1. Have you disabled dedupe and experienced the same issue? I would definitely turn dedupe off.
      2. Have you tested without jumbo frames and experienced the same issue? Making sure everything is back to 1500mtu?
      3. Have you tested using the Software iSCSI stack from the vSphere side instead of using hardware iSCSI? If so did you experience the same issue?

Leave a Reply

Your email address will not be published. Required fields are marked *

*