FreeBSD Raid

Disclaimer: This is very old information kept here for archival purposes. More current information can certainly be found elsewhere on this site or others.

When trying to setup a RAID subsystem for one of the servers at a company I work for, I had to do a lot of research on exactly which RAID controllers do and don't work. This is a summary of what I found by digging through the mailing lists, reading vendors notes, and so on. Hopefully others will find this usefull. I obviously haven't actually tried all of these devices, so this is mostly based on info I've gleaned from elsewhere and I can't guarantee the accuracy of it. If you notice anything on this page being out of date or wrong inform old address removed.

Manufact Model Comments Price *
Software Based Solutions
FreeBSD CCD Concatenated Disk Driver. Raid 0 and Raid 1 support only. See the CCD(4) and ccdconfig(8) man pages. $0
FreeBSD vinum Logical Volume Manager. Supports Concatenated plexes, Striped plexes, and RAID-5 plexes See the vinum(4) and vinum(8) man pages. Also http://www.lemis.com/vinum.html has some good information. NOTE: The RAID 5 portion is copyright Cybernet, Mail Cybernet directly for details. $0 (Raid 0/1)
$???? (Raid 5)
HOST Based Adapters
DPT PM3334UW All of the older DPT controllers are reported to work. The DPT SmartRAID V controllers DO NOT currently work, as they are I20 based and FreeBSD does not currently support the I20 spec (due to the restrictive licensing). For updates watch Simon Shapiro's web site. I do know that Simon is working on the SmartRAID V cards, no ETA yet though. 1 ch - $1241USD (S)
2 ch - $1431USD (S)
3 ch - $1636USD (S)
DPT PM3334UW/2-BB PM3334UW two channel PCI-to-Ultra Wide SCSI RAID Adapter (requires 4M Ram) $1431USD (S)
Compaq Smart 3200, 2/DH, and 2/P PCI adapters See http://www.doc.ic.ac.uk/~md/ida/ for details, These are NOT currently part of the standard distro, but must be added manually to the kernel (doesn't look to hard). Not a lot on how well this works, anyone?? See your Compaq
Reseller
SCSI-SCSI Based Adapters
All the SCSI-SCSI adapters sit in between the host adapter and the SCSI subsystem. It is recommended that you have a high performance SCSI controller in the host to get maximum throughput. People have reported good luck with Adaptec 2940/3940 controllers and Mylex (NON RAID!) controllers.
Infortrend The nice people at Media Integration have graciously given the BSD community special pricing on SCSI-SCSI Raid controllers.


Notes:
Here are some gotcha's with the Infortrend controllers (no show stoppers):

  • Make sure you have good and matching simms. The controller will hang with some memory chips. I've had bad luck with EDO (yeah yeah use ECC if you can afford it).
  • At least some Adaptecs controllers seem to think the UW (20M) controller can do 40M, while this may work some of the time, it can occasionally give errors. Lock the host controller speed/width to match the RAID SCSI device.
Mylex DAC960SX Dual controller hot failover, 16-265M cache, 1 channel to host, two channels to drives.
DACSUI-2W4M 8M cache (to 64M), 1 host, 5 SCSI $4309USD (L)
$3200USD (S)
DACSUI-5W-8M 4M cache (to 64M), 1 host, 2 SCSI $3236USD (L)
$2352USD (S)
DACSXIH-2W16M 16M cache (to 256M), Dual controller failover, 1 host, 2 SCSI, expandable $2995USD (L)
$2528USD (S)
DACSXIH-5W-64M 32M cache (to 256M), Dual controller failover, 1 host, 5 SCSI $5295USD (L)
$3331USD (S)
DAC960SU and DAC960SX I couldn't find anyone selling these, so I'm just noting they exist.
A note on CMD controllers A lot of people have been reporting unresolvable problems with the CMD controllers. A couple people have reported success, but not a lot. I can't specifically NOT recommend these controllers, but I can't really recommend them right now either. If you've had positive or negative experience with these controllers let me know and I'll anotate this as we go.
CMD CRD-5440 up to 256M RAM, Raid 0,1,0+1,5, 2-3 Disk and 1-3 host interfaces

ORD-5440-001 - with four UltraSCSI/Wide/Single-Ended channels any of which may be configured as host or disk channels

ORD-5440-002 - with two UltraSCSI/Wide/Differential host channels and two UltraSCSI/Wide/Single-Ended disk channels
CMD CRD-5500-001 Base CRD-5500 with no host or disk channel modules
CMD CRD-5530-000 Fast/Single-Ended SCSI Disk channel module
CMD CRD-5540-002 UltraSCSI/Wide/Single-Ended channel module
CMD CRD-5560-002 UltraSCSI/Wide/Differential channel module

Notes on Prices:

All prices are either list (L) price or estimated
street price (S) (wholesale with a 20% markup, yes I
know many can get better than that).

Network Buyers Guide List of SCSI Raid controllers.

CAVEATS

This is for CMD and Infortrend SCSI-SCSI systems:

One caveat is that if you use the Adaptec 2940/3940 card, disable
"allow disconnects."

The only FreeBSD issue you need to be aware of is that if you do
decide to use partitions, you need to make a little change to the
kernel to enable multiple LUN probing.  This is because FreeBSD views
partitions as being on separate LUNs.

See http://www.freebsd.org/handbook/handbook122.html#257 for more
details on enabling multiple LUN probing.  

FYI, I didn't add the struct as suggested in the handbook.  I just
changed the line in /sys/scsi/scsiconf.c:

maybe_more = 0; /* by default only check 1 lun */

to this:

maybe_more = 1; /* by default check all luns (time for a coffee break!) */



Stuff I haven't catagorized yet... 

Yes, we're using RAID solutions in production systems. We build scalable 
telecom systems, e.g. voice/fax-mailbox systems. I think the hardware 
we're using are DTC SCSI RAID controllers. They handle up to 7 scsidisks, 
RAID 0-5, I think. Hotswapping, and I think they could connect to two 
hosts as well. And they have a serialport, so you could control it using 
a VT-100 terminal, as well as using the display and buttons on the front 
for setting up the controller. Oh yeah, it has an audiable alarm :)
I've no idea about the prize anyway. But it's good stuff.




CMD Technology Inc. (http://www.cmd.com/) also makes SCSI-to-SCSI RAID
controllers.  They also have a turnkey system, the Daytona, which
includes a RAID controller and a drive enclosure with redundant power
supplies, hot-swappable drive canisters, and an extra slot for a tape
drive. 

The CMD CRD-5500 RAID controller is their top of the line product.  It
supports RAID levels 0, 1, 1+0, 4, and 5, and comes standard with 16MB
cache RAM (expandable to 512MB with 72-pin SIMMs).  It comes in the
standard 5.25" full-height form factor, so you can put it into any
standard case which has full-height bays.  This beast has 9 slots, in
which you can put the following SCSI interface modules:

CRD-5530 10MB/s single-ended FAST/SCSI-2 (disk channel only), $395
CRD-5540 20MB/s single-ended FAST/WIDE SCSI-2 (disk/host channel), $495
CRD-5560 20MB/s differential FAST/WIDE SCSI-2 (disk/host channel), $595
   **All prices are list prices

The CRD-5500 by itself (no modules) lists for $3,525.

The CRD-5500 controller supports any combination of interface modules,
with a maximum of 4 host channels and a maximum of 8 disk channels.
Yes, this means that this one RAID controller can be connected to
multiple FreeBSD machines.  I'm not exactly sure what the limitations
are with this, though.

CRD-550-015 CRD-5500 with 1 single-ended FAST/WIDE SCSI-2 host module
and 5 FAST SCSI-2 

The CRD-5500 can also be hooked up to another CRD-5500 using the
Failover kit (P/N: AAK-005500-000) for redundancy.

The Daytona contains a CMD CRD-5300 RAID controller and an enclosure
with support for either 4 3.5" half-height disks or 6 3.5" low profile
disks.  It comes with 16M cache, and has hot swappable redundant fans
and power supplies.  The following is a table of model numbers and
list prices for the Daytona:

HWS-005931-141  4 3.5" half-height canisters, single-ended FAST SCSI-2
list $5,995

HWS-005931-161  6 3.5" low profile canisters, single-ended FAST SCSI-2
list $5,995

HWS-005931-241  4 3.5" half-height canisters, single-ended FAST/WIDE SCSI-2
list $6,118

HWS-005931-261  6 3.5" low profile canisters, single-ended FAST/WIDE SCSI-2
list $6,118

HWS-005932-141  4 3.5" half-height canisters, differential FAST SCSI-2
list $6,395

HWS-005932-161  6 3.5" low profile canisters, differential FAST SCSI-2
list $6,395

HWS-005932-241  4 3.5" half-height canisters, differential FAST/WIDE SCSI-2
list $6,518

HWS-005932-261  6 3.5" low profile canisters, differential FAST/WIDE SCSI-2
list $6,518

  ** All Daytonas have 1 host port and 2 SCSI channels, and are
     expandable to 128M of cache

There will supposedly be a slave drive box (to be available soon) so
you can attach more drives to the Daytona.  No word on when that might
be available, though.

As for availability, CMD recently announced that it is distributing
the Daytona through Tech Data (large distributor), so it shouldn't be
*that* hard to find.

I apologize for the lengthy message, but some of this info isn't on
their web site (I had to get them to fax it to me).

This is your day.  I'm giving the following a spin.  I'm not much into C,
but I think I got all the fields right:

>            /*  Added to fix CRD-5440's lack of tags */
>            { T_DIRECT, SIP_MEDIA_FIXED, "CMD TECH", "CRD-5440*", "*"},
>               /*quirks*/0, /*mintags*/22, /*maxtags*/30
>       },
>       {

Oh, I should have been a little more clear there.  Go into
sys/cam/cam_xpt.c, and look around near the top of the file.  You'll see a
quirk table with entries like this:
        {
                /* Reports QUEUE FULL for temporary resource shortages */
                { T_DIRECT, SIP_MEDIA_FIXED, quantum, "XP39100*", "*" },
                /*quirks*/0, /*mintags*/24, /*maxtags*/32
        },
And, as far as numbers go, I'd suggest that you use exactly the numbers
that we use for the Atlas II, above.  They'll apply to both luns of your
RAID controller, so the available tag openings will be evenly split between
the two.

Adjile?????

http://www.caeneng.com/
http://www1.asacomputers.com/
http://www.dejanews.com/=zzz_maf/dnquery.xp?search=thread&svcclass=dnserver&recnum=%3cXFMail.981026204544.shimon_simon-shapiro.org@ns.sol.net%3e%231/1

CMD - 64 (32 per channel/total? I'm getting conflicting data here - check) tagged commands[CMD has some problems due to this], Mylex 128 tagged commands, the
Infortrend has 1024 (wow) tagged commands in current firmware.

Kingston DS400 RM - $572 wholesale
DE100I-SWD/B Carrier - $175 wholesale
DX100-SW/H Hot Swap option $53


STUFF I CAN'T GET/HAVEN'T GOTTEN ANY INFO ON YET.

=>  The Symbios SCSI<>SCSI raid systems we have at work have ethernet

StorComp RAID-7