Server Guide part 2: Affordable and Manageable Storage

SAS layers in the real world

The SAS layered model is much more than just theory. Below you can see how our LSI Logic card is able to combine a SATA RAID volume with two SATA disks and a SAS RAID volume with four 15000 rpm Fujitsu SAS hard disks in the same Promise Vtrak 300JS JBOD storage rack. Don't let the word JBOD confuse you: it does not refer to the RAID level "Just a Bunch Of Disks". It means that the storage rack doesn't have any raid capabilities, and that the RAID implementation needs to be on the HBA card. Also notice the word "expander" in the screenshot below, which refers to the circuit which is basically a SAS crossbar switch and a router at the same time. We will discuss the expander's functionality in more detail.


The bandwidth between the HBA and the SAS storage racks is also much higher than similar SCSI-320 configurations. Let us take a look at the external port of our HBA card. We find an industry standard 4x wide-port SAS connector, which combines four 300 MB/s SAS cables in one "SFF-8470 4X to 4X" external SAS cable. This gives us 1.2 GB/s bandwidth to any external storage rack, four times more than what would be possible with SCSI-320.


When we combine this wide port with a storage rack that has a built-in expander, magic happens. We get the advantages of the parallel SCSI architecture and those of the Serial SATA architecture. To understand why we say that "magic happens", let us take a concrete example. The storage rack that we tried out was the Promise Vtrack J300S, which can contain up to 12 3.5 inch drives. Take a look at the picture below.


In this configuration you get a big 1.2 GB/s pipe, called a SAS wide port, to your storage rack. If we use SATA without a port multiplier, we would only be able to attach four drives: each drives needs its own cable, its own point to point connection. If we use SATA with a port multiplier, we are able to use all 12 drives, but our maximum bandwidth to our HBA would be limited to 300 MB/s. This is OK for transaction based traffic, but it might introduce a bottleneck for streaming applications.

With SCSI we would be able to use up to 14 drives without a port multiplier, but we would have to add another SCSI HBA card if we ran out of space with our 14 SCSI drives. As hot swap PCI slots are very rare and expensive, this could mean we would have to take the server and storage down for some time.

Thanks to the built-in expander of the Vtrack J300s, not only can we address 12 drives with only 4 point to point connections, but we can also use up to four cascaded storage racks. So in this case, Promise has limited the SAS routing and SAS traffic distribution to 48 (4x 12) drives. In theory SAS can support up to 128 devices, but in practice HBA is limited to about 122 drives.

In other words SAS combines all the advantages of SATA and SCSI, without inheriting any of the disadvantages:

• You can use up to 122 drives instead of 14 (SCSI)
• You do not have to use a cable for each drive (SATA-1)
• Thanks to wide ports, the bandwidth of several channels can be combined into one big multiplexed pipe
• Thanks to the serial signalling of SAS, the bandwidth of wide ports will double in 2008 (4x600 MB/s instead of 4x 300 MB/s)
• You only need one SAS cable to attach external storage
• You can use cheaper SATA and fast SAS drives in the same storage rack

It is now crystal clear why SAS will completely replace SCSI-320 and why SAS is pretty popular among the drive manufacturers. Seagate, Fujitsu-Siemens and Hitachi have entered the SAS drive market with new SAS drives. Western Digital is the exception and has no SAS disk plans at all, but that doesn't mean they don't see a future for SAS. Western Digital views SAS racks as an ecosystem, a breeding place for their Raptors (10000 RPM Enterprise SATA disks). If it was up to Western Digital, SAS would only exist as cables and storage racks, filled with their Raptor disks.

Serial Attached SCSI or SAS has been available since late 2005 and is a logical evolution of the old parallel SCSI-320. However, it is quite revolutionary as it offers in some ways functionality that was only available with high end fibre channel storage.