Google Git
Sign in
android / platform / external / sg3_utils / 7502647d46389b2e0e659f658b51f56abd1f9511 / . / doc / sg_dd.html
blob: 0250541ed522120c1408781a5514fa4138186a45 [file] [log] [blame]
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>Linux sg_dd utility (a dd variant)</title>
<meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1">
<meta name="keywords" content="Linux, SCSI, sg_dd, dd variant">
<meta name="GENERATOR"
content="Mozilla/4.79 [en] (X11; U; Linux 2.5.31 i686) [Netscape]">
</head>
<body alink="#ff0000" background="paper.jpg" bgcolor="#ffffff"
link="#0000ff" text="#000000" vlink="#000080">
<center>
<h1><a class="mozTocH1" name="mozTocId228087"></a> The&nbsp; Linux
sg_dd utility<br>
</h1>
</center>
<a href="#Conclusion"></a>
<ol id="mozToc">
<!--mozToc h1 1 h2 2 h3 3 h4 4 h5 5 h6 6-->
<li><a href="#mozTocId228087"> The&nbsp; Linux
sg_dd utility
</a>
<ol>
<li><a href="#mozTocId370866">Introduction</a></li>
<li><a href="#mozTocId352668">dd like features</a></li>
<li><a href="#mozTocId703997">sg_dd extras</a></li>
<li><a href="#mozTocId657084">Retries</a></li>
<li><a href="#mozTocId437677">Continue on error
(coe)</a></li>
<li><a href="#mozTocId566094">Recovered errors</a></li>
<li><a href="#mozTocId899152">Verbose</a></li>
<li><a href="#mozTocId443896">sgp_dd, sgm_dd and
sg_read</a></li>
<li><a href="#mozTocId157981">Conclusion</a></li>
</ol>
</li>
</ol>
<h2><a class="mozTocH2" name="mozTocId370866"></a>Introduction</h2>
The <span style="font-weight: bold;">sg_dd</span> utility is a variant
of the standard Unix command <span style="font-weight: bold;">dd</span>
which
copies files. The sg_dd utility is specialized for devices that use the
SCSI command set in the Linux operating system. The sg_dd utility is
found in the <a href="sg3_utils.html">sg3_utils</a> package which
targets the linux kernel 2.4 and
2.6 series and has ports to various other operating systems.<br>
<br>
Only block oriented SCSI peripheral device types are supported by the
sg_dd utility. Obviously
those device types that deal with enclosures and medium changers don't
have addressable blocks and are not supported. The supported device
types are direct access devices (e.g. disks) and cd/dvd devices. The
SCSI tape device type is not be supported (but could be to a limited
degree). When instructed, the sg_dd utility issues SCSI
commands that are defined in SPC-3 (primary commands), SBC-2 (commands
for direct access devices (e.g. disks)) and MMC-4 (commands for CD/DVD
devices). These SCSI command sets can be found at <a
href="http://www.t10.org/">www.t10.org</a>
. <br>
<br>
It is becoming common for non-SCSI devices (e.g. ATA disks) to appear
as SCSI devices to an operating system via a protocol conversion in an
external enclosure and via some transport such as USB or IEEE 1394. The
sg_dd utility should work with most of these devices as it tends to use
exactly the same SCSI commands that the normal block layer would use.
However, advanced options (e.g. using the cdbsz=16 and iflag=fua) most
likely will be ignored. Apart from CD players over 10 years old, almost
all CD/DVD players use the Multi Media Command set (MMC or&nbsp;
MMC-2,3,4,5) as their native command set. The fact that the most common
transport for CD/DVD drives is ATAPI (i.e. the ATA packet interface) is
irrelevant to
the
sg_dd utility; they use SCSI command sets.<br>
<br>
This page outlines the features of the sg_dd utility version 5.60 found
in the <a href="sg3_utils.html">sg3_utils</a> version 1.23 package.<br>
<h2><a class="mozTocH2" name="mozTocId352668"></a>dd like features</h2>
The basic syntax of the sg_dd utility is the same as the dd command in
Unix. That said, the syntax of the dd command in Unix is different from
almost all other standard Unix commands. Those familiar with the dd
command
should not be too surprised by the syntax and semantics of the sg_dd
utility. Those not familiar with the dd syntax should be very careful,
especially with the 'of=' and 'seek=' options, both with dd and sg_dd.
The recent GNU implementation of the dd command is used as a reference
point.<br>
<br>
The main options of dd are:<br>
<br>
<table style="width: 100%; text-align: left;" border="1" cellpadding="2"
cellspacing="2">
<tbody>
<tr>
<td style="vertical-align: top;"><span style="font-weight: bold;">Standard
dd options</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">Default</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">Brief
description</span><br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">bs=<span
style="font-style: italic;">BS</span><br>
</td>
<td style="vertical-align: top;">512<br>
</td>
<td style="vertical-align: top;">Number of bytes in each block.<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">count=<span
style="font-style: italic;">COUNT</span></td>
<td style="vertical-align: top;">blocks in <span
style="font-style: italic;">IFILE</span><br>
</td>
<td style="vertical-align: top;">Number of blocks to copy.<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">if=<span
style="font-style: italic;">IFILE</span></td>
<td style="vertical-align: top;">stdin<br>
</td>
<td style="vertical-align: top;">file (or device) to read from.<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">of=<span
style="font-style: italic;">OFILE</span></td>
<td style="vertical-align: top;">stdout<br>
</td>
<td style="vertical-align: top;">file (or device) to write to.
sg_dd doesn't bother writing anything if <span
style="font-style: italic;">OFILE</span> is <span
style="font-family: monospace;">/dev/null</span> or
. (period).<br>
</td>
</tr>
</tbody>
</table>
<br>
<br>
When either dd or sg_dd are given these options with suitable
arguments, they will copy
(<span style="font-style: italic;">BS</span> * <span
style="font-style: italic;">COUNT</span>) bytes from the beginning
of <span style="font-style: italic;">IFILE</span> to the
beginning of <span style="font-style: italic;">OFILE</span>. One
restriction that sg_dd imposes
when either the <span style="font-style: italic;">IFILE</span> or <span
style="font-style: italic;">OFILE</span> are accessed
via SCSI commands is that <span style="font-style: italic;">BS</span>
(i.e. the block size) must match that of the
device. Further, if both the <span style="font-style: italic;">IFILE</span>
and <span style="font-style: italic;">OFILE</span>
are accessed via SCSI commands then <span style="font-style: italic;">BS</span>
must match that
of both devices. <br>
<br>
The following extensions are found in sg_dd. An <span
style="font-style: italic;">IFILE</span> of
"-" is interpreted as stdin; an <span style="font-style: italic;">OFILE</span>
of "-" is
interpreted as stdout while an <span style="font-style: italic;">OFILE</span>
of "." is
interpreted as <span style="font-family: monospace;">/dev/null</span>.
[dd interprets input and output file names of
"-" literally; dd interprets an output file of "." as the current
directory and will not accept it.] The sg_dd utility does not truncate
the <span style="font-style: italic;">OFILE</span> before starting the
copy (the dd command does
if it is a normal file). Hence a user may need to delete the output
file before using the sg_dd utility (if the size of that file is
greater than what sg_dd is going to copy over it).<br>
<br>
If the 'count=' option is not given then an attempt is made to
determine
the remaining blocks in the file, device or partition. If the input
file is stdin and no count is given then a copy will continue until an
EOF is detected on the input stream (or something else goes wrong). If
the 'count' option is not given then the remaining blocks on both the
input and output files are determined (if possible) and if both are
found then the minimum of
the two counts is used.&nbsp; The 'skip' option for an input file and
the 'seek' option for an output file are taken into account when
calculating the remaining number of blocks in a file, device or
partition.<br>
<br>
If the 'count' option is given then no further checks regarding the
remaining length of the input and output files are done and the sg_dd
will attempt to copy that number of blocks. A 'count=0' option is valid
and all the normal preparations are made including files being opened
but no copy takes place. Hence the 'count=0' option can be used to
check that the syntax is in order and that the files are present (see
the
"<a href="#mozTocId899152">Verbose</a>" section below).<br>
<br>
Other dd options supported by sg_dd:<br>
<br>
<table style="width: 100%; text-align: left;" border="1" cellpadding="2"
cellspacing="2">
<tbody>
<tr>
<td style="vertical-align: top;"><span style="font-weight: bold;">Standard
dd options</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">Default</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">Brief
description</span><br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">ibs=<span
style="font-style: italic;">BS</span></td>
<td style="vertical-align: top;">same as bs=<span
style="font-style: italic;">BS</span><br>
</td>
<td style="vertical-align: top;">number of bytes in each block of
<span style="font-style: italic;">IFILE</span><br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">iflag=<span
style="font-style: italic;">FLAGS</span><br>
</td>
<td style="vertical-align: top;"><br>
</td>
<td style="vertical-align: top;">similar
to option found in recent GNU dd versions, see below<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">obs=<span
style="font-style: italic;">BS</span></td>
<td style="vertical-align: top;">same as bs=<span
style="font-style: italic;">BS</span><br>
</td>
<td style="vertical-align: top;">number of bytes in each block of
<span style="font-style: italic;">OFILE</span></td>
</tr>
<tr>
<td style="vertical-align: top;">oflag=<span
style="font-style: italic;">FLAGS</span><br>
</td>
<td style="vertical-align: top;"><br>
</td>
<td style="vertical-align: top;">similar
to option found in recent GNU dd versions, see below<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">seek=<span
style="font-style: italic;">SEEK</span></td>
<td style="vertical-align: top;">0<br>
</td>
<td style="vertical-align: top;">block number (origin 0) in <span
style="font-style: italic;">OFILE</span> to commence writing<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">skip=<span
style="font-style: italic;">SKIP</span></td>
<td style="vertical-align: top;">0<br>
</td>
<td style="vertical-align: top;">block number (origin 0) in <span
style="font-style: italic;">IFILE</span> to commence reading</td>
</tr>
<tr>
<td style="vertical-align: top;">--help<br>
</td>
<td style="vertical-align: top;"><br>
</td>
<td style="vertical-align: top;">print usage message then exit<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">--version<br>
</td>
<td style="vertical-align: top;"><br>
</td>
<td style="vertical-align: top;">print version number and its
date then exit<br>
</td>
</tr>
</tbody>
</table>
<br>
If either ibs= and/or obs= are given to sg_dd then they must have the
same value
as bs=
(i.e. they are essentially dummies).&nbsp; The skip= option cannot be
used on an input stream
(e.g. stdin) while the seek= option cannot be used on an output stream
(e.g. stdout).<br>
<br>
All numeric arguments can take a multiplier suffix. From sg3_utils
version 1.13 sg_dd's multiplier prefixes have been brought into line
with those of GNU's dd. GNU's dd is found in coreutils and its
changelog notes the change to this suffix pattern on&nbsp; 2001-12-18:<br>
<br>
<table style="width: 50%; text-align: left;" border="1" cellpadding="2"
cellspacing="2">
<tbody>
<tr>
<td style="vertical-align: top;"><span style="font-weight: bold;">Multiplier</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">Meaning</span><br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">x&lt;n&gt;<br>
</td>
<td style="vertical-align: top;">*&lt;n&gt;<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">c<br>
</td>
<td style="vertical-align: top;">*1<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">w<br>
</td>
<td style="vertical-align: top;">*2<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">b<br>
</td>
<td style="vertical-align: top;">*512<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">k&nbsp; K&nbsp; KiB<br>
</td>
<td style="vertical-align: top;">*1024<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">KB<br>
</td>
<td style="vertical-align: top;">*1000<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">m&nbsp; M&nbsp; MiB<br>
</td>
<td style="vertical-align: top;">*1048576 </td>
</tr>
<tr>
<td style="vertical-align: top;">MB<br>
</td>
<td style="vertical-align: top;">*1000000<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">g&nbsp; G&nbsp; GiB<br>
</td>
<td style="vertical-align: top;">2**30<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">GB<br>
</td>
<td style="vertical-align: top;">10**9<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">t&nbsp; T&nbsp; TiB<br>
</td>
<td style="vertical-align: top;">2**40 </td>
</tr>
<tr>
<td style="vertical-align: top;">TB<br>
</td>
<td style="vertical-align: top;">10**12<br>
</td>
</tr>
</tbody>
</table>
<br>
The pattern that starts with "k" and proceeds to "m", "g" and "t" then
to "p", "e", "z" and "y"&nbsp; (not shown in above table). sg_dd only
implements as far as "p" (10**15 or 2**50). sg_dd only allows
multipliers based on "t" and "p" for 'count', 'skip' and 'seek'.<br>
<br>
sg_dd allows numeric arguments to be given in hexadecimal in which case
they must be prefixed by either "0x" or "0X". A numeric argument cannot
both be in hex and have a suffix multiplier. Hence "0x9" is interpreted
as hexadecimal 9 [not (0 * 9)==0]. This is valid "2x4x0xa" and yields
80 (but it isn't very clear).<br>
<br>
From sg3_utils version 1.19 (sg_dd version 5.46) hexadecimal numbers
can also be indicated by a trailing "h" or "H". The "h" suffix cannot
be used together with a suffix multiplier.<br>
<br>
The <span style="font-style: italic;">FLAGS</span> argument of
"iflag=" and "oflag=" is a comma
separated list of items chosen from one or more entries in this table:<br>
<br>
<table style="width: 100%; text-align: left;" border="1" cellpadding="2"
cellspacing="2">
<tbody>
<tr>
<td style="vertical-align: top;"><span style="font-weight: bold;">flag</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">sg
device</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">block
device</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">regular
file</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">comments</span><br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">append<br>
</td>
<td style="vertical-align: top;">ignored<br>
</td>
<td style="vertical-align: top;">applied on <span
style="font-style: italic;">OFILE</span><br>
</td>
<td style="vertical-align: top;">applied on <span
style="font-style: italic;">OFILE</span><br>
</td>
<td style="vertical-align: top;">use O_APPEND open flag.
Conflicts with 'seek=<span style="font-style: italic;">SEEK</span>'
when "<span style="font-style: italic;">SEEK</span> &gt; 0"<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">coe<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">applied if using SG_IO ioctl</td>
<td style="vertical-align: top;">ignored<br>
</td>
<td style="vertical-align: top;">continue on error; best effort
recovery then continue. N.B. can be used multiple times (e.g.
'iflag=coe,coe')<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">direct<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">use O_DIRECT open flag, no
effect with sg devices<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">dpo<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">applied if using SG_IO ioctl</td>
<td style="vertical-align: top;">ignored</td>
<td style="vertical-align: top;">"disable page out" set for READ
and/or WRITE SCSI commands</td>
</tr>
<tr>
<td style="vertical-align: top;">dsync<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">use O_SYNC open flag, no effect
with sg devices<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">excl<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">Use O_EXCL open flag<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">fua<br>
</td>
<td style="vertical-align: top;">applied<br>
</td>
<td style="vertical-align: top;">applied if using SG_IO ioctl<br>
</td>
<td style="vertical-align: top;">ignored<br>
</td>
<td style="vertical-align: top;">"force unit access" set for READ
and/or WRITE SCSI commands<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">sgio<br>
</td>
<td style="vertical-align: top;">ignored<br>
</td>
<td style="vertical-align: top;">use SG_IO ioctl<br>
</td>
<td style="vertical-align: top;">ignored<br>
</td>
<td style="vertical-align: top;">access block device via SG_IO
ioctl (only supported in lk 2.6 series)<br>
</td>
</tr>
</tbody>
</table>
<br>
Recent versions of GNU's dd command have these flags with similar
semantics:&nbsp; 'append', 'direct' and 'sync' (which is called 'dsync'
in sg_dd). So 'coe', 'dpo', 'excl', 'fua' and 'sgio' are sg_dd
extensions.<br>
<br>
The following dd options are <span style="font-weight: bold;">not</span>
supported by sg_dd:<br>
<ul>
<li>cbs=&lt;bytes&gt;</li>
<li>conv=&lt;keyword&gt;</li>
</ul>
Basically sg_dd does not supported the conversion features of dd. If a
conversion is really needed then sg_dd could be piped through dd (or
vice versa).<br>
<h2><a class="mozTocH2" name="mozTocId703997"></a>sg_dd extras</h2>
The extra options of sg_dd (not found in GNU's dd) are:<br>
<br>
<ul>
</ul>
<table style="width: 100%; text-align: left;" border="1" cellpadding="2"
cellspacing="2">
<tbody>
<tr>
<td style="vertical-align: top;"><span style="font-weight: bold;">extra
options in sg_dd</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">default</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">sgm_dd
</span>and<span style="font-weight: bold;"> sgp_dd</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">Brief
description</span><br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">blk_sgio=0 | 1<br>
</td>
<td style="vertical-align: top;">0<br>
</td>
<td style="vertical-align: top;"><br>
</td>
<td style="vertical-align: top;">when set access devices via SCSI
commands (SG_IO ioctl). May use iflag=sgio and/or oflag=sgio
instead.<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">bpt=<span
style="font-style: italic;">BPT</span><br>
</td>
<td style="vertical-align: top;">128 or 32<br>
</td>
<td style="vertical-align: top;">both<br>
</td>
<td style="vertical-align: top;">where <span
style="font-style: italic;">BPT</span> is the blocks per transfer
(granularity
of each IO). Default is 128 when <span style="font-style: italic;">BS</span>
&lt; 2048 (bytes) else the default
is 32. For block devices (<span style="font-style: italic;">BPT</span>
* <span style="font-style: italic;">BS</span>) is constrained by <span
style="font-family: monospace;">/sys/block/&lt;device&gt;/queue/max_sectors_kb</span>
.<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">cdbsz=6 | 10 | 12 | 16<br>
</td>
<td style="vertical-align: top;">10 or 16<br>
</td>
<td style="vertical-align: top;">both<br>
</td>
<td style="vertical-align: top;">cdb size of SCSI READ and/or
WRITE commands. Only applicable to sg devices or when the SG_IO ioctl
is being used (e.g. when blk_sgio=1). Defaults to 10 byte cdb unless
the largest address exceeds 32 bits or bpt exceeds 16 bits. In either
case a 16 cdb is used.<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">coe=0 | 1 | 2 | 3<br>
</td>
<td style="vertical-align: top;">0<br>
</td>
<td style="vertical-align: top;">sgp_dd<br>
</td>
<td style="vertical-align: top;">when non-zero, continue_on_error
for
sg devices and block devices using the SG_IO ioctl. May use iflag=coe
and/or oflag=coe
instead. An equivalent of coe=2 is 'iflag=coe,coe'<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">coe_limit=<span
style="font-style: italic;">CL</span><br>
</td>
<td style="vertical-align: top;">0<br>
</td>
<td style="vertical-align: top;">&nbsp;<br>
</td>
<td style="vertical-align: top;">number of consecutive "bad"
block errors allowed when reading and 'coe &gt; 0'. Default of 0 is
interpreted as no limit.<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">dio=0 | 1<br>
</td>
<td style="vertical-align: top;">0<br>
</td>
<td style="vertical-align: top;">both<br>
</td>
<td style="vertical-align: top;">direct IO (only via sg device
nodes)<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">odir=0 | 1<br>
</td>
<td style="vertical-align: top;">0<br>
</td>
<td style="vertical-align: top;"><br>
</td>
<td style="vertical-align: top;">O_DIRECT flag on open() when
set. Better to use iflag=direct and/or oflag=direct. </td>
</tr>
<tr>
<td style="vertical-align: top;">retries=<span
style="font-style: italic;">RETR</span><br>
</td>
<td style="vertical-align: top;">0<br>
</td>
<td style="vertical-align: top;"><br>
</td>
<td style="vertical-align: top;">number of times to retry an
error on a sg device READ or WRITE command<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">sync=0 | 1<br>
</td>
<td style="vertical-align: top;">0<br>
</td>
<td style="vertical-align: top;">both<br>
</td>
<td style="vertical-align: top;">when set, sends SYNCHRONIZE
CACHE SCSI command to <span style="font-style: italic;">OFILE</span>
if it is a sg device or accessed
via the SG_IO ioctl.<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">time=0 | 1<br>
</td>
<td style="vertical-align: top;">0<br>
</td>
<td style="vertical-align: top;">both<br>
</td>
<td style="vertical-align: top;">when set print elapsed time and
throughput calculation at the completion of the copy<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">verbose=<span
style="font-style: italic;">VERB</span><br>
</td>
<td style="vertical-align: top;">0<br>
</td>
<td style="vertical-align: top;">sgm_dd<br>
</td>
<td style="vertical-align: top;">larger <span
style="font-style: italic;">VERB</span> is the greater
the debug output. 1 and 2 print the cdbs for setup commands; 3 and 4
print the cdbs for all commands<br>
</td>
</tr>
</tbody>
</table>
<br>
<br>
The sg_dd utility examines the files it is given and treats them
differently depending on their file type. Depending on iflag=<span
style="font-style: italic;">FLAGS</span> and oflag=<span
style="font-style: italic;">FLAGS</span> settings: O_DIRECT,
O_SYNC, O_APPEND and O_EXCL flags may be added to the relevant open()
command.<br>
<br>
<table style="width: 100%; text-align: left;" border="1" cellpadding="2"
cellspacing="2">
<tbody>
<tr>
<td style="vertical-align: top;"><span style="font-weight: bold;">File
type</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">open
[when input]</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">open
[when output]</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">IO
method</span><br>
</td>
<td style="vertical-align: top;"><span style="font-weight: bold;">Notes</span><br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">normal<br>
</td>
<td style="vertical-align: top;">O_RDONLY<br>
<br>
</td>
<td style="vertical-align: top;">O_WRONLY | O_CREAT<br>
</td>
<td style="vertical-align: top;">Unix read() write()<br>
</td>
<td style="vertical-align: top;">N.B. A normal output file is
overwritten (not truncated).<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">stdin or stdout<br>
</td>
<td style="vertical-align: top;">[do nothing]<br>
</td>
<td style="vertical-align: top;">[do nothing]<br>
</td>
<td style="vertical-align: top;">Unix read() write()<br>
</td>
<td style="vertical-align: top;">hence open() flags have no
effect (e.g. 'oflag=direct' is ignored)<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;"><span
style="font-family: monospace;">/dev/null</span> or . (period)<br>
</td>
<td style="vertical-align: top;">O_RDONLY</td>
<td style="vertical-align: top;">[do nothing]<br>
</td>
<td style="vertical-align: top;">Unix read() if input<br>
</td>
<td style="vertical-align: top;">if output file then nothing is
written<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">block device<br>
</td>
<td style="vertical-align: top;">O_RDONLY</td>
<td style="vertical-align: top;">O_WRONLY | O_CREAT<br>
</td>
<td style="vertical-align: top;">Unix read() write()</td>
<td style="vertical-align: top;"><br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">block device [sgio flag given]<br>
</td>
<td style="vertical-align: top;">O_RDWR or O_RDONLY</td>
<td style="vertical-align: top;">O_RDWR</td>
<td style="vertical-align: top;">SCSI commands</td>
<td style="vertical-align: top;">Opens input O_RDONLY if O_RDWR
fails.&nbsp; The blk_sgio=1 option equates to both iflag=sgio and
oflag=sgio. Partitions ignored. </td>
</tr>
<tr>
<td style="vertical-align: top;">sg device<br>
</td>
<td style="vertical-align: top;">O_RDWR or O_RDONLY<br>
</td>
<td style="vertical-align: top;">O_RDWR<br>
</td>
<td style="vertical-align: top;">SCSI commands<br>
</td>
<td style="vertical-align: top;">Opens input O_RDONLY if O_RDWR
fails</td>
</tr>
<tr>
<td style="vertical-align: top;">raw device<br>
</td>
<td style="vertical-align: top;">O_RDONLY</td>
<td style="vertical-align: top;">O_WRONLY<br>
</td>
<td style="vertical-align: top;">Unix read() write()<br>
</td>
<td style="vertical-align: top;">O_DIRECT open flag in lk 2.6
series is a replacement for raw devices<br>
</td>
</tr>
<tr>
<td style="vertical-align: top;">scsi tape device<br>
</td>
<td style="vertical-align: top;">x<br>
</td>
<td style="vertical-align: top;">x<br>
</td>
<td style="vertical-align: top;">no IO<br>
</td>
<td style="vertical-align: top;">error reported<br>
</td>
</tr>
</tbody>
</table>
<br>
Some of the above combinations are not sensible (e.g. append=1 on a
block device). Raw devices were introduced in the lk 2.4 series and are
still available in the lk 2.6 series but opening a block device or a
normal file with O_DIRECT (in sg_dd using the option odir=1) is
preferred in the lk 2.6 series. If either the input or output file is a
raw device, or 'odir=1' is given then the internal buffers used by
sg_dd are aligned to a memory page boundary. A memory page is 4
kilobytes in the i386 architecture. This memory alignment is required
by both raw devices and normal block devices that
implement O_DIRECT.<br>
<br>
The blk_sgio=1 option (or iflag=sgio and/or oflag=sgio) is only
valid in the lk 2.6 series. This option
instructs the sg_dd utility to use the SG_IO ioctl to issue SCSI
commands even though the file type is not a scsi generic (sg) device.
Currently this will work for all cd/dvd devices irrespective of the
transport (except for some very old devices ( &gt; 10 years old)) and
block devices representing SCSI disks (e.g. <span
style="font-family: monospace;">/dev/sda</span>). Even though SCSI
tape (st) devices now accept the SG_IO ioctl in the lk 2.6 series, the
sg_dd utility expects direct access devices (whereas tapes are
sequential access) so sg_dd generates an error if a st device is
detected. When the input or output file is a sg device then the
blk_sgio option is ignored as IO with that file is always via SCSI
commands. When blk_sgio=1 is used with a block device and the <span
style="font-style: italic;">BPT</span> value
is too high then the first read may report an EIO (input/output)
error; the solution is to lower the <span style="font-style: italic;">BPT</span>
value. [The EIO error
occurred often with CD/DVD drives which have 2048 bytes sectors, so the
<span style="font-style: italic;">BPT</span> default value was reduced
for block size &gt;= 2048 bytes.]<br>
<br>
If a partition of block device is accessed (e.g. <span
style="font-family: monospace;">/dev/sda2</span>) when blk_sgio=0 (or
is not given) then logical block address 0 for the
purposes of sg_dd (and its skip and seek options) is the beginning of
that partition while the calculated count (e.g. when a 'count' option
is not given) is the extent of that partition. However if a partition
of block device is accessed (e.g. <span style="font-family: monospace;">/dev/sda2</span>)
when blk_sgio=1 then the
partition is ignored and the underlying device is accessed. This means
logical block address 0 for the purposes of sg_dd (and its skip and
seek options) is the beginning of the device while the calculated count
(e.g. when a 'count' option is not given) is the extent of that device.<br>
<br>
<span style="font-weight: bold;"></span>When a
CD is accessed via SCSI commands and the 'count' option is not given
then the READ CAPACITY SCSI command is used by sg_dd to determine the
number of blocks to read. Unfortunately some MMC standards allow a few
of the final blocks to be invalid ("run out" sectors) and this can
result in IO errors when the sg_dd utility is used to copy such a CD.
The isosize utility can be used to determine the actual size of an
ISO9660 file system residing on such a CD. It is often a few blocks
shorter than READ CAPACITY reports (for good reason).<br>
<br>
If a SIGUSR1 signal is sent to the process identifier (pid) of a
running sg_dd utility then the number of blocks copied to that point is
output. The copy continues.<br>
<br>
When the 'time=1' option is given, the elapsed time for the copy plus
the
throughput measured in megabytes (10**6 bytes)&nbsp; per second is
output when the copy is complete (or an error stops the copy). If a
SIGUSR1 signal is sent to the process identifier (pid) of a running
sg_dd utility which has the 'time' option set the elapsed time and the
throughput of the copy to that point is output and the copy continues.<br>
<h2><a class="mozTocH2" name="mozTocId657084"></a>Retries</h2>
Often retries are of little use, especially on medium errors, since the
device has probably already done multiple retries (probably
interspersed with moving the heads to the extreme ends of the media)
before the medium error is reported. However a transport error (e.g.
causing a CRC error in returned data) is not necessarily seen by the
device and a retry may quickly solve the problem. In SAS a Transport
Layer Retries (TLR) state machine is optional and requires both the
initiator and target to implement the capability. Most first generation
SAS disks do not implement TLR. So transport errors in the form of
"aborted commands" can be reported due to corruption (marginal cables)
or congestion.<br>
<br>
When the retries=<span style="font-style: italic;">RETR</span> option
is given and <span style="font-style: italic;">RETR</span> is greater
than 0 then most errors on a READ or WRITE SCSI command are retried up
to <span style="font-style: italic;">RETR</span> times. Device not
ready errors are not retried and "unit
attention" conditions are automatically retried (without looking at or
decrementing <span style="font-style: italic;">RETR</span>). Once the
number of retries is exhausted on the
same operation without success then sg_dd will refer to the 'coe'
option as to what to do next. Each new operation,<br>
READ or WRITE, or to a different logical block address has its own
retry count initialized to <span style="font-style: italic;">RETR</span>.<br>
<h2><a class="mozTocH2" name="mozTocId437677"></a>Continue on error
(coe)</h2>
Recent additions to the sg_dd utility allow it to be used as a copy "of
last resort" from failing media. Read errors on an input file
taking SCSI commands are "tolerated" when the 'coe' option is non-zero.
Write errors from SCSI commands are reported and ignored and the sg_dd
utility continues when the 'coe' option is non-zero. [In the case where
media errors are causing write errors the user should check the setting
of the AWRE bit in the SCSI "read write
error recovery" mode page (see SBC-2 at <a href="http://www.t10.org">http://www.t10.org</a>).]<br>
<br>
When a SCSI READ command detects an unrecoverable read error it
responds with a sense key of MEDIUM ERROR or HARDWARE ERROR.
Additionally it responds with the logical block address of the first
(lowest) block that it failed to read in the current READ command. Any
valid blocks prior to the "bad" block may or may not have been
transferred (depending on several other settings). If coe=0 then the
sg_dd utility will simply terminate at this point (with a reasonable
amount of debug information sent to stderr) <span
style="font-weight: bold;">and</span> good blocks prior to the bad
block may not be copied (depending on the setting of <span
style="font-style: italic;">BPT</span>). If 'coe'
is non-zero then the first
thing sg_dd will try to do is a truncated read up to, but not
including,
the
bad block. The remaining discussion in this section assumes 'coe' is
non-zero.<br>
<br>
The "bad" block may or may not be readable by the device. If it is
readable then most likely the associated ECC data indicates that it is
corrupt and the ECC data cannot recover it. [It is possible that all
the corruption is in the ECC data and the payload data is ok.] Such
blocks can be retrieved with the READ LONG command. If the READ LONG
fails then a block of zeros is
substituted for the bad block in the transfer buffer. When 'coe=1' then
READ LONG is not attempted and the "bad" block is replaced by zeros.
When coe=2'(or 'iflag=coe,coe') then a READ LONG is attempted. When
coe=3 then a READ LONG with its CORRCT bit set is attempted. Only SBC
devices (primarily SCSI disks) optionally support READ LONG. CD/DVD
devices (covered in the MMC-4 (draft) standard) do not, so a block of
zeros is substituted for them. Even if READ LONG succeeds on a "bad"
block, the recovered data may not be useful. There are no guarantees
that the user data will appear "as is" in the first 512 bytes.<br>
<br>
There still remain blocks after the "bad" block that need to be
fetched. Further bad blocks may be detected and if so
the algorithm in the last two paragraphs is repeated. The result of
this
process is an imperfect copy with blocks that were read properly placed
in the correct relative position in the output. When the coe=1 option
is given two addition counters are output on completion:<br>
<ul>
<li>unrecovered errors (number of bad blocks)<br>
</li>
<li>read_longs that "successfully" fetched part of those bad blocks<br>
</li>
</ul>
The first counter is the number of unrecovered (read) errors
encountered. Any number greater than zero flags an imperfect copy. The
second counter (always less than or equal to the first) is the number
of "successful" READ LONG SCSI commands performed. If the source media
is a CD or DVD then this number will be zero.<br>
<br>
The coe_limit=<span style="font-style: italic;">CL</span> option is
meant to stop sg_dd continuing ad
nausum if errors are being detected during reading. The input media may
be blank (unrecorded) or beyond its logical block address limit. The
coe_limit=<span style="font-style: italic;">CL</span> option is only
active on read operations when
'coe &gt; 0' which itself is only active on sg device nodes (or block
devices when blk_sgio=1 (or iflag=sgio) is given).<br>
<br>
<h2><a class="mozTocH2" name="mozTocId566094"></a>Recovered errors</h2>
Often errors are recovered using ECC data or by the device retrying
(usually re-reading) the media. Typically at the first sign of trouble,
recoverable errors lead to the block in question being reassigned to
another location on the media (automatically when the AWRE and ARRE
bits are set in the "read write error recovery" mode page).&nbsp; The
user of such a disk may be blissfully unaware that the disk may be
reaching the end of its useful life. Error counters are maintained in
the "Read error counter" and "Write error counter" logs pages which can
be viewed with smartctl (from smartmontools) and sg_logs (from the same
sg3_utils package that sg_dd comes from). Any block that is
automatically or manually re-assigned adds a new entry to the "grown"
defect list which can be viewed with '<span
style="font-family: monospace;">sginfo -G</span>' or '<span
style="font-family: monospace;">sg_reassign -g</span>'
(both found in the
sg3_utils package).<br>
<br>
A disk can be instructed to report RECOVERED ERRORs by setting the PER
bit in the "read write error recover" mode page. Most often this bit is
clear. When sg_dd detects RECOVERED ERRORs it reports them, counts
them and continues the copy. Only the lba of the last recovered error
in a READ or WRITE SCSI
command is reported so there could be more than one recovered error per
SCSI command. The bpt=1 option
could be chosen to limit every SCSI command to a single block transfer
(but that would slow things down a fair amount). If the count of
recovered errors is greater than zero at the end of the copy then this
count is output as well.<br>
<br>
There can be other reasons for a sense key of RECOVERED ERROR not
directly related to data being currently read or written. SMART alerts
(called in SCSI documents "Informational Exceptions") can be conveyed
via a RECOVERED ERROR sense key (see the MRIE field in the
Informational Exceptions mode page). Such alerts have additional sense
codes like "Failure prediction threshold exceeded" and those that
contain "impending failure".<br>
<h2><a class="mozTocH2" name="mozTocId899152"></a>Verbose</h2>
In the Unix style, sg_dd doesn't output anything (to stderr) during
large IO transfers. To get a progress report the SIGUSR1 signal can be
sent to the sg_dd process. In the Unix dd command style, sg_dd outputs
two lines on completion that show the number of full and partial
records
in (on the first line) and out (on the second line). Some modern
versions of the dd command also output elapsed time and throughput
figures (to get this with sg_dd use the "time=1" option).<br>
<br>
The sg_dd has a 'verbose' option whose default value is zero. When set
this option has the following actions:<br>
<ol>
<li>show categorization and INQUIRY data (where applicable) for the
input and output files. For files, other than streams, the file/device
size (and device block size) are output.<br>
</li>
<li>same output as 1 plus data for Unix and SCSI commands (cdbs) that
are not repeated (i.e. other than Unix read/write and SCSI READ/WRITE
commands). Increased error reporting for all SCSI commands</li>
<li>same output as 2 plus data for Unix and SCSI commands (cdbs) that
are repeated. For a large copy this will be a lot of output.</li>
<li>maximum amount of debug output. For a large copy this will be a
lot of output.</li>
</ol>
All verbose output is sent to stderr (so that sg_dd usage with "of=-"
(copy output to stdout) is not corrupted).<br>
<br>
Following is an example of using verbose=1 to find information about
<span style="font-family: monospace;">/dev/sda</span> . If no copy is
required then setting count=0 will see to
that. Since <span style="font-family: monospace;">/dev/sda</span> is a
block device then it would normally be
accessed via Unix system commands. The verbose=1 output is relatively
short when blk_sgio=0 (its default value). The second invocation is
with blk_sgio=1 and a lot more is output. That includes INQUIRY
standard response data (e.g. "SEAGATE ..." line).
See the SBC-2 drafts at <a href="http://www.t10.org/">www.t10.org</a>
for more information.<br>
<br>
<span style="font-family: monospace;">$ sg_dd if=/dev/sda of=. bs=512
verbose=1 count=0 blk_sgio=0</span><br style="font-family: monospace;">
<span style="font-family: monospace;">&nbsp;&gt;&gt; Input file type:
block device</span><br style="font-family: monospace;">
<span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
open input, flags=0x0</span><br style="font-family: monospace;">
<span style="font-family: monospace;">&nbsp;&gt;&gt; Output file type:
null device</span><br style="font-family: monospace;">
<span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
[bgs64] number of blocks=17781521 [0x10f5311], block size=512</span><br
style="font-family: monospace;">
<span style="font-family: monospace;">0+0 records in</span><br
style="font-family: monospace;">
<span style="font-family: monospace;">0+0 records out</span><br
style="font-family: monospace;">
<span style="font-family: monospace;"><br>
<br>
$ sg_dd if=/dev/sda of=. bs=512 verbose=1 count=0 blk_sgio=1</span><br
style="font-family: monospace;">
<span style="font-family: monospace;">&nbsp;&gt;&gt; Input file type:
block device</span><br style="font-family: monospace;">
<span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
open input(sg_io), flags=0x2</span><br style="font-family: monospace;">
<span style="font-family: monospace;">&nbsp;&nbsp;&nbsp; /dev/sda:
SEAGATE&nbsp;&nbsp;
ST39173LC&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1234&nbsp;
[pdt=0]</span><span style="font-family: monospace;"></span><br
style="font-family: monospace;">
<span style="font-family: monospace;">&nbsp;&gt;&gt; Output file type:
null device</span><br style="font-family: monospace;">
<span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
number of blocks=17781521 [0x10f5311], block size=512</span><br
style="font-family: monospace;">
<span style="font-family: monospace;">0+0 records in</span><br
style="font-family: monospace;">
<span style="font-family: monospace;">0+0 records out</span><br>
<br>
Some of the output above is out of order, the "number of blocks" line
relates to the input file.<br>
<br>
Mode page settings can be examined and changed with a utility like <a
href="sdparm.html">sdparm</a> .<br>
<h2><a class="mozTocH2" name="mozTocId443896"></a>sgp_dd, sgm_dd and
sg_read</h2>
These three sg3_utils utilities are closely related to sg_dd . They all
have their own man pages.<br>
<br>
<span style="font-weight: bold;">sgp_dd</span> uses POSIX threads
(default: 4 threads) to potentially speed the copy. The syntax and
semantics of sgp_dd are very similar to sg_dd. The "continue on error"
processing in sgp_dd is simpler than what is described above for sg_dd.
Instead of the verbose=<span style="font-style: italic;">VERB</span>
option used by sg_dd, sgp_dd uses
'deb=<span style="font-style: italic;">VERB</span>' which does a
similar thing.<br>
<br>
<span style="font-weight: bold;">sgm_dd</span> uses memory mapped IO to
speed copies from or to sg device nodes. Memory mapped IO will bypass
the kernel buffers used by the sg driver for normal data transfers.
With memory mapped IO the Host Bus Adapter (HBA) DMA mechanism will
transfer data directly from the external bus (e.g. fibre channel) to
the user space. Even though this may seem like a major performance win,
there are various low level details (e.g. alignment and signal
processing) that complicate the picture.<br>
<br>
<span style="font-weight: bold;">sg_read</span> uses a similar syntax
to sg_dd but has no 'of=<span style="font-style: italic;">OFILE</span>'.
It is designed to measure the
performance of device (e.g. disk) caches, the throughput of the
transport and command overhead. It perform a read (either a SCSI READ
command or a Unix read()) from the given skip block address (default is
logical block address 0) and continues doing reads until count=<span
style="font-style: italic;">COUNT</span> is exhausted. If multiple
reads are&nbsp;
performed (i.e. when "<span style="font-style: italic;">COUNT</span>
&gt; <span style="font-style: italic;">BPT</span>") then they are all
from the same
logical address. For example, with SCSI READs, exactly the same READ
cdb is issued for each read operation. The data is thrown away. Zero
block reads can also be issued using a negative count value. This is
useful for measuring SCSI command overhead.<br>
<h2><a class="mozTocH2" name="mozTocId157981"></a>Conclusion</h2>
The sg_dd utility maintains syntactic and semantic compatibility with
the Unix/GNU dd command while allowing precise control over SCSI
devices
such as SCSI disks and CD/DVD drives. Amongst other uses, it can copy
data from failing media, continuing as best it can in the presence of
MEDIUM errors, while recovering as much data as possible.<br>
<br>
<p>Return to <a href="index.html">main</a> page. </p>
<center>
<p>Last updated: 24th January 2007<br>
<br>
</p>
</center>
</body>
</html>