So, the cheap big SAS drive you bought don’t appear on your OS
All right. First things first. You have to make sure that the drive isn’t formatted 520 bytes. You can start by following this instructions: https://github.com/ahouston/setblocksize/wiki
A pdf copy just in case the site is gone:
Hint: You don’t have to build the setblocksize if you have the sg_format command worked.
Ok, I did all that, and my drive refuses to create partition or fs. Give me all kinds of io errors. What do I do?
Please, remain calm. Understand that is the price you pay for your cheapness.
Well, there are some possible solutions. But sometimes, don’t.
HGST models usually have custom firmware. And we can’t update with different firmware. Even if you have an eprom reader-writer and soldering skills. I learned that the fw data lies all over the disk, not just the EEPROM chip. Areas we cannot access without devices that cost 800 times the price you paid for your 8x10TB SAS disks. Or more.
That said, let’s try to make your HD work.
First things first: Using the explanation from the site I linked above, reformat your HD from 520 to 512 bytes. Or 4096 bytes, if possible. 4096 bytes is a little better if you’re using ZFS.
The format process takes forever. And forever, I mean, FOREVER! 10+ hours to format a 4TB HD. So, keep your computer plugged on a UPS, and avoid doing it on rainy days.
Ok, done? Reboot your computer, use # fdisk -l or # lsblk to see if your HD appeared. It worked? Nice! Try to create a GPT partition with # fdisk /dev/sdX (or parted). All good? Nice, go watch a movie or do some exercises, you fat lazy. Failed? Well, prepare your Doritos and your Coca-Cola Zero.
Probable Causes / Possible Solutions
Mode Sense (Block Descriptor). There are some old HDs that use 6-bit mode sense instead of 10-bit mode sense (default). Long shot, takes forever and may be a waste of time. Know what you’re doing. Be aware: Use only as the last resource.
# sg_format -6 --format -b 512 /dev/sgXProtection type 1 and type 2
At this time, I assumed you have all sg3_util installed. And as the nerd you are, I assume you know when there’s a # before the command, you have to run it as a root. I’m too lazy to increase sudo in front of the command.
So, run:
# sg_readcap -l /dev/sgXIt will return something like this:
Read Capacity results:
Protection: prot_en=0, p_type=0, p_i_exponent=0
Logical block provisioning: lbpme=0, lbprz=0
Last LBA=7814037167 (0x1d1c0beaf), Number of logical blocks=7814037168
Logical block length=512 bytes
Logical blocks per physical block exponent=0
Lowest aligned LBA=0
Hence:
Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB, 4.00 TB
See, at the line Protection, we have prot_en, p_type, p_i_exponent. They MUST be 0. If some of them are different, your problem may lie here.
Case “Type 1 protection“
Read Capacity results:
Protection: prot_en=1, p_type=0, p_i_exponent=0Oh, shit, there’s a number 1 where should be 0. But only one 1. Lucky you. Run:
# sg_format --format -s 512 --pfu=0 /dev/sgXCase “Type 2 protection”
Read Capacity results:
Protection: prot_en=1, p_type=1, p_i_exponent=0Oh shit, there’s two number 1! Two!!! Lucky you. Run:
# sg_format --format -s 512 --fmtpinfo=0 /dev/sgXChoose your problem, run the proper tool, wait for one eternity, and test. If works, go to the pool and play with your dog or something.
Nah, I’m a nerd and I want to know why in the world these disks had protections enabled someday.
Ok, I help you. Most of these disks are from data centers, where people pay lots of money to buy equipment to rent virtual space for customers like you have somewhere to save your porn stuff. And dog pictures, of course. This is one of the several never-to-be-used functions that try to protect information inside the disk.
Read the brochure below, Seagate can explain better:
After all that, were you able to use your disk? No? Well, there are other attempts, but more low-level. I will try to post some how-to to modify the fw to ignore the protections. But not today. I’m tired.
Useful shit
Figuring out why one disk worked and another don’t.
Comparing parameters:
# sdparm -al /dev/sgXOutput:
/dev/sg1: HITACHI DKW2E-H4R0SS AHAH
Direct access device specific parameters: WP=0 DPOFUA=1
Read write error recovery [rw] mode page:
AWRE 0 [cha: y, def: 0, sav: 0] Automatic write reallocation enabled
ARRE 0 [cha: y, def: 0, sav: 0] Automatic read reallocation enabled
TB 0 [cha: y, def: 0, sav: 0] Transfer block
RC 0 [cha: y, def: 0, sav: 0] Read continuous
EER 1 [cha: y, def: 0, sav: 1] Enable early recovery (obsolete)
PER 1 [cha: y, def: 1, sav: 1] Post error
DTE 0 [cha: y, def: 0, sav: 0] Data terminate on error
DCR 0 [cha: y, def: 0, sav: 0] Disable correction (obsolete)
RRC 1 [cha: y, def: 1, sav: 1] Read retry count
COR_S 0 [cha: y, def: 0, sav: 0] Correction span (obsolete)
HOC 0 [cha: y, def: 0, sav: 0] Head offset count (obsolete)
DSOC 0 [cha: y, def: 0, sav: 0] Data strobe offset count (obsolete)
LBPERE 0 [cha: n, def: 0, sav: 0] Logical block provisioning error reporting enabled
MWR 0 [cha: n, def: 0, sav: 0] Misaligned write reporting
WRC 1 [cha: y, def: 1, sav: 1] Write retry count
RTL 500 [cha: y, def: 0, sav:500] Recovery time limit (ms)
Disconnect-reconnect (SPC + transports) [dr] mode page:
BFR 128 [cha: y, def: 0, sav:128] Buffer full ratio
BER 255 [cha: y, def: 0, sav:255] Buffer empty ratio
BIL 0 [cha: y, def: 0, sav: 0] Bus inactivity limit
DTL 0 [cha: n, def: 0, sav: 0] Disconnect time limit
CTL 0 [cha: y, def: 0, sav: 0] Connect time limit
MBS 1040 [cha: y, def: 0, sav:1040] Maximum burst size (512 bytes)
EMDP 0 [cha: n, def: 0, sav: 0] Enable modify data pointers
FA 0 [cha: n, def: 0, sav: 0] Fair arbitration
DIMM 0 [cha: n, def: 0, sav: 0] Disconnect immediate
DTDC 0 [cha: n, def: 0, sav: 0] Data transfer disconnect control
FBS 0 [cha: n, def: 0, sav: 0] First burst size (512 bytes)
Format (SBC) [fo] mode page:
TPZ 11716 [cha: n, def:11716, sav:11716] Tracks per zone
ASPZ 0 [cha: n, def: 0, sav: 0] Alternate sectors per zone
ATPZ 0 [cha: n, def: 0, sav: 0] Alternate tracks per zone
ATPLU 0 [cha: n, def: 0, sav: 0] Alternate tracks per logical unit
SPT 3306 [cha: n, def:3306, sav:3306] Sectors per track
DBPPS 512 [cha: n, def:512, sav:512] Data bytes per physical sector
INTLV 1 [cha: n, def: 1, sav: 1] Interleave
TSF 100 [cha: n, def:100, sav:100] Track skew factor
CSF 22 [cha: n, def: 22, sav: 22] Cylinder skew factor
SSEC 0 [cha: n, def: 0, sav: 0] Soft sector
HSEC 1 [cha: n, def: 1, sav: 1] Hard sector
RMB 0 [cha: n, def: 0, sav: 0] Removable
SURF 0 [cha: n, def: 0, sav: 0] Surface
Rigid disk (SBC) [rd] mode page:
NOC 308240 [cha: n, def:308240, sav:308240] Number of cylinders
NOH 10 [cha: n, def: 10, sav: 10] Number of heads
SCWP 0 [cha: n, def: 0, sav: 0] Starting cylinder for write precompensation
SCRWC 0 [cha: n, def: 0, sav: 0] Starting cylinder for reduced write current
DSR 0 [cha: n, def: 0, sav: 0] Device step rate
LZC 0 [cha: n, def: 0, sav: 0] Landing zone cylinder
RPL 0 [cha: n, def: 0, sav: 0] Rotational position locking
ROTO 0 [cha: n, def: 0, sav: 0] Rotational offset
MRR 7200 [cha: n, def:7200, sav:7200] Medium rotation rate (rpm)
Verify error recovery (SBC) [ve] mode page:
V_EER 0 [cha: n, def: 0, sav: 0] Enable early recovery (obsolete)
V_PER 0 [cha: y, def: 0, sav: 0] Post error
V_DTE 0 [cha: y, def: 0, sav: 0] Data terminate on error
V_DCR 1 [cha: y, def: 0, sav: 1] Disable correction (obsolete)
V_RC 0 [cha: y, def: 1, sav: 0] Verify retry count
V_COR_S 0 [cha: n, def: 0, sav: 0] Verify correction span (obsolete)
V_RTL -1 [cha: y, def: 0, sav: -1] Verify recovery time limit (ms)
Caching (SBC) [ca] mode page:
IC 0 [cha: y, def: 0, sav: 0] Initiator control
ABPF 0 [cha: y, def: 0, sav: 0] Abort pre-fetch
CAP 0 [cha: y, def: 0, sav: 0] Caching analysis permitted
DISC 0 [cha: y, def: 0, sav: 0] Discontinuity
SIZE 0 [cha: y, def: 0, sav: 0] Size enable
WCE 1 [cha: y, def: 0, sav: 1] Write cache enable
MF 0 [cha: y, def: 0, sav: 0] Multiplication factor
RCD 0 [cha: y, def: 0, sav: 0] Read cache disable
DRRP 0 [cha: y, def: 0, sav: 0] Demand read retention priority
WRP 0 [cha: y, def: 0, sav: 0] Write retention priority
DPTL -1 [cha: y, def: -1, sav: -1] Disable pre-fetch transfer length
MIPF 0 [cha: y, def: 0, sav: 0] Minimum pre-fetch
MAPF -1 [cha: y, def: -1, sav: -1] Maximum pre-fetch
MAPFC -1 [cha: y, def: -1, sav: -1] Maximum pre-fetch ceiling
FSW 0 [cha: y, def: 0, sav: 0] Force sequential write
LBCSS 0 [cha: y, def: 0, sav: 0] Logical block cache segment size
DRA 0 [cha: y, def: 0, sav: 0] Disable read ahead
SYNC_PROG 0 [cha: n, def: 0, sav: 0] Synchronous cache progress indication
NV_DIS 0 [cha: n, def: 0, sav: 0] Non-volatile cache disable
NCS 4 [cha: y, def: 8, sav: 4] Number of cache segments
CSS 0 [cha: y, def: 0, sav: 0] Cache segment size
Control [co] mode page:
TST 0 [cha: n, def: 0, sav: 0] Task set type
TMF_ONLY 0 [cha: n, def: 0, sav: 0] Task management functions only
DPICZ 1 [cha: y, def: 0, sav: 1] Disable protection information check if protect field zero
D_SENSE 1 [cha: y, def: 0, sav: 1] Descriptor format sense data
GLTSD 0 [cha: n, def: 0, sav: 0] Global logging target save disable
RLEC 0 [cha: n, def: 0, sav: 0] Report log exception condition
QAM 1 [cha: y, def: 0, sav: 1] Queue algorithm modifier
NUAR 0 [cha: n, def: 0, sav: 0] No unit attention on release
QERR 0 [cha: y, def: 0, sav: 0] Queue error management
RAC 0 [cha: n, def: 0, sav: 0] Report a check
UA_INTLCK 0 [cha: n, def: 0, sav: 0] Unit attention interlocks control
SWP 0 [cha: n, def: 0, sav: 0] Software write protect
ATO 0 [cha: y, def: 0, sav: 0] Application tag owner
TAS 0 [cha: n, def: 0, sav: 0] Task aborted status
ATMPE 0 [cha: y, def: 0, sav: 0] Application tag mode page enabled
RWWP 0 [cha: y, def: 0, sav: 0] Reject write without protection
SBLP 0 [cha: n, def: 0, sav: 0] Supported block lengths and protection information
AUTOLOAD 0 [cha: n, def: 0, sav: 0] Autoload mode
BTP 0 [cha: y, def: 0, sav: 0] Busy timeout period (100us)
ESTCT 37452 [cha: n, def:37452, sav:37452] Extended self test completion time (sec)
Control extension [coe] mode page:
DLC 0 [cha: n, def: 0, sav: 0] Device life control
TCMOS 0 [cha: n, def: 0, sav: 0] Timestamp changeable by methods outside standard
SCSIP 0 [cha: n, def: 0, sav: 0] SCSI timestamp commands take precedence over other methods
IALUAE 0 [cha: n, def: 0, sav: 0] Implicit asymmetric logical unit access enabled
INIT_PR 0 [cha: n, def: 0, sav: 0] Initial command priority
MSDL 0 [cha: y, def: 0, sav: 0] Maximum sense data length
Application tag (SBC) [atag] mode page:
AT_LAST 1 [cha: y, def: 1, sav: 1] Last
AT_LBAT 0xffff [cha: y, def: 0xffff, sav: 0xffff] Logical block application tag
AT_LBA 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block address
AT_COUNT 0xffffffffffffffff [cha: y, def: 0xffffffffffffffff, sav: 0xffffffffffffffff] Logical block count
AT_LAST.1 0 [cha: y, def: 0, sav: 0] Last
AT_LBAT.1 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.1 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.1 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block count
AT_LAST.2 0 [cha: y, def: 0, sav: 0] Last
AT_LBAT.2 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.2 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.2 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block count
AT_LAST.3 0 [cha: y, def: 0, sav: 0] Last
AT_LBAT.3 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.3 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.3 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block count
AT_LAST.4 0 [cha: y, def: 0, sav: 0] Last
AT_LBAT.4 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.4 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.4 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block count
AT_LAST.5 0 [cha: y, def: 0, sav: 0] Last
AT_LBAT.5 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.5 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.5 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block count
AT_LAST.6 0 [cha: y, def: 0, sav: 0] Last
AT_LBAT.6 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.6 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.6 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block count
AT_LAST.7 0 [cha: y, def: 0, sav: 0] Last
AT_LBAT.7 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.7 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.7 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block count
AT_LAST.8 0 [cha: y, def: 0, sav: 0] Last
AT_LBAT.8 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.8 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.8 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block count
AT_LAST.9 0 [cha: y, def: 0, sav: 0] Last
AT_LBAT.9 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.9 0x0 [cha: y, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.9 0xff7f0000 [cha: y, def: 0x5c107bcd, sav: 0x0] Logical block count
AT_LAST.10 1 [cha: y, def: 1, sav: 0] Last
AT_LBAT.10 0x0 [cha: n, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.10 0x0 [cha: n, def: 0x0, sav: 0xa3205f5c107bcd] Logical block address
AT_COUNT.10 0x800000000000ffff [cha: y, def: 0x800000000000ffff, sav: 0xffffffff00000000] Logical block count
AT_LAST.11 1 [cha: n, def: 0, sav: 0] Last
AT_LBAT.11 0xffff [cha: n, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.11 0xffffffffffffffff [cha: y, def: 0xffffffffffffffff, sav: 0x300000000000000] Logical block address
AT_COUNT.11 0x800000000000ffff [cha: n, def: 0x0, sav: 0x100000000000000] Logical block count
AT_LAST.12 1 [cha: n, def: 0, sav: 1] Last
AT_LBAT.12 0xffff [cha: n, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.12 0xffffffffffffffff [cha: n, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.12 0x800000000000ffff [cha: n, def: 0x0, sav: 0xd0effff6ff7f0000] Logical block count
AT_LAST.13 1 [cha: n, def: 0, sav: 0] Last
AT_LBAT.13 0xffff [cha: n, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.13 0xffffffffffffffff [cha: n, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.13 0x800000000000ffff [cha: n, def: 0x0, sav: 0x0] Logical block count
AT_LAST.14 1 [cha: n, def: 0, sav: 1] Last
AT_LBAT.14 0xffff [cha: n, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.14 0xffffffffffffffff [cha: n, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.14 0x800000000000ffff [cha: n, def: 0x0, sav: 0x0] Logical block count
AT_LAST.15 1 [cha: n, def: 0, sav: 0] Last
AT_LBAT.15 0xffff [cha: n, def: 0x0, sav: 0x0] Logical block application tag
AT_LBA.15 0xffffffffffffffff [cha: n, def: 0x0, sav: 0x0] Logical block address
AT_COUNT.15 0x800000000000ffff [cha: n, def: 0x0, sav: 0x20f2fff6ff7f0000] Logical block count
Protocol specific logical unit [pl] mode page:
LUPID 6 [cha: n, def: 6, sav: 6] Logical unit's (transport) protocol identifier
Protocol specific port [pp] mode page:
PPID 6 [cha: n, def: 6, sav: 6] Port's (transport) protocol identifier
Power condition [po] mode page:
PM_BG 0 [cha: y, def: 0, sav: 0] Power management, background functions, precedence
STANDBY_Y 0 [cha: y, def: 0, sav: 0] Standby_y timer enable
IDLE_C 0 [cha: y, def: 0, sav: 0] Idle_c timer enable
IDLE_B 0 [cha: y, def: 0, sav: 0] Idle_b timer enable
IDLE 0 [cha: y, def: 0, sav: 0] Idle_a timer enable
STANDBY 0 [cha: y, def: 0, sav: 0] Standby_z timer enable
ICT 20 [cha: y, def: 20, sav: 20] Idle_a condition timer (100 ms)
SCT 0 [cha: y, def: 0, sav: 0] Standby_z condition timer (100 ms)
IBCT 6000 [cha: y, def:6000, sav:6000] Idle_b condition timer (100 ms)
ICCT 0 [cha: y, def: 0, sav: 0] Idle_c condition timer (100 ms)
SYCT 0 [cha: y, def: 0, sav: 0] Standby_y condition timer (100 ms)
CCF_IDLE 1 [cha: y, def: 1, sav: 1] check condition on transition from idle
CCF_STAND 1 [cha: y, def: 1, sav: 1] check condition on transition from standby
CCF_STOPP 2 [cha: y, def: 2, sav: 2] check condition on transition from stopped
Informational exceptions control [ie] mode page:
PERF 0 [cha: y, def: 0, sav: 0] Performance (impact of ie operations)
EBF 0 [cha: y, def: 0, sav: 0] Enable background function
EWASC 0 [cha: y, def: 0, sav: 0] Enable warning
DEXCPT 0 [cha: y, def: 0, sav: 0] Disable exceptions
TEST 0 [cha: y, def: 0, sav: 0] Test (simulate device failure)
EBACKERR 0 [cha: y, def: 0, sav: 0] Enable background (scan + self test) error reporting
LOGERR 0 [cha: y, def: 0, sav: 0] Log informational exception errors
MRIE 0 [cha: y, def: 0, sav: 0] Method of reporting informational exceptions
INTT 0 [cha: y, def: 0, sav: 0] Interval timer (100 ms)
REPC 0 [cha: y, def: 0, sav: 0] Report count (or Test flag number [SSC-3])
Background control (SBC) [bc] mode page:
S_L_FULL 0 [cha: y, def: 0, sav: 0] Suspend on log full
LOWIR 0 [cha: y, def: 0, sav: 0] Log only when intervention required
EN_BMS 0 [cha: y, def: 0, sav: 0] Enable background medium scan
EN_PS 0 [cha: y, def: 0, sav: 0] Enable pre-scan
BMS_I 0 [cha: y, def: 0, sav: 0] Background medium scan interval time (hour)
BPS_TL 0 [cha: y, def: 0, sav: 0] Background pre-scan time limit (hour)
MIN_IDLE 0 [cha: y, def: 0, sav: 0] Minumum idle time before background scan (ms)
MAX_SUSP 0 [cha: y, def: 0, sav: 0] Maximum time to suspend background scan (ms)
Try not to mess too much. If you want to change:
# sdparm --set PARAM=X /dev/sgX
Didn't work? Reset the drive: # sg_reset /dev/sgX
Want to make it permanent?
sdparm --set PARAM=x --save /dev/sgX
Hello! I just wanted to ask if you ever have
any problems with hackers? My last blog (wordpress)
was hacked and I ended up losing several weeks of hard work due to
no data backup. Do you have any methods to stop hackers?
Hello. Yes, I had problem with hackers in the past. Lots. My network is running on a home server, what I do is lazy, not professional. But, I keep it behind a physical firewall and I try to keep my security patches more up-to-date as possible, among one or two other security best practices. Oh, and I try to keep at least one daily backup of everything.