TrueNas SCALE. How to fix this error?

Question

linux

TrueNas SCALE. How to fix this error?

CRITICAL
Pool purple state is ONLINE: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected.

spoiler

[email protected][~]# smartctl -a /dev/sdc

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 1
3 Spin_Up_Time 0x0027 209 174 021 Pre-fail Always - 4516
4 Start_Stop_Count 0x0032 096 096 000 Old_age Always - 4068
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0
9 Power_On_Hours 0x0032 069 069 000 Old_age Always - 23356
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 163
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 73
193 Load_Cycle_Count 0x0032 199 199 000 Old_age Always - 4265
194 Temperature_Celsius 0x0022 122 105 000 Old_age Always - 28
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0

SMART Error Log Version: 1
Warning: ATA error count 62865 inconsistent with error log pointer 2

ATA Error Count: 62865 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 62865 occurred at disk power-on lifetime: 23252 hours (968 days + 20 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
04 61 08 38 37 00 e0 Device Fault; Error: ABRT 8 sectors at LBA = 0x00003738 = 14136

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
c8 00 08 38 37 00 e0 00 02:15:17.621 READ DMA
ef 10 02 00 00 00 a0 00 02:15:17.556 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 00 02:15:17.555 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 02:15:17.555 SET FEATURES [Set transfer mode]
ef 10 02 00 00 00 a0 00 02:15:17.527 SET FEATURES [Enable SATA feature]

Error 62864 occurred at disk power-on lifetime: 23252 hours (968 days + 20 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
04 61 02 00 00 00 a0 Device Fault; Error: ABRT

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ef 10 02 00 00 00 a0 00 02:15:17.556 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 00 02:15:17.555 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 02:15:17.555 SET FEATURES [Set transfer mode]
ef 10 02 00 00 00 a0 00 02:15:17.527 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 00 02:15:17.526 IDENTIFY DEVICE

Error 62863 occurred at disk power-on lifetime: 23252 hours (968 days + 20 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
04 61 46 00 00 00 a0 Device Fault; Error: ABRT

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ef 03 46 00 00 00 a0 00 02:15:17.555 SET FEATURES [Set transfer mode]
ef 10 02 00 00 00 a0 00 02:15:17.527 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 00 02:15:17.526 IDENTIFY DEVICE
c8 00 08 30 37 00 e0 00 02:15:17.480 READ DMA
ef 10 02 00 00 00 a0 00 02:15:17.464 SET FEATURES [Enable SATA feature]

Error 62862 occurred at disk power-on lifetime: 23252 hours (968 days + 20 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
04 61 02 00 00 00 a0 Device Fault; Error: ABRT

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ef 10 02 00 00 00 a0 00 02:15:17.527 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 00 02:15:17.526 IDENTIFY DEVICE
c8 00 08 30 37 00 e0 00 02:15:17.480 READ DMA
ef 10 02 00 00 00 a0 00 02:15:17.464 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 00 02:15:17.464 IDENTIFY DEVICE

Error 62861 occurred at disk power-on lifetime: 23252 hours (968 days + 20 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
04 61 08 30 37 00 e0 Device Fault; Error: ABRT 8 sectors at LBA = 0x00003730 = 14128

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
c8 00 08 30 37 00 e0 00 02:15:17.480 READ DMA
ef 10 02 00 00 00 a0 00 02:15:17.464 SET FEATURES [Enable SATA feature]
ec 00 00 00 00 00 a0 00 02:15:17.464 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 02:15:17.464 SET FEATURES [Set transfer mode]
ef 10 02 00 00 00 a0 00 02:15:17.458 SET FEATURES [Enable SATA feature]

SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 23356 -
# 2 Extended offline Completed without error 00% 23268 -
# 3 Extended offline Completed without error 00% 23260 -
# 4 Short offline Completed without error 00% 23252 -
# 5 Extended offline Interrupted (host reset) 90% 23208 -
# 6 Short offline Completed without error 00% 23180 -
# 7 Short offline Completed without error 00% 23127 -
# 8 Short offline Completed without error 00% 22985 -
# 9 Short offline Completed without error 00% 22827 -
#10 Short offline Completed without error 00% 22659 -
#11 Short offline Completed without error 00% 22509 -
#12 Short offline Completed without error 00% 22341 -
#13 Short offline Completed without error 00% 22178 -
#14 Extended offline Completed without error 00% 21828 -
#15 Extended offline Completed without error 00% 20518 -
#16 Extended offline Completed without error 00% 19829 -
#17 Extended offline Completed without error 00% 19109 -
#18 Extended offline Aborted by host 80% 18416 -
#19 Extended offline Completed without error 00% 17703 -
#20 Extended offline Completed without error 00% 16959 -
#21 Extended offline Aborted by host 80% 16234 -

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

I reinstalled several times, checked the HDD on another VictoriaHDD PC, there are no BADs.