I
I
IlliumIv2020-05-07 09:48:36
PowerShell
IlliumIv, 2020-05-07 09:48:36

Why does Remove-Item (PowerShell) in a cycle eat up free space equivalent to the volume of files being deleted?

Good afternoon, ladies and gentlemen.

I needed to delete a large number of files according to a pre-created list, and to solve the problem, I wrote a simple PowerShell script:

$ListFiles = @(Get-Content C:\list.txt)
ForEach($FileFullName in $ListFiles)
{
  $FileFullName
  Remove-Item $FileFullName
}


The file list.txe contained ~480,000 lines, one line = one file. The total volume of deleted files was ~40 TB. Content example:
c:\026fa350fa7840df849070b594e8ff38\1\1\d2987171fade4b60985a0afae845c4d0.fidx
c:\026fa350fa7840df849070b594e8ff38\1\1\d2987171fade4b60985a0afae845c4d0\996df0dcfab94cf0bc4184b33b3bf806
c:\026fa350fa7840df849070b594e8ff38\1\1\d2987171fade4b60985a0afae845c4d0\af971067f0b34776b137def9a5dfc691
c:\026fa350fa7840df849070b594e8ff38\1\1\d2987171fade4b60985a0afae845c4d0\c78b71204c5c4e12a81fd1eb9ac40726
c:\06cfa7e917964f50ae1c0e39395ee808.dsc
c:\14c26905802b43b8bd4464bbf7203fd5\1\1\4664a71b596146989626e51f282b25d6.fidx
c:\14c26905802b43b8bd4464bbf7203fd5\1\1\4664a71b596146989626e51f282b25d6\0093053c2abc4ea981ae27aef43e06cb


After I ran the script, I observed a very strange picture for me: the amount of space occupied on the storage system did not decrease, but increased, and about 1 to 1, depending on the number of deleted files. Storage for 100 TB, with 58 occupied before the start of the script at the end of its work, was filled with 98 TB - that is, the volume of deleted files. Upon completion of the script, the place was cleared - instantly the amount of occupied space decreased from 98 to 18 TB.

The storage system is a set of disks connected to the hardware raid controller, assembled in RAID6.

The question is simple: why is this behavior observed when using Remove-Item and what would happen if there was not enough space on the storage system?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
M
MaxKozlov, 2020-05-07
@MaxKozlov

I didn’t notice before, but
Remove-Item [-Path] <String[]>
so you can transfer the entire list to it at once, and not twist it in a loop

A
azarij, 2020-05-07
@azarij

and if you do not erase the files through the turn? cmd'shnym del for example, explorer or total commander what.
I think it's not about remove-item, but about how the storage system is connected.
how is it connected? directly (controller in the same server, block - iscsi / fc, SMB ball)?
what OS was the script run from?
Are storage volumes thick or thin? If applicable in this case.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question