View Single Post
  #3   (View Single Post)  
Old 26th February 2014
vermaden's Avatar
vermaden vermaden is offline
Administrator
 
Join Date: Apr 2008
Location: pl_PL.lodz
Posts: 1,056
Default

@J65nko

Its not a database problem, its ZFS problem for all data.

Typical write is:
Code:
POOL: WRITE METADATA1
POOL: WRITE DATA1
POOL: ERASE METADATA1
POOL: WRITE METADATA2
POOL: WRITE DATA2
POOL: ERASE METADATA2
As You can imagine there are blocks of free space after METADATA has been erased, which beautifully leads to fragmentation. This can be limited by adding a ZIL device, even on a single disk, then it would look like that:
Code:
 ZIL: WRITE METADATA1
POOL: WRITE DATA1
 ZIL: ERASE METADATA1
 ZIL: WRITE METADATA2
POOL: WRITE DATA2
 ZIL: ERASE METADATA2
So there would be no 'holes' ... but casual fragmentation is as usual.

You can check current ZFS fragmentation level with this script:

zfs-fragmentation.sh
Code:
#! /bin/sh

if [ ${#} -ne 1 ]
then
  echo "usage: $( basename ${0} ) POOL|DATASET"
  exit 1
fi

zdb -ddddd ${1} | awk --non-decimal-data \
'
/Indirect blocks/ {
  file_number++;
  next_block = 0;
}

/L0/ {
  split($3, fields, ":");
  this_block = ("0x"fields[2])+0;
  this_block_size = ("0x"fields[3])+0;
  total_blocks++;
  if( next_block != 0 ) {
    if( next_block == this_block ) {
      not_fragmented++;
    }
    else {
      fragmented++;
    }
  }
  next_block =  this_block + this_block_size;
}

END {
  total_fragment_blocks = fragmented + not_fragmented;
  printf("There are %d files.\n", file_number );
  printf("There are %d blocks and %d fragment blocks.\n", total_blocks, total_fragment_blocks );
  printf("There are %d fragmented blocks (%2.2f%%).\n", fragmented, fragmented*100.0/total_fragment_blocks );
  printf("There are %d contiguous blocks (%2.2f%%).\n", not_fragmented, not_fragmented*100.0/total_fragment_blocks );
}
'
The REAL solution to ZFS fragmentation is 'Block Pointer Rewrite', which will allow defragmentation and other great features [1] but it did not happened yet.

[1] Rebuild 4-way RAID5 into 5-way RAID5 or remove disks from pool.
__________________
religions, worst damnation of mankind
"If 386BSD had been available when I started on Linux, Linux would probably never had happened." Linus Torvalds

Linux is not UNIX! Face it! It is not an insult. It is fact: GNU is a recursive acronym for “GNU's Not UNIX”.
vermaden's: links resources deviantart spreadbsd
Reply With Quote