Update: Linux VM template best practices

I had meant to post this sooner than now but unfortunately work is keeping me occupied 28×7 (that’s not a typo!)

This is a partial update to this post

Attila seems to be a scripting whiz and he’s made a change to my dd script to not utilize bc and added better tunable parameters – ddsleep, ddbs and ddsize. Then he emailed me to share with you! He has tested the default values on some systems and they seemed good: the progress was acceptable and it didn’t generate a high load, the system remained very responsive (in fact it was nearly not noticeable at all):

#!/bin/bash

# by default we skip /
doroot=0

# we want to keep 10 percent free
freepercent=5

# we want to keep at least 100 megabytes free
freespace=100m

# low priority
ddnice=19

# the file to create
file=.vtools-zerovmdk.bin

# the amount of time to sleep between two dd
ddsleep=0.4

# the block size to use with dd
ddbs=64k

# the size by which to increment the file in each turn
ddsize=10m

# if the file is already there, we continue to grow it
resume=0

fss=

while [ $# -gt 0 ]; do

	case $1 in
		--do-root)
			doroot=1
			shift 1
			;;

		--resume)
			resume=1
			shift 1
			;;

		--free-percent)
			freepercent=$2
			shift 2
			;;

		--free-space)
			freespace=$2
			shift 2
			;;

		--dd-nice)
			ddnice=$2
			shift 2
			;;

		--file)
			file=$2
			shift 2
			;;

		--dd-sleep)
			ddsleep=$2
			shift 2
			;;

		--dd-bs)
			ddbs=$2
			shift 2
			;;

		--dd-size)
			ddsize=$2
			shift 2
			;;

		--fss)
			fss=$2
			shift 2
			;;

		*)
			echo "Invalid parameter: $1"
			exit 1
			;;

	esac

done

function resolve_size
{
	local size=$1
	local m=0

	if ! (echo "$size" | grep -q "[0-9]*"); then
		echo "Invalid size: $size" >&2
		exit 1
	fi

	case $size in
		*g)
			m=3
			;;
		*m)
			m=2
			;;
		*k)
			m=1
			;;
	esac

	if [ $m -gt 0 ]; then
		local l=${#size}
		l=$(( $l - 1 ))
		size=${size:0:$l}
		local i=0
		while [ $i -lt $m ]; do
			size=$(( $size * 1024 ))
			i=$(( $i + 1 ))
		done
	fi
	echo $size
}

function calc_percent
{
	local v=$(($1 * $2))
	local l=${#v}
	l=$(($l - 2))
	echo ${v:0:$l}
}

function calc_fsfree
{
	df -B 1 $fs | tail -n 1 | awk -F" " '{print $4}'
}

function calc_fsmax
{
	df -B 1 $fs | tail -n 1 | awk -F" " '{print $2}'
}

freespace=`resolve_size $freespace`
ddbs=`resolve_size $ddbs`
ddsize=`resolve_size $ddsize`

test -z "$ddnice" && ddnice=`nice`

if [ ! -z "$fss" ]; then
	doroot=1
else
	fss=`mount | egrep "type (ext3|ext2|xfs)" | awk -F" " '{ print $3 }'`
fi

for fs in $fss; do

	if [ ! -d "$fs" ]; then
		echo "No such directory: $fs" >&2
		exit 1
	fi

	if [ "$fs" = "/" ]; then
		test "$doroot" = "0" && continue
	fi

	if [ $resume -eq 0 -a -f "$fs/$file" ]; then
		echo "File exists: $fs/$file" >&2
		exit 1
	fi

	fsmax=`calc_fsmax $fs`
	fskeep=`calc_percent $fsmax $freepercent`
	if [ $freespace -gt $fskeep ]; then
		fskeep=$freespace
	fi

	fsfree=`calc_fsfree $fs`
	fszero=$(($fsfree - $fskeep))

	#
	# If the last chunk will not reach
	# the space amount we need to free then
	# then we may enter an endless loop
	last=0

	while [ $fszero -ge 0 ]; do

		test $last -eq 1 && break

		ddcnt=$ddsize

		if [ $fszero -lt $ddsize ]; then
			ddcnt=$fszero
			last=1
		fi

		# this is the step where we may loose precision (see 'last' variable)
		ddcnt=$(( $ddcnt / $ddbs ))

		nice -n $ddnice dd count=$ddcnt bs=$ddbs if=/dev/zero of="$fs/$file" oflag=append \
conv=notrunc status=noxfer 2>&1 | sed '/^[0-9]\++[0-9]\+.*\(records\|rekord\)/d' >&2

		test ! -z "$ddsleep" && sleep $ddsleep

		fsfree=`calc_fsfree $fs`
		fszero=$(($fsfree - $fskeep))
	done

	nice -n $ddnice rm -f "$fs/$file"
done

The script is also downloadable from here.

Attilla, many thanks.

Cheers,

Leo