This post describes a project that began in 2015 to crop the background from screen captures of dialog boxes. I use this process for versions with major user interface releases because there are around 150 dialogs to document in our user reference documentation.

The aim of this project was to capture the screens of the Abiquo product and automatically crop them to make major release documentation faster and easier!

Use case to crop images

The dialog boxes in the Abiquo user interface are bright against the darker background of the web application. In the documentation, we crop screen captures to display only the dialogue boxes (see the Abiquo wiki). We used Imagemagick to crop the backgrounds. Imagemagick runs on many operating systems, and you can use the Imagemagick commands in scripts, programs, or plugins.

In 2015, we were capturing the screen using the Firefox browser or Shutter on Linux, with a BASH plugin to crop the screenshots. In 2020, we were taking the screenshots using Snagit on a Mac and using a ZSH script to run imagemagick commands. A screenshot may contain a dialog box or it could be a main screen with no dialog box, so the script also tests for the type of screenshot.

Note that the imagemagick convert commands may not work properly if the background contains objects in high contrast, for example, if the mouse pointer is included in the background area.

Breakdown of commands to crop images from a shell script

So the first step was to try converting the image to find out if there is a dialog box. If there is no dialog box, the following convert command will make a large change to the image size.

convert "${directory}${ifile}" \
-bordercolor gray75 -compose copy -border 3 \
-bordercolor gray58 -compose copy -border 3 \
-fuzz 50% -trim +repage "${output_directory}$ifileout"

So after we perform the convert command, we use the identify command to calculate the new image width.

new_width=$(magick identify "${output_directory}$ifileout" | awk -F "[ ]\|x" '{print $3}')

new_height=$(magick identify "${output_directory}$ifileout" | awk -F "[ ]\|x" '{print $4}')
change_width=$(( width - new_width ))
print "changewidth: $change_width"

If the change is greater than 20 pixels, the image has no dialogue box, so we can copy it straight to the output directory

# For files with no dialogue box
if [[ change_width -ge 20 ]]; then
cp ${directory}$ifile ${output_directory}$ifileout
else

Otherwise, we perform one of two convert commands to remove the image background.
# for weird files that it doesn't work for (vdc and vapp)
if [[ change_width -le 1 ]]; then
convert "$directory${ifile}" -crop "${new_width}x${new_height}+0+0" +repage \
-bordercolor gray75 -compose copy -border 3 \
-bordercolor gray58 -compose copy -border 3 \
-fuzz 50% -trim +repage \
-fuzz 50% -trim +repage "$output_directory${ifileout}"
else
convert "${directory}${ifile}" \
-bordercolor gray75 -compose copy -border 3 \
-bordercolor gray58 -compose copy -border 3 \
-fuzz 50% -trim +repage \
-fuzz 50% -trim +repage "${output_directory}${ifileout}"
fi
fi

This command works on the majority of screenshots of the Abiquo 5.0 UI that contain the browser screen.

What the commands are doing

Imagemagick is, you know, magic, but here is a brief description of what the commands are apparently doing. The convert command adds some borders to the image. The trim line of the command uses fuzz to take the colour of the corner pixels of the image and blurs pixels within 50% of that colour, then trims the blurred area, and this trim is run twice.

Previous work with this process

Originally, I took the screenshots using Shutter of the Chromium browser. So the original Imagemagick convert command had three steps: crop the browser tab and navigation area off the browser window, trim some of the dark area and repeat the trim.

convert "${FILE}" -crop "${WIDTH}"x"${BCROP}"+0+62 +repage \
-fuzz 50% -trim +repage \
-fuzz 50% -trim +repage "${FILE}"

This command required the screenshots to have a tiny grey browser border around them, a black area and a dark area and then a bright dialogue box in the middle. This original command would not work on a screenshot without a grey browser border, for example, one taken when the browser window is in full-screen mode or even enlarged to occupy the whole screen.

Previously, we used Linux and created screenshots with Shutter in Ubuntu 14.04 or above. But sadly, with the release of Ubuntu 18.04, Shutter was no longer maintained.

In Shutter, you can easily go to the Session tab, select multiple images and run a plugin on all of them at once. Shutter includes an Autocrop plugin, which is a BASH script that will “remove the empty borders from the image” using the following Imagemagick trim command, where “${FILE}” is the image file name.

convert "${FILE}" -trim +repage "${FILE}"

It was easy to create a Shutter plugin by copying and modifying this existing plugin. See for example, this ask Ubuntu post. This post also contains useful information about optimising your screenshots! And we used Inkscape to create a logo to identify the plugin and saved it in an SVG file.

Conclusion

I hope this post can inspire you to experiment with automating the processing of your own screen captures in some way! All in all, the process saved me many hours of screenshot editing.