GStreamer / Windows · December 11, 2019

EnableDelayedExpansion Breaks GStreamer Pipelines

One might often find themselves throwing together quick GStreamer pipelines in the course of prototyping a media streaming project, which quickly grows tedious once the correct combination of elements and caps are mastered. Inevitably the developer will move to automate some of this tedium by migrating the pipeline into a shell script as a temporary measure on the journey towards authoring a custom application. This should be as trivial as dumping the launch line into a text file and optionally adding parameters for configurable parts of the pipeline (e.g. file names, network ports, bitrate settings), but in this article you will see how this can sometimes go wrong. The example pipeline we will use for demonstration is a simple image transcoder:

gst-launch-1.0 filesrc location=lena.png ! pngdec ! jpegenc quality=60 ! filesink location=lena.jpg

If one is to dump this pipeline directly into a batch file, then all is well, but such a pipeline is not so convenient for repeated invocation unless some parameters are added. This contrived example performs single or batch processing of images, relying on EnableDelayedExpansion for wildcard support.

@ECHO off
SETLOCAL EnableDelayedExpansion

REM Check argument counts
IF [%2] == [] GOTO usage
IF NOT [%4] == [] GOTO usage

REM Prevent mixing of wildcard inputs with fixed output filename
SET /a _count=0
FOR %%f IN (%2) DO SET /a _count=_count+1
IF NOT [%3] == [] IF !_count! NEQ 1 GOTO usage

GOTO parse

:usage
ECHO Usage: %0 QUALITY [PATTERN ^| INFILE [OUTFILE]]
EXIT /b 1

:parse
SET _quality=%1
SET _pattern=%2
SET _outfile=%3

REM set GST_DEBUG=GST_PIPELINE:5
FOR %%f IN (%_pattern%) DO (
SET _infile=%%~f
IF [%_outfile%] == [] SET _outfile=%%~nf.jpg
CALL :launch !_infile! !_quality! !_outfile!
)
EXIT /b %errorlevel%

:launch
REM launch SUBROUTINE infile quality outfile
ECHO Processing %1 ... %3
gst-launch-1.0 -q filesrc location=%1 ! decodebin ! jpegenc quality=%2 ! filesink location=%3
GOTO:EOF

When run, the pipeline fails in an unexpected way:

>transcode.bat 90 *.png
Processing lena.png ... lena.jpg
ERROR: from element /GstPipeline:pipeline0/GstFileSrc:filesrc0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstFileSrc:filesrc0:
streaming stopped, reason not-linked (-1)
ERROR: pipeline doesn't want to preroll.

Setting ECHO ON doesn’t reveal anything unexpected about how the command is being parsed. Increasing levels of GST_DEBUG don’t offer much insight at the log levels where one can actually hope to parse the extremely verbose output — nevertheless I got lucky with GST_DEBUG=GST_PIPELINE:4

>set GST_DEBUG=GST_PIPELINE:4 && transcode.bat 90 *.png
Processing lena.png ... lena.jpg
0:00:00.040143800 7684 0000000003058B10 INFO GST_PIPELINE gstparse.c:337:gst_parse_launch_full: parsing pipeline description 'filesrc location=lena.png jpegenc quality=90 filesink location=lena.jpg '
ERROR: from element /GstPipeline:pipeline0/GstFileSrc:filesrc0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstFileSrc:filesrc0:
streaming stopped, reason not-linked (-1)
ERROR: pipeline doesn't want to preroll.

One can now see that the pipeline being parsed by GStreamer is not as expected — there are no element links, and the decodebin element is missing. This is because delayed expansion parses text surrounded by pairs of exclamation points as variables to be evaluated, even if separated by whitespace.

The solution is to escape element links in the batch script with double carets (^) like so:

gst-launch-1.0 -q filesrc location=%1 ^^! decodebin ^^! jpegenc quality=%2 ^^! filesink location=%3

Through much googling, I was surprised to find that nobody has ever come across this problem specifically in conjunction with GStreamer — or thought to document it.

The Windows Command Prompt is full of strange quirks like this (for example, placing a caret at the end of a batch file will leak all available memory), yet I still find myself reaching instinctively for Command Prompt over Powershell.