Gstreamer Small Tutorial

Arash Shafiei

What gstreamer is.

Gstreamer is a multimedia framework. A multimedia framework is a software which provides an abstraction layer for the developers of multimedia applications. To build a multimedia application, developers need some basic components. The components include sources and destinations of media stream as well as some manipulators. These components in the terminology of gstreamer are called Element. An element can have several roles. An element which is the source of stream is called source element, a manipulator element is called filter element, and finally the destination of stream is called sink element. Having a complete set of elements, one can implement any multimedia application. Since multimedia framework must be e tensible by users, gstreamer is implemented with a plugin! based architecture. "n the language of gstreamer, a user can implement a filter which suits his application and uses it through gstreamer framework. "n the following picture you can see the role of gstreamer.

Gstreamer is implemented in # and for its ob$ect!oriented architecture it uses G%b$ect. G%b$ect provides an ob$ect system for # developers. "t is part of a broader Glib which provide many basic A&"s needed for any # developer.

Why using gstreamer.

"magine different media players. They all consist of demu ers, decoders, converters, encoders, mu ers. %ne may notice that all of them have the same components. Each implements these components in its manner. 'ith gstreamer the media players can get their components and focus on design and functionalities of the player. The same analogy goes for any kind of application such as streaming servers or ()( rippers.

Gstreamer foundations.
Element As mentioned Elements are the smallest working blocks of gstreamer. *y chaining several elements together and flowing streams of data, one may achieve a specific goal for implementing an application. There are different types of elements such as+ source element+ the source of stream sink element+ the destination of stream filter+ the manipulator of streams Each element can change state during the e ecution of application. The states are+ null+ no resource is allocated ready+ resources are allocated, but the streams are not opened pause+ streams are opened but no processed play+ streams are being processed The following figure depicts a chain of elements.

Bin *in is an element which contains other elements. Since it is an element it can have the same states as mentioned for element. *y changing the state of a bin it automatically changes the states of its elements.

Pipeline &ipeline is a top level *in. Each application must have one pipeline. The following figure demonstrates a pipeline for an ogg player.

Pad The pads are element,s interface. Streams flow from an element,s source pad to another element,s sink pad. &ads can be built before e ecution or during runtime. &ads can have different capabilities, so before connecting two elements together we must make sure that two elements can so to speak talk together. Bus Each element can communicate with other elements via a bus. *ins usually transfer the bus messages of their children to applications. Buffers They can pass streaming data between elements in the bin. They always travel downstream. Events %b$ects used to notify elements which are waiting for a particular event to occur. They can be sent from application and can travel downstream and upstream. Message %b$ects posted on pipeline bus to notify the application of any particular information such as bugs, state changes, end of streams, etc.. Query %b$ects sent from application or between elements to ask a particular information such as duration, positions, etc.. "n a bin they can travel downstream and upstream.

How to write a simple media player.

This code is a simple media player. 'e will e plain how it actually works.
#include <gst/gst.h> /* Structure to contain all our information, so we can pass it to callbacks */ typedef struct _Custom ata ! "st#lement *pipeline$ "st#lement *source$ "st#lement *%ideo_con%ert$ "st#lement *audio_con%ert$ "st#lement *%ideo_sink$ "st#lement *audio_sink$ & Custom ata$ /* 'andler for the pad(added signal */ static void pad_added_handler )"st#lement *src, "st*ad *pad, Custom ata *data+$ int main)int argc, char *arg%,-+ ! Custom ata data$ "st.us *bus$ "st/essage *msg$ "stStateChange0eturn ret$ gboolean terminate 1 234S#$ /* 5nitiali6e "Streamer */ gst_init )7argc, 7arg%+$ /* Create the elements */ data.source 1 gst_element_factor8_make )9uridecodebin9, 9source9+$ data.audio_con%ert 1 gst_element_factor8_make )9audiocon%ert9, 9audio_con%ert9+$ data.audio_sink 1 gst_element_factor8_make )9autoaudiosink9, 9audio_sink9+$ data.%ideo_con%ert 1 gst_element_factor8_make)9ffmpegcolorspace9,9%ideo_con%ert9+$ data.%ideo_sink 1 gst_element_factor8_make)9auto%ideosink9,9%ideo_sink9+$ /* Create the empt8 pipeline */ data.pipeline 1 gst_pipeline_new )9test(pipeline9+$ if ):data.pipeline ;; :data.source ;; :data.audio_con%ert ;; :data.audio_sink ;; :data.%ideo_con%ert ;; :data.%ideo_sink+ ! g_printerr )9<ot all elements could be created.=n9+$ return (>$ & /* .uild the pipeline. <ote that we are <?@ linking the source at this * point. Ae will do it later. */ gst_bin_add_man8 )"S@_.5< )data.pipeline+, data.source, data.audio_con%ert , data.audio_sink, data.%ideo_con%ert,

data.%ideo_sink, <B44+$ if ):gst_element_link )data.audio_con%ert, data.audio_sink++ ! g_printerr )9#lements could not be linked.=n9+$ gst_obCect_unref )data.pipeline+$ return (>$ & if ):gst_element_link)data.%ideo_con%ert, data.%ideo_sink++ ! g_printerr)9#lements could not be linked.=n9+$ gst_obCect_unref)data.pipeline+$ return (>$ & g_obCect_set )data.source, 9uri9, 9httpD//docs.gstreamer.com/media/sintel_trailer( /* Set the B05 to pla8 */ <B44+$

EFGp.webm9,

/* Connect to the pad(added signal */ g_signal_connect )data.source, 9pad(added9, "_C344.3CH )pad_added_handler+, 7data+$ /* Start pla8ing */ ret 1 gst_element_set_state )data.pipeline, "S@_S@3@#_*43I5<"+$ if )ret 11 "S@_S@3@#_C'3<"#_2354B0#+ ! g_printerr )9Bnable to set the pipeline to the pla8ing state.=n9+$ gst_obCect_unref )data.pipeline+$ return (>$ & /* 4isten to the bus */ bus 1 gst_element_get_bus )data.pipeline+$ do ! msg 1 gst_bus_timed_pop_filtered )bus, "S@_C4?CH_@5/#_<?<#, "S@_/#SS3"#_S@3@#_C'3<"# ; "S@_/#SS3"#_#00?0 ; "S@_/#SS3"#_#?S+$ /* *arse message */ if )msg :1 <B44+ ! "#rror *err$ gchar *debug_info$ switch )"S@_/#SS3"#_@I*# )msg++ ! case "S@_/#SS3"#_#00?0D gst_message_parse_error )msg, 7err, 7debug_info+$ g_printerr )9#rror recei%ed from element JsD Js=n9, "S@_?.K#C@_<3/# )msg(>src+, err(>message+$ g_printerr )9 ebugging informationD Js=n9, debug_info L debug_info D 9none9+$ g_clear_error )7err+$ g_free )debug_info+$ terminate 1 @0B#$ break$ case "S@_/#SS3"#_#?SD g_print )9#nd(?f(Stream reached.=n9+$ terminate 1 @0B#$ break$ case "S@_/#SS3"#_S@3@#_C'3<"# D /* Ae are onl8 interested in state(changed messages from the pipeline */ if )"S@_/#SS3"#_S0C )msg+ 11 "S@_?.K#C@ )data.pipeline++ ! "stState old_state, new_state, pending_state$ gst_message_parse_state_changed )msg, 7old_state, 7new_state, 7pending_state+$ g_print )9*ipeline state changed from Js to JsD=n9, gst_element_state_get_name )old_state+, gst_element_state_get_name )new_state++$ & break$ defaultD /* Ae should not reach here */ g_printerr )9BneMpected message recei%ed.=n9+$ break$ & gst_message_unref )msg+$

& & while ):terminate+$

&

/* 2ree resources */ gst_obCect_unref )bus+$ gst_element_set_state )data.pipeline, "S@_S@3@#_<B44+$ gst_obCect_unref )data.pipeline+$ return G$

/* @his function will be called b8 the pad(added signal */ static void pad_added_handler )"st#lement *src, "st*ad *new_pad, Custom ata *data+ ! "st*ad *sink_pad_audio 1 gst_element_get_static_pad )data(>audio_con%ert, 9sink9+$ "st*ad *sink_pad_%ideo 1 gst_element_get_static_pad )data(>%ideo_con%ert, 9sink9+$

"st*ad4ink0eturn ret$ "stCaps *new_pad_caps 1 <B44$ "stStructure *new_pad_struct 1 <B44$ const gchar *new_pad_t8pe 1 <B44$ g_print )90ecei%ed new pad NJsN from NJsND=n9, "S@_*3 _<3/# )new_pad+, "S@_#4#/#<@_<3/# )src++$ /* 5f our audio con%erter is alread8 linked, we ha%e nothing to do here */ if )gst_pad_is_linked )sink_pad_audio++ ! g_print )9 Ae are alread8 linked. 5gnoring.=n9+$ goto eMit$ & /* 5f our %ideo con%erter is alread8 linked, we ha%e nothing to do here */ if )gst_pad_is_linked )sink_pad_%ideo++ ! g_print )9 Ae are alread8 linked. 5gnoring.=n9+$ goto eMit$ & /* Check the new padNs t8pe */ new_pad_caps 1 gst_pad_get_caps )new_pad+$ new_pad_struct 1 gst_caps_get_structure )new_pad_caps, G+$ new_pad_t8pe 1 gst_structure_get_name )new_pad_struct+$ if )g_str_has_prefiM )new_pad_t8pe, 9audio/M(raw9++ ! /* 3ttempt the link */ ret 1 gst_pad_link )new_pad, sink_pad_audio+$ if )"S@_*3 _45<H_2354# )ret++ ! g_print )9 @8pe is NJsN but link failed.=n9, new_pad_t8pe+$ & else ! g_print )9 4ink succeeded )t8pe NJsN+.=n9, new_pad_t8pe+$ & & else if )g_str_has_prefiM )new_pad_t8pe, 9%ideo/M(raw9++ ! /* 3ttempt the link */ ret 1 gst_pad_link )new_pad, sink_pad_%ideo+$ if )"S@_*3 _45<H_2354# )ret++ ! g_print )9 @8pe is NJsN but link failed.=n9, new_pad_t8pe+$ & else ! g_print )9 4ink succeeded )t8pe NJsN+.=n9, new_pad_t8pe+$ & & else ! g_print )9 5t has t8pe NJsN which is not raw audio. 5gnoring.=n9, new_pad_t8pe+$ goto eMit$ & exitD /* Bnreference the new padNs caps, if we got them */ if )new_pad_caps :1 <B44+ gst_caps_unref )new_pad_caps+$ /* Bnreference the sink pad */ gst_obCect_unref )sink_pad_audio+$ gst_obCect_unref )sink_pad_%ideo+$

&

-irst thing we need to do is to design our application. Here,s the schema+

.et,s first e plain the /01 mark. /uridecodebin1 actually is a bin which consists of decoder and demu er elements. "t makes our task very easy. Since it has demu er, it may have an arbitary number of source depends on the input media which it receives. So it dynamically create source pads and we must write a callback function which connects the source pads to corresponding sink pads of converters. "f there is an unknown output of demu er 2for instance subtitle stream3 we don,t deal with it. 4ow let,s start walking through the code.
/* Structure to contain all our information, so we can pass it to callbacks */ typedef struct _Custom ata ! "st#lement *pipeline$ "st#lement *source$ "st#lement *%ideo_con%ert$ "st#lement *audio_con%ert$ "st#lement *%ideo_sink$ "st#lement *audio_sink$ & Custom ata$

'e keep the elements in a data structure so we can pass it to callback function. This structure contains all the elements that we need.
/* 'andler for the pad(added signal */ static void pad_added_handler )"st#lement *src, "st*ad *pad, Custom ata *data+$

This is the signiture of the callback function. 'e will e plain this function later on.
int main)int argc, char *arg%,-+ ! Custom ata data$ "st.us *bus$ "st/essage *msg$ "stStateChange0eturn ret$ gboolean terminate 1 234S#$ /* 5nitiali6e "Streamer */ gst_init )7argc, 7arg%+$ /* Create the elements */ data.source 1 gst_element_factor8_make )9uridecodebin9, 9source9+$ data.audio_con%ert 1 gst_element_factor8_make )9audiocon%ert9, 9audio_con%ert9+$ data.audio_sink 1 gst_element_factor8_make )9autoaudiosink9, 9audio_sink9+$ data.%ideo_con%ert 1 gst_element_factor8_make)9ffmpegcolorspace9,9%ideo_con%ert9+$ data.%ideo_sink 1 gst_element_factor8_make)9auto%ideosink9,9%ideo_sink9+$

gst_init

initiali5es the gstreamer. 6sing gst_element_factory_make we can create the elements.

/* Create the empt8 pipeline */ data.pipeline 1 gst_pipeline_new )9test(pipeline9+$ if ):data.pipeline ;; :data.source ;; :data.audio_con%ert ;; :data.audio_sink ;; :data.%ideo_con%ert ;; :data.%ideo_sink+ ! g_printerr )9<ot all elements could be created.=n9+$ return (>$ &

'e create the pipeline and make sure that all the elements has been created.
/* .uild the pipeline. <ote that we are <?@ linking the source at this * point. Ae will do it later. */ gst_bin_add_man8 )"S@_.5< )data.pipeline+, data.source, data.audio_con%ert , data.audio_sink, data.%ideo_con%ert, data.%ideo_sink, <B44+$ if ):gst_element_link )data.audio_con%ert, data.audio_sink++ ! g_printerr )9#lements could not be linked.=n9+$ gst_obCect_unref )data.pipeline+$ return (>$ & if ):gst_element_link)data.%ideo_con%ert, data.%ideo_sink++ ! g_printerr)9#lements could not be linked.=n9+$ gst_obCect_unref)data.pipeline+$

return (>$

7 'e add all elements to the pipeline and link the sink elements to convert elements. 4ote that at the moment we don,t know how many source pad the demu er element would have. So we have to link them in runtime.
/* Set the B05 to pla8 */ g_obCect_set )data.source, 9uri9, 9httpD//docs.gstreamer.com/media/sintel_trailer(EFGp.webm9, <B44+$

4ote that g_obCect_set is a G%b$ect function, which sets the variable values. Since gstreamer is using G%b$ect system, all the ob$ects inherit the properties of their ancestors which is G%b$ect and therefore we can pass GstElement as a parameter to this function. Here we need to set the value of uri variable.
/* Connect to the pad(added signal */ g_signal_connect )data.source, 9pad(added9, "_C344.3CH )pad_added_handler+, 7data+$

Again g_signal_connect is a G%b$ect function to connect a signal on an element to a callback function. This means that whenever a pad is added to the element, the callback function is called.
/* Start pla8ing */ ret 1 gst_element_set_state )data.pipeline, "S@_S@3@#_*43I5<"+$ if )ret 11 "S@_S@3@#_C'3<"#_2354B0#+ ! g_printerr )9Bnable to set the pipeline to the pla8ing state.=n9+$ gst_obCect_unref )data.pipeline+$ return (>$ &

'e change the state of the pipeline to playing state. 4ote that intermediate state changing will be done automatically since the system can not directly goes from null to play state.
/* 4isten to the bus */ bus 1 gst_element_get_bus )data.pipeline+$ do ! msg 1 gst_bus_timed_pop_filtered )bus, "S@_C4?CH_@5/#_<?<#, "S@_/#SS3"#_S@3@#_C'3<"# ; "S@_/#SS3"#_#00?0 ; "S@_/#SS3"#_#?S+$

'e listen to the bus of the pipeline and catch all the messages.
/* *arse message */ if )msg :1 <B44+ ! "#rror *err$ gchar *debug_info$ switch )"S@_/#SS3"#_@I*# )msg++ ! case "S@_/#SS3"#_#00?0D gst_message_parse_error )msg, 7err, 7debug_info+$ g_printerr )9#rror recei%ed from element JsD Js=n9, "S@_?.K#C@_<3/# )msg(>src+, err(>message+$ g_printerr )9 ebugging informationD Js=n9, debug_info L debug_info D 9none9+$ g_clear_error )7err+$ g_free )debug_info+$ terminate 1 @0B#$ break$ case "S@_/#SS3"#_#?SD g_print )9#nd(?f(Stream reached.=n9+$ terminate 1 @0B#$ break$ case "S@_/#SS3"#_S@3@#_C'3<"# D /* Ae are onl8 interested in state(changed messages from the pipeline */ if )"S@_/#SS3"#_S0C )msg+ 11 "S@_?.K#C@ )data.pipeline++ ! "stState old_state, new_state, pending_state$ gst_message_parse_state_changed )msg, 7old_state, 7new_state, 7pending_state+$ g_print )9*ipeline state changed from Js to JsD=n9, gst_element_state_get_name )old_state+, gst_element_state_get_name )new_state++$ & break$ defaultD /* Ae should not reach here */ g_printerr )9BneMpected message recei%ed.=n9+$

break$ & gst_message_unref )msg+$ & & while ):terminate+$

'e parse all messages and print the corresponding messages on the screen.
/* 2ree resources */ gst_obCect_unref )bus+$ gst_element_set_state )data.pipeline, "S@_S@3@#_<B44+$ gst_obCect_unref )data.pipeline+$ return G$ &

-inally we release all the resources and change the state of the system to null.
/* @his function will be called b8 the pad(added signal */ static void pad_added_handler )"st#lement *src, "st*ad *new_pad, Custom ata *data+ ! "st*ad *sink_pad_audio 1 gst_element_get_static_pad )data(>audio_con%ert, 9sink9+$ "st*ad *sink_pad_%ideo 1 gst_element_get_static_pad )data(>%ideo_con%ert, 9sink9+$ "st*ad4ink0eturn ret$ "stCaps *new_pad_caps 1 <B44$ "stStructure *new_pad_struct 1 <B44$ const gchar *new_pad_t8pe 1 <B44$ g_print )90ecei%ed new pad NJsN from NJsND=n9, "S@_*3 _<3/# )new_pad+, "S@_#4#/#<@_<3/# )src++$ /* 5f our audio con%erter is alread8 linked, we ha%e nothing to do here */ if )gst_pad_is_linked )sink_pad_audio++ ! g_print )9 Ae are alread8 linked. 5gnoring.=n9+$ goto eMit$ & /* 5f our %ideo con%erter is alread8 linked, we ha%e nothing to do here */ if )gst_pad_is_linked )sink_pad_%ideo++ ! g_print )9 Ae are alread8 linked. 5gnoring.=n9+$ goto eMit$ &

4ow we e plain the callback function. -irst we get the sink pad of our converters. "f they are already connected we don,t need to do anything.
/* Check the new padNs t8pe */ new_pad_caps 1 gst_pad_get_caps )new_pad+$ new_pad_struct 1 gst_caps_get_structure )new_pad_caps, G+$ new_pad_t8pe 1 gst_structure_get_name )new_pad_struct+$ if )g_str_has_prefiM )new_pad_t8pe, 9audio/M(raw9++ ! /* 3ttempt the link */ ret 1 gst_pad_link )new_pad, sink_pad_audio+$ if )"S@_*3 _45<H_2354# )ret++ ! g_print )9 @8pe is NJsN but link failed.=n9, new_pad_t8pe+$ & else ! g_print )9 4ink succeeded )t8pe NJsN+.=n9, new_pad_t8pe+$ & & else if )g_str_has_prefiM )new_pad_t8pe, 9%ideo/M(raw9++ ! /* 3ttempt the link */ ret 1 gst_pad_link )new_pad, sink_pad_%ideo+$ if )"S@_*3 _45<H_2354# )ret++ ! g_print )9 @8pe is NJsN but link failed.=n9, new_pad_t8pe+$ & else ! g_print )9 4ink succeeded )t8pe NJsN+.=n9, new_pad_t8pe+$ & & else ! g_print )9 5t has t8pe NJsN which is not raw audio. 5gnoring.=n9, new_pad_t8pe+$ goto eMit$ &

Then we check the capability of the new pad which has been created. 'e check if it,s of type of raw video or raw audio. "f the source pad which is created is raw video we link it to the sink pad of video converter. "f it,s of the type of raw audio we link it to the sink pad of the audio converter.
exitD /* Bnreference the new padNs caps, if we got them */ if )new_pad_caps :1 <B44+ gst_caps_unref )new_pad_caps+$

/* Bnreference the sink pad */ gst_obCect_unref )sink_pad_audio+$ gst_obCect_unref )sink_pad_%ideo+$

-inally we release the resources. 4ow you can compile this code and see a media player which actually works. 8ou can use this command to compile+
gcc pla8er.c (o pla8er Opkg(config ((cflags ((libs gstreamer(G.>GO

How to test elements in terminal.

4ow imagine you want to test the elements before starting to develop an application. Gstreamer provides a tool named gst!launch which can be used for this purpose. -or e ample you want to write an application to convert mp9 to ogg. 8ou can create your pipeline and test all the needed elements using this command+
gst(launch filesrc location19test.mpP9 : mad : audiocon%ert : %orbisenc : oggmuM : filesink location19test.ogg9

To play the output you can use+

gst(launch filesrc location19test.ogg9 : oggdemuM : %orbisdec : audiocon%ert : audioresample : alsasink

Sour es
:;< GStreamer Application (evelopment =anual 2;.>.93 , 'im Taymans , Steve *aker, Andy 'ingo, ?onald S. *ult$e, Stefan @ost :A< GStreamer S(@ documentation 2http+BBdocs.gstreamer.comBdisplayBGstS(@BHome3