Any markup that is added to Widget titles in a sidebar depends upon your theme.
Here, for example, is the code which twentysixteen uses to register 3 widget areas, for each of which 'before_title' and 'after_title' are used to set the markup which wraps the widget titles.
<function twentysixteen_widgets_init() {
register_sidebar( array(
'name' => __( 'Sidebar', 'twentysixteen' ),
'id' => 'sidebar-1',
'description' => __( 'Add widgets here to appear in your sidebar.', 'twentysixteen' ),
'before_widget' => '<section id="%1$s" class="widget %2$s">',
'after_widget' => '</section>',
'before_title' => '<h2 class="widget-title">',
'after_title' => '</h2>',
) );
register_sidebar( array(
'name' => __( 'Content Bottom 1', 'twentysixteen' ),
'id' => 'sidebar-2',
'description' => __( 'Appears at the bottom of the content on posts and pages.', 'twentysixteen' ),
'before_widget' => '<section id="%1$s" class="widget %2$s">',
'after_widget' => '</section>',
'before_title' => '<h2 class="widget-title">',
'after_title' => '</h2>',
) );
register_sidebar( array(
'name' => __( 'Content Bottom 2', 'twentysixteen' ),
'id' => 'sidebar-3',
'description' => __( 'Appears at the bottom of the content on posts and pages.', 'twentysixteen' ),
'before_widget' => '<section id="%1$s" class="widget %2$s">',
'after_widget' => '</section>',
'before_title' => '<h2 class="widget-title">',
'after_title' => '</h2>',
) );
}
add_action( 'widgets_init', 'twentysixteen_widgets_init' );
I'm not sure what theme you are using. The Toolset starter theme doesn't register any markup to wrap widget titles, and if you dynamically create new widget areas in layouts they will not either.
You can register a widget area in your theme's functions.php file much like the sample from twentysixteen and then use that widget area with your widget area cell, and the markup for the title will be applied.