Automation via Slicebot

Use Slicebot to monitor one or more directories for new files. Once a file is detected, it is automatically sliced and uploaded to the cloud.

Slicebot is included with the Slicer installation.

Slicebot is not officially supported on Windows.

Quick Start

Setting up Slicebot consists of the following steps:

  1. Create a configuration file that defines the directories to be monitored and how content will be processed.
  2. Start Slicebot by running the following command:

Basic Information

This section explains:

Configuration Overview

Configuration settings may be defined through the following mechanisms:

Settings will be applied in the following order:

As a result, settings defined within a file name will take precedence over any conflicting settings defined elsewhere.

Main Configuration File

Slicebot requires a configuration file that contains settings that define:

View configuration settings.

Key information:

A sample configuration file is provided below.

[global]

enc_wait = 1

mail_host = smtp.gmail.com:587

done_emails = sys@mycompany.com, me@mycompany.com

fail_emails = me@mycompany.com

[~/t/slice/a/]

[~/t/slice/b/]

enc_wait = 0

username = me@email.com

apikey = abcDEFghiJKLmnoPQRtuvWXYz123ABCdefGHIJKL

File-Specific Configuration File

Instructions on when and how a specific file is processed may be defined by creating a configuration file that contains the desired configuration settings.

Although most configuration settings may be defined within a file-specific configuration, there are exceptions (e.g., halt_on_error and ignore). The description associated with each setting indicates any limitations on where it may be implemented.

Key information:

A sample configuration file is provided below.

enc_wait = 0

done_emails = marketing@mycompany.com, joe@mycompany.com

fail_emails = jane@mycompany.com

File Name Configuration

Instructions on when and how a specific file is processed may be defined by including the desired settings within the source media's file name.

Slicebot will only honor settings defined within a file name when the allow_inline setting has been enabled via either the main or a file-specific configuration file.

Although most configuration settings may be defined within a file name, there are exceptions (e.g., halt_on_error and ignore). The description associated with each setting indicates any limitations on where it may be implemented.

Syntax:

FileName^Setting1=Value1^Setting2=Value2^SettingN=ValueN.ext

Example:

Original file name:

marketing.mp4

New file name:

marketing^max_slices=3^external_id=1FA1272b.mp4

Follow these guidelines when defining a setting within a file name:

Testing

Use --dry-run option to test your configuration without processing files.

./slicebot --dry-run slicebot.cfg

Ending a Slicebot Session

Slicebot runs indefinitely until one of the following conditions is true:

Configuration Settings

This section describes the supported configuration settings that can be used to control how slicebot processes files.

Key information:

The following configuration settings may be leveraged when defining:

Setting

Description

Example

allow_inline

Directory Monitoring

Determines whether the source media's file name may contain configuration settings.

Learn more.

Valid values are:

  • 1|true: Indicates that the file name may contain configuration settings.
  • 0|false: Indicates that settings defined within the file name will be ignored.

Default value: 1

allow_inline = 0

apikey

Required

Requires Slicer version 17111500 or higher

Authorization

Sets the secret API key through which Slicebot authenticates to the CMS.

Bypass this requirement by setting the SLICER_USER and the SLICER_APIKEY environment variables.

This API key should correspond to the user defined by the username setting.

Syntax:

apikey = abc...apikey = abcDEFghiJKLmnoPQRtuvWXYz123ABCdefGHIJKL

auto_replace

Asset Replacement (Library)

Determines whether asset(s) with the same external ID will be automatically replaced.

Valid values are:

  • 1|true: Ensures that the external ID assigned to an asset starts with _replace. This will cause it to replace existing assets with the same external ID.
  • 0|false: Indicates that the external ID assigned to an asset will not be modified.

auto_replace = 1

autoexpire

Asset Retention (Library)

Marks an asset for deletion once the specified number of hours have elapsed.

The sample configuration, shown to the right, will delete assets 48 hours after they have been encoded.

autoexpire = 48

breaks

Ad Breaks (Asset)

Inserts an ad break for each specified time range.

Key information:

  • Start and stop times are specified in seconds.
  • Specify multiple ad breaks by using a comma to delimit time ranges.
  • The original content corresponding to each ad break will be removed.
  • Ad breaks may be used to insert ads during live or on-demand playback.

Single Ad Break Syntax:

Multiple Ad Breaks Syntax:

breaks = StartTime1-StopTime1,StartTime2-StopTime2,StartTimeN-StopTimeN

breaks = 10.1-30.58,105-151.332

bug

Asset

Overlays an image on the video at the specified start time for the given duration.

Key information:

  • Only RGBA PNG images may be overlaid on to video.
  • The image overlay effect respects the transparency defined in the PNG image.
  • It is important that the specified PNG image match the resolution of the source video.
  • The start time and the duration of the image overlay effect is specified in milliseconds.
  • Use a negative number to specify start time from the end of the asset.

    For example, a start time of -5000 will start the image overlay effect 5 seconds prior to the end of the asset.

  • Specify multiple image overlays by using 3 commas as a delimiter.
  • If multiple images overlap, then Z-order is determined by the order in which they are listed.

Syntax:

The sample configuration, shown to the right, will add overlay1.png at the start of the asset for 5 seconds and overlay2.png five seconds before the end of the asset. If these two overlay effects overlap, then overlay2.png would appear on top of overlay1.png.

bug = overlay1.png,0,5000,,,overlay2.png,-5000,5000

description

Asset (Library)

Assigns a name to the asset generated from the source media.

This setting is typically used when defining file-specific configurations.

By default, assets are named after the source media's file name.

description = Marketing Event 2016

done_directory

Post-Processing

Defines the directory to which the source media will be moved after it has been successfully processed.

Key information:

  • If the specified directory does not exist, then it will be created.
  • After the source media has been successfully processed, it will be deleted if one of the following conditions are true:

    • This setting has not been defined.
    • This setting is set to none.

      Syntax:

      done_directory = none

done_directory = ~/media/success

done_emails

Notifications

Defines a comma-separated list of email addresses to which an email notification will be sent when a file is successfully processed.

This setting requires the mail_host setting.

done_emails = fred@gmail.com, george@example.com

done_url

Notifications

Defines the URL to which a HTTP POST request will be submitted when a file is successfully processed.

Learn more.

done_url = http://myserver.net/myservice

enc_wait

Slicebot

Determines whether Slicebot will wait until after the cloud finishes encoding an asset before it starts processing the next source media file.

Valid values are:

  • 1: Indicates that Slicebot must wait until after encoding is completed.
  • 0: Indicates that Slicebot may move on to the next source media file as soon as slicing is completed.

Default value: 0

enc_wait = 1

external_id

Asset (Library)

Assigns an external ID to the asset generated from the source media.

By default, the same external ID may be assigned to multiple assets. Override this behavior by adding the "_replace:" prefix to the desired external ID. This will cause the new asset to replace existing assets with the same external ID.

external_id = ep20115e_12

external_id = _replace:ep20115e_12

fade

Asset

Fades audio and/or video in/out from the specified start time.

Key information:

  • Determine whether this setting will fade in or out by setting the Method parameter to either of the following values:

    • in: Fade in
    • out: Fade out
  • This fade effect may be applied to audio and/or video. Valid values for the MediaType parameter are:

    • audio: Fades audio in from silence.
    • video: Fades video in from black.
    • both: Fades in both audio and video.
  • The start time and the duration of the fade effect is specified in milliseconds.
  • Use a negative number to specify start time from the end of the asset.

    For example, a start time of -5000 will start the fade effect 5 seconds prior to the end of the asset.

  • Specify multiple fade effects by using 3 commas as a delimiter.

Syntax:

The sample configuration, shown to the right, will fade in audio/video at the start of the asset for 5 seconds and then fade out 5 seconds before the end of the asset.

fade = in,both,0,5000,,,out,both,-5000,5000

fail_directory

Post-Processing

Defines the directory to which the source media will be moved when it is not successfully processed.

Key information:

  • If the specified directory does not exist, then it will be created.
  • After all attempts to process the source media have failed, it will be deleted if one of the following conditions are true:

    • This setting has not been defined.
    • This setting is set to none.

      Syntax:

      fail_directory = none

fail_directory = ~/media/failure

fail_emails

Notifications

Defines a comma-separated list of email addresses to which an email notification will be sent when a file is unsuccessfully processed.

This setting requires the mail_host setting.

fail_emails = ops@company.org

fail_url

Notifications

Defines a URL to which a HTTP POST request will be submitted when a file is unsuccessfully processed.

Learn more.

fail_url = http://myserver.net/myservice2?status=fail

force_aspect_ratio

Asset

Forces the video to the specified aspect ratio.

Syntax:

force_aspect_ratio = 16:9

halt_on_error

Source Media Processing

Determines whether processing will stop when an issue occurs.

This setting may only be defined within the main configuration file.

Valid values are:

  • 1|true: Indicates that Slicebot will stop processing when either of the following occurs:

    • A job fails due to an issue with the source media.
    • A job fails due to a configuration issue.
  • 0|false: Indicates that Slicebot will always continue to the next job.

Default value: 0

halt_on_error = 1

helper_noext

Directory Monitoring

Determines whether supporting files must include the source media's file extension.

Valid values are:

  • 1|true: Indicates that the supporting file's file name should exclude the source media's file extension.

    Example:

    Source media's file name:

    marketing.mp4

    File-specific configuration file name:

    marketing.cfg
  • 0|false: Indicates that the supporting file's file name should include the source media's file extension.

    Example:

    Source media's file name:

    marketing.mp4

    File-specific configuration file name:

    marketing.mp4.cfg

helper_noext = 1

ignore

Configuration File

Determines whether a directory will be monitored for new files.

This setting should only be defined under a directory-specific section within the main configuration file.

Valid values are:

  • 1|true: Indicates that the directory corresponding to the current section will not be monitored.
  • 0|false: Indicates that directory corresponding to the current section will be monitored.

ignore = 1

ignoreName

Directory Monitoring

Ignores files in a monitored directory with the specified file name.

Key information:

  • File names are case-insensitive.
  • Specify multiple files by using a comma to delimit each one.

ignoreName = thumbs.db,.DS_Store

ignoreSuffix

Directory Monitoring

Ignores files in a monitored directory with the specified file extension(s).

Key information:

  • File extensions are case-insensitive.
  • Specify multiple file extensions by using a comma to delimit each one.

ignoreSuffix = db,txt,jpg

libraries

Asset (Library)

Adds an asset to one or more shared libraries.

Key information:

  • A library may be specified either by name or GUID.
  • Library names must be an exact match and are case-sensitive.
  • Add an asset to multiple libraries by using a comma to delimit each one.

Add to Library by Name Syntax:

libraries = LibraryName

Add to Library by GUID Syntax:

libraries = LibraryGUID

Add to Multiple Libraries Syntax:

libraries = 357...,8b0...libraries = 35700c786f2b4f03abb880d2b0bf70fb, 8b085b99d89a4617969682fc23757da2

mail_from

Notifications

Determines the sender email address (i.e., from) for email notifications.

mail_from = automation@company.org

mail_host

Notifications

Defines the hostname for the mail server through which email notifications will be sent.

A port number may be appended to the specified host. Mail servers typically use the following ports:

  • 25: Standard emails
  • 587: Secure emails

If a mail server requires authorization before sending out emails, then the mail_password and mail_username settings must also be defined.

Syntax:

mail_host = smtp.company.com:587

mail_password

Notifications

Defines the password for the user defined by the mail_username setting.

This setting is only required when a mail server requires authorization before sending emails.

mail_password = G183hIU39331f

mail_username

Notifications

Defines the name of the user account on the mail server through which email notifications will be sent.

This setting is only required when a mail server requires authorization before sending emails.

mail_username = robo-emails@example.com

max_slices

Slicing

Limits the number of temporal slices to generate from the source media.

Key information:

  • By default, there is no limit on the number of slices that will be generated from the source media. Rather, the entire file is sliced.
  • This setting is useful when either troubleshooting or when slicing short segments of media.
  • This setting defines an approximate limit. The Slicer may generate a few additional slices.

max_slices = 20

meta

Asset (Library)

Adds metadata to the asset.

Key information:

  • Field names must meet the following requirements:

    • Unique values.
    • Alphanumeric characters and underscores are allowed.
    • Letters must be specified in lower-case.
  • Enclose values containing spaces with quotes.
  • Assign multiple metadata fields to an asset by using three commas to delimit each field/value pair.

Syntax:

An alternative method for assigning metadata is through a JSON file.
Learn more.

meta = rating=TV-13

meta = rating=TV-13,,,air_date=20160105

meta_int

Asset (Library)

Adds metadata containing an integer value to the asset.

Key information:

  • Field names must be unique and are case-insensitive.
  • Non-integer values will be set to 0.
  • Assign multiple metadata fields to an asset by using three commas to delimit each field/value pair.

Syntax:

meta_int = is_ad=1

meta_int = is_ad=1,,,year=2013

mix_atracks

Asset

Mixes multiple audio tracks together into a single track.

A sample use case is when the source audio has separate left and right audio tracks instead of a single stereo audio track.

Key information:

  • An audio track is a zero-based number where 0 represents the first audio track in the source file.
  • Specify the desired channel using one of the following values:

    • L
    • R
    • C
    • RL
    • RR
    • SL
    • SR
    • LFE
  • Use a comma to delimit each specified audio track.

Syntax:

mix_atracks = 0=L,1=R,2=C,3=RL,4=RR,5=LFE

multibot

Directory Monitoring

Indicates whether multiple instances of Slicebot are monitoring the same directory.

Valid values are:

  • 0: Indicates that only a single instance of Slicebot is monitoring any given directory.
  • 1: Indicates that more than one instance of Slicebot is monitoring a directory.

    All instances of Slicebot consuming source media from the same folder must be configured to enable multibot mode.

    Each instance of Slicebot should run on a separate computer.

Default value: 1

multibot = 1

notify_retries

Notifications

Determines the maximum number of attempts to send out a HTTP notification before proceeding to the next file. Notifications are defined through the following settings:

Default value: 3

notify_retries = 10

notify_retry_wait

Notifications

Determines the number of seconds that must pass after a failed notification before reattempting to send out a notification.

Default value: 10

notify_retry_wait = 30

password

Deprecated

This setting and the SLICER_PASS environment variable have been deprecated and should not be used. Please use the apikey setting or the SLICER_APIKEY environment variable instead.
Learn more.

Defines the password that corresponds to the user defined by the username setting.

password = 1@D208PneG63f

poster

Asset

Generates a poster image from the video frame corresponding to the specified time.

This parameter is ignored when poster_file has been defined.

Syntax:

poster = 5000

poster_file

Asset

Sets the asset's poster image to the specified file.

Syntax:

poster_file = /images/poster.png

poster_size

Asset

Sets the size of the poster image.

Key information:

  • This parameter is ignored when poster_file has been defined.
  • If both the poster and poster_file parameters have not been defined, then the poster image will be a thumbnail of your on-demand content.
  • Specifying a resolution higher than the source image will not result in upscaling. The poster image will remain the same resolution as the source image.
  • Poster images are scaled down to preserve aspect ratio.

Syntax:

poster_size = 512x512

require_captions

Processing Prerequisite

Determines whether a closed captioning file is required.

Learn more.

Valid values are:

  • 1|true: The source media will not be processed unless a corresponding closed captioning file is found.
  • 0|false: Source media will be processed regardless of whether a corresponding closed captioning file is found.

require_captions = 1

require_config

Processing Prerequisite

Determines whether a file-specific configuration file is required.

Learn more.

Valid values are:

  • 1|true: The source media will not be processed unless a corresponding file-specific configuration file is found.
  • 0|false: Source media will be processed regardless of whether a corresponding file-specific configuration file is found.

This is an invalid setting for file-specific configuration files.

require_config = 1

skip_drm

Asset

Determines whether the asset's playback URLs must be signed.

Valid values are:

  • 1: Encrypts the asset, but playback does not require a signed playback URL.

    This configuration is useful for ads that require playback independent from a playback session.

  • 0: Playback for that asset requires a signed playback URL.

skip_drm = 1

start_url

Defines the URL to which a HTTP POST request will be submitted whenever slicing is initiated on a file.

Learn more.

start_url = http://myserver.net/myservice

thumbnails

Asset

Adds one or more thumbnails.

Key information:

  • A default thumbnail will be created regardless of whether this setting is defined.
  • The file name for this thumbnail will be assigned the specified prefix.
  • This thumbnail will be bounded by the specified width and height. The thumbnail will fit within the specified dimensions, while maintaining the source video's aspect ratio.
  • Add multiple thumbnails by delimiting each one with a comma.
  • Generating additional thumbnails may incur additional encoding and storage costs. Additionally, it may impact processing speed.

Syntax:

thumbnails = tiny:50x50,med:400x400

timedmeta

Asset

Adds timed metadata at the specified time.

Single Field Syntax:

Multiple Fields Syntax:

timedmeta = Time1:Field1=Value1,Time2:Field2=Value2,TimeN:FieldN=ValueN

timedmeta = 0:TIT2=MyTitle, 2000:TCOM=Myself

trim

Asset

Trims the content from the source media that occurs prior to the specified start time. Optionally, duration may be specified to trim content that exceeds the specified time period.

Specify StartTime and Duration in milliseconds.

The source media's file type may affect video frame accuracy by a few milliseconds.

Trim Content Prior to Start Time Syntax:

Trim Content by Start Time and Duration Syntax:

The sample configuration, shown to the right, will exclude the first 3 seconds of the source media and truncate it so that the asset's total duration is 52.5 seconds.

trim = 3000,52500

username

Required

Authorization

Defines the user name corresponding to the account to which the asset will be uploaded.

Bypass this requirement by setting the SLICER_USER and SLICER_APIKEY environment variables.

username = joe@example.com

Closed Captions

You can provide slicebot with closed caption information by placing closed caption data in a TTML file in the same directory as the media file. TTML files are simple text files that can be created with third party tools or with any text editor:

<?xml version="1.0" encoding="utf-8"?>
<tt xml:lang="en" xmlns="http://www.w3.org/2006/04/ttaf1" xmlns:tts="http://www.w3.org/2006/04/ttaf1#styling">
  <head>
    <styling>
      <style id="defaultSpeaker" tts:fontSize="24" tts:fontFamily="Arial" tts:fontWeight="normal" tts:fontStyle="normal" tts:textDecoration="none" tts:color="white" tts:backgroundColor="black" tts:textAlign="center" />
      <style id="defaultCaption" tts:fontSize="24" tts:fontFamily="Arial" tts:fontWeight="normal" tts:fontStyle="normal" tts:textDecoration="none" tts:color="white" tts:backgroundColor="black" tts:textAlign="center" />
    </styling>
  </head>
  <body style="defaultCaption" id="thebody">
    <div xml:lang="en">
      <p begin="00:00:01.420" end="00:00:02.620"><metadata ccrow="14" cccol="3" /><span tts:fontStyle="italic">THIS IS SOME TEXT...</span></p>
      <p begin="00:00:06.620" end="00:00:09.790"><metadata ccrow="13" cccol="1" />(Doorbell rings)</p>
      <p begin="00:00:11.100" end="00:00:13.000"><metadata ccrow="14" cccol="8" />♪ LA LA LA SINGING ♪</p>
      <p begin="00:00:20.460" end="00:00:22.170"><metadata ccrow="14" cccol="0" />THIS IS MORE TEXT</p>
    </div>
  </body>
</tt>

The require_captions config setting can be used to ensure that slicebot will skip a file if it does not have a corresponding TTML file. As with other settings, it can be used in the global, directory, or per-file context.

The TTML files use a naming convention similar to config files: take the name of the associated media file and add a ".ttml" extension to the filename. For example, if slicebot is processing a file named /tmp/foo/film.mp4, it will look to see if there is a file named /tmp/foo/film.mp4.ttml and, if so, use it for the source of closed caption data. If no corresponding TTML file exists, the video will not have any closed caption data. Note that the helper_noext setting can alter the naming convention for TTML files.

Only hh:mm:ss.sss time formats are currently supported in the TTML file.

Metadata Files

You can assign metadata key-value pairs to an asset using individual settings; for larger amounts of metadata, however, this can be cumbersome, so it may be more convenient to make use of a metadata file. This file is a JSON text file that has the metadata you want associated with the media file. As with caption and config files, the name of the metadata file is the media filename with an additional extension: .meta.json. For example, if a media file is named video123.mp4, then an associated metadata file for it would be named video123.mp4.meta.json.

The contents of the metadata file must be a valid JSON object, or processing the file will produce an error. Structured metadata is supported, although presently it is not fully displayed or editable in the CMS UI.

{
    "genre" : "comedy",
    "network" : "ZNN",
    "season" : 1,
    "episode" : 13
}

In general, a given media file will use individual metadata settings or a metadata file, but not both. However, in the event that both are specified for a particular file, metadata from a file is applied first, followed by any metadata settings.

HTTP Notifications

HTTP notifications provide an additional way to integrate a system with your content production workflow. This type of notification allows Slicebot to submit a HTTP POST request whenever one of the following events takes place:

Enable HTTP notifications by setting one or more of the above settings to the URL that will be requested.

Email notifications may also be sent whenever slicing completes (done_emails) or when an error occurs (fail_emails).

HTTP POST Request

Slicebot will submit a HTTP POST request using the default Internet media type (i.e., application/x-www-form-urlencoded). The body of this request will consist of a single line with the following parameters:

Parameter Description

filename

Indicates the full path and the filename of the file.

asset_id

Indicates the system-generated unique ID of the new asset.

external_id

Indicates the external ID defined in the configuration. If one has not been defined, then it will be set to an empty string.

description

Indicates the file's description defined in the configuration. If one has not been defined, then it will be set to an empty string.

status

Indicates the result of the slicing operation. Valid values are either "ok" or "error."

In addition to the above parameters, any custom query string parameters defined in the configuration setting (i.e., start_url, done_url, and fail_url) will be automatically stripped from the request and inserted into the request body. This allows custom parameters to be passed to the remote server as needed.

View a sample scenario.

Custom query string parameters whose names match one of the above system-defined parameters (e.g., filename or description) will be ignored.

To allow for future expansion, the server process that receives the HTTP POST should silently ignore unrecognized parameters.

Notification Issues

Slicebot will log an error under the following conditions:

After which, it will reattempt the HTTP notification several times as determined by the notify_retries setting. After each attempt, it will pause for a few seconds as determined by the notify_retry_wait setting. If it is unable to successfully submit the HTTP POST request, then it will move on to the next file.

Sample HTTP Notification Scenario

This sample HTTP notification scenario assumes that the done_url setting is configured to:

http://www.example.com/slicing/handler?custom=1234&other=5678

The following sample HTTP POST request will be submitted once a file called "video.m4v" has been successfully sliced.

POST /slicing/handler HTTP/1.1

Host: www.example.com

Accept-Encoding: identity

Content-Length: 161

Content-Type: application/x-www-form-urlencoded

Connection: close

User-Agent: upLynk-slicebot/1.0

status=ok&other=5678&description=The%20file&filename=%2Fopt%2Ft%2Fmyvideos%2Fvideo.m4v&external_id=dummy_id&custom=1234&asset_id=a9119d7afc5242319d028794ae283ea7

Notice that the custom query string parameters (i.e., custom and other) defined in the done_url setting were automatically stripped from the request and inserted into the request body.