MediaConch

Implementation and policy checking
on FFV1, Matroska, LPCM, and more

Jérôme Martinez, MediaArea

Experience Workshop - November 2016
PREFORMA logo European Commission logo FP7-ICT Programme logo

What is MediaConch?

MediaConch is a conformance checker

  • Implementation checker
  • Policy checker
  • Reporter
  • Fixer

What is MediaConch?

Implementation and Policy reporter
Reporter

What is MediaConch?

Implementation report:
Reporter
Policy report:
Reporter

What is MediaConch?

General information about your files
MediaInfo

What is MediaConch?

Inspect your files
Trace

What is MediaConch?

Policy editor
Policies

What is MediaConch?

Public policies
Public Policies

What is MediaConch?

Fixer

  • Segment sizes in Matroska
  • Matroska “bit flip” correction
  • FFV1 “bit flip” correction
    video_bitflip_detected

Integration

Archivematica is an integrated suite of open-source software tools that allows users to process digital objects from ingest to access in compliance with the ISO-OAIS functional model
video_bitflip_detected

MediaConch interfaces

  • Graphical interface
  • Web interface
  • Command line
  • Server (REST API)
  • (Work in progress) a library (.dll/.so/.dylib)

MediaConch output formats

  • XML (native format)
  • Text
  • HTML
  • (Work in progress) PDF
  • Tweakable! (with XSL)

Open source

  • GPLv3+ and MPLv2+
  • Relies on MediaInfo (metadata extraction tool)
  • Use well-known open source libraries: Qt, sqlite, libevent, libxml2, libxslt, libexslt...

Supported formats

  • Priorities for the implementation checker

    • Matroska
    • FFV1
    • PCM
  • Can accept any format supported by MediaInfo for the policy checker

    • MXF + JP2k
    • QuickTime/MOV
    • Audio files (WAV, BWF, AIFF...)
    • ...

Supported formats

Can be expanded

  • By plugins
    • Support of PDF checker: VeraPDF plugin
    • Support of TIFF checker: DPF Manager plugin
    • You use another checker? Let us know
  • By internal development
    • More tests on your preferred format is possible
    • It depends on you!

Versatile

Several input formats are accepted

  • FFV1 from MOV or AVI
  • Matroska with other video formats
  • (Work in progress) Extraction of a PDF or TIFF attachement from a Matroska container and analyze with a plugin (e.g. VeraPDF and DPF Manager)
  • ...

Versatile

Input can be from:

  • Files (local/network)
  • FTP/FTPS/SFTP
  • HTTP/HTTPS
  • Amazon S3

Versatile

Binaries are provided for:

  • Windows
  • Mac
    Homebrew users:"brew install mediaconch", that's all!
  • Linux (Ubuntu, Debian, Fedora, OpenSUSE...)
    Since Ubuntu 16.04 and Debian Testing/9 users:
    "apt-get install mediaconch" or in Ubuntu Store, that's all!
    (it is in the official distros repository)
  • Embedded devices? Doable
    (we tested it on a Raspberry Pi )
  • Can be ported on other distros (BSD...)

Standardization

  • Matroska is widely used but not (yet) standardized
  • FFV1 is gaining increasing usage in preservation contexts but is not (yet) standardized

CELLAR: IETF workgroup

FFV1 performance

  • NOA tested on SD 8-bit content:
    • i7-2600 (4 cores+HT, 3.4-3.8 GHz)
    • 3-4x real time
    • 4-5x decoding speed increase compared to JP2k
  • VIAA is testing on SD 10-bit content (FFmpeg 3.2):
    • E5-2698V3 (16 cores+HT, 2.3-3.6 GHz)
    • 0.7x real time/thread, 11-12x real time/all cores+HT
    • 3-4x decoding speed increase compared to JP2k
    • Better compression ratio by 8-10% compared to JP2k

FFV1 performance

  • This is an average, results varies depending on the content of files
    • From 0.4x to 2.4x (average 0.7x) real time/thread (encoding/decoding)
    • From 0.7x to 16x (average 3.5x) the speed of JP2k (FFmpeg)
  • Not convinced?
    • Test on your own files
    • MediaArea will provide test scripts
    • We can perform tests for you

Worldwide

  • 2 project leaders

    • Jerôme Martinez (Digital Media Analysis Specialist, France)
    • Dave Rice (Archivist, USA)
  • Presentations worldwide

    • IASA, France
    • FIAT/IFTA, Austria
    • FOSDEM, Belgium
    • AMIA, USA
    • Code4Lib, USA
    • JTS, Singapore
    • (3-6 October 2016) IPRES, Switzerland
    • (25-29 September 2016) IASA, USA

Matroska research corpus

What's next?

  • Continue to improve handling of huge collections
  • Continue to improve user interface
  • Support of embedded attachments
  • Statistics
  • Finish standardization of Matroska and FFV1
  • More conformance tests
  • More fixing cases

And after PREFORMA sponsorship?

It depends on you!

  • This is open source
  • Driven by user requests
  • Everyone can develop or sponsor a development
  • Potential features:
    • Support of tests for your prefered format
      (MOV? MXF? JP2k? WAV?)
    • Support of other checkers
      (BWF MetaEdit? QCTools?)
    • Integration in your workflow
    • ...

Example (Plugins)


Example (Plugins)

Example (Plugins)

Stay in touch

MediaArea: https://mediaarea.net, @MediaArea_net

MediaConch: https://mediaarea.net/MediaConch, @MediaConch

Jérôme Martinez: jerome@mediaarea.net

Slides: https://mediaarea.net/Events

License: CC BY