Open source tools

 

Some examples of open source projects

 

Jérôme Martinez
MediaArea

 

CNA Heritage Event
Technoport Luxembourg, November 2018

The four freedoms

  • The freedom to run the program as you wish, for any purpose
  • The freedom to study how the program works, and change it so it does your computing as you wish
  • The freedom to redistribute copies so you can help your neighbor
  • The freedom to distribute copies of your modified versions to others

FFmpeg

Tool for manipulating A/V files

Very versatile

Lot of formats supported

Lot of possibilities to manipulate A/V files

Command line tool

ffmprovisr

FFmpeg has a steep learning curve

ffmprovisr helps users through the command generation process so that more people can reap the benefits of FFmpeg

Maintained by archivists like you

https://amiaopensource.github.io/ffmprovisr/

ffmprovisr

QCTools

Helps users analyze and understand their digitilzed video files through use of audiovisual analytics and filtering

Based on FFmpeg

Graphical interface or command line

https://www.bavc.org/preserve-media/preservation-tools/qctools

(Development snapshots on
https://MediaArea.net/QCTools)

QCTools

QCTools

Colors of digitilzed video are weird? Let's check...

MediaInfo

Convenient unified display of the most relevant technical and tag data for video and audio files.

Graphical interface or command line or software library

https://MediaArea.net/MediaInfo

MediaInfo

MediaInfoOnline

MediaConch

Implementation and policy checking on FFV1, Matroska, LPCM (and more)

Based on MediaInfo

Graphical interface or command line

https://MediaArea.net/MediaConch

MediaConch

Implementation and Policy reporter
Reporter

MediaConch

Implementation report:
Reporter
Policy report:
Reporter

MediaConch

General information about your files
MediaInfo

MediaConch

Inspect your files
Trace

MediaConch

Policy editor
Policies

MediaConch

Public policies
Public Policies

BWF MetaEdit

Embedding, validating, and exporting of metadata in Broadcast WAVE Format (BWF) files

Supports the FADGI Broadcast WAVE Metadata Embedding Guidelines

Graphical interface or command line

https://MediaArea.net/BWFMetaEdit

BWF MetaEdit

BWF MetaEdit

AVI MetaEdit

Embedding, validating, and exporting of metadata in AVI (Standard and OpenDML) files

Supports the U.S. National Archives Guidelines recommandations

Graphical interface or command line

Is similar to BWF MetaEdit? Right, code from BWF MetaEdit was reused, limiting development cost

https://MediaArea.net/AVIMetaEdit

AVI MetaEdit

MOV MetaEdit

Embedding and editing of metadata in MOV (Apple QuickTime) or MP4 (ISO/IEC 14496-14 a.k.a. MPEG-4 Part 14) files.

It is currently focused on Universal Ad ID metadata and Pixel Aspect Ratio edition

Sponsors were not interested in more. Could be expanded on request

Graphical interface or command line

https://MediaArea.net/MOVMetaEdit

MOV MetaEdit

vrecord

Tool to digitise analogue video

Make videotape digitization or transfer easier.

Blackmagic Design capture cards currently supported

https://github.com/amiaopensource/vrecord

vrecord

VirtualDub FFV1

VirtualDub is a Windows transcoding tool with a GUI

No FFV1 supported by current version

Developers abandonned the tool (no more development)

But it is open source

"Nobody wants to update the tool? Let's add FFV1 support ourselves"

http://www.av-rd.com/projects/2017-virtualdub_ffv1.html

VirtualDub FFV1

RAWcooked

Encodes RAW audio-visual data (DPX/TIFF) into a losslessly compressed file (no more unplayable TAR!)

Metadata accompanying the RAW data are preserved (reversiblility)

Sidecar files, like MD5, LUT or XML, are in container attachments

https://MediaArea.net/RAWcooked

RAWcooked

  • Final package is 1.5-3x (usually 2x) smaller than DPX/TIFF
  • Checksum by "Cluster" (usually 1 second) at container level
  • Checksum by "Slice" (you choose how many per frame) at video level
  • Files are natively playable by lot of tools (FFmpeg, VLC...)

RAWcooked

  • Storage
    Save HDD/LTO space: either ~2x less cost for same redundacy or 2x more redundancy for ~ same cost
  • Transport
    Encode, transport, decode; you save bandwidth (€... and transfer speed) without changing something else in your workflow (same files after revert to DPX/TIFF)

RAWcooked

~20 DPX flavors (RGB/RGBA * bitdepth * filled/packed * Big/Little endian) supported

2 TIFF flavors (RGB * 16-bit bitddepth * Big/Little endian) supported

Uses FFmpeg FFV1 encoder (internal encoder planned), uses internal FFV1 decoder

Transparent development, all at https://github.com/MediaArea/RAWcooked

RAWcooked

Developed by MediaArea
https://MediaArea.net

Main sponsorship by AV Preservation by reto.ch
https://reto.ch

With additional financial support from some other archives:

Projects have similar patterns

This is open source

One can peek code from another project

Lot of libraries are shared

A communuity (e.g. archives from different countries) can build an ecosystem

Funding

Driven by user requests

Most of tools were funded after a need is detected by users

Why paying for something you don't control? (is your current choice future proof?)

Everyone (you included) can develop or sponsor a development

You can fork if you think you have a better idea than others

Users have full control

Actors

Funding: European Commission, MoMA, National Endowment for the Humanities, the Knight Foundation, Library of Congress and FADGI, NARA (National Archives and Records Administration), CNA (National Audiovisual Centre of Luxembourg), Nasjonalbiblioteket (National Library of Norway), IFI (Irish Film Institute), Northwestern University Libraries and lot of small sponsoring from tens of other small to large entities

Actors

Management: MediaArea, AVP, Bay Area Video Coalition...

Development: MediaArea, lot of individual developers, and lot of archivists!

Funding, management, development: not always same people. You decide.

Stay in touch

MediaArea: https://MediaArea.net, @MediaArea_net

Jérôme Martinez: jerome@MediaArea.net

Slides: https://MediaArea.net/Events

License: CC BY