Introduction

dvbindex is a program which is meant to read a number of MPEG-TS files,
probably with DVB metadata embedded inside them, and put this metadata along
with stream information inside an SQLite database.

It uses ffmpeg for obtaining information about the streams, and libdvbpsi for
reading the DVB metadata.

Why?

This program is mostly meant to be used by people working on STBs and/or TVs
which support DVB. Quite often, in order to reproduce a specific issue, a
specific stream is needed, for example one which has a number of table version
changes, or which has a service with a subtitle/audio stream in a very specific
language accompanied with a video stream in a particular resolution.

People who do this kind of work usually have lots of different streams, however
finding one with a particular combination of parameters is a tedious task : it
needs opening each file in a stream analyzer and looking at its parameters.
dvbindex is meant to make this task easier : by making appropriate queries
against the database it creates, one can significantly reduce the number of
streams that need to be inspected manually.

Since stream files can be quite large sometimes, care was taken to ensure that
the whole file is never read twice : instead, the file being read is analyzed
simultaneously by ffmpeg and libdvbpsi.

Compiling

Compilation was only tested under Linux, with ffmpeg 3.2.2, libdvbpsi 1.3.0,
and sqlite 3.16.2. Your mileage may vary with other OSes and/or library
versions : please submit bug reports if something doesn’t work with your
configuration.

Since dvbindex uses CMake, the whole process is limited to generating the
project on your platform and building it. If you’re using make as the build
backend, just do :

mkdir build
cd build
cmake ..
make

Usage

dvbindex should be handed the name of the database file to create (or modify)
as its first parameter. All the following parameters are paths to directories
and/or files for indexing. Very brief status information is output to the
standard error stream. After the program exits, the database file will contain
all information it could capture.

Any following invocations of the program don’t cause it to rebuild the database
from scratch : it skips all files that have already been indexed based on their
name and size.

Testing

Testing consists of running test/dvbindex-test.sh and passing the path to the
executable as one of its arguments. The stream repository that’s used to create
the reference database is available upon request.

Example queries

I’m not a SQL wizard, so I’m sure much more complicated (and useful) queries
can be thought of.

Get the number of services declared as non-scrambled in each of the indexed
files :

SELECT f.name, count(0) AS num_non_scrambled FROM services s
JOIN sdts sdt ON s.sdt_rowid = sdt.rowid
JOIN pats pat ON sdt.pat_rowid = pat.rowid
JOIN files f ON pat.file_rowid = f.rowid
WHERE s.scrambled = 0
GROUP BY f.rowid;

Get the list of files which have at least one 6-channel audio track :

SELECT f.name FROM aud_streams a
JOIN files f ON a.file_rowid = f.rowid
WHERE a.channels = 6
GROUP BY f.rowid;

Get the list of services which have a HD video track :

SELECT s.name FROM vid_streams v
JOIN pats pat ON v.file_rowid = pat.file_rowid
JOIN pmts pmt ON pmt.pat_rowid = pat.rowid AND pmt.program_number = (
  SELECT program_number FROM elem_streams es
  JOIN pmts pmt_for_es ON es.pmt_rowid = pmt_for_es.rowid
  WHERE es.pid = v.pid
)
JOIN sdts sdt ON sdt.pat_rowid = pat.rowid
JOIN services s ON s.sdt_rowid = sdt.rowid AND s.program_number = pmt.program_number
WHERE v.width = 1920 AND v.height = 1080

Get the list of services which have at least one teletext stream :

SELECT s.name, count(*) AS num_teletext_streams
FROM ttx_pages t
JOIN elem_streams es on t.elem_stream_rowid = es.rowid
JOIN pmts pm ON es.pmt_rowid = pm.rowid
JOIN pats pa ON pm.pat_rowid = pa.rowid
JOIN sdts sd ON sd.pat_rowid = pa.rowid
JOIN services s ON pm.program_number = s.program_number AND sd.rowid = s.sdt_rowid
GROUP BY es.pid, pm.program_number, sd.onid

Missing features

Lots. Currently, dvbindex only reads PAT and PMT, and SDT tables. Support for
all the other tables will be added in the future, along with their export to
the database.

There is no way to obtain any information about scrambled streams, so expect
these to have mostly NULLs in place of actual data.

Logging could probably be a bit more verbose.

Bugs

Lots, probably. I have about 10 streams that I used for testing, but
practically every stream is a totally different world. If you encounter any
incorrect behaviour, like crashes or missing metadata, please submit a bug
report along with the stream that was being read.