Teaser- A tool for decoding and analyzing IR signals

[3FG]

Teaser is a utility Java program which decodes IR signals, using IR protocol descriptions in the text file IRdefs.asc. Users may use the syntax of IRdefs.asc to write custom protocol definitions to decode unrecognized signals. Teaser also can also do some classification of IR bursts.

Compared to DecodeIR.dll, the decoding engine in Teaser

• Decodes a superset of protocols
  Is easily extensible by a user without recompilation
  Allows a quick way to investigate a new IR protocol. e.g. to find if it is lsb or msb or to try various partitioning of bits into Device/Subdevice/OBC.
  For each input signal, it can display all possible decodes for known IR protocols
  Prefers to work on a group of related IR signals to improve the accuracy of selection from the candidate decodes.

In future, it should be practical to augment the decoder engine to generate RMDU files automatically for all executors in protocols.ini.

[vickyg2003]

Hi Dave,
Wow this does sound interesting.

Allows a quick way to investigate a new IR protocol. e.g. to find if it is lsb or msb or to try various partitioning of bits into Device/Subdevice/OBC.

Quite frankly I don't know how I would get to that it has been a long time since I've seen an unknown signal, thanks to your hard work.

I had a trial run with your teaser program and do have some feedback.

First thing I would suggest, is that you add a filter to the dialog box for the load file, so that you don't see every file in a folder. I realize that I can type *.Lirc in the search box, and see only Lirc file types, but most people would be looking for a file drop down, Lirc, TXT, ICT....

Then I tried to open the only LIRC file I had on my system other than the ones that were included in your test file. Your program didn't like it. IRScrutinizer was able to open and process this file and decode it as Nec 1.253 type signals. I've uploaded the file for your inspection.
https://www.hifi-remote.com/forums/dload ... e_id=14519
I didn't read the instructions thoroughly, is there an error file I should be submitting too?

And lastly I see that you will do a pronto hex from a paste. Is that only for learns, or does it do short forms too? I suppose short forms are not applicable to an unknown protocol. And is there a reason it doesn't do UEI learns as well?

Vicky

[3FG]

Hi Vicky,
Thanks for trying it out. I'll add the filter to the dialog box.

LIRC files have two forms, and Teaser is intended to only work with the "raw" form. These raw files contain the string "RAW_CODES", and the data is a series of integer durations in microseconds. The LIRC file you tried is not of this type; it instead describes an already decoded signal, although not in the way we usually write decodes. I don't know how to reliably convert this latter type of file programmatically; the LIRC documentation is pretty opaque. So I have chosen not to try. This form only works with "gap" signals like NEC or RC5, if I understand the docs correctly.

I would like to accept UEI learns directly rather than using the RMIR Timing Summary. However, UEI has used three different learned formats. The RDF files tell RMIR which format a particular remote uses, and RMIR converts and displays the learns in the original format, but AFAIK we can only access one learn at a time. Nor is there a way to write the learned signals data to a file. So I think the Summary is the only practical way to get the learned data.

[Barf]

Dave, thank you very much for posting this, including the sources. I have to congratulate you for getting your program out before mine. I have found remarkably few bugs (some signals with only a NEC intro decode as NEC1 (instead of NEC)).

I think this thread should be located in the "software" forum.

You must have some license. Legally speaking, having no license at all means (AFAIK) in most countries that the receiver has no rights at all, including the usage of the program. It is "found-on-the-street-ware"? Here is a page that discusses different free software licenses. FYI, RM as well as my work use GPL,

Please consider maintaining the sources at SourceForge or Github, or another alternative. Git is considered a more modern version management system than subversion, but it has a "learning curve". Likewise, Github is surpassing SF, both in volume and in features.

Please consider using a more standard source tree layout, putting the "com" directory into a "src" directory", and supplying am Ant- or Maven build.

To Lirc: I have explained this a few time in the past. The problem is that no-one really knows what Lirc does with its files. Different keywords has been added, with unclear semantic. Although the main keywords are documented, this is not complete. Note the disclaimer therein:

LIRC was designed to collect IR data and save it in a private, compact, yet human readable format with the purpose of being able to re-transmit (or re-recognize) these signals. It was not designed with the goal of providing a well documented and tested configuration file format that could be used e.g., to generate arbitrary IR signals or to convert them to other formats. The configuration file should thus not be considered a public interface to LIRC.

This manpage should be understood with this in mind. It was authored by reading the code of LIRC, and does not constitute an authoritative specification of the behavior of LIRC and its configuration file. Also, some less commonly used flags and parameters are not documented.

(OK, I was the one who told the maintainer to put it there...) With this in mind, my standpoint is that only Lirc can decode Lirc. I translated the relevant parts of Lirc to Java, and called the outcome "Jirc" ("Lirc" with "J"(ava) instead of "L"(inux).) This is contained in IrScrutinizer, and can also be used as an independent part in other programs, subject to the conditions of its license (GPL).

I think we should try to write tools that fit into each other. There is no need to implement Lirc import functionally again.

You have chosen not to use the IRP notation, but instead invented your own language. I do not think this was a wise decision. IRP definitely has its quirks (in particular the repeat model), but we know it (including its quirks) fairly well, and it is well documented and implemented. You are suggesting another Lirc (in the sense above). How are we going to check the consistency between "teaserish" and IRP? (For example, in your Nova Pace, the IRP comment is not consistent with the code (D:11 should be D:10)). It is important to keep a common data base for both rendering and decoding, and possibly also for other applications, like code generation (PB?). Or should I support "teaser-ish" in IrScrutinizer?

I tried for some time yesterday to understand your Epson protocol, and gave up...

Oh, the name .... not very well chosen, IMHO.

Again, thank you very much for this.

Bengt

[3FG]

Bengt,
Thanks very much for taking the time to look at this. BTW, no congratulations are in order--I have backup files from February of 2013 with the core algorithm already programmed, so my progress has been very slow. As you have seen, my approach and motivations differs from yours.

My intent has been to provide a decoder which uses a single engine to do the decoding, rather than e.g. having one routine for "gap" signals and a different one for biphase signals. I believe that facilitates adding new protocols without much danger of breaking the algorithm(s). Secondly, I want a decoder which accepts protocol definitions from a text file, so that a user can add a new protocol easily. Third, since my interests are strongly aligned with UEI remotes, I want a way in which decoded signals can be automatically converted for use with UEI executors. (Both RMIR and IrScrutinizer offer some capability to do this, but only for simple executors.) I think the approach used in Teaser demonstrates that the first two desires are achievable, and I am confident that the third is also. The third desire also explains why the protocol definitions in IRdefs.asc yield a decode of "NEC1 Missing ditto frame" rather than "NEC"-- the UEI executors need to have either NEC1 or NEC2 specified. IMO, "NEC1 Missing ditto frame" is both an accurate description and clearer than "NEC".

When I started this effort, I planned to use IRP as the input. I had difficulty in getting your code to compile (I didn't understand how Antler worked), so I postponed using IRP as the input while I worked on the internal data structures and program flow. As time went on, however, I concluded that IRP was not a good choice.

• It can't represent the B&O protocol
  It is awkward in representing protocols with variable lead in structure, e.g. Zenith
  It describes burst sequences in time order from left to right, but by default parameters are described in lsb, which is the reverse time order.
  It appears that while parsers can recognize erroneous syntax, they have difficulty in pinpointing the error
  And that's important, because Bengt and Graham seem to be the only two people on the planet who can confidently write complicated IRP
  It doesn't have a mechanism for describing acceptable tolerances when decoding.
  Doesn't describe what to do when decoding a partial signal.
  Has no way to describe e.g. Gray coded signals

IRdefs.asc recognizes two part Pioneer signals and can distinguish these from a repeated single part Pioneer. At present, IRP appears to not be able to specify this difference. Of course, I'm currently the only person who can write in "teaserish", so that's a deficiency, but it demonstrably meets the need for at least the protocols definitions in IRdefs.asc. IRP doesn't, although I think simple IR protocols expressed in IRP could be imported, using suitable defaults for the non-specified items.

I don't agree that we need a single data base for decoding and rendering. It's desirable for consistency, but IMO it complicates handling newly discovered protocols. And the rendering I'm interested in (employing UEI executors) seems to me to be fundamentally different than e.g generating Pronto Hex.

In spite of the disagreements, I really do appreciate your comments. Thanks.

[The Robman]

Nice thread, moving to software forum.

[Barf]

Bengt,
I have backup files from February of 2013 with the core algorithm already programmed, so my progress has been very slow.

well, the Github repository for IrTransmogrifier was created in September 2015 (existed om my local disk possibly a year before), 370 commits, > 17k lines Java (4.5k of that is test code), and still no first version...

My intent has been to provide a decoder which uses a single engine to do the decoding, rather than e.g. having one routine for "gap" signals and a different one for biphase signals. I believe that facilitates adding new protocols without much danger of breaking the algorithm(s).

Sorry, do not follow.

Secondly, I want a decoder which accepts protocol definitions from a text file, so that a user can add a new protocol easily.

OK

Third, since my interests are strongly aligned with UEI remotes, I want a way in which decoded signals can be automatically converted for use with UEI executors.

Well, I have previously argued that this "mapping" contains decisions that are better left to the user. Nevertheless, having a smarter, possibly heuristic, mapping of a set of IR signals ("a remote" in IrScrutinizer terminology) might be a good idea, at least as long as the user can reject it. There can be a data base of rules how to map set of IR signals to UEI executors. BUT, isn't this a completely different usecase from decoding and identifying IR signals?

The third desire also explains why the protocol definitions in IRdefs.asc yield a decode of "NEC1 Missing ditto frame" rather than "NEC"-- the UEI executors need to have either NEC1 or NEC2 specified. IMO, "NEC1 Missing ditto frame" is both an accurate description and clearer than "NEC".

Seems like you like to have what DecodeIR calls "NEC" called "NEC1 Missing ditto frame" instead.

When I started this effort, I planned to use IRP as the input. I had difficulty in getting your code to compile (I didn't understand how Antler worked),

Instructions are here. If you have problems after reading that, I'll help. But please note that it might be a better idea to use the public API.

It can't represent the B&O protocol

Disagree

It is awkward in representing protocols with variable lead in structure, e.g. Zenith

Don't follow you here. Zenit is a PITA, but IMHO because of the variable length bitfield.

It describes burst sequences in time order from left to right, but by default parameters
are described in lsb, which is the reverse time order.

IIRC, when I write IrpMaster, I discovered that there was a discrepancy between the documentation ("the general spec says how it is to be interpreted") and the "established" IRP protocols (I think took them MSB), so I followed the latter. There may be some stuff to be clarified here. (It is "all academics"... which to me does not mean that it is unimportant.)

It appears that while parsers can recognize erroneous syntax, they have difficulty in pinpointing the error

IMHO, IrScrutinizer is not worse, possibly better, than comparable programs.

And that's important, because Bengt and Graham seem to be the only two people on the planet who can confidently write complicated IRP

May be some truth here () but we are also the two "mathematicians" of the forum. Reformulating algorithms into another setting (like the expression setting of IRP) is not something that you will be good at just because you master the target language.

It doesn't have a mechanism for describing acceptable tolerances when decoding.

Was not intended for that either. Such an attribute is easily added to the protocol data base. Oh, I think it is overkill to have a tolerance on every single durations. (You seem to have 30% on everything anyhow...)

Doesn't describe what to do when decoding a partial signal.

Was not meant for that either. In IrTransmogrifier I have some "decode-only" protocols like "NEC" (in the DecodeIR sense), Denon{1] and Denon{2}, which is how I envision that you tackle that problem.

Has no way to describe e.g. Gray coded signals

Agree. I would also like to mention CRC checksums, cf. Glow-with-the show. The way the latter was "solved" was with overparametrization: the user is required to enter the CRC value. This is not an attractive solution.

Actually, my criticism is somewhat orthogonal to yours: I think that it contains too may possibilities to express "sillyness" that we never would like to implement, in particulat w.r.t. repetitions, like (...)* ... (...)* and, even worse, (... (...)* ....)* (hierarchical repeats). (FYI: IrTransmogrifier rejects multiple repeats).

But most importantly, the few shortcomings of IRP, is that a reason to throw it all out and start all over again? If you throw something out and replace it, at least you should "prove" that it is better than what it replaces.

You approach amounts to introducing keywords with special hard-soldered semantics.

BTW, I found you collection of test files quite useful; thanx for that.

Still have some more things on my mind, but this is getting long enough...

Nice to have some more substantial discussion.

[3FG]

But most importantly, the few shortcomings of IRP, is that a reason to throw it all out and start all over again? If you throw something out and replace it, at least you should "prove" that it is better than what it replaces.

Yes, I did conclude that IRP is inadequate for specifying a IR signal decoder. I make no claim that teaserish is optimal (or even close), but it is adequate for specifying every IR protocol currently covered by DecodeIR, and more. I think the "IRP" you provided for B&O demonstrates quite well my issues with IRP:

Code: Select all

{455k,3125,msb}<200u,-zeroGap,zeroGap=2,oneGap=3,() | 200u,-oneGap,zeroGap=1,oneGap=2> 
(200u,-1,200u,-1,200u,-5,D:9,F:8,200u,-4,200u,-100m) 
{zeroGap=1, oneGap=3} 
[D:0..511,F:0..255]

It is IMO illegal, certainly undefined in the IRP specification, and very difficult to read. How many people could write it?

[Barf]

It is IMO illegal, certainly undefined in the IRP specification

There is no such thing as "IMO illegal", either it is valid (conforming with the specification), or not. It is valid (legal). In some detail: A bitspec consists of one of more (here two) bare_irstreams. A bare_irstream is a sequence of irstream items. Irstream items are, for example,assignments. QED.

and very difficult to read. How many people could write it?

Possibly; no-one has ever claimed that IRP is reasy to read. After all, you are cramming half a page (or more) of English text into one (or a few) lines. But, my bottom line is that this is, as opposed to the previous item, really an "IMHO". It is not a hard argument.

With your approach, you are instead putting the semantics into keywords, each of them having a short sentence of descriptions. The true meaning is in the Java code only. This is (possibly "IMHO" is adequate here) the mistakes of Lirc done again.

Although somewhat disappointed, I respect your decision. Again thank you very much for publishing this, including the sources and the test cases. And the discussion

[Barf]

According to the previous discussing,

0000 0009 0016 0000 005B 058E 005B 058E 005B 1BC5 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 058E 005B 1638 005B B1BC

is not a valid B&O signal (repeated encoded 0, however without using the repeat burst). However, teaser accepts it as

Code: Select all

                              B&O                    Dev=   0      OBC=   0         0 - 44
                              B&O repeating  Dev=   0      OBC=   0         0 - 42

[3FG]

Yes, that is not a valid B&O signal, if I understand the B&O document correctly. However, IRdefs.asc allows this as a reasonable decode. We do this for lots of signals that don't conform to specification. For example, with a few exceptions, signals with the wrong frequency are accepted provided the timing is close. Missing terminator frames and incorrect checksums may be deemed acceptable, although typically with a note. In this case, I have supposed that the chance of a mis-decode is quite small, given the singular nature of the B&O protocol.

With a gap signal like B&O, a repeat burst pair could be handled in two ways. For example, given the fragment 110000, and assigning A=0, B=1, and R as the repeat, the transmitted signal could be
BRARRR which ensures that the total duration of every signal is nearly constant since the duration of R is midway between those of 0 and 1.
or
BRARAR where the rule is that no burst pair is sent twice consecutively. This presumably would allow better error detection. We know that many B&O signals follow the first rule, and I think their specification calls for it, but we don't know if all of their receiving equipment only accepts the first rule. I don't see a good reason to return "unknown" for the above Pronto Hex. The may be a case for including a note that the signal is (probably) invalid, but I didn't include that possibility in Teaser.

[Barf]

There are two different use cases here:

1. decoding (in the sense of identifying valid signals), and
2. "guessing" what is meant by an invalid, but somehow "close to valid", signal.

There is nothing wrong by addressing the second use case, however, the user should request which use case he is interested in, possibly by an option in the program. "Near misses" should be clearly identified as such. A near miss is still a miss...

I would advocate that a program like this should be like a witness in court: telling "the truth, the whole truth, and noting but the truth", leaving the interpretation to the judge, or the user. Classifying an invalid signal as valid, possibly even without a message, is a bug.

But this also shows a fundamental problem with your approach of having the behavior defined by the implementation of (sometimes insufficiently documented) keywords.

We probably agree that one of the main problems with DecodeIR is that its decision rules are intransparent and inflexible while hard implemented in the C++ code. Your approach seems to (partially) suffer from the same sort of problems. Assume that someone does not like the "forgiving" behavior of the B&O decoding: there is no other possibility but to hack the Java code. (On the side, if we had a strict B&O recognizer, defining a "lax B&O protocol" would probably be fairly simple.)

I want to stress again that this is meant as constructive criticism. Great that you are doing this, and are sharing the result!