JP1 Remotes • IrpTransmogrifier: new program/library for IRP protocols

Page 1 of 14

IrpTransmogrifier: new program/library for IRP protocols

Posted: Mon Aug 12, 2019 1:14 pm

by Barf

After almost four years: First public release of IrpTransmogrifier, called version 1.0.1.!

This is a completely new program, written from scratch, that is intended to replace IrpMaster, DecodeIR, and (most of) ExchangeIR, and much more, like potentially replacing hand written decoders/renders. The project consists of an API library that is also callable from the command line as a command line program. It does not come with a GUI, but is intended to be integrated into GUI programs, like IrScrutinizer.

It has a data base file of protocols in IRP format. This drives both the rendering and the decoder. So, in contrast to DecodeIR, the behavior can be (almost) completely customized from that file, together with a number of parameters.

As a command line program, it is a program with sub-commands. Sub-commands are, for example

* render (for rendering signals from protocol/parameters),
* decode (decode signals like DecodeIR)
* analyze (from one or many signals, compute IRP form)
* expression (compute IRP expression)
* list (list information in the protocol data base)

and more.

Here are some bat files as example.

I intend to do a Youtube video within then next few days demonstrating the command line program.

Online reference manual

This thread continues this one.

A questions from that thread:

The Robman wrote:
Here's what I tried:

irptransmogrifier decode "0000 006C 0024 0002 0159 00AD 0015 0016 0015 0016 0015 0016 0015 0016 0015 0016 0015 0016 0015 0016 0015 0041 0015 0016 0015 0041 0015 0041 0015 0016 0015 0016 0015 0041 0015 0016 0015 0016 0015 0016 0015 0016 0015 0016 0015 0041 0015 0016 0015 0016 0015 0016 0015 0041 0015 0041 0015 0041 0015 0041 0015 0016 0015 0041 0015 0041 0015 0041 0015 0016 0015 06C0 0159 0057 0015 0E81 0159 0057 0015 0E81" > output.txt
Actually, it is without the double quotes (I will see if I can make it grok them anyhow). But if you remove them, you will still get -- nothing. This is not an error but means that no decode has been found. In the strict sense, this is correct: the signal is a NEC1 with one copy of the repeat-ditto tucked onto the intro, and there is no known protocol that looks like that. I am not saying this is the ideal behavior of a decoder. Instead, the use case of loose matching, "guessing", I have simply not yet implemented. I know that DecodeIR, and also Dave's teaser presently do a better job here. It is planned.

It now works with AND without the quotes. AND the somewhat flawed signal poses no problem any more:

Code: Select all

$ irptransmogrifier decode 0000 006C 0024 0002 0159 00AD 0015 0016 0015 0016 0015 0016 0015 0016 0015 0016 0015 0016 0015 0016 0015 0041 0015 0016 0015 0041 0015 0041 0015 0016 0015 0016 0015 0041 0015 0016 0015 0016 0015 0016 0015 0016 0015 0016 0015 0041 0015 0016 0015 0016 0015 0016 0015 0041 0015 0041 0015 0041 0015 0041 0015 0016 0015 0041 0015 0041 0015 0041 0015 0016 0015 06C0 0159 0057 0015 0E81 0159 0057 0015 0E81
        NEC1-rnc: {D=128,F=136,S=38}
        NEC1: {D=128,F=136,S=38}

The program can be downloaded here, just unpack the file IrpTransmogrifier-1.0.1-bin.zip into an empty directory and execute irptransmogrifier.bat (Windows) or irptransmogrifier.sh (Linux & MacOs) from the command line. Installing the newest version 2.0.1 of IrScrutinizer will also give access to the program.

Keep transmogrifying!!

Posted: Thu Aug 22, 2019 12:10 pm

by mathdon

Barf, I am trying to incorporate IrpTransmogrifier into RMIR as an alternative to DecodeIR, using the .jar file as a Java library. It works on a few protocols, but not on many others. I am baffled by what is going on. The heart of the code is:

Code: Select all

        irSignal = Pronto.parse( ps.toStringArray());
        Decoder decoder = new Decoder( new IrpDatabase( ( String )null ) );
        Map<String, Decode> sigDecodes = decoder.decodeIrSignal( irSignal );

As far as I can tell, this is what the command line does with a Pronto string. This works with some protocols, such as MCE and Panasonic but not others, such as Sony. Here is a Sony signal:

Code: Select all

0000 0067 0000 0010 0060 0018 0018 0018 0018 0018 0018 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0030 0018 0018 0018 0030 0018 0030 0018 0018 0018 0018 0018 0018 0335

Whether I feed this into Pronto.parse as a string or as a string array, I get no decodes. But if I feed it into a command line then it decodes as Sony15. Why the difference? What am I doing wrong?

I have tried creating the IrSignal by other means, as I don't want to have to convert to Pronto and then to IrSignal. Each way behaves the same. I've written this using Pronto as that is the direct comparison I can make with the command line version, not because that is my final intention.

A separate question is that DecodeIR outputs a hex field as well as an OBC. I can't see such a field from IrpTransmogrifier. I take it that is because the hex value is really a property of the executor, not of the protocol. Is that the case?

Posted: Thu Aug 22, 2019 2:34 pm

by Barf

mathdon wrote:Barf, I am trying to incorporate IrpTransmogrifier into RMIR as an alternative to DecodeIR, using the .jar file as a Java library.

Kewl! I feel honoured!

It works on a few protocols, but not on many others. I am baffled by what is going on. The heart of the code is:
Code: Select all
        irSignal = Pronto.parse( ps.toStringArray());
        Decoder decoder = new Decoder( new IrpDatabase( ( String )null ) );
        Map<String, Decode> sigDecodes = decoder.decodeIrSignal( irSignal );

Looks sane, but I cannot reproduce the problem. Please show me your code. This works:

Code: Select all

package javaapplication7;

import java.util.Map;
import org.harctoolbox.ircore.IrSignal;
import org.harctoolbox.ircore.Pronto;
import org.harctoolbox.irp.Decoder;
import org.harctoolbox.irp.Decoder.Decode;
import org.harctoolbox.irp.IrpDatabase;

public class JavaApplication7 {

    public static void main(String[] args) {
        try {
            IrSignal irSignal = Pronto.parse("0000 0067 0000 0010 0060 0018 0018 0018 0018 0018 0018 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0030 0018 0018 0018 0030 0018 0030 0018 0018 0018 0018 0018 0018 0335");
            IrpDatabase irpDatabase = new IrpDatabase((String) null);
            System.out.println("We have " + irpDatabase.size() + " protocols in the data base!");
            Decoder decoder = new Decoder(irpDatabase);
            Map<String, Decode> sigDecodes = decoder.decodeIrSignal(irSignal);
            for (Decode decode :  sigDecodes.values())
                System.out.println(decode);
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }
}

...
Whether I feed this into Pronto.parse as a string or as a string array, I get no decodes. But if I feed it into a command line then it decodes as Sony15. Why the difference? What am I doing wrong?

Show me the code and I will tell you.

I have tried creating the IrSignal by other means, as I don't want to have to convert to Pronto and then to IrSignal.

There are a number of constructors for IrSignal. If you have a more concrete question ("how do I ..."), I will be happy to answer. (Make sure you get the difference between IrSequence, ModulatedIrSequence, and IrSignal.)

A separate question is that DecodeIR outputs a hex field as well as an OBC. I can't see such a field from IrpTransmogrifier. I take it that is because the hex value is really a property of the executor, not of the protocol. Is that the case?

Yes. The "hex" is in the problem domain of executors and thus RM. There is also no "hex" in your IRP spec. Any suggestions?

Posted: Thu Aug 22, 2019 4:22 pm

by mathdon

Many thanks for your quick reply. I thought I was probably making some mistake that would be obvious to you, but at least I know now that is not the case and that my approach appears, on the surface at least, to be sensible. I will do some more experiments and if I can't resolve it then I will post the full code for you to look at. I won't have time to do this till the weekend but will get back to you when I have done so.

On the matter of the hex, I feel that as it is not a true part of the decode, the Learned Signals tab in RMIR should not show it. It can be positively misleading as different executors can use different hex for the same signal. But I need to investigate whether this would cause problems elsewhere in RMIR, before making any change of this sort.

Posted: Fri Aug 23, 2019 2:22 am

by Barf

OK, the weekend is only hours away

I have a gut-feeling that what could have gone wrong is the IrpDatabase, loading from the IrpProtocols.xml contained in the jar. (Therefore I print the .size() above.) But it is only a guess.

Hex: One reasonably clean way of implementing such an "additional computed quantity" would be to have an extra XML element in the IrpProtocols.xml, like (contained within the NEC1 protocol)

Code: Select all

<irp:aux-parameter name="hex_005A">~F:-8</irp:aux-parameter>

(In English: F (= OBC) is bit-inverted and reversed (in 8-bit context) producing the hex parameter of the 005A executor.) There should then be some API function for a Protocol to compute the value, given the input values. Opinions?

Posted: Fri Aug 23, 2019 3:57 am

by mathdon

I have found a little time this morning to do a bit of testing, and now have got it working. I think there was more than one issue, as I am sure that earlier the decodes were not being generated, but the final one was down to my carelessness. My post-processing to convert the decodes into the form needed by RMIR was requiring D, S and F parameters all to be present in the decode, so protocols with no subdevice were not showing in RMIR. There must have been other issues too, though, as the missing S parameter was causing an error message in rmaster.err and yesterday I was checking rmaster.err and finding no errors. But now it is working, I'm not going to anguish over just what was wrong. Many thanks for your help. I'm happy with the IrSignal creation as I am now using the duration list and the sizes of the onetime and repeat sections, so avoiding the need for Pronto.

A quick glance has shown that there are some learned signals that are decoded by DecodeIR but not by IrpTransmogrifier. I will look at these when I have more time to see if I can see a reason, but will probably need to get back to you on it. I haven't had time to think about your suggestion regarding hex yet, and again will get back to you on it.

Posted: Fri Aug 23, 2019 5:16 am

by Barf

mathdon wrote:... requiring D, S and F parameters all to be present in the decode, so protocols with no subdevice were not showing ...

You appear to have found a/the solution, but just "for the rest of you":

Code: Select all

Decoder.DecoderParameters decoderParams = new Decoder.DecoderParameters();
decoderParams.setRemoveDefaultedParameters(false);
... = decoder.decodeIrSignal(irSignal, decodeParams);

A quick glance has shown that there are some learned signals that are decoded by DecodeIR but not by IrpTransmogrifier.

I am sure there are. A subset of these are surely "false positives" from DecodeIR. There is one issue I know of. Please post the questionable (non-)decodes here.

Posted: Fri Aug 23, 2019 12:20 pm

by mathdon

I have found some more time today, so here are some signals decoded by DecodeIR but not IrpTransmogrifier. The decodes by DecodeIR are almost certainly valid as they are in a sequence of learned signals with the same protocol. I am currently using a setup by mdavej on an XSight Touch that includes 152 learned signals. Of these, 10 are not decoded by DecodeIR or IrpTransmogrifier but there are 22 that are decoded by DecodeIR but not IrpTransmogrifier. I have listed some of them below, converted to Pronto by RMIR, together with their DecodeIR decodes:

Code: Select all

0000 0067 000B 0010 0015 0018 0030 0018 0018 0018 0030 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0030 02D5 0060 0018 0030 0018 0018 0018 0030 0018 0030 0018 0030 0018 0030 0018 0018 0018 0030 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0030 02D5 (Sony15, D=151, F=61)

0000 0067 0006 000D 000A 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 0424 0060 0018 0018 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 0424 (Sony12, D=1, F=4)

0000 0071 0008 0032 0010 0010 0010 0010 0010 0010 0010 0030 0010 0030 0010 0010 0010 0030 0010 0AA3 0080 0040 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0030 0010 0030 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0030 0010 0030 0010 0010 0010 0030 0010 0AA3 0080 0040 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0AA3 (Panasonic, D=176, S=16, F=17)

0000 006D 0000 000B 004C 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0013 074C 004C 0026 0026 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0013 074C (OrtekMCE, D=21, F=21)

0000 006D 001A 0000 0157 00AC 0013 0055 0013 00AC 0013 0055 0013 00AC 0013 00AC 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 00AC 0013 0055 0013 00AC 0013 0055 0013 0498 0157 0055 0013 0D24 0157 0055 0013 0D24 0157 0055 0013 0D24 0157 0055 0013 1365 (G.I. Cable, D=0, F=26)

The 3rd and 4th of these report "Inconsistent length in Pronto Hex" in IrpTransmogrifier but are accepted back by RMIR as new learned signals and decoded as listed. I don't know whether it is RMIR or IrpTransmogrifier that is wrong.

Posted: Sat Aug 24, 2019 2:17 am

by Barf

First, "inconsistent lengths" are detected when parsing the Pronto into an IrSignal, not during decoding as such. You may like to have a look at the consequences of accepting such erroneous signals in RM, and/or their origin.

0000 0067 000B 0010 0015 0018 0030 0018 0018 0018 0030 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0030 02D5 0060 0018 0030 0018 0018 0018 0030 0018 0030 0018 0030 0018 0030 0018 0018 0018 0030 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0030 02D5 (Sony15, D=151, F=61)

This is (computed by "irptransmogrifier analyze 0000 ... ")
{40.2k,594,msb}<1,-1|2,-1>(A:11,-17.421m,(4,-1,B:15,-17.421m)*){A=0x2e9,B=0x5ee9}
So we see that the intro sequence is junk, the repeat very well corresponds to the claimed Sony15. Remove the intro sequence, and it decodes. DecodeIR appears to "forgive" the junk intro sequence.

0000 0067 0006 000D 000A 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 0424 0060 0018 0018 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 0424 (Sony12, D=1, F=4)

In principle the same thing: junk as intro, remove and it decodes.

0000 0071 0008 0032 0010 0010 0010 0010 0010 0010 0010 0030 0010 0030 0010 0010 0010 0030 0010 0AA3 0080 0040 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0030 0010 0030 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0030 0010 0030 0010 0010 0010 0030 0010 0AA3 0080 0040 0010 0010 0010 0030 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0010 0AA3 (Panasonic, D=176, S=16, F=17)

Inconsistent lengths, junk both at the start and at the end (indicated as italics). After removing, decodes as expected.

0000 006D 0000 000B 004C 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0013 074C 004C 0026 0026 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0013 074C (OrtekMCE, D=21, F=21)

Inconsistent lengths, so I changed the third number to 000B. Then it almost decodes:

Code: Select all

$ irptransmogrifier --loglevel fine  decode -p ortekmce  0000 006D 000B 000B 004C 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0013 074C 004C 0026 0026 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0013 074C
[org.harctoolbox.irp.ParameterCollector add] FINE: Name inconsistency: P, new value: 1&11, old value: 0&0 (0)
[org.harctoolbox.irp.Decoder lambda$decodeIrSignal$3] FINE: Protocol OrtekMCE did not decode: Conflicting assignments of P, new: 1, old: 0

In English: the first P (recall the IRP {38.6k,480}<1,-1|-1,1>([P=0][P=1][P=2],4,-1,D:5,P:2,F:6,C:4,-48m)+{C=3+#D+#P+#F}[D:0..31,F:0..63] ) is supposed to be 0, but is 1. Erroneous signal, false positive for DecodeIR.

0000 006D 001A 0000 0157 00AC 0013 0055 0013 00AC 0013 0055 0013 00AC 0013 00AC 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 00AC 0013 0055 0013 00AC 0013 0055 0013 0498 0157 0055 0013 0D24 0157 0055 0013 0D24 0157 0055 0013 0D24 0157 0055 0013 1365 (G.I. Cable, D=0, F=26)

This decodes (??!)

Code: Select all

$ irptransmogrifier decode 0000 006D 001A 0000 0157 00AC 0013 0055 0013 00AC 0013 0055 0013 00AC 0013 00AC 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 00AC 0013 0055 0013 00AC 0013 0055 0013 0498 0157 0055 0013 0D24 0157 0055 0013 0D24 0157 0055 0013 0D24 0157 0055 0013 1365 
G.I.Cable: {D=0,F=26}, beg=0, end=51, reps=4

Posted: Sat Aug 24, 2019 4:26 pm

by mathdon

There is something strange happening with that final G.I.Cable decode. If I put that pronto string into a command line, as you have done, then it works. If I put it into the Java app you gave in your first reply to me above, then it doesn't work. But I thought they were identical?

I have thought further about the hex issue, and still think it is not appropriate to include an extra XML element to handle it. The hex is nothing to do with the protocol, it concerns the executor. I have been experimenting with using the command translators in protocols.ini to generate the hex. This requires linking the protocol from the decode to an executor in protocols.ini, but in many cases these have the same name. I have that working for such protocols and in the signals in mdavej's XSight setup it gives the same hex as does DecodeIR. The non-matching cases still have to be handled, and there are other special cases too, but I don't think it will be any more effort than adding extra XML elements, and probably less. What do you think of this?

Posted: Sun Aug 25, 2019 6:23 am

by mathdon

As a further thought on the hex issue, I think the identification between IrpTransmogrifier protocols and protocols.ini executors has a simple solution. I have discovered that the "uei-executor" XML element can be read from the IrpDatabase by code, and as far as I can tell, it is used only to include it in the HTML file. If its values in the XML file are brought up to date and include the variant name, such as 0058:2, then they can be used to identify the executor. There are still a few ambiguities where there is more than one protocol with the same executor and using different command translators, but I think these are combos that I would not need to use. Unless you see a problem with this, I intend to edit my copy of the XML file to test this out with some examples, and if it works then I would be prepared to do the full updating, if you would be happy with that. Any further entries needed to generate hex from the IrpTransmogrifier decode would go into protocols.ini, which is their natural home, rather than the XML file.

I am even beginning to think that it might be possible to make the conversion of learned signals to a device upgrade, which at present is pretty unreliable in RMIR, might be made to work reliably. That would be a longer term aim, though.

Posted: Sun Aug 25, 2019 11:18 am

by Barf

mathdon wrote:There is something strange happening with that final G.I.Cable decode. If I put that pronto string into a command line, as you have done, then it works. If I put it into the Java app you gave in your first reply to me above, then it doesn't work. But I thought they were identical?

You have not done anything wrong. This more hairy than I first thought. The G.I.Cable protocol consists of a long intro and a short ditto as repeat. Your signal consists of the intro and four copies of the dittos, all declared to be the signals intro. The repeat of your signal is declared empty. (Crappy learn really.) To classify this as a G.I.Cable signal is not entirely straightforward... The interactive program does this:

Code: Select all

 if (!strict && (irSignal.introOnly() || irSignal.repeatOnly())) {
                ModulatedIrSequence sequence = irSignal.toModulatedIrSequence();
                ....
                decodeIrSequence(decoder, sequence, name, maxNameLength);
}

i.e. transforms it to a ModulatedIrSequence (stripping the repeat info), and works on that -- successfully.

I am right now not quite sure which way is the correct one. (See the snippet at the end; if decoding as IrSignal fails, it tries as ModulatedIrSequence),

I have thought further about the hex issue, and still think it is not appropriate to include an extra XML element to handle it. The hex is nothing to do with the protocol, it concerns the executor. I have been experimenting with using the command translators in protocols.ini to generate the hex. This requires linking the protocol from the decode to an executor in protocols.ini, but in many cases these have the same name. I have that working for such protocols and in the signals in mdavej's XSight setup it gives the same hex as does DecodeIR. The non-matching cases still have to be handled, and there are other special cases too, but I don't think it will be any more effort than adding extra XML elements, and probably less. What do you think of this?

I have also been thinking... You can use the existing ItpTransmogrifier API, to compute extra variables without any changes, see the snippet at the end.

More explicitly, define an instance of the NameEngine class -- this is pretty much "definitions" in the specs. They contains "Expression"s (bare expression in the specs), and can be evaluated in the context of the just recovered protocol parameters. The code snippet below (appendParameters) computes some extra parameters and extends the map of the retrieved parameters.

I have discovered that the "uei-executor" XML element can be read from the IrpDatabase by code, ...

Strictly speaking, the XML element is "parameter", which has an attribute "name", which sometimes have the value "uei-executor". The idea is that you can have "arbitrary" parameters there, belonging to any problem domain. Irptransmogrifier just ignores what it does not understand. The user can enter whatever stuff he or she wants. The values for uei-executor is simply just taken from DecodeIR.html, without thinking. I would of course welcome if you submit a fixed version.

See the snippet at the end.

I am even beginning to think that it might be possible to make the conversion of learned signals to a device upgrade, which at present is pretty unreliable in RMIR, might be made to work reliably. That would be a longer term aim, though.

Kewl! Possibly you also can have a shot at improving this use case too?.

Appendix. This code summarizes the discussion so far.

Code: Select all

package javaapplication7;

import java.io.IOException;
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
import org.harctoolbox.ircore.InvalidArgumentException;
import org.harctoolbox.ircore.IrSignal;
import org.harctoolbox.ircore.ModulatedIrSequence;
import org.harctoolbox.ircore.Pronto;
import org.harctoolbox.ircore.ThisCannotHappenException;
import org.harctoolbox.irp.Decoder;
import org.harctoolbox.irp.Decoder.Decode;
import org.harctoolbox.irp.Expression;
import org.harctoolbox.irp.InvalidNameException;
import org.harctoolbox.irp.IrpDatabase;
import org.harctoolbox.irp.IrpParseException;
import org.harctoolbox.irp.NameEngine;
import org.harctoolbox.irp.NameUnassignedException;

public class MyDecoder {

    public static void main(String[] args) {
        try {
            MyDecoder dec = new MyDecoder();
            dec.decode("0000 0067 0000 0010 0060 0018 0018 0018 0018 0018 0018 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0030 0018 0018 0018 0030 0018 0030 0018 0018 0018 0018 0018 0018 0335");
            dec.decode("0000 006C 0022 0002 015B 00AD 0016 0016 0016 0016 0016 0041 0016 0041 0016 0016 0016 0016 0016 0016 0016 0016 0016 0041 0016 0041 0016 0016 0016 0016 0016 0041 0016 0041 0016 0041 0016 0041 0016 0016 0016 0041 0016 0016 0016 0016 0016 0016 0016 0041 0016 0016 0016 0016 0016 0041 0016 0016 0016 0041 0016 0041 0016 0041 0016 0016 0016 0041 0016 0041 0016 05F7 015B 0057 0016 0E6C");
            dec.decode("0000 006D 001A 0000 0157 00AC 0013 0055 0013 00AC 0013 0055 0013 00AC 0013 00AC 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 0055 0013 00AC 0013 0055 0013 00AC 0013 0055 0013 0498 0157 0055 0013 0D24 0157 0055 0013 0D24 0157 0055 0013 0D24 0157 0055 0013 1365");
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }

    private final IrpDatabase irpDatabase;
    private Decoder decoder;
    private Decoder.DecoderParameters decoderParams;

    /**
     * Map from protocol names to executor (identified by String (to avoid parsing stuff with/without colons :-))
     */
    private Map<String, String> protocolToExecutor;

    /**
     * For an executer, a collection of additional parameter to compute.
     */
    private Map<String, NameEngine> executorParams;

    public MyDecoder() throws IOException, IrpParseException, InvalidNameException {
        irpDatabase = new IrpDatabase((String) null);
        if (irpDatabase.size() == 0)
            throw new ThisCannotHappenException("Empty protocol data base!");
        String s = irpDatabase.getFirstProperty("Aiwa", "uei-executor");
        decoder = new Decoder(irpDatabase);
        decoderParams = new Decoder.DecoderParameters();
        decoderParams.setRemoveDefaultedParameters(false);


        protocolToExecutor = new HashMap<>();
        protocolToExecutor.put("nec1", "005A");

        executorParams = new HashMap<>();
        NameEngine exec005AParams = new NameEngine("{hex = ~F:-8, rawData = D:-32 | (S<<8):-32 | (F<<16):-32 | (~F<<24):-32}");
        executorParams.put("005A", exec005AParams);
    }

    public void decode(String pronto) throws Pronto.NonProntoFormatException, InvalidArgumentException, NameUnassignedException {
        decode(Pronto.parse(pronto));
    }

    public void decode(IrSignal irSignal) throws NameUnassignedException {
        //Decoder.setDebugProtocolRegExp("G.I.Cable");
        Map<String, Decode> sigDecodes = decoder.decodeIrSignal(irSignal, decoderParams);
        for (Decode decode : sigDecodes.values()) {
            appendParameters(decode);
            System.out.println(decode);
        }

        if (sigDecodes.isEmpty()) {
            ModulatedIrSequence seq = irSignal.toModulatedIrSequence();
            decode(seq);
        }
    }

    public void decode(ModulatedIrSequence irSequence) {
        Decoder.DecodeTree decodes = decoder.decode(irSequence, decoderParams);
        System.out.println(decodes);
    }

    private void appendParameters(Decode decode) throws NameUnassignedException {
        String protocolName = decode.getName().toLowerCase(Locale.US);
        NameEngine originalParameters = new NameEngine(decode.getMap());

        String executor = irpDatabase.getFirstProperty(protocolName, "uei-executor");
        if (executor == null)
            executor = protocolToExecutor.get(protocolName);

        if (executor != null) {
            NameEngine executorParameters = this.executorParams.get(executor);
            if (executorParameters != null) {
                for (Map.Entry<String, Expression> entry : executorParameters) {
                    String name = entry.getKey();
                    Expression exp = entry.getValue();
                    long value = exp.toLong(originalParameters);
                    decode.getMap().put(name, value);
                }
            }
        }
    }
}

Posted: Sat Aug 31, 2019 7:25 am

by mathdon

mathdon wrote:I am even beginning to think that it might be possible to make the conversion of learned signals to a device upgrade, which at present is pretty unreliable in RMIR, might be made to work reliably. That would be a longer term aim, though.

That has now become a short-term aim. Indeed, I have upped my target. I now believe it should be possible to make RMIR do any conversion of learned signals to upgrade that can be done manually, including conversion to combo executors that can handle several device codes in the same upgrade. This is a progress report on this aim.

I have extended the syntax of the uei-executor parameter in the IRP database as follows. It now has four parts, of which the last three are optional. These are a qualified PID such as 005E:2, an executor name in parentheses (), additional assignments in braces {} and executor parameter expressions in brackets [].

The executor name is required in a few instances where there is more than one executor with the same qualified PID. In these cases the qualified PID identifies the processor code but the device and command translators differ. I have a default matching between protocol name (from the IrpTransmogrifier database) and executor name (from protocols.ini) that handles most cases but probably not all.

The additional assignments define parameters auxiliary to those of the IrpTransmogrifier decode if these are needed for the parameter expressions.

The executor parameter expressions consist of two comma-separated lists, one for the device parameters and the other for the command parameters of the executor. These are in one-to-one correspondence with the parameters defined in protocols.ini, the two lists being separated by a semicolon. An expression may be omitted for a parameter if the default value is to be used, but the separating commas must be included. The expressions give the parameter values, avoiding any need for the parameter names to be given and so avoiding the naming inconsistencies present in protocols.ini. A very simple example of a uei-executor entry is:

Code: Select all

009E[S;D,F]

for the Aiwa Combo executor. This gives only the subdevice (S) in the Device Parameters, the device (D) and OBC (F) are both Command Parameters.

Many combo protocols are more complex than this, however, such as by having several device code values in their Device Parameters with an index given in the Command Parameters to specify which is to be used. In these cases the Device Parameters have to be determined from the full set of learned signals to be included in the upgrade. One signal alone is not sufficient. In this situation there is more than one possible mapping from the decode parameters to the Device Parameters of the executor. To allow for this, I have extended the IRP syntax of IrpTransmogrifier, the extensions being handled by a preprocessor so that they are never seen by the expression evaluator of IrpTransmogrifier. I will give full details of these extensions later, as they are still under development, but as a taster, here are entries for the Sony 12 and Sony 15 protocols to use the Sony 12/15 executor:

Code: Select all

Sony 12:  00CA[D?0,0?0,D?1,0?1;??,F]
Sony 15:  00CA{X=D<32?1:0}[D?0,X?0,D?1,X?1;??,F]

The executor has two pairs of Device Parameters, each pair consisting of a device code and a "Force Sony15" boolean. Device codes >=32 are always Sony 15, those <32 can be either protocol. The executor sends them as Sony 12 unless the Force flag is set. The additional assignment X in Sony 15 is 1 (= true) when D<32, else 0. Expressions ending in ?n for some integer n denote alternates, distinguished by the value n. The ?? expression then takes the value n. So the Sony 15 entry represents two alternates:

Code: Select all

{X<32:1:0}[D,X, , ;0,F] and {X<32:1:0}[ , ,D,X;1,F]

where I have converted the < required in the XML of the IrpTransmogrifier database back to the < sign it represents.

This is all implemented and has been tested to generate a combo protocol from three Sony 12 signals with device code 1 and three Sony 15 signals with device code 26, with the two protocols mixed in the order they are presented to the upgrade generator.

Barf, after incorporating the fixes you have given to handle the previous set of unrecognised signals, I still have 13 learned signals out of the 152 in mdavej's setup that are identified by DecodeIR but not IrpTransmogrifier. I hope we can in due course find out why this is so, but at present I am concentrating on this upgrade conversion.

Posted: Sun Sep 01, 2019 4:23 am

by Barf

IrpTransmogrifier version 1.1.0 has been released! This release contains a number of semi-important improvements and bug fixes. Release notes:

Code: Select all

Release Notes for IrpTransmogrifier 1.1.0.

The notation #n refers to issues number n at
https://github.com/bengtmartensson/IrpTransmogrifier/issues

===============================================================================
2019-08-31: Version 1.1.0

* API example code added in directory contributed.
* Code generation code are now in a separate project IrProtocolCodeGeneration #108
* Pronto signals are per default loosely parsed #118, #117
* Bugfixes/improvements in Analyzer/Biphase* #116
* IrSignal.toString produces silly result in the presence of ModulatedIrSequence #115
* Refacturing of main class IrpTransmogrifier #114
* XML basedir problem fixed. (IrScrutinizer #314)
* Logging for debugging of (non-) decodes vastly improved #93
* Removed (non-owned) schemas and entitys files, #101.
* Fixes for non-recognized options in decode and analyze. #110.
* analyze now has --girr option #103
* New protocol classification (list --classify), constant intro/repeat #102
* Modulation/demodulation support: new command demodulate, command render has new
  option --modulate; #100
* Remove support for the old ini file format, including convertconfig. #107
* Remove IctImporter #109

Logging of non-decodes (using --loglevel fine) is now working much better ("why did this not decode as xxx"). With the OrtekMCE example above:

Code: Select all

$ irptransmogrifier --loglevel fine  decode -p ortekmce 0000 006D 000B 000B 004C 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0013 074C 004C 0026 0026 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0026 0026 0013 0013 0026 0026 0026 0013 074C
[org.harctoolbox.irp.NamedProtocol recognize] FINE: Protocol: OrtekMCE: "{38.6k,480}<1,-1|-1,1>([P=0][P=1][P=2],4,-1,D:5,P:2,F:6,C:4,-48m)+{C=3+#D+#P+#F}[D:0..31,F:0..63]", actual data: Freq=38029Hz[+1998,-999,+999,-999,+999,-999,+500,-500,+999,-999,+999,-999,+999,-999,+999,-500,+500,-999,+999,-999,+500,-49121][+1998,-999,+999,-999,+999,-999,+999,-999,+500,-500,+999,-999,+999,-999,+999,-500,+500,-999,+999,-999,+500,-49121][]
[org.harctoolbox.irp.BitspecIrstream decode] FINE: intro >Level 0: "<1,-1|-1,1>(P=0,4,-1,D:5,P:2,F:6,C:4,-48m)*", IrSequence: [1998,999,999,999,999,999,500,500,999,999,999,999,999,999,999,500,500,999,999,999,500,49121]
[org.harctoolbox.irp.BareIrStream decode] FINE: >Level 1: "P=0,4,-1,D:5,P:2,F:6,C:4,-48m"
[org.harctoolbox.irp.Assignment decode] FINE: >Level 2: "P=0"
[org.harctoolbox.irp.Assignment decode] FINE: <Level 2: "P=0", result: 0
[org.harctoolbox.irp.Duration decode] FINE: >Level 2: "4"
[org.harctoolbox.irp.Duration decode] FINE: <Level 2: "4"
[org.harctoolbox.irp.Duration decode] FINE: >Level 2: "-1"
[org.harctoolbox.irp.Duration decode] FINE: <Level 2: "-1"
[org.harctoolbox.irp.FiniteBitField decode] FINE: >Level 2: "D:5"
[org.harctoolbox.irp.FiniteBitField decode] FINE: <Level 2: "D:5", result: 21
[org.harctoolbox.irp.FiniteBitField decode] FINE: >Level 2: "P:2"
[org.harctoolbox.irp.ParameterCollector add] FINE: Name inconsistency: P, new value: 1&11, old value: 0&0 (0)
[org.harctoolbox.irp.Decoder lambda$decodeIrSignal$3] FINE: Protocol OrtekMCE did not decode: Conflicting assignments of P, new: 1, old: 0

which hopefully is (somewhat) understandable.

[Graham: getting back to you later today]

Posted: Sun Sep 01, 2019 5:43 am

by Barf

Graham, interesting. Some comments:

I envision that the elements in IrpProtols.xml can look something like
this:

Code: Select all

  <irp:protocol c-name="Sony15" name="Sony15">
        <irp:parameter name="uei-executor"><![CDATA[
	    00CA ; D < 32; { hex = ..., Force = D < 32, ... }
	    ]]>
	</irp:parameter>
        <irp:parameter name="uei-executor"><![CDATA[
	    00DE ; ... ; { hex = ..., Force = D < 32, ... }
	    ]]>
	</irp:parameter>
        <irp:irp><![CDATA[{40k,600}<1,-1|2,-1>(4,-1,F:7,D:8,^45m)*[D:0..255,F:0..127]]]></irp:irp>
        <irp:documentation/>
    </irp:protocol>

(if using CDATA you do not need to write < for < etc.)
There can be more than one element <irp:parameter name="uei-executor">: it is up to subsequent processing to select one candidate.

There are three parts: the executor name (just a string for now, hexadecimal ID with or without version, with or without name), a predicate that, evaluate after the extra parametes, evaluates to non-zero or zero if the executor can be used or not, and finally the parameters, following the syntax and semantics of the definitions of the IRP notation. So can the executor parameters be computed, as well as "additional assignments".

I doubt that a distinguishing between "additional assignments" and "executor parameter" is meaningful.

I strongly recommend to keep within the (extended) IRP stuff, see this. For example, your ?n operator can be replaced by nested ?: like this for the resistor colour code:

Code: Select all

n == 0 ? black : n == 1 ? brown : n == 2 ? red : n == 3 ? orange ... : white

I really doubt that a preprocessor is a sensible solution. What problem are you adressing with it?

Barf, after incorporating the fixes you have given to handle the previous set of unrecognised signals, I still have 13 learned signals out of the 152 in mdavej's setup that are identified by DecodeIR but not IrpTransmogrifier. I

I will be happy to have a look at them. But IMHO decoding of IR signals/sequences is not the same use case as forensics (leading garbage etc)... Still, I may be able to give some support for the forensics case.

Minor comment: In IRP (and C) do not write "X<32 ? 1 : 0", just "X<32" suffices (this is not Java...).

To avoid misunderstandings: You do what you want with RM. I am just suggesting things.