RMIR: The Next Generation

[mathdon]

As there has been only one minor comment on development build 7 of RMIR v2.08, I intend to issue it unchanged as an official release. That will be the last build of RMIR that is compatible with Java 7. The next release will be of RMIR v2.09 and will require Java 8 or later.

This change is required as v2.09 will incorporate Barf's IrpTransmogrifier as the default decoder for learned signals, in place of DecodeIR that will remain as a selectable option. IrpTransmogrifier requires Java 8, so the combination with RMIR also does so. This combination makes great things possible that were impossible with DecodeIR. The first great thing, which will be in RMIR v2.09 build 1, is conversion of learned signals to device upgrade that really works. This feature has been present in RMIR for quite some time now, but hidden on an Advanced menu as it works on only very simple cases. It will now be a permanently visible button on the Learned Signals tab. And when I say really works, I mean that the intention is that it should be able to create any device upgrade from a set of learned signals that could be created manually. This includes upgrades using combo protocols (executors) that support signals that do not all have the same device and subdevice codes, and even those that support multiple protocols (in the true meaning of the word).

There will be exceptions, but only a very few. Consider the Sony DSP executor whose entry in protocols.ini says

If your learned signal decode displays 3 repeats of Sony15 36,96, then 3 Sony8 entries followed by one more Sony15 entry, then you need to use this Sony DSP protocol. The first Sony8 entry gives you the device code (should be 195 for DSP), the second Sony8 entry gives you the function code (should be 81 for DSP), and the 3rd Sony8 entry gives you the OBC.

I see no way of handling this algorithmically. There will also be upgrades produced that require manual fine tuning after their creation. An example is the Barco protocol, where the boolean use toggle is a device parameter. Again I see no way of determining algorithmically how this should be set, so a created Barco upgrade will use the default value of false. Apart from such cases, I expect upgrades that are incorrect to be the result of bugs in either the coding or the table entries that are used. The changes and additions to RMIR involved in this new feature are so substantial that there are bound to be such bugs. When this new version is released, I hope users who find such bugs will post them so that they can be investigated and, if possible, fixed.

[The Robman]

This sounds exciting. When you started mentioning Sony DSP I thought you were about to describe that you COULD handle it, and I was going to be totally amazed!

[mathdon]

When you started mentioning Sony DSP I thought you were about to describe that you COULD handle it, and I was going to be totally amazed!

No, but I think I can handle Panasonic Mixed Combo (001F:8) and that is pretty weird too .

[Barf]

Kewl!! I feel honored to be able to contribute to RM in such a substantial way!

Would it be possible to include two wrapper scripts (one for Windows, one for Linux and Mac) that would allow the user to start IrpTransmogrifer as command line program, too? They would essentially say

Code: Select all

"%JAVA%" -cp RemoteMaster.jar org.harctoolbox.irp.IrpTransmogrifier  %*

Consider the Sony DSP executor whose entry in protocols.ini says

If your learned signal decode displays 3 repeats of Sony15 36,96, then 3 Sony8 entries followed by one more Sony15 entry, then you need to use this Sony DSP protocol. The first Sony8 entry gives you the device code (should be 195 for DSP), the second Sony8 entry gives you the function code (should be 81 for DSP), and the 3rd Sony8 entry gives you the OBC.

(The second sentence contradicts the first, Using the second for this example.)
Actually, in the context it would not be too hard. In IrpProtocols.xml add an entry like

Code: Select all

 <irp:protocol name="SonyDSP">
       <irp:parameter name="uei-executor">
              <rm:deployment name="Sony DSP">
                     <rm:assignment name="OBC">F</rm:assignment>
                      <rm:hex>F:-8</rm:hex>
               </rm:deployment>
        </irp:parameter>
        <irp:irp><![CDATA[{40k,600}<1,-1|2,-1>((4,-1,195:8,^45m)3,(4,-1,81:8,^45m)3,(4,-1,F:8,^45m)3,)[F:0..255]]]></irp:irp>
        <irp:documentation/>
    </irp:protocol>

(in the context of the IrpTransmogrifier thread). (Warning, have not tested this, not even the syntax) With this entry, IrpTransmogrifier will decode appropriate signals as "SonyDSP". Mapping this on an executor is in this case trivial, OBC=F, hex is this number backwards ("lsb").

[mathdon]

I have now posted RMIR v2.09 build 1 in the RMIR Development folder on SourceForge. It is a full installation package. I have posted it to allow testing of the learned to upgrade conversion before general release. It has not had as much testing as I would like, and in particular I have not uploaded any upgrades that it has produced to check that they work, but they look fine. I have also posted three .rmir files, two of XSight remotes and one of URC-6440, for testing the conversion process. Files 1 and 2 each include over 130 real learned signals. These are old user files posted during the development of XSight support in RMIR, to show bugs that are now fixed. File 3 includes three constructed learned signals, and there are also six such signals at the end of file 1. These are to test conversion situations not occurring in any of the real learned signals.

I suggest you begin testing as follows. Open test file 1 and select the top 6 files, which are Sony 12 and Sony 15 with device codes 1 and 26, then press the conversion button. You will be offered a choice of using either the Sony 12/15 or Sony Combo (12/15/20) executor. Choose Sony 12/15. After they are converted, select the six files at #19, which are more Sony 12 and Sony 15 with device codes 1, 151 and 164. Again opt for Sony 12/15 executor. You will now be asked if you want to append these to the upgrade already created, or create a new one. Select append. You will now be told that four of the files (device codes 151 and 164) are not consistent with the device parameters already set. You have a choice of appending the two that are consistent or of aborting the conversion. Choose whichever you wish.

Now repeat the entire process but selecting the Sony Combo (12/15/20) executor. Initially you will be asked if you want to append this to an existing device upgrade that uses this executor, or create a new one. Say new, but when adding the second set, append to the upgrade from the first set. Sony 12/15 only supports two different device codes but the combo executor, with two command bytes, supports any number, so the append will succeed completely. (The new device codes will be wrong, but I've traced that to a longstanding bug in RMIR that is nothing to do with the conversion process and which I only found after I had posted this build).

This will give you a taste of how the process works, so you can then try whatever you wish. Test file 1 has, at the bottom of the learned list, six learned signals created by the render facility of IrpTransmogrifier. These illustrate features that the real learned signals do not. All the real Panasonic signals have device 176, subdevice 16. This is not a good enough test for Panasonic Mixed Combo, which is one of the available executors so appending the two extra ones gives a better test. There are also two Pioneer-Mix signals that test the Pioneer Mix (007E:4) executor and two SonyDSP signals (see, I did it , spurred by Barf's suggestions!). Pioneer Mix (007E:5) is a more challenging variant but not available in the XSight, so test file 3 is a URC6440 setup with three Pioneer-Mix signals that test this variant.

You will find that there is a new entry Set IR decoder on the Options menu. This enables you to select DecodeIR instead of IrpTransmogrifier, but the conversion button is then disabled. DecodeIR is more forgiving than IrpTransmogrifier, so this option may be useful for signals that IrpTransmogrifier does not decode, and in particular for XMP signals for which DecodeIR has some correction algorithms to improve the decoding ability. You can change decoder "on the fly" while the Learned Signals tab is open, and the decodes will change accordingly.

Finally, at Barf's suggestion the RMIR installation folder has a batch file irptransmogrifier.bat which will run its command line interface in the same way as the file of the same name in the separate IrpTransmogrifier distribution. I hope Barf can provide the corresponding Linux .sh file.

[Barf]

Good job!! Some initial feedback (more probably later):

Please let Pronto, not UEI learned, be the default on the Learned Signals Detail popup. Or at least proved a selection.

Details on the signals not (uniquely) decoded by IrpTransmogrifier:

TransmogrifierTest1.rmir:
#9 leading garbage (we discussed that previously), however it decodes with the new option --ignoreLeadingGarbage (API: decodeParameter.setIgnoreLeadingGarbage(true))

#19 {40.2k,596,msb}<1,-1|2,-1>(A:17,-129m){A=0x12b47} unknown

#46 Leadout missing, decodes with --min-leadout 1300 (or less). Which is probably a very bad default, though.

#61 No leadin, makes no sense.

#62 Leadin missing, however second part decodes using --ignoreLeadingGarbage (new, not in 1.1.0).

#139 Unknown: {38.0k,500,msb}<-1,1|1,-1>(4,-1,A:16,-48.752m)*{A=0xd3f9}

#144 Unknown: {38.0k,500,msb}<-1,1|1,-1>(4,-1,A:16,-48.752m)*{A=0xd7a1}

transmogrifierTest1.rmir:
#54, #66, #88 The multiple decodes (NEC1 & NEC-rnc) can be prevented by putting <irp:parameter name="prefer-over">NEC1-rnc</irp:parameter> within the NEC1-protocol.

#85 & #112: Appears complete garbage. Tell me if otherwise.

I am a bit unsure if ignoreLeadingGarbage is a sensible default or not.

[The Robman]

I haven't had a chance to play with this yet, but there was one bug in the old Learned to Upgrade process for NEC signals where, if no sub-device was present, it put -1 or something like that in the field, which would cause the upgrade to not work if it wasn't removed.

Do you know if that was fixed in this new version?

[mathdon]

@Barf
Many thanks for all the new info, though I haven't had time to study it yet.

@Rob
Rob, this isn't a fix of what Vyrolan provided. It is a totally new approach. There is no way that a fully functioning conversion was possible with DecodeIR, so that wasn't Vyrolan's fault. IrpTransmogrifier has changed the picture completely. The NEC issue is trivial compared to such things as Sony DSP, Panasonic Mixed Combo and Pioneer MIX, all of which are handled in the new version. It is sufficiently complex that there are bound to be teething bugs, so please report any that you find, but I am confident that any bugs found will be fixable.

[mathdon]

In testing the learned to upgrade conversion for the many Pioneer executors, I have discovered that the explanatory text in protocols.ini for the Pioneer DVD and DVD2 executors is wrong. They both say

Any OBC with bit 5 set can be sent (by this protocol) only as a one part signal and will force the "Prefix" columns to say "none" and will force the device column to be Device 1 from the Setup sheet.

Any OBC with bit 5 clear can be sent (by this protocol) only as the second part of a two part signal and will force the "Prefix" columns to indicate the first part of the two part signal and will force the device column to be Device 2 from the Setup sheet.

These are the wrong way round. It is bit 5 clear that can be sent only as a one part signal and bit 5 set that can be sent only as the second part of a two part signal. This can be confirmed easily by creating a new upgrade and seeing what happens on the function page. It can also be seen from the command translators. These translators all include the "comp" argument, which I suspect is what gave rise to the error, but a little thought shows that the relationship between bit 5 of the OBC and the flag indicating a two part signal involves two "comp"s, which cancel one another out.

I will correct this in the next build of RMIR, unless there are any objections.

[Barf]

Bug: If DecodeIR is not found, the "Learned Signals" is greyed out, which is not really meaningful. However, Options -> Set IR Decoder happily lets you select it.

[mathdon]

Bug: If DecodeIR is not found, the "Learned Signals" is greyed out, which is not really meaningful. However, Options -> Set IR Decoder happily lets you select it.

Fixed, for the next build. The Learned Signals tab is now present and enabled whenever the remote supports learned signals, whether or not DecodeIR is found, since IrpTransmogrifier is always present. Also, if DecodeIR is not present, the radio button to select it is disabled and the IrpTransmogrifier button is forced to be selected.

Barf, at your request in another thread, I have now also made Pronto be the default on Learned Signals Details dialog.

[mathdon]

Barf, many thanks for your comments in an earlier post on learned signals that did not decode. In your list for TransmogrifierTest1.rmir, you did not mention numbers 92, 97, 98, 99, 101, 104, 105, 109, 110 and 132. These all decode as OrtekMCE with DecodeIR but give no decode with IrpTransmogrifier 1.1.0. I have just tried adding a further protocol into the IrpDatabase:

Code: Select all

<irp:protocol c-name="OrtekMCE2" name="OrtekMCE{2)">
        <irp:parameter name="uei-executor">021B:JP1</irp:parameter>
        <irp:irp><![CDATA[{38.6k,480}<1,-1|-1,1>([][P=1][P=2],4,-1,D:5,P:2,F:6,C:4,-48m)+{C=3+#D+#P+#F}[D:0..31,F:0..63]]]></irp:irp>
        <irp:documentation>The <A href="#repeat">repeat frames</A> are not all identical. P is a position code: 0 for the start frame of a repeat sequence, 2 for the end frame and 1 for all frames in between. C is a checksum, 3 more than the number of 1 bits in D, P, F together.  DecodeIR v2.37 and later check P and will report in the Misc field if either the start or end frame, or both, is/are missing.</irp:documentation>
    </irp:protocol>

which is your OrtekMCE with "[P=0]" replaced by []". All these signals except #101 decode as OrtekMCE{2} but I do not entirely understand why. I had expected it to catch those signals that DecodeIR reports as with "no start frame", but it also catches those reported as "only start frame". Also, IrpTransmogrifier reports a value P=2 for a few Ortek and Ortek{2} signals but not for most of them and this doesn't correlate with anything I can see from DecodeIR. I thought the decodes from IrpTransmogrifier only gave values to names in the domain section in (square) brackets at the end of the IRP, which doesn't include P for the OrtekMCE protocol.

Can you shed any light on this, and do you think OrtekMCE{2} is a useful addition, or is it superseded by your ignoreLeadingGarbage option?

BTW I do think that option is a useful default for decoding learned signals in RMIR, as it should help when the learn did not catch the beginning of the signal.

[Barf]

In your list for TransmogrifierTest1.rmir, you did not mention numbers 92, 97, 98, 99, 101, 104, 105, 109, 110 and 132. These all decode as OrtekMCE with DecodeIR but give no decode with IrpTransmogrifier 1.1.0.

You are right, I just forgot But on the other hand we have been discussing the issue before,

Generally speaking, defining "relaxed protocols" is definitely the way to handle these "almost correct" signals. I am thinking of having some convention for naming of the "relaxed protocols", just adding a number is not very creative. I have already used "_relaxed" and "noCheck" as suffix for such protocols, leaning towards "_relaxed".

What you ard doing in the "OrtekMCE{2}" is not exactly clean: First, P, as a virgin variable is read from a bitfield (during intro), then (starting repeat) overwritten by an assignment, next time it is read it is a check for the decoder (it checks that the number read from the bitfield is the same as the assigned value). For example, you can't render such a protocol. I am considering if the parser should even reject it at initial parsing.

Right now I am not sure how to express "read a value, and accept any value, or junk it". Will have to think about it.

Edit: "relaxed protocols" should probably be decode-only.

[mathdon]

What you ard doing in the "OrtekMCE{2}" is not exactly clean: First, P, as a virgin variable is read from a bitfield (during intro), then (starting repeat) overwritten by an assignment, next time it is read it is a check for the decoder (it checks that the number read from the bitfield is the same as the assigned value). For example, you can't render such a protocol. I am considering if the parser should even reject it at initial parsing.

I'm not sure I understand this. It is a long time since I wrote the IRP Specification, so you probably know it better than I do. But I was relying on the last sentence of section 12.3 on the semantics of variations. It says

If the bare irstream of an alternative is empty then its execution consists of skipping the remainder of the IRstream and proceeding to its next execution.

So I interpreted OrtekMCE{2} to mean nothing being sent on button down (remainder after [] being skipped), P=1 being repeated while button is pressed and P=2 on button release. In other words, the start frame is absent and no reason why this cannot be rendered.

[Barf]

If the bare irstream of an alternative is empty then its execution consists of skipping the remainder of the IRstream and proceeding to its next execution.

So I interpreted OrtekMCE{2} to mean nothing being sent on button down (remainder after [] being skipped), P=1 being repeated while button is pressed and P=2 on button release. In other words, the start frame is absent and no reason why this cannot be rendered.

You are right. Sorry for that,

... which is your OrtekMCE with "[P=0]" replaced by []". All these signals except #101 decode as OrtekMCE{2} but I do not entirely understand why. I had expected it to catch those signals that DecodeIR reports as with "no start frame", but it also catches those reported as "only start frame". Also, IrpTransmogrifier reports a value P=2 for a few Ortek and Ortek{2} signals but not for most of them and this doesn't correlate with anything I can see from DecodeIR. I thought the decodes from IrpTransmogrifier only gave values to names in the domain section in (square) brackets at the end of the IRP, which doesn't include P for the OrtekMCE protocol.

I cannot directly relate. Can you please give me something I can reproduce?