IrScrutinizer: capturing, generating, analyzing, import, exp

[Barf]

... But how can I export that tree as a single Girr file containing a RemoteSet with multiple remotes?

You can't. The remote editor simply does not have that structure. Cf. this issue, from 2015...

On the other hand, using the Java API for the Girr library, you can do that in around 10 lines of Java, or you can do it as an XML transformation. Or possibly can allow the file selector in RM to select directories.

[mathdon]

I have now posted development build 10 of RMIR v2.10 in the RMIR Development folder on SourceForge for you to try. It is just a RemoteMaster.jar file to replace that in build 9. I am not announcing it elsewhere as it is really intended just for you, but it is there if anyone else is following this thread.

It supports import of Girr files into both RM and RMIR. In RM this is through the menu item "File > Import Girr File" and in RMIR through the button "Import Girr" on the Devices tab. The process follows closely that of the conversion of Learned Signals to a device upgrade. The Name field of the signal in IrScrutinizer becomes the function name in RM or RMIR. There is no assignment of functions to buttons in the imported device upgrade, as there is no button information in IrScrutinizer, so that needs to be done separately in RM or RMIR after import.

Thank you for the ability to uncheck Remove Defaulted Parameters. It is not necessary to uncheck this for the import process to work, but there are rare circumstances in which an imported signal may be corrupted if this is not unchecked. An example would be an import with two NEC1 signals with the same Device parameter, one with the Subdevice specified by default and the other with an explicit value different from the default. The import process assumes defaulted values to be unspecified and so can be overwritten. This is a necessary feature of the way that conversions to combo protocols are determined, so is not a defect in the import process.

[Barf]

First, I would prefer if you check in the stuff to SVN, and just tell me to check it out there. You do not need and dummy releases. If I test stuff, I run it through the debugger; binary-only is a no-go.

Second,

there are rare circumstances in which an imported signal may be corrupted if this is not unchecked.

I would not consider it a satisfying solution to say that you have a Girr importer, but it fails if there are defaulted parameters. I just implemented a new function Protocol.fillInDefaults(Map<String,Long>), that should do exactly what you want. In your previous code snippet:

Code: Select all

NamedProtocol np = LearnedSignal.getTmDatabase().getNamedProtocol( md.getProtocolName() );
Map< String, Long > parms = cmd.getParameters();
Map< String, Long> paramsWithDefaulted = np.fillInDefaults(params);
... 

PS. Have you discovered my collection?

[mathdon]

First, it is in the SVN and was there for 3 hours before you posted your reply. The binary was posted first. Next time I will be sure to do it the other way round.

Second, I did not say it was satisfactory. I actually hoped you would come up with a solution, so thank you for doing so.

Third, I had not discovered your collection.

[mathdon]

I have updated the build in the SVN in several ways. It uses your IrpTransmogrifier snapshot (library named IrpTransmogrifier-1.2.7-d94d0bc-jar-with-dependencies.jar if I understand your request about versioning it correctly) with the new np.fillInDefaults(..) function, which works perfectly. I have also used cmd.checkForParameters() to attempt to handle Girr files saved without the Parameters option. Commands found to be undecodable should be shown as such in the table of unconverted signals which is displayed when there are such failures. And I have tested the case of Girr files containing more than one remote. This is presumably an irrelevance as I understand that there is no means yet of creating such a file, but I created one anyway by manual editing for testing purposes.

I look forward to any further comments and suggestions for improvement.

[mathdon]

Oh, I forgot to mention one further addition. I have used rmProtocols.xml to add uei-executor entries for the Dyson and Dyson2 protocols. There wasn't any point in doing this before having Girr import, as UEI learned signal format can't handle them but I have tested the import with a Girr file and that works fine.

[mathdon]

I feel that Girr import to RM is largely done, pending further comments and suggestions from you, so my mind has turned to thinking about Girr export from RM. Is there merit in doing this? If so, are there any examples of creating Girr files as I cannot understand Girr.XmlExporter. I see that one of the constructors for RemoteSet says

This constructor sets up a RemoteSet from a given Map of Remotes, so that it can later be used through the xmlExport or xmlExportDocument to generate an XML export.

but I cannot find xmlExport or xmlExportDocument either.

[Barf]

I have updated the build in the SVN in several ways. It uses your IrpTransmogrifier snapshot (library named IrpTransmogrifier-1.2.7-d94d0bc-jar-with-dependencies.jar if I understand your request about versioning it correctly)

Perfect!

with the new np.fillInDefaults(..) function, which works perfectly.

I have also used cmd.checkForParameters() to attempt to handle Girr files saved without the Parameters option.

It may not be necessary, since (for example) both getProtocolName() and getParameters() are calling chdckForParameters(). But no problem, the call is idempotent.

Commands found to be undecodable should be shown as such in the table of unconverted signals which is displayed when there are such failures.

Should be no problems, just to catch the right exceptions at the right place.

And I have tested the case of Girr files containing more than one remote. This is presumably an irrelevance as I understand that there is no means yet of creating such a file, but I created one anyway by manual editing for testing purposes.

I do not think it is a good idea to base such decision on what the current IrScrutinizer can do. "Everyone" can write a program that generates such files, in particular when using the library. (Although I do not exactly expect to see IrScope or AnalysIR with Girr support. ) Also have you considering in the fileselector to allow the user to select a directory? -- that would return a RemoteSet with several Remotes.

Also, I would suggest that you say "A Girr file" instead of "A Girr file from IrScrutinizer" (or possibly "A Girr file, for example from IrScrutinizer").

In SVN, you forgot to check in Girr.jar.

Possibly you can do something with the meta data, for example stuff it into Notes.

When importing this file, containing NEC1 commands with two D's, I am offered to use 00 5A for it. Needless to say, this fails. There comes an error popup, stating some signals could not be converted. But there is only 3, when many more signals failed.

When importing this Denon/Denon-K file, basically everything goes wrong...

[Barf]

I feel that Girr import to RM is largely done, pending further comments and suggestions from you, so my mind has turned to thinking about Girr export from RM. Is there merit in doing this?

Absolutely. There are two ways: either "dumb": just call the executor a protocol and export parameters as hex and the protocol parameters. This is done in org.harctoolbox.irscrutinizer.importer.RmduImporter.main(), that can be called from the command line to generate a Girr File from (many) rmdus. (Check the version from today.)

Or, the considerably harder "smart" way, resolving the executors to protocols...

If so, are there any examples of creating Girr files as I cannot understand Girr.XmlExporter.

It is quite simple, just use RemoteSet.toDocument(...) to create an XML Document, and print it with XmlUtils.printDOM (from IrCore).

XmlExporter is just some string constants (possibly the name is misleading...).

I see that one of the constructors for RemoteSet says

This constructor sets up a RemoteSet from a given Map of Remotes, so that it can later be used through the xmlExport or xmlExportDocument to generate an XML export.

but I cannot find xmlExport or xmlExportDocument either.

Sorry for this, the content after the comma is both obsolete and at the wrong place. Nuked.

[mathdon]

Commands found to be undecodable should be shown as such in the table of unconverted signals which is displayed when there are such failures.

Should be no problems, just to catch the right exceptions at the right place.

Sorry, I didn't express this very well. What I meant is that this is already implemented, the "should" means that I hope it works properly.

Also have you considering in the fileselector to allow the user to select a directory? -- that would return a RemoteSet with several Remotes.

I see no great benefit in this. All that happens with such a RemoteSet is that the user is asked to choose which Remote to import, which is the same as choosing which file from the directory to import. I have given RM and RMIR the ability to handle such a RemoteSet precisely because there may be means outside IrScrutinizer to create this.

In SVN, you forgot to check in Girr.jar.

It is there as lib/org.harctoolbox.girr.jar, my hand-created jar file. I haven't found your Girr.jar. Where is it? If I click on Girr in your collection, I just get a list of protocols.

Thanks for the files that create errors. I knew there would be teething problems. I will investigate.

Or, the considerably harder "smart" way, resolving the executors to protocols...

That was what I was intending. I like a challenge . Is it worth the effort, do you think?

[mathdon]

When importing this file, containing NEC1 commands with two D's, I am offered to use 00 5A for it. Needless to say, this fails. There comes an error popup, stating some signals could not be converted. But there is only 3, when many more signals failed.

Yes, you will be offered 00 5A. That list is the set of executors that can support each individual signal separately. It is not until the conversion process is attempted that it attempts to find a set of device parameters that is consistent with all the signals. To do otherwise would require attempting the conversion for every executor in the list currently shown, a time-consuming process that could well result in no executors being found that can handle ALL the signals.

However, it was of course a bug that it got the conversion wrong and reported only 3 non-convertible signals. That bug is nothing to do with Girr files, it was present also for learned signal conversion. Thank you for identifying it. I have fixed it (not yet in SVN, pending my investigating the Denon file) and it now correctly reports 146 signals not converted with the 00 5A executor.

I haven't resolved the Denon situation yet. I find these to be very complicated executors and suspect the issue may be specific to Denon. I will report again when I have further info.

[Barf]

What I meant is that this is already implemented, the "should" means that I hope it works properly.

Have to disappoint you. Consider this file. I think it bails out by the ninth signal "LCD cycle Dim, Bright, Off" (possibly because of the weird name?), and nothing is delivered.

I haven't found your Girr.jar. Where is it?

I have just set up a CI build

Thanks for the files that create errors. I knew there would be teething problems. I will investigate.

There are more For example, the philips_vr100 file (an RC5/D=5 with many Fs), the apple_remote, ALL panasonic, possibly others.

Or, the considerably harder "smart" way, resolving the executors to protocols...

That was what I was intending. I like a challenge . Is it worth the effort, do you think?

I think it would be a fantastic thing. Not only would it possible to export the JP1 "library" to non-jp1 world, but it would also be possible to write a software emulator for a jp1-remote -- remember the emulator in ProntoEdit?

There are, as I see it, two ways to approch this problem: Either have a micro processor emulator (one per supported architechture), or write IRPs for the executors, and feed it through IrpTransmogrifier. The first I think is extremely hard -- should then try to find something someone else has done. In particular, we do currently do not have a Kevin or a Marcin...

The second way appears more tractable. An executor is really nothing else than a protocol in the sense of a mapping from a set of input parameters to the space or IR signals; it is just that the parameters are more evolved. Possibly some extensions, like conditional IRStreams ( variable ? true_IRStream : false_IRStream) will be necessary for example for combi protocols. (We discussed this a few years ago, but could not find it when I searched.) But, while Girr-import is/was not particularly hard, this probably is.

[mathdon]

I am sure there are many more files that don't work, but I hope only one bug. Now I have the correct fix for the NEC file, all Denon signals import. There is an issue with the Denon Girr file in that there are duplicate command names, so only the later one gets included. There is also an issue with the uei-executor entry for Denon, so till I fix that, the hex imports wrongly. But those two issues are specific to this file and protocol, the bug that the NEC file brought to light will affect many files. I'll do more testing before I post any fixes to the SVN.

[mathdon]

The version in the SVN should now import all the files you mentioned as causing errors. Here are some notes on them:

The NEC1 file showed up a bug that has been present all along in the learned signal conversion process.
The Denon/Denon-K file showed up issues with the uei-executor entries for Denon, Sharp and Denon-K in irpProtocols.xml.
The philips_vr1100 file showed up a bug specific to the Girr import process.
The Panasonic files showed up a typo in the uei-executor entries for Panasonic and JVC-48.
With these fixed, the other files then worked.

The SVN includes a rmProtocols.xml patch file to fix the irpProtocols.xml errors, but if I understand the patch syntax correctly then it shows an issue with the patch process. My recollection is that if I put <irp:parameter/> and follow it with replacement <irp:parameter> entries then the existing entries will be deleted and replaced by the new ones. I seem to recall testing that and it worked in the past, but now the new entries seem to get appended without the deletion occurring. Here is an excerpt from an rmaster.err file:

Code: Select all

001F:8(Panasonic Mixed Combo;;
        D.S=0.0,0.1,0.2,0.3.0.4,1.0,1.1,1.2,1.3,1.4,2.0,2.2,2.3,2.4,3.0,3.3,3.4,4.0,4.4,5.0)
		[D?0D,D?1D|||S?0S,D?2D|||S?1S,D?3D|||S?2S,D?4D|||S?3S,D?5D|||S?4S,2,32;??D,??S,F]
*** Error: sequencer has invalid syntax
00CD:2[D;S,F]
No protocol found for executor descriptor 00CD:2[D;S,F]
00CD:JP1[D;S,F]
001F:8(Panasonic Mixed Combo;;
        D.S=0.0,0.1,0.2,0.3,0,4,1.0,1.1,1.2,1.3,1.4,2.0,2.2,2.3,2.4,3.0,3.3,3.4,4.0,4.4,5.0)
		[D?0D,D?1D|||S?0S,D?2D|||S?1S,D?3D|||S?2S,D?4D|||S?3S,D?5D|||S?4S,2,32;??D,??S,F]
*** Error: sequencer has invalid syntax
*** Error: sequencer has invalid syntax
00CD:2(Panasonic Combo)[D;S,F]

You will see that the entry from irpProtocols.xml was processed first, 001F:8 invalid as there is a dot where there should be a comma and 00CD:2 missing the executor name, followed by corrected versions of both from rmProtocols.xml. I am not sure why there are so many invalid sytax messages but think they all stem from the 001F:8 typo. Have I misremembered how the patch process works?

I haven't tested any files other than the ones you mentioned. I cannot find a way to download files from your Girr collection. I seem to have to copy the text to a text editor and save it as a .girr file, which is rather tedious. I would welcome you letting me know of any files that still give errors after these fixes.

On the export of Girr files from RM, I have my own ideas that differ from yours but I need to do some testing to see if they are viable.

[Barf]

Have not checked your fixes yet, but here are some quick answers:

... Have I misremembered how the patch process works?

An empty <irp:parameter name="..."> removes an existing parameter, while a non-empty appends an existing parameters. What happens if you have one empty and one non-empty is ... undefined (at least for now).

I cannot find a way to download files from your Girr collection. I seem to have to copy the text to a text editor and save it as a .girr file, which is rather tedious.

Either you install a Git client (on Windows, I think TortoiseGit is popular) and clone the repository, or you just press "Clone or download" on the main page on GitHub (bright green, it is easy to find), and select "download ZIP".