JP1 Remotes

I will make a version of the spreadsheet tomorrow with no formula in it.

All I know is that if I don't restart the program each time, if I do a category with a lot of files and then follow it with a category with fewer files, many of the files from the previous category will get re-downloaded.

Try doing the Cable category and then do Cameras to see what I mean.

Rob,
Yes I'm holding off.
Especially that I see that you've changed things quite a bit. It's shaping up into a fabulous lookup tool.
BUT, I have a problem if you plan to keep all upgrade categories in one sheet.
I prefer the category tabs you had earlier.
Has to do with filtering speed.
When I wanted to find one thing using the AutoFilter dropdown, I saw Excel chugging along at 50% utilization, calculating and calculating. Usually 2-5%.
Using XP with 1 gig RAM, 3MHz speed hyperthreading usually fast fast machine.

3FG,
Re: descriptions from the download page -
When I was downloading manually I saw some very useful descriptions about the upgrade when there was something unusual.
The problem will be a limit of 255 characters in a cell. Yes I know we can put 65535 chracters in a cell but it will not then allow copying a sheet, and wordrap fails beyond 255 characters. So that's a rough one.
Possible option is to put such notes into a note tied to a cell. In VBA it'll be something like cell.comment.text or activeCell.AddComment.Text "blah blah blah" (I don't remember really)

ElizabethD wrote:Rob,
Yes I'm holding off.
Especially that I see that you've changed things quite a bit. It's shaping up into a fabulous lookup tool.
BUT, I have a problem if you plan to keep all upgrade categories in one sheet.
I prefer the category tabs you had earlier.
Has to do with filtering speed.
When I wanted to find one thing using the AutoFilter dropdown, I saw Excel chugging along at 50% utilization, calculating and calculating. Usually 2-5%.
Using XP with 1 gig RAM, 3MHz speed hyperthreading usually fast fast machine.

Hmm, I found the new format to be much easier for me to read and find things. With the tabs, I was searching forever. Perhaps Excel is not the best storage mechanism for this database.

Version 0.7

Intended to fix the need to exit the program between downloads (thanks for the clear description), includes both long and short forms of the links, nulls removed, web page descriptions added. Downloading takes about twice as long, because the description is on a different page than the actual file, so there are double the number of pages to download.

Still doesn't handle files with semicolon separators, combo protocols, or zip files.

The links and web descriptions are stored in link.csv, and so that information is determined at the time of download. Other information is collected from the downloaded files at the time of processing HDD files. If files have been removed from the website, but haven't been erased from the local copy on the hard disk, upgrades.csv will list the stale file, but there won't be a link.

Liz, I've implemented this as CSV files, rather than Excel files, because I think CSV files are recognized by all editions of Excel, and frankly it is a lot easier to program and executes faster. I don't know of a way to add notes to CSV files.

Here's the no-formula version of JP1 Master:
https://www.hifi-remote.com/forums/dload ... le_id=8487

Liz, it was always my intention to combine all of the separate tabs into a master tab so I could sort them by protocol and device code. Once the spreadsheet is finished, we can always create separate tabs again.

One of the other tasks I intend to do using the info collected is to go through all of the zip files and where the zip is not necessary, replace it with a single file.

3FG wrote:... includes both long and short forms of the links, nulls removed, web page descriptions added. Downloading takes about twice as long, because the description is on a different page than the actual file, so there are double the number of pages to download.

We don't really need the long description, we need the short description because it's how we connect the file that we're looking at back to the one listed on the web.

You shouldn't need to go to any extra pages to get this. I assume you start by going to the main index page for the category (eg, here for cameras). This page lists the short descriptions (ie, the ones we need) along with various other bits of info, such as when it was last updated and the icon, and the URL. My suggestion would be to use this info to start the table of data. Then you can start downloading the files to get the files and the real file names.

Here's a list of the data that I'd like to see collected:
1) Category (eg, Cameras)
2) Short description (eg, Aiptek Action HD GVS HD camcorder)
3) File id (eg, 6119, don't need the full URL)
4) Icon GIF (eg, rm, km-xls, xls, zip, txt, etc)
5) Date (eg, 16 Jan 2009 03:56 pm)
6) Real file name (eg, Aiptek Action HD GVS.rmdu)
7) File type (eg, rmdu, zip, txt, etc)

3FG wrote:Still doesn't handle files with semicolon separators, combo protocols, or zip files.

Maybe you could add an extra column where you list why you didn't process the file, as this will help us go back and clean them up.

Also, could you start thinking about what it would take to have the tool run totally automatically. I'm thinking that it would be great if we could just click a button and it would download all of the files from all of the folders in the Device Upgrades section (apart from the WAV folder) and then automatically process them, with the results ending up in one big csv master file.

I'm thinking that I'll use the results that I have now to go through and un-zip files or update old KM files with the semi-colons, etc and then when I'm done, I'd like to be able to just re-run without needing to do all the "hand holding" that the app currently requires.

Btw, I'm already thinking ahead to other ways that we can use this app. I think there are still several files over in the old Yahoo groups that have not been ported, so there might be a use for an adapted version of the app even after we're done with this particular project. Hey, maybe it could also download all of the CCF files from the Pronto section at Remote Central with one button click! And maybe it could automatically run John's DecodeCCF against them. See what you've started!

I just went through and updated all the KM files that the app couldn't handle from the following folders (will do the rest later):

A/V Selectors
Audio
Cameras
Combo Audio Systems
Digital STBs

The Robman wrote:Here's the no-formula version of JP1 Master:
https://www.hifi-remote.com/forums/dload ... le_id=8487

Aha, it's nice and fast. No issues with this one. Sorry for false alarm.

The Robman wrote:Liz, it was always my intention to combine all of the separate tabs into a master tab so I could sort them by protocol and device code. Once the spreadsheet is finished, we can always create separate tabs again.

No problem. What was I thinking of you ask? well, when it was so sluggish, I figured making smaller sets will do the job ... Of course one sheet is better, so I'm totally with you and Vicky on that one.

I just completed fixing the rest of the old KM files, so they should all be compatible with the upgrade processor in the app now.

I also converted several text files into upgrades.

Rob,
You're probably wondering why I haven't understood your item 2). Here's why: 2) is actually called "File" rather than "description", both on the main index page and on the file upload page. But both the upload page and Rmdu files have something called description, and I thought one of those must be what you wanted. I think it is clear now.

Regarding 4) Icon GIF, is text like "rm.gif" sufficient?


Thanks for the list of desired items. That helps a lot.

The entire process can be automated so that only 1 click is required; in fact my first version worked that way. But the website needs to be working reliably, or the program needs a whole lot of logic to continue an interrupted download. It also would help if I can ever figure out how to cancel an in-progress download......

rm.gif is fine. It would be better if you could trim the suffix off, but if you can't, I can do it in the spreadsheet.

I've just completed a massive effort to replace as many zip files and other non-upgrades as I can.

I've updated the non-formula version of jp1-master also.

Is this cross reference list something that RMIR could use? Is it going to be the data that drives some sort of online search that rmir could hook into to help users find, download and import device upgrades into their remotes? Or I suppose rmir could download it, parse it and provide a search Ui, then download and import any upgrades found.

I don't know what other uses it could be put to. My primary goal is to document all the upgrades that we have and clean up some of the non-standard files.

Now, when someone posts a single learn from a device, we'll have a much better chance of seeing if we have a match.

Greg, I think so, it should be useful for all sorts of things. Link lookup? Not sure, it might be a site link similar to Vicky's Lookup tool. I do not think though, that as is, RMIR could grab the cells into protocols.ini because the device vs subdevice columns is a bit muddy distinction at this point. Over the few days of development it's changing a lot, so keep looking. I keep looking to see if Rob tells me to do something more and how The, our usual, undefined scope creep, and I don't mean this in any negative way in case somebody thinks I did.