Home | Ham Radio |     Share This Page
IcomProgrammer II

A Python resource for programming Icom radios

— All content Copyright © 2018, P. LutusMessage Page

Version 1.6 (01-30-2018)

Introduction | Communications Issues | Defining the Memory Data | Configuration | Technical Notes | Version History

(double-click any word to see its definition)

Introduction

Before I introduce this project I want to list all the Icom programs I've written over the years:

  • EasyTuner (1998) — An Excel spreadsheet with an associated communications library that allowed users to make spreadsheet entries for each desired memory. Windows only.
  • RadioComm (1999) — A desktop application that controlled an Icom radio as well as programming its memories. Windows only.
  • IcomControl (2000) — A radio controller that only controlled the Icom IC-PCR1000 headless radio. Windows only.
  • Icom Programmer (2010, Ruby) — This program's predecessor, it was a Ruby project meant to program an Icom radio's memories, not be a radio controller in the usual sense.
  • JRX (2012) — My most recent general-purpose Icom virtual radio, written in Java, much more sophisticated than its predecessors, but it controls the radio, it doesn't program its memories (reasons below).

IcomProgrammer II is meant to program an Icom radio's memories, not be a control panel, and it doesn't have a desktop user interface — for a true virtual radio that does, see JRX. This is the most recent, and by far the best, of my efforts to program the memories of my Icom radios. It's written in Python 3 and was developed on a Linux platform, but it should function on Windows with reasonable care during installation and operation.

When I wrote the earlier Ruby program I had a smaller set of radios that needed programming, but at the moment I have these Icom radios:

  • IC-7000
  • IC-706MKII
  • IC-746
  • IC-756Pro
  • IC-R8500
  • IC-PCR1000
  • IC-PCR1500

The IC-7000 is my most recent acquisition, and it is a terrific radio. I bought it so I could have a compact, sophisticated radio on my boat to replace the IC-706MKII, which, although fully operational, is far behind the times in a technical sense. But to fully exploit this new radio, I needed to replace my previous programming method written in Ruby. Among the problems with the prior method are that it didn't program memory names (alphanumeric labels), for those Icom radios that have this ability. (As it turns out, every one of my Icom transceivers accepts memory labels except the IC-706MKII.) Another problem was that the Ruby program didn't communicate with the radio very fast for a reason I didn't understand at the time. Both these issues are addressed in this new project.

Communications Issues

This project relies on a Python 3 program named "icom_programmer.py", which can program many different Icom radios in a basic sense (meaning provide memory definitions), and it can also program memory labels for some Icom radios. For many users the memory-programming ability will suffice, but for me personally, one of the primary reasons I embarked on this project was to be able to program memory labels.

Memory Labels

In the past, reliant as I was on the IC-706MKII while boating, and because I couldn't program memory labels, I would print a radio memory table that correlated memory numbers with frequencies and names. In modern times, with complete virtual worlds flying through the air at the speed of light, this was a terrible and embarrassing handicap.

Digression: one of my Litmus tests for a project's technological sophistication is to ask, "Do trees have to fall for this to work?" If so, then the project isn't at (or near) the pinnacle of technology. I can't possibly describe or even recall how many memory tables I printed out, so that the printed table agreed with the radio. I was more than ready to program memory labels into the radio itself.

Differences Between Radios

Now the bad news. Even though all modern Icom radios provide a computer interface, they use different commands and protocols. For the most basic functions like specifying a memory location, a frequency and an operating mode, Icom radios are in reasonable agreement, so the programmer doesn't have to write different code for each model (although there are some model-specific adjustments required — see below about the IC-R8500).

But when it comes to programming memory labels, one must almost write one method per radio. Here's a link to a site that has some information about programming memory labels, broken down by Icom model. Visitors to the site will notice that (a) the radios all require different approaches to this problem, and (b) many popular radio models don't have published methods yet.

Because I have so many radios available, I was able to write and test my program on all of them, and I can say it works with all my radios that have a memory label capability:

  • IC-7000
  • IC-746
  • IC-756Pro
  • IC-R8500

Beyond this, even though I can't test radios I don't have, it's very likely that my program will work with related models, like newer radios in the IC-746 and IC-756 lines as well as newer versions of the IC-7000.

Icom Radio Communications / CI-V

Virtually all Icom radio communication is by way of a special box called a "CI-V Interface". As it happens, Icom sells a CI-V box, but I can tell you from personal experience that third-party products are both better and less money — the Icom CI-V box costs US$190.00, while a very good third-party product is just US$59.00. (I don't have any association with Black Cat Systems, I just like their product.)

While shopping for Icom radio accessories, don't worry about offending Icom — if they keep building reliable radios and we keep buying them, I'm sure they won't mind if we comparison-shop for peripherals, especially in a case like this, where there's no meaningful comparison.

But let's move on to the virtual parts of Icom communications — the actual signals. Icom radios communicate by way of a serial stream of data, a communications pathway that's one bit high and as wide as time. When expressed like that it doesn't sound very efficient, but consider that all radio communications are also serial streams — one bit high, and as wide as time.

As to serialization and time, here's a description of a typical exchange between a computer and an Icom radio — all the data streams are in hexadecimal notation:

  • Computer to radio: fe fe 70 e0 08 00 01 fd
    Meaning: "Hello Fifi (fe fe), please pass this along to radio (70): this is the computer (e0), please switch your memory address (08) to address 1 (00 01), end of message (fd)."
  • Radio to Computer: fe fe 70 e0 08 00 01 fd
    Meaning: "Hello Fifi (fe fe), just to make sure the computer and I understand each other, please pass along this exact copy of what I received (70 e0 08 00 01), end of message(fd)."
  • (The radio processes the received packet ...)

  • Radio to computer: fe fe e0 70 fb fd
    Meaning: "Hello Fifi (fe fe), please pass this along to the computer (e0): this is radio (70), and what you just sent was all right (fb), end of message (fd)."
    Or if something goes wrong: "Hello Fifi (fe fe), please pass this along to the computer: this is radio (70), and what you just sent was wrong and wasn't acted on (fa), end of message (fd)."
The basic outline of all Icom communications is as described above:
  • The computer sends a data packet to the radio.
  • The radio echoes back the entire packet for error detection.
  • After anayzing and possibly acting on the packet, the radio usually replies with a six-byte packet that indicates whether any errors were detected. As shown above, if an error is detected, code fa is sent, otherwise fb.
  • There's one exception: if the computer asks the radio for data, the radio replies with the requested data instead of an error / confirmation packet.
  • And (guess what) there's an exception to the exception: If the computer's data request has an error, the radio replies with an error reporting packet instead of the requested data. This means the computer program needs to be able to detect a packet that isn't the expected data, but an error report. This issue came up in development during this project.

Speed Issues

Here are some more details, including a method for speeding up serial communications:

  • During my earlier work with the CI-V system, I quickly noticed that the CI-V interface didn't support any kind of hardware handshaking, one of the standard schemes to prevent either side from exceeding the receiving ability of the other.
  • To be able to program a radio, I realized I had to avoid sending data faster than the radio could process it, and in the absence of handshaking I had no signal to tell me the radio was ready for more data.
  • So to make the system work, I introduced time delays sufficient to assure that the data were successfully received and acted on.
  • The problem with this approach is that the hardwired time delay had to accommodate the worst possible delay by any radio the program must work with.

The results were reliable, but the program took forever to program a radio that had more than a dozen memories.

In this project I use a different method, one based on what was described above. Before moving on to the next packet of data, I wait until the radio has echoed the previously sent data, acted on it, and replied with its standard error reporting message. This scheme greatly speeds up data transfers, so that a transaction time (for one radio memory definition) of less than 100 milliseconds is possible, depending on the radio (200 milliseconds is more typical). This represents a big improvement in the speed of my earlier program, even though that program didn't even try to define memory labels.

Defining memory labels takes more time than simply defining a frequency and operating mode, but I think users will see that, on a modern radio that might have between 100 and 500 definable memories, having memory labels is more than worth it.

Defining the Memory Data

In earlier programs I tried a number of ways to edit and store radio memory data, but my favorite approach has always been a spreadsheet — it's easy to edit and it's a user-friendly environment. One of my earliest Icom projects, named "EasyTuner", relied on a spreadsheet for input and storage, but it required people to buy a copy of Microsoft Excel, which I always thought was a serious drawback.

That was 1998, and much has changed. There is now a free office suite called LibreOffice (formerly OpenOffice). LibreOffice includes a spreadsheet program that has every possible advantage over the Microsoft product:

  • It's free.
  • Its data files are based on a transparent, portable, internationally agreeed-on "open document" format that any other program can read.
  • Because of the use of the open document format, If OpenOffice / LibreOffice were to evaporate, users can still recover their data.
  • Also, it's free.
  • Also because of the open document format, a program like mine can easily read the contents of a spreadsheet created by LibreOffice.
  • Did I forget to mention that it's free?

So in summary, this scheme gets its data from LibreOffice spreadsheets the user has created, talks to the user's radio(s) over a serial interface connected to a CI-V interface connected to a radio, and the radio's memories are programmed as the user wishes. In the next section we will discuss data creation and system layout.

Configuration
IcomProgrammer II is Copyright © 2018, P. Lutus, and is released under the GPL.
  • First, download the heart of the system, a Python program available at this link:
    • Right-click the link and choose "Save link as ..." or similar wording that differs from browser to browser.
    • When you have the file, put it in a convenient empty directory, rename the file "icom_programmer.py" and make it executable.
  • Next, make sure you have Python 3.x installed:
    • If you're running Linux, don't worry, it's installed (much of Linux's system software is written in Python).
    • If you're running Windows or another primitive operating system, get Python here.
    • Be sure to download and install Python 3.x, not 2.x.
  • For both Linux and Windows, be sure to install the Python 3 "PySerial" library. This is the only Python library this program requires that's not likely to be installed by default.

As explained above, put the program "icom_programmer.py" in a convenient, empty directory, preferably in your user-document area, so its contents will be backed up when you do system backups. You do perform system backups, yes? I ask because creating frequency tables is a fair amount of work, and it would be a shame if the directory containing your data files were located somewhere that isn't ever backed up.

Directory Layout

Once fully populated, here is what the IcomProgrammer II system layout looks like, at the level below the directory you choose:

You don't have to create these directories, they're created for you by icom_programmer.py when it first runs — in fact, once you have chosen a base directory and moved icom_programmer.py into it, you may want to run the program once before going on, so it will create its directories.

Even though the directories are shown above containing files, most of them are automatically created by IcomProgrammer II. Only one file needs to be present at the start — one frequency-definition spreadsheet file (in the above example listing, that file is "general_hf.ods"). You can have any number of frequency definition spreadsheet files for different purposes, but at least one needs to be present.

Spreadsheet Table Layout

Inside the frequency definition spreadsheet file, created by LibreOffice, you would create a table with at least these minimum fields:
Mode RxFreq
am 0.1200
am 0.1315
am 2.5000
am 5.0000
am 10.0000
am 15.0000
am 20.0000
am 25.0000
lsb 3.9000
lsb 7.2000
usb 14.2000
usb 21.2000
usb 28.2000

The column titles should be as shown — icom_programmer.py uses the titles to locate the data it needs. The program recognizes many more column names than these two, but these two are the minimum that must be present to allow memory definition. Here is an example spreadsheet with every column title that icom_programmer.py recognizes, plus some more. You don't need to limit yourself to the column titles that icom_programmer.py needs, but to get your radio programmed as you want, you do need to use the names that the program recognizes.

Here are the special column names and what they mean:

  • Mode: Reception mode, can have one of these values: am,fm,wfm,cw,rtty,lsb,usb
  • RxFreq: Reception frequency in MHz.
  • TxFreq: For transceivers, this entry is for split-frequency operation. If this field has no entry, the reception frequency is defined for transmission and split mode is disabled.
  • RxTone: Squelch tone (CTCSS) for reception. If no entry, CTCSS squelch is disabled.
  • TxTone: CTCSS tone for transmission. If no entry, no tone is used during transmission.
  • MemTag: A label that, if present, will be included with the other values and (if your radio supports it) will be displayed by the radio when this memory is in use. This label can typically be from eight to ten characters long, but if your labels are longer, don't worry — icom_programmer.py will make them the right length to suit the radio's requirements.

Remember, again, you don't need to have all these columns in your spreadsheet, only those you want to have, but two as a minimum: "Mode" and "RxFreq". And again, you can have any number of other columns with other names for any purpose you want. Also, the order of the columns isn't important, only the names — put them in any order that's convenient.

Program Configuration

There is one more step — you need to define your configuration in the icom_programmer.py source file, at a location that originally will look more or less like this:

    # User configuration table
    self.radio_configurations = {
      'IC-706-Boat' : ('IC-706',('general_hf','marine_hf','marine_vhf_short','E')),
      'IC-706-Home' : ('IC-706',('general_hf','marine_hf','vhf_repeaters_short','E')),
      'IC-7000-Boat' : ('IC-7000',('general_hf','general_uhf','marine_hf','marine_vhf_long','cb','E')),   
      'IC-746' : ('IC-746',('general_hf','marine_hf','marine_vhf_short','vhf_repeaters_short','E')),
      'IC-756Pro' : ('IC-756Pro',('general_hf','marine_hf','E'))
    }

            

The idea of this configuration is that there is one line per desired configuration, and a given radio can have more than one configuration. When the program runs, each configuration line appears in a menu of choices, and by selecting that line, you tell icom_programmer.py to program your radio with that configuration. Here's a color-coded breakdown of a configuration line:

'IC-706-Boat' : ('IC-706',('general_hf','marine_hf','marine_vhf_short','E')),
            
  • The red field is a name you choose — it will appear in a menu for you to select it. You might want to use a name that describes the radio and the purpose of the configuration, but that's up to you.
  • The green field must be an Icom radio name from the list farther down in the icom_programmer.py source file, a list that appears below the name "icom_codes". The name you enter should be an exact copy — in fact, copying the name you want is recommended.
  • The blue fields are names of LibreOffice spreadsheet files that are located in the "frequency_data_ods" subdirectory, but without their ".ods" suffixes. For example, if you want to refer to a file named "general_hf.ods", you would enter 'general_hf' in your configuration. You can enter any number of file names, but remember that, in this case, the order is important — the definitions in the earliest named files are given the lowest-numbered memory locations.
  • The purple field 'E' isn't a file name, it's an optional flag that tells icom_programmer.py to erase any unused memory locations beyond the last defined memory.
  • Your definition lines should have the exact syntax of these examples — all the punctuation should be the same as above.
  • Special case: if you include the name of just one spreadsheet file, you need to follow it with a comma, like this:
    'IC-706-Boat' : ('IC-706',('general_hf',)),
                    

    I know this looks a bit odd, and it shouldn't really be necessary — it's a Python quirk. I mention this because I encountered it and remembered that, in Python, ('name') just doesn't work — it has to be ('name',).

Once you have created your configuration, you can run icom_programmer.py and a menu will appear that will look more or less like this:

Options:
    1: Program IC-7000-Boat
    2: Program IC-7000-Home
    3: Program IC-706
    4: Program IC-746
    5: Program IC-756Pro
    6: Program IC-R8500
    7: Create All Radio Frequency Lists
    8: Create All CSV/TSV Frequency Lists
    9: Quit
Select (1 - 9):

            

Assuming you have written your configuration correctly, and assuming you have the spreadsheet file(s) referred to there, and assuming you have your radio(s) connected to your computer, you should be ready to program your radios.

The option named "Create All Radio Frequency Lists" automatically generates CSV, TSV and HTML versions of your selected radio configurations. The option named "Create All CSV/TSV Frequency Lists" does the same for each of the original spreadsheet files that are used to program the radios. These features are meant to maximize the number of export and display options for the radio frequency lists.

HTML and PDF Output

For convenient access, this program creates HTML files containing the present configuration of your radio(s). These HTML pages are located in the "radio_html_tables", but the program will also create PDF versions of the files if a PDF formatter is present on your system. The formatter is named "wkhtmltopdf" and it's available here.

Technical Notes

Because different Icom radios use different communications protocols, programming Icom radios isn't easy, but in the case of defining memory labels, it's much more difficult. Therefore, as mentioned earlier, even though I have quite a few Icom radios and I have tested them all, I expect that a radio sufficiently different than those I own won't be compatible with this program.

On the other hand, because Python is self-documenting, those with some programming ability may be able to modify this program to accommodate their radios. And remember, this program is released under the GPL, which means you have the right to make your own versions.

Some of the differences between Icom radios take your breath away. For example, every Icom radio uses one-based bank and memory numbering (i.e. banks 1 through n, memories 1 through n for n banks and memories), except the IC-R8500 receiver, which uses zero-based numbering (banks 0 through n-1, memories 0 through n-1 for n banks and memories). This sort of thing, all too common, makes programming for Icom radios not unlike juggling weights while your hair is on fire. And it explains why there are so few decent memory programming utilities for Icom radios.

But Icoms are very nice radios. In spite of all I have said here, I wouldn't have any other kind of radio. I was turned into an Icom fan during my around-the-world sail, when (in the humid, salty air on board a small sailboat, for years on end) every kind of radio I owned died, except the Icom.

Version History
  • Version 1.6 01-30-2018. Two changes: (1) Rewrote this application in Python 3. (2) Fixed a bug that skipped one frequency entry per bank on most multi-bank Icom radios. With the exception of the Icom R-8500, these radios use 1-based indexing and therefore have 99, not 100, locations per bank.
  • Version 1.5 01-24-2018. Fixed a bug that defaulted to frequency split mode when no transmit frequency was specified. In some cases this prevented correct operation of automated antenna tuners.
  • Version 1.4 03-06-2013. Added more file types to those that are automatically created.
  • Version 1.3 12-25-2012. Tuned the HTML formatting code.
  • Version 1.2 12-16-2012. Optimized the PDF creation code, added more features.
  • Version 1.1 11-24-2012. Added more Icom models to the known-configuration lists (of label and bank sizes).
  • Version 1.0 11-18-2012. Initial public release.

Home | Ham Radio |     Share This Page