Well, it's time to write your own protocol. Music-X's Librarian is billed as "universal": it can communicate with any MIDI instrument, provided that you supply the right information using the built-in Protocol Editor. Although this process is documented in the Music-X manual, it is still tricky to do. In addition, the manual is more of a reference than a tutorial.
This article will teach you how to write a Music-X protocol and contains plenty of examples. I won't repeat straightforward information found in the manual, so I recommend having it near you while you work through this article. Also, I won't cover every single detail about protocols; instead, I'll concentrate on the more practical uses (and a few less common ones) so you can start writing protocols as quickly as possible.
Since this is a technical journal, I'll assume that you know how to use an ASCII table and hexadecimal numbers. In the text, all hex numbers will be preceded by a dollar sign to distinguish them from decimal numbers. (For example, hex $42 equals 66 in base ten.) In addition, some familiarity with MIDI system exclusive data will be helpful.
Both libraries and protocols may be saved as separate files on your disk. The following diagram illustrates the relationship of library and protocol files.
Although library files are physically separate from protocol files, every library file has a COPY of some protocol inside it. If you modify the copy inside a library, and then save the library, the protocol file on disk is not affected. Similarly, modifying the protocol file on disk does not affect any libraries. Thus, if you have a protocol that is used by ten different libraries, and you want to modify that protocol, you must change it in all ten libraries! This is a shortcoming of the implementation.
Music-X comes supplied with protocols for several popular instruments. In particular, there is a "generic" protocol called Generic.Single that works for almost all instruments. "Well," you say, "if this generic protocol is so versatile, why not use it for all my instruments?" One reason is that Generic.Single cannot request patches from your instrument -- you must initiate the transfer from the instrument's front panel (assuming this is possible). A second reason is that it cannot distinguish legal from illegal data. This means that if there is a transmission error between your instrument and your Amiga, the generic protocol will not notice.
However, Generic.Single is a good starting point for writing your own specific protocol. Let's take a look at the generic protocol by making a generic library. Run Music-X, and choose Librarian from the Mode menu. Then choose New... from the File menu, move to the appropriate directory, and select the file Generic.Single. Finally, choose Modify Protocol from the Edit menu, and we have arrived at the Protocol Editor:
The important part of this protocol is found in the Data strings in the SEND and RECEIVE panels. Briefly, the RECEIVE Data entry says to read data in 16-megabyte ($1000000-byte) chunks until an $F7 is received. The SEND Data entry says to send 16-megabyte chunks of data until there is none left, and then send $F7. (Note: $F7 is a special MIDI byte meaning "end of system exclusive data.")
In other words, this protocol is rather mindless. It grabs or sends as much data as it can find, regardless of what the data looks like. If you could hand it a Deluxe Paint file, for example, this protocol would happily send it to your MIDI synthesizer. (No, the picture would not appear on your synth's front panel!)
You do not need to answer all of the questions: only those that apply to your instrument. For example, some instruments transmit all their data in one big message, but others require an elaborate "conversation" between the sender and the recipient. You must read your instrument's system exclusive documentation to determine which questions need to be answered. For some help with this, see Phil Saunders' "Medley" columns in the April and May 1991 issues of AMAZING COMPUTING.
P The patch number that you set on the Librarian Page. N The MIDI channel that you set on the Librarian Page.and others have their values maintained by Music-X:
M Total blocks sent. V The last value received. K Checksum of a specified sequence of bytes.A complete list of numeric variables is found in the manual.
In general, to store k bytes of data, you use the expression:
(X.k)For example, this is how you tell Music-X to send or receive 25 ($19) bytes of data:
(X.19)Here is how you tell Music-X to SKIP past 25 bytes of data, not storing it:
(X.19.19)Here is how to tell Music-X to break 50 bytes of data into nybbles, delete all the most-signficant nybbles, and concatenate the remaining data into 25 ($19) bytes:
(Y.19)
A second, very useful string variable is the Program string, which is represented as PG. To use it, fill in the strings labeled Normal Program and Preview Program with anything you like. The value of the PG variable will be either of these two strings. You toggle between the two values by pressing the PREVIEW button on the Librarian Page. This provides a convenient way for the user to change the behavior of the protocol without entering the Protocol Editor.
Two general-purpose string variables are known merely as #1 and #2. Fill them with any data you like, and use them anywhere on the SEND and RECEIVE panels. If you don't actually need these strings for protocol data, they provide a convenient place to write comments to yourself about the protocol.
An Initiate string says, "Hello, I would like to begin a patch transfer." To determine its value, consult your instrument's system exclusive documentation, and look for a message called "Request to send," "Request for bulk dump," or something similar. For the SEND Initiate string, one often uses the header from a patch dump.
If your instrument does not require any handshaking, then you will need to use only the Initiate and Data strings, and you are now ready to write your Data strings. These contain the X and/or Y data variables for sending or receiving the patch data and storing it as a library entry. The case studies in the next section will provide several examples.
If you instrument does require handshaking, then you must also provide Confirm and/or Ack ("Acknowledge") strings. These strings each represent the message "I am ready!" Confirm is sent by the instrument (and so Music-X must be told how to recognize it), and Ack is sent by Music-X. Depending on the system exclusive implementation, your instrument expects a particular sequence of Confirm/Data/Ack messages. The Music-X manual is really quite good about explaining this mechanism; however, you will need to spend some time with your instrument's system exclusive documentation to figure out what messages need to be used.
Finally, if your instrument requires a special message if the patch transfer is aborted, you should fill a value for the Cancel string.
Reading the T8 manual, we find that the following message will ask the T8 to send a particular patch via MIDI:
$F0 "Begin system exclusive" byte. $01 Sequential Circuits' manufacturer ID. $00 Request to send a patch, please. ... The patch number. $F7 "End of system exclusive" byte.Music-X needs to send this message when it wants to receive a patch. So, we fill in the following data in the RECEIVE Initiate box:
F0, 01, 00, P, F7Now, Music-X must be prepared to receive data from the T8. A patch contains 68 ($44) bytes of data, followed by $F7. So, in the RECEIVE Data box, we write:
(X.44), F7Now, we could have written (X.45) and grabbed the $F7 byte at the same time. However, by writing $F7 explicitly, we cause Music-X to check that an $F7 is received as the 69th byte. Also, since we have specified the $F7 separately, not as part of an "X" variable, it will not be stored as part of the patch. We must remember to append an $F7 when we send our patch data back to the T8.
To fill in the SEND panel, we must examine the T8 patch data dump format:
$F0 "Begin system exclusive" byte. $01 Sequential Circuits' manufacturer ID. $03 This is data for one patch. ... The patch number. ... 68 bytes of patch data. $F7 "End of system exclusive" byte.To transmit the patch back to the instrument, we could simply send everything we have stored. However, this is a bad solution because the original patch number is stored inside the patch data. Thus, we would be able to send the patch back to its original location ONLY! This is not very versatile, especially since the Librarian will let us change the patch number on the Librarian Page if we use the "P" variable.
To solve this problem, we'll use the first part of the data dump format as our SEND Initiate string, skip the original first four bytes of the stored patch, and then send the rest of the data. So our SEND Initiate value is now:
F0, 01, 03, Pand our SEND Data value, which skips past the first 4 bytes, is:
(X.4.4), (X.40), F7Note that we remembered to append the $F7 that earlier we chose not to store.
Since the first two bytes of both Initiate values are the same -- $F0, $01 -- let's define the PR ("Prefix") string variable to have this value, and use PR in both Initiate commands. They become:
RECEIVE Initiate: PR, 00, P, F7 SEND Initiate: PR, 03, Pand our T8 protocol is now complete.
Using Generic.Single to grab 1 patch, and reading the documentation, we find out the following information:
o A single patch dump contains 398 bytes plus $F7. o The patch name is found in bytes 382-398. o The format for requesting 1 single patch is: $F0 "Begin system exclusive" byte. $10 Oberheim's manufacturer ID. $02 This is an Xpander. $00 Send me a single patch, please. ... The patch number (0-99). $F7 "End of system exclusive" byte. o The format of a single patch dump is: $F0 "Begin system exclusive" byte. $10 Oberheim's manufacturer ID. $02 This is an Xpander. $01 Here comes a patch. $00 It is a single patch. ... The patch number (0-99). ... 392 bytes of patch data. $F7 "End of system exclusive" byte.Now we are ready to start writing the protocol. Since both the SEND and RECEIVE command will begin with the same bytes, we define the PR ("Prefix") variable once again -- this time to be:
F0, 10, 02On the RECEIVE panel, we set the Initiate value to:
PR, 0, 0, P, F7We now expect an entire patch dump of 398 ($18E) bytes to follow, ending with an $F7, so we can set the RECEIVE Data value to:
(X.18E), F7For sending data, we use the same trick as in the T8 case study: skip over the first several bytes (containing the old patch number) and substitute our own in the SEND Initiate value. This value is:
PR, 1, 0, PWe now skip the first 6 bytes of data (the header) and send only the 392 ($188) bytes of patch data itself, plus the $F7 we stripped off during the RECEIVE.
(X.6.6), (X.188), F7Finally, let's tell the protocol how to locate the patch name. It is found in bytes 382-398 of the patch data. Thus, give Name Offset a value of 382 and Name Length a value of 16. The Librarian will now extract and display the name of each patch as it is received.
Normally, we would be done now. However, Xpander patch names are stored in a particular manner that requires us to do a little more work. At the moment, only the first character of the patch name will actually be displayed on the Librarian Page. We shall solve this problem in Example #2 under HANDLING PATCH NAMES, below.
In this protocol, we will use checksums and see an alternate, more reliable, method for handling the Data strings.
To request a patch from the SPX-90, the command is:
$F0 "Begin system exclusive" byte. $43 Yamaha's manufacturer ID. $2n Use MIDI channel n, increased by $20. $7E Send me some kind of patch dump. "LM 8332" I am an SPX-90 (there are 2 spaces). "M" Give me one patch dump. ... The patch number. $F7 "End of system exclusive" byte.By now, we are experts and can convert this into a RECEIVE Initiate string quickly, except for the line that says "Use MIDI channel n." Music-X provides the N variable which always contains the number of the MIDI channel that we set on the Librarian Page. Once we add $20 to it, we're ready to type in the string:
F0, 43, (20+n), 7E, "LM 8332", "M", P, F7We will need some of these bytes again later, so let's store the first two bytes as the Prefix, and the two strings as Substring #1. This makes our RECEIVE Initiate value:
PR, (20+n), 7E, #1, P, F7The format of the SPX-90 patch dump is the most complicated we've seen so far:
$F0 "Begin system exclusive" byte. $43 Yamaha's manufacturer ID. n Use MIDI channel n. $7E This is some kind of patch dump. $00 Together with the next byte... $58 ...this is the data length: 88 bytes. "LM 8332" I am an SPX-90 (there are 2 spaces). ... The patch number. ... 32 bytes holding the 16-character patch name. ... 46 bytes of patch data. ... Checksum $F7 "End of system exclusive" byte.Our RECEIVE Data string needs to read 16 ($10) bytes of header, 32 ($20) bytes of patch name, 46 ($2E) bytes of patch data, and 1 checksum byte. Using the same method as the previous two protocols, and ignoring the checksum issues for now, we would have a RECEIVE Data value of:
(X.10), (X.20), (X.2F), F7or more simply:
(X.5F), F7However, this time we will build more intelligence into the RECEIVE Data string by describing the format of the patch dump. This approach has two advantages: it checks the incoming data more closely, and it saves space by not storing the header bytes inside each patch. A small disadvantage is that the patch library data is now more heavily dependent on the protocol; thus, if you plan to write your own programs to interpret Music-X library files, you may have a tougher time doing it.
There are 32+46+1 ($4F) bytes of data after the patch number. Once again ignoring the checksum, we write the RECEIVE Data string as follows:
F0, 43, N, 7E, 0, 58, "LM 8332", "M", P, (X.4F), F7This is now an accurate description of the data we shall expect, so Music-X will complain if it reads any non-matching bytes. Using our PR and #1 substrings, this becomes:
PR, N, 7E, 0, 58, #1, P, (X.4F), F7Since everything before the (X.4F) is not captured by the X data variable, the header will not be stored inside each library entry. This fact is the reason for the advantages and disadvantages I listed earlier.
Now, let's handle the checksum byte. Yamaha forms its checksum value by adding all the bytes between #1 and (X.4E) inclusive, negating the value, applying a logical "and" operation with $7F, and truncating to one byte. This operation is notated as (-K&7F.1). If the checksum received does not match this value, Music-X complains.
To form this checksum, we surround the relevant bytes with curly braces. Thus, our finished RECEIVE Data string is:
PR, N, 7E, 0, 58, {#1, P, (X.4E)}, (-K&7F.1), F7
Now it's time to build the SEND portion of the protocol. We must
remember that the header bytes and the checksum have not been stored in the
library entry, so we must construct and send them manually. Well, guess
what? We can use the same patch description we made for the RECEIVE Data
string. There is no need for any SEND Initiate string in this case.
To save typing, we store the RECEIVE Data string as string #2, and then just put #2 in our RECEIVE Data and SEND Data strings. We are now done except for handling the patch name, which we discuss in Example #3 of HANDLING PATCH NAMES, below.
Even though Music-X knows WHERE your patch name is, it may not know how to INTERPRET it. If your MIDI instrument uses ASCII and stores its patch name one character per byte, then no special interpretation will be necessary. However, suppose your instrument uses the values 1 through 26 to represent the letters 'A' through 'Z', values 27 through 36 to represent the digits '0' through '9', and 37 to to be a question mark character. This is NOT the ASCII code. So, you need to tell Music-X how to TRANSLATE between your MIDI instrument's internal code for the patch name, and the standard ASCII code that Music-X (and most computers) use. This is done with a Character Map.
To construct a Character Map, picture the numbers 0 through 127 all lined up in a row. These are the numbers that your instrument uses for characters in patch names. Beneath each number, write the character that the number SHOULD represent to Music-X. The resulting two-row table is a Character Map. Here's the Map for our example above:
0 1 2 3 4 5 6 7 8 9 10 ... 25 26 27 28 29 30 31 32 33 34 35 36 37 A B C D E F G H I J ... Y Z 0 1 2 3 4 5 6 7 8 9 ?Now that we have the table, how do we fill in the Char Map field on the Protocol Editor screen? This field contains a list of RANGES OF CHARACTERS, separated by commas. For example, AZ means "all characters from 'A' to 'Z', inclusive," and 06 means "all characters from '0' to '6', inclusive." You may also indicate a range of numeric values by using two hexadecimal numbers preceded by backslashes.
Now look at your two-row table and pack your second row into ranges, as described above. In our example given, you wind up with this Character Map:
\00\00,AZ,09,??The first range, \00\00, is only one character long. It says that the value 0 should be translated into 0. Is this detail necessary? Yes, because a Character Map is always assumed to begin at zero. If we used the Map
AZ,09,??instead, then the value 0 would be translated into 'A', 1 into 'B', etc. This is wrong -- it's off by 1.
The second range, AZ, says that the values 1-26 get translated into the characters 'A' to 'Z'. The third range, 09, translates the values 27-36 into the characters '0' through '9'. The last range, ??, is only one character long, and translates the value 37 into a question mark character. What happens to the values 38-127? The manual does not say, so I play it safe and specify that they translate to themselves:
\00\00,AZ,09,??,\26\7FCharacter Maps are versatile but have a few serious limitations. First of all, they cannot handle characters that are not stored one per byte. For example, the Prophet VS synthesizer stores its name in tightly packed, 5-bit characters, which Music-X cannot translate. Second, Character Maps are insufficient if your instrument's internal character set is wildly unlike ASCII. For instance, if a synth has an internal character set like this:
0 1 2 3 4 5 6 7 ... 'A' 'Q' '7' '%' 'R' 'M' '2' '*' ...with characters placed arbitrarily in the table, there is no Character Map that can represent this translation.
But, you cry, why can't we define a separate range for each character, containing ONLY that character, like this?
AA,QQ,77,%%,RR,MM,22,**, ...In theory, you can; unfortunately, the Char Map gadget can hold only 79 character! Thankfully, today's MIDI instruments are constructed around popular microprocessors that use ASCII. Perhaps a future version of Music-X will address these limitations.
Here are some examples of constructing character maps.
\00\1B,19,00,\26\34,az,\4F\52,AZ,\6D\7F
The ranges are 0-27 ($0-$1B) untouched, the digits '1' to '9', the
digit '0' in its own range, 38-52 ($26-$34) untouched, the letters a-z,
79-82 ($4F-$52) untouched, and the letters A-Z. To be safe, we specify that
the remaining values 109-127 ($6D-$7F) translate to themselves.
To remedy this, I used a Character Map to translate zeroes into space characters (ASCII $20). It leaves all other characters untouched. Here's the Character Map, assuming that the Xpander uses plain ASCII for the rest of its characters (which it does):
\20\20,\01\7FNow the patch names are displayed correctly, though a blank space appears after each of the 8 characters. We can eliminate the last space at least by claiming that the name length is 15 instead of 16.
Just for fun, let's modify our Char Map to translate capital letters into small letters. The Xpander uses only capital letters internally. On the ASCII table, capital letters are found in positions 65-90 ($41-$5A), so we isolate this range:
\20\20,\01\40,AZ,\5B\7Fand then translate the letters to lower case:
\20\20,\01\40,az,\5B\7F
Anyway... can we make Music-X extract this patch name? Turning to the manual, we find that Music-X can indeed understand this "nybblized" data (a nybble is half a byte) by using the Y data variable (page 384). It can grab two bytes, extract their least-significant nybbles, and join them together into a single byte -- exactly what we need. Hooray!
Unfortunately, our success is short-lived. Music-X assumes that the first byte arriving is the least-significant, and the second is the most-significant. This is exactly the opposite of what the SPX-90 sends! In other words, in our $45 example above, Music-X will interpret this as the number $54 (the letter 'T'). Our method won't work.
In fact, our method has a second problem. If we use the Y variable to extract nybbles, this changes the actual data stored by Music-X, and therefore alters the value of our computed checksum (K variable)! In order to make this work, we would have to disable the checksum handling in our protocol, and treat the checksum as an ordinary data byte.
As of this writing, I still have not managed to get Music-X to understand SPX-90 patch names. If anybody else can figure out a method, please contact me.
If you choose Load... in the Librarian, and then click on CANCEL in the requestor, your current library disappears from the screen (and is replaced by "No Page"). I consider this a bug because "CANCEL" should return the program to the exact state it was in before the requestor appeared. To bring back your library, use the Set Display command in the Edit menu.
If you are in the Protocol Editor and you choose Load... or Save... from the File menu, sometimes the file requestor's Directory gadget contains an incorrect value. This causes the message "-- Not a valid directory --" to be displayed in the requestor. If you examine the directory name, you will find that the end of the directory name has been replaced by the PROTOCOL's name! This happens if your directory name is longer than 32 characters.
I have managed to put Music-X into an infinite loop (although its menu bar keeps working) by choosing Load in the Librarian when there is already a library loaded. The problem is intermittent and may be related to the above directory bug.
- barrett@cs.umass.edu