yamahasynth.com forums

This is the place to talk about all things related to Yamaha Synthesizers!
  1. Michael Trigoboff
  2. Montage
  3. Wednesday, 11 July 2018
I've been having trouble keeping track of my Live Sets. So I wrote a little app that displays an overview of all of them, read from a Montage file. It is proving to be really helpful for finding Live Sets that I've either lost or forgotten.

First, download the .zip file, extract livesets.py from it, and put livesets.py into a directory containing your Montage files.

Then you will need to install a Python interpreter. You can download a Windows one from here, or a Mac one from here,

For Windows 10, I suggest you download the 64-bit version. It probably wouldn't hurt to download the 64-bit version for a Mac as well.

Once you have the Python interpreter installed, open a command-line window (cmd.exe or PowerShell on Windows, Terminal Window on a Mac), navigate to the directory containing your Montage file(s) using the 'cd' command, and type this:

python livesets.py

The app will print instructions for how to use it.

At the moment it prints Live Sets in outline form. In the future, I'm thinking I might have it print in a grid form that matches what you see on the Montage Screen.

I've only tested it with .X7U files. It will only work with files created with version 2.0 or later of the Montage firmware. I might extend it to work with earlier versions.

If any of you use it, let me know how it goes. I'll be happy to look into and fix any bugs you find.
Attachments (1)
Responses (23)
Jason Accepted Answer Pending Moderation
I haven't downloaded or cracked open the zip - but there has also been an on-line live set viewer for X7L and X7U files. It does not cross-reference the Performance names -- so that's a downside (your utility likely does). The benefit of not doing this is that the utility was not affected when the MSB/LSB pairs changed in FW v2.00. That's what you're referencing about supporting older firmware. The downside (in the old online tool) is that the Perf names aren't there - so live set that is used seems "blank".

https://jyamaha.alwaysdata.net/
(or http://jyamaha.alwaysdata.net/ ) I usually use non-secure because there is no personal information or passwords exchanged and it's faster for me to leave off the https. At some point browsers will redirect on their own - and this site should be ready for that.
  1. one week ago
  2. Montage
  3. # 1
Michael Trigoboff Accepted Answer Pending Moderation
I just tried the website with a recent .X7U file (the one I was using to test my Python app), and this is all it did:

Upload Response
File upload size: Bytes: 2474464

Performances numentries: 112
The problem with Montage files created with pre-version 2.0 firmware is not just the MSB/LSB pair change. Something more significant must have been changed because the older files caused my app to crash, which the MSB/LSB change would not have done. Maybe that's why the website didn't work.

My app does print Performance names, but only for User Performances (I have an idea for how I might be able to print the names for Preset Performances, but I don't know if I'll bother). For Library Performances, I just print 'LIB', since I have not yet been able to figure out how those are encoded in the file. It prints '???' for Live Set entries that do not point to a valid Performance.

I've attached a sample of the text output it currently produces to this post.
Attachments (1)
  1. one week ago
  2. Montage
  3. # 2
Michael Trigoboff Accepted Answer Pending Moderation
Here is v1.0 of the app.

It now prints accurate information for Library Performances (although not their names). "Accurate information" means that it prints the same information as John Melas' Waveform Editor.
Attachments (1)
  1. one week ago
  2. Montage
  3. # 3
Jason Accepted Answer Pending Moderation
I do not think the parser online is having an issue with the new version. The parser throws away uploaded files - so I have no way to retest. I do have an error log of the script which shows a problem with a character in a string and my decoder ('ascii') was not tolerant of this. There's a chance that other files can suffer from similar problems in different places - but I've used a different decoder to try to account for non-ascii characters in the string as happened with your upload (meaning: there are other places I use "ascii" as the decoder - and I didn't change all of them - just the one place your particular file had issue). You can retry and see if it's fixed.
  1. 6 days ago
  2. Montage
  3. # 4
Michael Trigoboff Accepted Answer Pending Moderation
That same file works now. It might’ve been my use of the vertical bar character in a performance name.
  1. 6 days ago
  2. Montage
  3. # 5
Jason Accepted Answer Pending Moderation
I'm not sure - the decoder claimed there was an "0xFF" in the Performance name and "ascii" only decodes to 0x7F (7-bit) and bails out if there's a character above 127. The fact that it's working now is good enough for me.

Resolving library names requires the library. You would need to parse through an X7A if you want to stick to purely file decode. Or have the user load (in order) the library files to your parser. I do not believe they self-identify their slot numbering.

There's another a way without using X7U/X7L/X7A files - which is slow. To use MIDI messages to ask Montage for the Performance names of the library slots. This is a slow process if there are lots of library entries. And deciding how to terminate is difficult - requiring to scan through all possible library performances (640 x # libraries).

The Steinberg/Yamaha USB driver allows for file access of Montage - which is much faster. I wish I had some API information on this interface to leverage it. I'm not sure if it allows for general access - or is limited to just MIDI files. It would be good to use this interface to transfer performance names - if you could - since it's much faster than the (slow clock) serialized MIDI data.
  1. 6 days ago
  2. Montage
  3. # 6
Michael Trigoboff Accepted Answer Pending Moderation
I ran into that 0xFF thing. I discovered that some of the performance names start with four bytes of 0xFF. I just automatically get rid of them (Python lstrip) if they are there. I know there were some of those in my test file. Perhaps your new decoder is ignoring them.
  1. 6 days ago
  2. Montage
  3. # 7
Michael Trigoboff Accepted Answer Pending Moderation
It would be really nice if Yamaha chose to release some of their internal documents (like file format specifications and APIs). It could enable people to make some really cool apps, and I don’t see how Yamaha could lose by it. I can’t see where a file format specification could cause damage to Yamaha if it were released.
  1. 6 days ago
  2. Montage
  3. # 8
Jean Accepted Answer Pending Moderation
Hello

You can download from this site old version of my program "MontagePerformanceDump" at https://www.yamahasynth.com/forum/montageperformancedump-for-os-2-0 or from Moessieurs site :
http://www.moessieurs.com/mise-a-jour-5.html

With it you can scan a X7A, X7L or X7U file and produce a listing with all live sets with performances names (preset, user or lib) and the name of belonging librairies.

The listing show also waveforms, performances, arpeggios, curves, motion sequence, micro tuning, favorite and songs (actually missing audition phrases).

This is not the primary goal of the program, which is dump performances by midi for browsing, analyse dependencies, stats, etc.

As there is no interest for this program (request for improvements or bug reports), I have stopped sharing it and I improve it for my own needs.

But if there is an interest I can continue to share the latest versions.
  1. 6 days ago
  2. Montage
  3. # 9
Michael Trigoboff Accepted Answer Pending Moderation
Perhaps those of us who are programmers feel a need for tools that most musicians don’t necessarily want. Or maybe we create tools for ourselves that are too difficult for non-programmers to want to bother with.

Of the three “wheels“ we’ve (re)invented, I’d say Jason’s is the easiest to use, Jean’s is the most complete, and I prefer the way mine displays Live Sets.

I’m currently using mine to help me reorganize my Live Sets. It’s going very well.
  1. 5 days ago
  2. Montage
  3. # 10
Jason Accepted Answer Pending Moderation
I've added cross-reference to the user data in live sets. It's not online yet - but working. Also adjust for pre-2.0 vs 2.0 and later fw. I key off the file version. There's also a "guide" before the live set which shows how to decode the MSB:LSB:PC which isn't correct for v2.0, so I fixed that.

Again, not online yet.

I also noticed I do not tell the user what the page names are. I just spit them out - so I'll probably add a label like Live Set Page Name: XXXXX.
  1. 5 days ago
  2. Montage
  3. # 11
Michael Trigoboff Accepted Answer Pending Moderation
What’s the difference in the file format, 2.0 vs pre-2.0?

Maybe this would be a good place for us programmers to exchange information and help each other.
  1. 4 days ago
  2. Montage
  3. # 12
Jean Accepted Answer Pending Moderation
I think Jason speak about Bank select change in 2.0 (page 184 for datalist of 1.6 and page 187 for datalist of 2.0).

I don't know if there is really difference between file format in 2.0 and prior, but I read some informations differently for pre-2.0 versions. In short a shift of 4 bytes in the calculation of the offsets of the names of wavs, perfs, arps, etc.

I say I don't know if this is difference of file format because all is speculations about format and my interpretation of some data structure may be erratic.

Also, not different file format, but versions add some entries in catalog. eg : favorites, audition phrases.

And as new versions often add performances, effects and arpeggios I have four tables of presets in my program depending on version.

I am ok to exchange information but I have difficulties with english.

(Do you speak french ?)
  1. 4 days ago
  2. Montage
  3. # 13
Jason Accepted Answer Pending Moderation
This is correct - the format has not changed. But the interpretation of MSB:LSB:PC for presets/user/library performances (single and multi) has changed. My previous version of the live set information just spit out the raw MSB:LSB:PC - and at the top of the live set section gave a "decoder ring" for which MSB:LSB pairs were for presets/user/or library. That was wrong for fw v2.0+ -- but the new, not live yet, version corrects this. Also I internally store each entry from a number index based on MSB:LSB:PC from 0-639. To calculate this index, one must use a different formula due to the MSB:LSB:PC differences between the two versions.

I have no table for internal performances as the utility is focused on user information. I just still spit out raw information on presets (even in the latest version). I could have a table for each. Generating such a table is simple enough using the data list. Such as what I have done with the "cheat sheets" I've published showing preset Performance PART utilization.
  1. 4 days ago
  2. Montage
  3. # 14
Jean Accepted Answer Pending Moderation
Just a question : Is sharing information about file format legal ?

I have not read the copyrigth, but it is possible that reverse engineering of file format is not legal.

So I think Yamaha don't like we share these informations on his forum ?
  1. 4 days ago
  2. Montage
  3. # 15
Michael Trigoboff Accepted Answer Pending Moderation
(Do you speak french ?)
Je suis désolé je parle seulement un peu français.
  1. 4 days ago
  2. Montage
  3. # 16
Michael Trigoboff Accepted Answer Pending Moderation
Just a question : Is sharing information about file format legal ?

I have not read the copyrigth, but it is possible that reverse engineering of file format is not legal.

So I think Yamaha don't like we share these informations on his forum ?
I think it would be unimaginably stupid for Yamaha to take legal action against three of their customers who are providing free tools to help Montage users. But who knows the depths of stupidity that lawyers might be capable of?

I am personally willing to take the risk, since I'm not running a business that could be affected. My impression is that Jason is in the same situation as I am. Jean is in a different situation.

John Melas is either reverse-engineering the file formats, or perhaps he has made a licensing deal with Yamaha. I am going to ask him, and maybe he'll tell me. If it turns out that he's reverse-engineering the files, then we're all safe because Yamaha seems to be fine with what he's doing.

Or maybe someone from Yamaha could notice this conversation and tell us what Yamaha officially thinks?
  1. 4 days ago
  2. Montage
  3. # 17
Jean Accepted Answer Pending Moderation
I am not in a different situation. I am retired and I do not sell anything.
My post is just a precaution.
And actually if there is no reaction from Yamaha, "who does not say word consents" as we say in French.
  1. 4 days ago
  2. Montage
  3. # 18
Jean Accepted Answer Pending Moderation
I think John Melas is in the same situation as us and he is making efforts to decrypt the format.

I say this because after buying the Waveform Editor, and loaded one of my X7A files, I found that the performances of an x library referenced waveforms of a library y.
But I knew that the performances of the library x of my file referenced the waveforms of the library x.

So I wrote him and sent my file.

I added, if it could help, that I had written some software and that I had to play with two index to find the libraries to which elements belonged. A "natural" index that follows the order of MSB.LSB.PC and an internal index of Yamaha that I do not understand, but that I know how to find.

He told me that was the problem and asked me how to find this index.
I therefore conclude that he has no more information from Yamaha than us.
Of course I communicated the information to him and he quickly sent me a patch.

In the past I also asked him if he could modify his software for Moxf to merge the arpeggios of different files as he did in his software for the Motif. (An impossible manipulation directly on the Motif and the Moxf).
He told me that Moxf's internal arpeggio format was completely different from Motif's, and that he will study the question later but he did not have time now.

I was lucky enough to be able to decrypt this format enough to write an utility allowing this manipulation (which I shared here and on Motifator).
Again I concluded that he did not have more information than me from Yamaha, the internal format, for arpeggios, was indeed completely different from the Motif.
  1. 3 days ago
  2. Montage
  3. # 19
Jean Accepted Answer Pending Moderation
Hi

Here is my first contribution about file format.

Hope you can found something useful.

https://drive.google.com/open?id=1GKajdGJbKJmXMOs85LVSyvP9tOQf3r2i

https://drive.google.com/open?id=1CV1t2qBqm2e8PX0laY5NF-d65no_HKxI
  1. 3 days ago
  2. Montage
  3. # 20
  • Page :
  • 1
  • 2


There are no replies made for this post yet.
However, you are not allowed to reply to this post.
2018 © Yamaha Corporation of America and Yamaha Corporation. All rights reserved.