Describe the problem. What did you expect? What do you see?

As described here: http://www.fretsonfire.net/forums/viewtopic.php?f=11&t=50698&start=420#p600014

and here: http://www.fretsonfire.net/forums/viewtopic.php?f=11&t=50698&p=600020#p600020

The user indicates that a chart that uses hi-hat control phrases added by EOF, FoF won't play it, it will return to the song selection screen. EOF exports them as Sysex events that should rightly be ignored by any software that doesn't have their own handling for those events. However, it appears FoFiX will just fail to handle the MIDI entirely.

List the steps to recreate the problem. 1. Play a chart where the MIDI has Sysex events

What version are you using? Include the FoFiX version, the Python version, your operating system, and whether it is Git, alpha, beta, RC, or final release. (If it's Git, please include the commit hash.)

The user indicated this occurs with FoFiX 3.121 and FoFiX 4.0.

If this is a bug report, please include the log file!

The user provided the FoFiX 4 log: [ 0.000000] (D) Logging initialized: Tue Oct 04 16:16:39 2011 [ 0.515000] (W) Pitch bending is not supported; install john.stumpo's pitchbend module (r7 or higher) if you want it. [ 0.515000] (W) OggStreamer not found. Falling back to legacy pyogg/pyvorbis based ogg streamer. [ 0.640000] (W) Missing pyaudio or pypitch - microphone support will not be possible [ 0.703000] (D) GameEngine class init (GameEngine.py)... [ 0.703000] (D) FoFiX v4.0.0 alpha 1 starting up... [ 0.703000] (D) Python version: 2.6.6 [ 0.703000] (D) Pygame version: 1.9.1release-svn2575 [ 0.703000] (D) PyOpenGL version: 3.0.1 [ 0.703000] (D) Numpy version: 1.5.1 [ 0.703000] (D) PIL version: 1.1.7+ [ 0.703000] (D) sys.argv: ['C:\Documents and Settings\Joe Santoli\Desktop\fofix-4.0.0alpha1-win32\fofix-4.0.0alpha1\FoFiX.exe'] [ 0.703000] (D) os.name: nt [ 0.703000] (D) sys.platform: win32 [ 0.703000] (D) win32api.GetVersionEx(1): (5, 1, 2600, 2, 'Service Pack 2', 2, 0, 256, 1, 0) [ 0.719000] (D) Disabling screensaver. [ 0.719000] (D) Initializing audio. [ 0.765000] (D) Audio configuration: (44100, -16, 2) [ 0.781000] (D) Initializing pygame.mixer & audio system at 44100 Hz. [ 0.781000] (D) Initializing video. [ 0.859000] (D) 0 joysticks found. [ 0.875000] (W) No MIDI input ports found. [ 1.547000] (D) Theme font not found: title.ttf [ 1.547000] (D) Theme font not found: streak.ttf [ 1.547000] (D) Theme font not found: song.ttf [ 1.562000] (D) Theme font not found: songlist.ttf [ 1.562000] (D) Theme font not found: songlist.ttf [ 1.562000] (D) Theme font not found: streakphrase.ttf [ 1.562000] (D) 6 guitscw sounds found in sounds: guitscw1.ogg - guitscw6.ogg [ 1.562000] (D) Theme sound not found: bassdrum.ogg [ 1.578000] (D) Theme sound not found: battleused.ogg [ 1.578000] (D) Theme sound not found: crash.ogg [ 1.578000] (D) Theme sound not found: clapsound.ogg [ 1.578000] (D) Theme sound not found: coopfail.ogg [ 1.578000] (D) Theme sound not found: failsound.ogg [ 1.578000] (D) Theme sound not found: rescue.ogg [ 1.578000] (D) Theme sound not found: rocksound.ogg [ 1.578000] (D) Theme sound not found: staractivate.ogg [ 1.578000] (D) Theme sound not found: stardeactivate.ogg [ 1.594000] (D) Theme sound not found: starlost.ogg [ 1.594000] (D) Theme sound not found: starpowerready.ogg [ 1.594000] (D) Theme sound not found: tom01.ogg [ 1.594000] (D) Theme sound not found: tom02.ogg [ 1.594000] (D) Theme sound not found: tom03.ogg [ 1.594000] (W) MegaLight V4/sounds/crowdcheers.ogg not found -- using data/sounds/crowdcheers.ogg instead. [ 1.594000] (W) No stages\ folder found, forcing None setting for Animated Stage. [ 1.594000] (D) theme.ini loaded [ 1.609000] (D) Ready. [ 1.609000] (D) 6 bassscw sounds found in sounds: bassscw1.ogg - bassscw6.ogg [ 1.687000] (D) 8 drumscw sounds found in sounds: drumscw1.ogg - drumscw8.ogg [ 4.000000] (D) View: Push: MainMenu [ 4.000000] (D) View: Push: Menu [ 6.281000] (W) Your theme does not appear to properly support the solo graphical submenu. Check to be sure you have the latest version of your theme. [ 6.281000] (D) View: Push: Menu [ 7.390000] (D) View: Pop all [ 7.390000] (D) View: Pop: MainMenu [ 7.390000] (D) View: Pop: Menu [ 7.390000] (D) View: Pop: Menu [ 7.390000] (D) View: Pop: MainMenu [ 7.406000] (D) View: Pop: Menu [ 7.515000] (D) View: Push: Lobby [ 7.515000] (W) No theme class for themeLobby - Loading default... [ 13.484000] (D) View: Push: LoadingSplashScreen [ 13.484000] (W) No theme class for setlist - Loading default... [ 13.484000] (W) Your theme does not appear to properly support the setlist graphical submenu. Check to be sure you have the latest version of your theme. [ 13.484000] (D) Loading libraries in C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs [ 13.500000] (D) Song.getAvailableLibraries function call...library = C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs [ 13.500000] (D) Loading songs in C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs [ 13.547000] (D) Song C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\8. Shadow Gallery - The Crusher\song.ini was not found in the cache. [ 13.547000] (D) Retrieving parts from: C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\8. Shadow Gallery - The Crusher\notes.mid [ 13.703000] (D) MIDI Type identified as GH [ 13.703000] (D) Retrieving sections from: C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\8. Shadow Gallery - The Crusher\notes.mid [ 13.844000] (D) Writing out cache for song C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\8. Shadow Gallery - The Crusher\song.ini. [ 13.844000] (D) Song C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Closure in Moscow - Reindeer Age\song.ini was not found in the cache. [ 13.844000] (D) Retrieving parts from: C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Closure in Moscow - Reindeer Age\notes.mid [ 13.969000] (D) MIDI Type identified as GH [ 13.969000] (D) Retrieving sections from: C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Closure in Moscow - Reindeer Age\notes.mid [ 14.078000] (W) Song.py: Using auto-generated note count sections... [ 14.078000] (D) Writing out cache for song C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Closure in Moscow - Reindeer Age\song.ini. [ 14.078000] (D) Song C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Milk+ - Celadoar\song.ini was not found in the cache. [ 14.078000] (D) Retrieving parts from: C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Milk+ - Celadoar\notes.mid [ 14.094000] (W) Note file not parsed correctly. Selected part and/or difficulty may not be available. [ 14.094000] (D) Retrieving sections from: C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Milk+ - Celadoar\notes.mid [ 14.094000] (W) Song.py: Unable to retrieve section names for practice mode selection: MidiSectionReader instance has no attribute 'sysex_event' [ 14.094000] (D) Writing out cache for song C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Milk+ - Celadoar\song.ini. [ 14.328000] (D) Setlist loaded. [ 14.328000] (D) View: Pop: LoadingSplashScreen [ 14.328000] (W) Unable to load image file: C:\Documents and Settings\Joe Santoli\Desktop\fofix-4.0.0alpha1-win32\fofix-4.0.0alpha1\data\themes\MegaLight V4\setlist\item.dae [ 14.328000] (W) Unable to load image file: C:\Documents and Settings\Joe Santoli\Desktop\fofix-4.0.0alpha1-win32\fofix-4.0.0alpha1\data\themes\MegaLight V4\setlist\label.dae [ 14.328000] (W) Unable to load image file: C:\Documents and Settings\Joe Santoli\Desktop\fofix-4.0.0alpha1-win32\fofix-4.0.0alpha1\data\themes\MegaLight V4\setlist\library.dae [ 14.328000] (W) Unable to load image file: C:\Documents and Settings\Joe Santoli\Desktop\fofix-4.0.0alpha1-win32\fofix-4.0.0alpha1\data\themes\MegaLight V4\setlist\library_label.dae [ 14.578000] (D) View: Push: SongChoosingScene [ 14.578000] (D) View: Pop: Lobby [ 20.578000] (D) Song C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Milk+ - Celadoar\song.ini successfully loaded from cache. [ 20.578000] (D) Song C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Milk+ - Celadoar\song.ini successfully loaded from cache. [ 20.578000] (D) View: Push: PartDiffChooser [ 20.578000] (W) No theme class for partDiff - Loading default... [ 26.203000] (D) View: Pop: PartDiffChooser [ 26.219000] (D) View: Pop: SongChoosingScene [ 26.219000] (D) GuitarScene init... [ 26.219000] (D) Song C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Milk+ - Celadoar\song.ini successfully loaded from cache. [ 26.219000] (D) View: Push: LoadingSplashScreen [ 26.250000] (D) Battle Objects Enabled: [4, 2, 3, 7, 8, 6, 5] [ 26.250000] (E) Attempted to load theme variable fret_press - no default found. Traceback (most recent call last): File "Theme.pyo", line 58, in getattr KeyError: 'fret_press'

[ 26.641000] (D) themes\MegaLight V4\notes exists! [ 27.031000] (D) Simple tails used; complex tail loading error... [ 27.109000] (D) GuitarScene keysList: [[64, 256, 1024, 4096, 16384, 128, 512, 2048, 8192, 32768]] [ 27.984000] (D) loadSong function call (song.py)... [ 27.984000] (D) Song C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Milk+ - Celadoar\song.ini successfully loaded from cache. [ 28.000000] (D) Song with only a single audio track identified - single-track miss volume applied: 1.0 [ 28.000000] (D) Retrieving notes from: C:\Documents and Settings\Joe Santoli\Desktop\FoFiX 3.121 Final (py 2.4 ogl2)\data\songs\Milk+ - Celadoar\notes.mid [ 28.141000] (E) Terminating due to unhandled exception: Traceback (most recent call last): File "FoFiX.py", line 298, in <module> File "FoFiX.py", line 253, in main File "GameEngine.pyo", line 1119, in run File "GameEngine.pyo", line 1083, in main File "GameEngine.pyo", line 1111, in doRun File "GameEngine.pyo", line 1078, in _runTask File "Input.pyo", line 335, in run File "Input.pyo", line 246, in broadcastEvent File "SongChoosingScene.pyo", line 843, in keyPressed File "SongChoosingScene.pyo", line 633, in startGame File "World.pyo", line 127, in createScene File "SceneFactory.pyo", line 38, in create File "GuitarScene.pyo", line 852, in init File "Resource.pyo", line 229, in load File "Resource.pyo", line 100, in load File "GuitarScene.pyo", line 852, in <lambda> File "Song.pyo", line 3753, in loadSong File "Song.pyo", line 2527, in init File "midi\MidiInFile.pyo", line 48, in read File "midi\MidiFileParser.pyo", line 173, in parseMTrkChunks File "midi\MidiFileParser.pyo", line 135, in parseMTrkChunk File "midi\EventDispatcher.pyo", line 67, in sysex_event AttributeError: MidiReader instance has no attribute 'sysex_event'

The Phase Shift developers and I had decided to try to implement custom markers with Sysex events because it wouldn't cause any conflicts with the current system of using normal MIDI notes. However this becomes a problem if FoFiX isn't handling the MIDI correctly. The fix should be very simple, FoFiX's MIDI parser can just skip each Sysex event, each event explicitly defines how many bytes large it is.

It looks like this problem has been in FoF's code base for at least 5 years, as it's mentioned in this Italian forum post: http://forumtgmonline.futuregamer.it/showthread.php?s=2bbc0cb0260deb60abda803a03aa5855&t=27300

The crash is coming from the midi code that comes from here that has been used since the beginning.

http://www.mxm.dk/products/public/pythonmidi/

I emailed maxm, the listed contact for that Python package.

Maxm didn't offer to correct the code, but gave his consent for us to do so. I don't know python, so if one of the FoFiX developers could correct this, it would probably be the best way to go.

I don't have the time to learn enough Python to fix this myself, it really would be best for one of FoFiX's developers to make corrections to the MIDI parser. At this time, I do not plan to remove EOF's use of Sysex events for notations that are not universally supported by PC rhythm games, because that would be the opposite of progress.

NewCreature has provided a patch that corrects the issue by removing the function call that tries to process the Sysex event: http://www.fretsonfire.net/forums/viewtopic.php?p=608206#p608206

So i guess i will mark this issue as fixed.

Since the fix is so simple, could it be back-ported to FoFiX 3.x or any earlier releases? Please do us know when a release that includes this fix is made, until then EOF's documentation will have to maintain that all versions of FoF and FoFiX have this problem.

it would be simple to back-port to any fofix release, as the midi library hasn't changed any since the original FoF. An hour ago or so before i saw this, i was thinking about adding newcreature's fix to the maintenance_3x branch of fofix, plus a few other things. So there might be another 3.12x release at some point.

I was talking a little bit with nhydock(blazingGamer) about making another release of 4.0 with this fix, and others. We both agreed it was a good idea.

Sounds good to me.