I used to lead a jazz fusion group during the 1990's and early 2000's... myself and the core members of the group stayed together throughout the decade and we had developed an uncanny ability to free-compose very elaborate pieces that rivaled the pieces that I had actually formally composed and written out in traditional form.
During this period I had devised a simple hand sign system in order to signal my fellow musicians as to any change that I wanted to go to in real time... of course we could also switch off between members so that anyone could be the signal caller at any given time.
These signals would generally be given anywhere from a bar or more before the desired change in order to insure equal balance of the sections in terms of phrase lengths and standard bar groupings. (i.e., 4 bar intros, 8 bar melodies, 16 bar vamps, endings, etc...)
The basic premise is that keys are represented by fingers either pointing up or down... for example two fingers pointing upwards like a peace sign would represent two sharps for the key of D major... one finger pointing down for one flat as the key of F major and so on... The key of C major is simply a closed fist or zero sign!
Modes are represented in the same manner only the fingers are pointed horizontally... for example four fingers pointing sideways would represent the Lydian mode and so on...
Since we obviously only have five digits to use and we do have the need to signal keys with upwards of six and seven sharps or flats as well as modes up to seven. These are signaled by forming a closed fist with the thumb placed between the index and middle fingers for the number six... the thumb placed between the middle and ring fingers for the number seven.
The designated sign caller of the moment would always signal the key change first and then the mode to be used within that chosen key. (i.e., four fingers upward for the key of E major followed quickly by two fingers to the right representing dorian... therefore we would all be playing and improvising over and with a F#m7 chord.)
Please try it for yourself and record your sessions for transcribing the pieces later in order to add these new compositions to your groups repertoire. The legendary bassist Jaco Pastorius used to say "Let's play it now and figure it out later" and trust me he was absolutely right in this type of scenario.
Note: I also created additional activator signs to represent other scales such as Melodic and Harmonic Minor, Diminished, Whole-tone, etc... which I more than likely will post at a later date.
Thank you and there are a variety of tracks demonstrating this at Soundcloud with my fusion group
"The Shadow Prophets" https://soundcloud.com/the-shadow-prophets