Programmable real-time audio signal processing application
Nelze vybrat více než 25 témat Téma musí začínat písmenem nebo číslem, může obsahovat pomlčky („-“) a může být dlouhé až 35 znaků.

notes.txt 17KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397
  1. CONTENTS:
  2. 1) TODO List
  3. 2) OSX Install Notes
  4. SPAT LAB SETUP:
  5. + 1. Stereo
  6. a. TotalMix - select 'diff. stereo' snapshot
  7. b. mixer: Recall preset 91 ('stereo diff')
  8. + 2. Direct - 24 channels - each channel goes in and out on same number.
  9. a. TotalMix - select 'DTS_24ch'
  10. b. mixer: Recall preset 96 ('24ch firecac ufx')
  11. STRATEGY:
  12. + implement highly parallel version - which can take advantage of multiple processors
  13. to do more - acoustic pitch tracking, source separation based fades
  14. TODO:
  15. + When a cmMsgList item is selected it should send out the selected index
  16. after the individual data items rather than before them - this way the index
  17. can be used as a definitive signal to perform some action on the previously
  18. transmitted data items.
  19. + cmJson and cmLex should report the name of the file when with syntax error messages.
  20. + Use CSV for mod score file format (to eliminate need for labels)
  21. + Autoload of default pgm cfg, program, enable audio, sequence, score location.
  22. + (done) Circuit switcher patch.
  23. + Review and document the app threading and locking during audio file loading.
  24. + Remove old performance evaluation code from cmScore.
  25. + All programs should be able to reload without crashing via enabling/disabling audio.
  26. + (in new version) Select and save audio devices.
  27. + (done) Mac Conversion
  28. + (done) Live MIDI - to test audio/MIDI delay.
  29. + (done) As threshold increases overall volume decreases - add a compensation algorithm.
  30. + There are unintialized variable warnings on the release build.
  31. + Add preset delete.
  32. + All uses of va_copy() should have a complimentary va_end()
  33. + The list boxes do not show the currently selected item.
  34. + (done)MIDI note messages are sent but do not trigger notes on OSX.
  35. *** Usage Notes
  36. live - Disconnects WT cmd input (WT will not receive an 'on' msg)
  37. Disconnects TL reset input (TL will not receive a 'reset' msg)
  38. Disconnects MFP sel input (MPF will not receive an 'on' msg)
  39. Switches audio input to KR from WT to AIN.
  40. Turn the 'meas' checkbox 'on'.
  41. simulate - Turn the 'audio in' checkbox 'on'.
  42. Switch MFP output from SF to Nano.
  43. (SF MIDI input then comes from the MIDI port.)
  44. audio in - Switch audio input to KR from WT to AIN.
  45. meas - Instruct the SF to generate measurement outputs for the active meas. unit.
  46. Otherwise the measurements must be loaded from the recorded measurment list.
  47. print - Print a report from the SF.
  48. quiet - Turn off SF output.
  49. LA Secs - Fragment recd/play unit look-ahead time in seconds.
  50. Fade DbpSec - Framgent recd/play unit fade out time
  51. (time to fade to 0 following a fade msg)
  52. *** Cross-fade Notes
  53. Cross-fades are initiated by sending any msg to the 'AvailCh.trig' input.
  54. The 'AvailCh' object then toggles the parameter router channels and xfader gates.
  55. Only after this should the new parameter values be transmitted either from
  56. the 'ActiveMeas' object (through the scale range chains) or directly from the
  57. 'ScMod' object.
  58. Sending parameter values prior to triggering the 'AvailCh' will result in the
  59. parameters being sent to the currently active 'Kr'. This will result in two
  60. possibly unintended effects:
  61. 1) The effect of the parameter will be heard immediately - possibly resulting in distortion.
  62. 2) If a subsequent trigger is sent to 'AvailCh' the parameters will be routed to
  63. the fade-out (current) channel rather than the fade-in (next) channel.
  64. **** Live Test Score
  65. + Line 1048 has a red G#5 immediiately following another G#5. Is this correct?
  66. For now the second G#5 has been marked as a 'skip'.
  67. + Changed Tempo sections 25,26,27 to 40,47,47a
  68. + Measurements are taken for sections 51-54 but these sections follow bar 136 and
  69. are therefore outside the test. These sections have therefore been redirected
  70. to the downbeat of 201-204.
  71. *** Testing Notes:
  72. + Equipment List:
  73. Four Microphones: Four performance/ Four recording
  74. 2 inside 2 inside
  75. 2 overhead 2 overhead
  76. 4 powered speakers
  77. 2 Mixers (1 performance 1 recording)
  78. Performance Computer (harpo)/ Audio Interface (delta1010) / MIDI interface (Fastlane)
  79. Recording Computer (crel) / Audio Interface (delta 1010) / MIDI interface (???)
  80. Sensor Strip
  81. + Performance Setup
  82. +------ +
  83. Mic0 ----------->| |
  84. Mic1 ----------->| | sends +-----+ +-------+
  85. Mic2 ----------->| |------->| A/D | | | +------+ +--------+
  86. Mic3 ----------->| Mixer |------->| |------>| harpo |<-------| MIDI |<-----| sensor |
  87. aux | | | | | | +------+ | strip |
  88. Spkr0 <----------| |<-------| |<------| | +--------+
  89. Spkr1 <----------| |<-------| | +------ +
  90. main | | | |
  91. Spkr2 <----------| |<-------| D/A |
  92. Spkr3 <----------| |<-------| |
  93. +-------+ +-----+
  94. + Software Development
  95. - Create Score File
  96. - Create Recording Program (test with long MIDI playback generating audio - look for drift)
  97. Record the index of each MIDI event at it's location in an audio channel.
  98. - Allow all variables and patch connections to be set from the scMod script and have
  99. multiple scripts with varynig effects setups.
  100. - When scanning past ramping variables in scMod the end value should be
  101. taken as the next variable(???) - this is not necessarily correct
  102. because one never knows where a timed change may end - maybe ramped
  103. variables should include a 'skip value' giving the next ambient value
  104. for the ramped variable - experiment with this to figure out what works.
  105. - Add comb filters tuned according to the current MIDI notes as an additional effect.
  106. - Add EQ output stage (use mixer).
  107. - Add an input Compressor.
  108. - The dry signal should be able to be routed to seperate output channels - around the compressor.
  109. (Better would be to output a delayed version of the dry signal that was in sync with the transformed
  110. signal - this might mean simply passing the dry version as separate outputs from KR).
  111. - Create a mode in scMod which increments values based on an onset detector. So that
  112. changes only happen on attacks. This still doesn't help if the pedal is down (or
  113. if notes are sustained) but otherwise might be a better way to ramp parameters.
  114. - (DONE) The ability for the measurements to be called at the correct time must be built in.
  115. (or alternatively to use stored effects).
  116. - (DONE) Effects applied to the playback fragments.
  117. - (DONE?) CROSSFADE BUG - This may be fixed by the change to cmDspAvailCh which
  118. handles the case where no channels are available by sending an error message
  119. but not actually changing the state of the cross fader.
  120. + Experiments:
  121. 1) Speaker placement and live/electronics mix.
  122. 2) Sliver mix level
  123. 3) Try varying degree's of effects
  124. *** 11/19
  125. + The recd/play fade should be able to trigger from a capture note as well as
  126. playback note. An offset might also be useful.
  127. (Should be a default fade for each fragment - keyed to the input. This
  128. will be the fade that will occur when
  129. + Allow setting fade time in the score.
  130. + Allow setting fade rate based on 1.0 to 0.0 from fade point to end point.
  131. + Allow for multiple fades markers per fragment. (what does this mean?)
  132. + Write code to ignore playback when the score follower is not stable -
  133. or to throw out fragments where there is a mistake.
  134. + At the end of each fragment recording the fragment should be truncated by the look-ahead time
  135. to avoid capturing the attack of the marked note.
  136. + Part 2 data analysis: analyze the order of notes in counter rhythms.
  137. + Allow 'evenness' sequences to have non-even relationships.
  138. *** 11/1
  139. + Change the wavetable to read stereo files or add a second wavetable to play the other channel.
  140. + The console window is not always updating from the bottom.
  141. + The 'Dump' button results are not going to the console window.
  142. + Put dry signal into separate output channels.
  143. + Add 'adaptive' mode parameters to scale/range mappings.
  144. + The 'meas'->'parameter' mappings should changable from scMod (mod0.js)
  145. + The 'adaptive' mode parameters (e.g. offset and invert) need to be connected in the patch.
  146. + Create a mode in scMod which increments values based on an onset detector. So that
  147. changes only happen on attacks. This still doesn't help if the pedal is down (or
  148. if notes are sustained) but otherwise might be a better way to ramp parameters.
  149. + Mark all notes in the score according to how well they would act as places to transition.
  150. Notes held while the pedal is down would not be good places to transition.
  151. These indicators would then be used to determine where a section change can occur
  152. when the actual section change is missed.
  153. + (done) All score_loc's and event indexes in meas0.js that beginning with
  154. score location 743 must be decremented by two. (e.g. loc 743 becomes 741 ...)
  155. (score_loc_1.txt is now the correct score file)
  156. + When scanning past ramping variables in scMod the end value should be
  157. taken as the next variable(???) - this is not necessarily correct
  158. because one never knows where a timed change may end - maybe ramped
  159. variables should include a 'skip value' giving the next ambient value
  160. for the ramped variable - experiment with this to figure out what works.
  161. *** 10/17
  162. Select bar 129.
  163. Start on F5 before 129.
  164. Score follower jumps to loc. 978 then backs up to 973.
  165. *** 9/27
  166. * (DONE) Implement live recording for use in part 2.
  167. * Implement a delay between when a new section is set to trigger and when
  168. it actually does. This might allow transitions to be set up prior to when they are heard.
  169. * (DONE) OS-X version crashes when the printf("PROCSET ...) is removed from _cmScProcSets().
  170. * Add ability to set mappings and perf. measure settings to scMod.
  171. * The scMod should play through all changes up to the cur starting location so that
  172. we can mimic the state of playing the piece through but allow starting from any location.
  173. * Experiment with changing settings using the scMod ramping functions.
  174. * Make a source separation based fade using an filter/inverse filter based on the
  175. spectrum prior to the cross-fade. As an extra feature notice the state of the pedal
  176. and decay appropriately.
  177. * The electronic score needs the ability to specify that the output is sent to the audible
  178. cross-fade channel rather than the inactive channel.
  179. * Add a score mode that performs some action (e.g. incr/decr) on each incoming score follower
  180. event.
  181. * In AvailCh what happens when no available channels are found - this may be the cause
  182. of the cross-fade cut-out problem.
  183. * Add eq stage to output.
  184. * Setting the upper slope to a negative value is effective.
  185. * Demo Material: Seq 8 Mark 204 (1024-4 2048-4) Fade 10ms Cost->Threshold.
  186. - recd7 Mark 151
  187. - recd8 Mark 145 - same settings a previous take.
  188. - (recd9) Mark 145 - w/ changing xfade switch to mode 4 at 38
  189. - recd10 Mark 145 - same w/ no mode change
  190. - recd11 Mark 145 - a. 3 different takes w/ score constant - no perf. parameters used - fixed threshold=65
  191. - recd12 Mark 148 b.(same settings as recd9)
  192. - recd13 Mark 151 c.
  193. - recd14 Mark 161 Section 40 - fixed changes see mod0.js for note
  194. - Seq 6 Mark 145 M-92 (38,39,39a) (recd16) (recd17) (recd18) (recd19)
  195. - Seq 7 Mark 167 M-100
  196. *** 8/13
  197. * Cross-fade was cutting out during demo.
  198. * Missing takes between seq 7 and seq 8.
  199. * The MIDI is mis-aligned against the audio.
  200. * Are cost / tempo working? ... test changing mode.
  201. * Add an automatic volume adjustment to prevent parameter changes from causing large volume changes.
  202. * What can we actually do between after a MIDI note is received? Is it already too late to
  203. send parameters w/o affecting the attack of the note.
  204. * Live Test material
  205. Part 1: Meas:76 - (one measure) to get measurments for later sections
  206. Part 1: Meas:94 (sect 38 (Seq 6)) through Meas:136 (include 136 stop at 137).
  207. Part 2: ("/Users/kevin/temp/piano score part 2 draft 1 master m 232 - 241 1st 2 bts.sib")
  208. The downbeat of part two aligns with the 3rd beat (in 4/4) of 122 in the first part.
  209. Ends on measure 131 at end of beat 3.
  210. MISSING MIDI for measures 114:126
  211. *** July 10
  212. ** (done) Make separate mappings and scale/range controls for left and right.
  213. ** Add EQ output stage.
  214. ** Work out the speaker setup.
  215. ** Add capture/playback. Analyze notes on capture and do not play if there are
  216. any wrong notes. (Skipped notes however are acceptable.)
  217. **
  218. ** When a section transition is occurs late - (e.g. due to dropped notes) do not apply
  219. the transformation all at once - instead either ramp it in or step it in on subsequent
  220. attacks. Section transitions which are positively identified are intended to have
  221. dramatic changes so applying the updated parameters immediately is acceptable - but
  222. when the parameters are changed mid-section they should be much applied subtely.
  223. ** (done) Build a database of measurements and setup the program to be able to
  224. apply a given measurement at it's assigned section.
  225. ** Redevelop spectral distortion algorithm to use a spline as the transform.
  226. ** use Log frequency frequency transform instead of FFT.
  227. ** Allow for a continous window size via zero padding.
  228. ** Add a 'write' preset file button - so the preset file can be saved prior to next crash.
  229. ** measurement values can generate MAX_DBL - be sure that are not being sent through to the audio algorithm.
  230. (see ln:965 cmDspKr.c for a hack to fix this)
  231. ** add invert to scale/range to cause output to go in opposite direction.
  232. ** non-grace eveneess are used to generate a measurement value from previous tempo calibration section.
  233. (non-grace evenness notes therefore have two scores: 'evenness' and 'overall-duration').
  234. ** note that the default setting for dyn and even.
  235. ** add dyn,even,tempo,cost number boxes to allow artificial setting of these parameters.
  236. add dyn,even,tempo,cost as modulator variables to allow them to be set from the modulator
  237. With these additions we can simulate apply measurements at the 'application' section.
  238. ** add begining and ending measure numbers to 'seq' labels
  239. ** Gain compensation for mode 4.
  240. ** Do IFFT using cos()^c + i * sin()^c - these bases are orthognal but cause harmonic distortion.
  241. To be efficient this might involve writing an FFT function.
  242. // May 22
  243. ** Crashers (Should be tested but are probably already fixed following score follower debugging.)
  244. Seq 2 m24 Mark 36 & 38
  245. Seq 4: Mark 115
  246. Seq 4: m76-79 Mark 129 (W/ meas: even & dyn -> thresh change min Thresh to 40) - use 4th seq w/ b1
  247. Seq 6: m92 Mark 143
  248. Seq 7: m103 Mark 173
  249. Seq 7: Mark 172 Meas 103 - always crashes on playback.
  250. Seq 4: Mark 76 Meas 40 - crash!
  251. Seq 2: First mark meas 23
  252. Crash seems to happen in cmProc4.c: _cmScMatchInitMtx() ln:1311.
  253. It looks like a memory overrun. Looks like the first line is wrong
  254. shouldn't:
  255. if( rn >p->mrn && cn > p->mcn )
  256. be
  257. if(rn*cn > p->mrn*p->mcn)
  258. BUGS:
  259. // Apr 20
  260. The tempo measurement can produce invalid values. Set score to 22 then play Mark 38.
  261. First tempo measurement is a non-sense value - probably produced by an div by zero.
  262. Also: Mark 8, Meas 10.
  263. Crash on playing Mark 37.
  264. Click on list control outside below list item - crash!
  265. Select Mark 171 (Seq 7) Section 43, m103 crash
  266. // Feb 27
  267. + Audio seems to preceded MIDI by around 250ms this probably arises
  268. from a delay that was inserted by 'mas'. Can the delay be removed?
  269. // Feb 25
  270. + Fix the audio file input/output ports
  271. //
  272. // Feb 6 & 7
  273. //
  274. + Performed notes which arrive which about 50ms could be
  275. considered chords. Extra notes notes which were not part of
  276. the chord are probably common and should be discounted during
  277. the cost analysis.
  278. + (done) Add alignment cost as a 4th variable along with dyn,even,tempo.
  279. + (done) In the score print out (score_loc.txt) Section 2 is starting on Bar 5
  280. when it should start inside Bar 7.
  281. + In Take 1 the 2nd dynamics set is not triggered.
  282. + In Take 3 Eveness 2 the C2 and F#4 are NOT missing althrough they are in the
  283. evalation. Also E7 which ends that set is not marked with an 'e'.
  284. + It is possible to have even-non-grace sets where the note rythm value's are not
  285. all the same (e.g. bar 20 )
  286. + User soft-thresholds for the dynamics categories.
  287. + Set 39 even measure 25 shows the first note as G#2 when it should be C#1
  288. + Missig MIDI note sounded:
  289. C1 score-loc:132
  290. A#2 140
  291. A5 173
  292. C#2 195 Marker 36
  293. E5,G33,A#2,C#1 212-218 Marker 36
  294. E3 185 Marker 37
  295. F5,C#4,G#5 986 Marker 204
  296. -------------------------------------------------------------------------------
  297. OSX - Install Notes
  298. -------------------------------------------------------------------------------
  299. 1. Install macports
  300. 2. Install git-core (sudo port install git-core)
  301. 3. sudo port install fftw-3
  302. 4. sudo port install fftw-3--single
  303. 5. port select --list gcc (which gcc is active)
  304. 6. sudo port install gcc47
  305. 7. sudo port --set gcc mp-gcc47
  306. 8. sudo port install fltk
  307. 9. sudo port install xorg-libX11
  308. 10.sudo port install git-core
  309. 11.sudo port install emacs +x11
  310. 10 install ~/Library/Preferences/org.larke.kc.txt (is this required?)
  311. 11 create ~/Library/Preferences/kc
  312. 12 Install ~/Library/Preferences/time_line.js, time_line_preset.js, time_line_preset.csv