A dll to know when there is a beat in a music ?
Blitz3D Forums/Blitz3D Beginners Area/A dll to know when there is a beat in a music ?
| ||
| Hi, :) For one of my games, there is a strobe light, and i want the strobe light to be shown or to be hidden depending on the kind of sound emitted by the sound card. Basically i want to show the light when there is a beat and hide it otherwise. What i plan to do is to code a routine that analyzes the sounds of a music and that stores in an array the time (millisecs) when there is a beat so that the program doesn't have to analyze the sounds in realtime but know when the beats happen. Do you know if there is a dll or software able to analyze the sounds of a music and to track when there is a beat ? Last edited 2012 |
| ||
| Look for "BlitzBass Studio" in the toolbox here on this site. BBS does require a license to use, though, for commercial projects, so be aware of this. |
| ||
| Thanks for the suggestion Adam, i will take a look at Blitz Bass Studio. Do you think it is possible to analyze a sound with the windows functions ? |
| ||
| Hey RemCoder, I'm really not sure anything exists currently to do what you're asking for - I was looking for something similar myself. I'm sure there must be a solution around, since there's a few (non-Blitz I should mention) programs that analyse sounds in such a way. However, I've never come across anything that was readily avbailable/usable by Blitz :( As for Windows, well, even the simpple Windows Sound Recoreder (SndRec.exe) can analyse a simple waveform of a sound, but whether this is an accessable feature,. or just a bult-in part of the recorder, I don't know... In the end, I decided to abandon the search and try to code something myself. It's a difficult process, and, I've not gotten anything finished yet - Thoiough I have a plan of action, which may be of help? Basically, I need to go through the following steps. Personally, I'm starting with MP3 files but they are really complicated, so I would recommend maybe starting with something like an uncompressed WAV file if possible. 1) Determine which file type (encoder used) you will work with. Since the format of the code within each will vary greatly depending on this type. 2) Look up around the web (wikipedia is actually pretty handy for this) to obtain the HEADER aqnd file-structure details for that particular type. You will need this information to be able to decode the actual data in the file 3) From the Header and structure code, you should hopefully have aqn idea of the following aspect of the particular file: a) SampleRate (Sound Resolution) b) BitRate (Data Resolution) c) Duration d) Channels (Mono/Stereo) e) Depth (8-bit, 16-bit etc.) 4) There's no way to 'analyse' the particular sound without the above values. some encoding formats may vary the bitrate too (mp3 for example) meaning you need to locate the bitrate for each 'Audio Frame'... In theory, once you have these values, you should then be able to take the block of code that represents the sound itself, and identify the sequences of bytes that make it up. Beware here, some formats use a Little-Endian and others may use Big-Endian words for groups of bytes. This means that if, say, the sound is 16-Bit depth, then it refers to a short integer, or 2 x 8-bit bytes which can represent a value from 0-65535 LittleEndian will mean the first byte will represent (0 to 255) and the second byte will represent ((0 to 255)*256) The LEAST Important Byte is the First Byte. LittleEndian will mean the first byte will represent ((0 to 255)*256) and the second byte will represent (0 to 255) The MOST Important Byte is the First Byte. 5.) Check on Wikipedia or somewhere again, and you should be able to ascertain the format of the byte structure. It may be that any stereo channels are processed 'simultaneously', i.e.: BYTE1: 1st Byte LEFT CHANNEL BYTE2: 2nd Byte LEFT CHANNEL BYTE3: 1st Byte RIGHT CHANNEL BYTE4: 2nd Byte RIGHT CHANNEL or, a more confusing structure such as: BYTE1: 1st Byte LEFT CHANNEL BYTE2: 1st Byte RIGHT CHANNEL BYTE3: 2nd Byte LEFT CHANNEL BYTE4: 2nd Byte RIGHT CHANNEL Either way, you'll probably want to extract the bytes into an array or Type that can separate the LEft channel from Right channel bytes in the correct order. 6.) Next is to deal with the bitrate and sample rate. The bitrate is generally the 'speed' of the data when converted into sound. If the bitrate is, say, 128kbps (kilo-BITS per second) then around 131072 bits make one second. That's 16384 8-bit bytes or, if the depth is 16-bits like the example above, then it's 8192 short-integer 16-bit-bytes. 7) Now we know how much data there is per second, we need to know how much of that data is used to represent the amplitude of the sound at what frequency. The Sample rate is suually written as 44.1 or 22.050 etc. this is in kHz. Meaning 44.1 (which is typical sampling rate for CDs) per second. This frequency is the actual range of frequencies that can be represented by the data. 8) So wat does it mean for our 16384 Bytes to represent 441 00 different frequencies? Well, instead of looking at the whole 'one second' for a moment, let's consider the smallest amount of data we can : Since we're using 16-bit bytes for this example, our smallest data 'block' is 2 8-bit-bytes. 16 bits in total. This single 2-byte-short-integer can store any value from 0 to 65535. There are way more than 44100 values! (Actually, some bits of each byte may be used for other purposes - again, check wikipoedia or similar for details!) 9) We should now be able to convert each byte into a value representing the proportionate frequency. Once we have done this for every group of 8192 bytes (or 16384 perhaps for stereo) then we have groups of sound frequencies for each second! 10) At this point, the dataq may best be viewed or analysed by plotting it on a graph, or some other means of identifying particularly low-frequency sounds (i.e. representing a bass drum beat for instance) 11) You could then ;calculate the tempo according to the repetition of patterns discovered in the sounds by the time intervals at which they occur. PHEW! So you can see, it's quite a big task - You can understand why I'm stiulll not finished with trying to get it going myself - though I am not about to give up any time soon! If you have enough patience to wait for me - which could be months to be honest - then I have no problem sharing the code - otherwise, this is the best I can offer right now. |
| ||
| Hi _PJ_, :) Interesting but it seems to be really complicated and time consuming to do. I have found another possible way to track the beats in a music : To track the millisecs when playing the music and to press a key manually when a beat occurs. The program creates a file in realtime where for each millisec, either a normal state or a beat state is saved. Then when playing the sound during the game, the program reads the file to know when there is a beat or not. I know this is not a programmer way, but for a game it only need to be perceived as a good effect not a perfect effect. Last edited 2012 |
| ||
| Couldn't you monitor the sound, and detect any sharp volume increases? |
| ||
| I'm glad you have an easy way that works, RemCoder. Ross, even though it would be a whole lot simpler than the method mentioned above (which I must admit was based mainly on the ability to identify the complete waveform of the entire sound rather than JUST the beats) - 'Monitoring' the sound would still need some kinda DLL or ability to read the sound as it's playing - something which seems to be sorely absent from anything I've seen so far regarding Blitz! It's good to see that there's some workarounds and such for dealiong with more "esoteric" sound functionality within B3D though :) |
| ||
| True! Stupid suggestion really, if there's no blitz way of doing it. |
| ||
| Ross>> You give me an idea, thanks :) : Maybe i can use a strobe light when playing the music in a pitch black room and record a video of the wall of the room. With this i will have two kinds of images, either a black image or a bright image. Then i can analyse each image of the video to determine when there is a beat or not. By analyzing only a few pixels of each image, it should be fast enough to analyze all the frames. The limitation is that i have a camera which can only records at 30fps so i don't know if it will be really precise. 30fps <> 1000 millisecs ! Oh this gives me another idea ! Maybe i can use a usb port to receive a signal from the strobe light and to record it depending on the millisecs passed. (easy to say, but i don't really have the electronic skills, maybe i can outsource this task) |
| ||
| Any new way to achieve this ? I suppose that the best way would be to analyze the file... But how ? Also is there a way to detect the "volume" of the low pitch melody and of the high pitch melody ? Thanks, |
| ||
| I suppose that the best way would be to analyze the file... But how ? if you have the raw audio data you may try some "low pass filter algorithm" play somewhat with the smooth factor. i did a small search an find this article, i do not read it, simply the image on it is very clear about the results of a low pass filter on audio data. https://kiritchatterjee.wordpress.com/2014/11/10/a-simple-digital-low-pass-filter-in-c/ low pass filters may range from simple averaging samples to more complicated algos (look for kalman filter for example) https://en.wikipedia.org/wiki/Kalman_filter [edit] lookin forward... perhaps this software is usabel for you http://vamp-plugins.org/sonic-annotator/ https://code.soundsoftware.ac.uk/projects/sonic-annotator/wiki don't know how hard/help it be! [/edit] |
| ||
| @Charrua>>Thanks for the links, but a little too complicated for me. I have found this : https://www.youtube.com/watch?v=jZoQ1S73Bac and also i may have found a way to detect the beats in a music track by using a combination of a windows media player visualization plugin + an audio/video capture software + a custom procedure in Blitz3d. Not tried but i think it can work. The idea is to detect when there is a beat by capturing a small area of the visualization effects (vertical bars) and if the color is black there is no beat, and if the color is green, there is a beat. An ugly macgyver workaround i have to admit :P |
| ||
| Some time ago I tried to draw the wave audio from a .wav file with B3D, it worked but I didn't manage to sync it with the audio. I may give it a try with MP3. |
| ||
| With the method i describe, there should be no synchronization problem... But there may be a synchronization problem when the fps drops too low compared to the music which is playing, so my idea is to have a variable for each 100ms step to know if it corresponds to a beat or not. But i am not sure if Blitz3d has a function to get the current time of a music/sound in milliseconds... |
| ||
| using bass.dll you should be able to analyze the spectrum. if you just check the low frequency, you'll get what you need. It's not a hard task and has been used for tons of spectrum vizualizer. I really think you can get it working easily But i am not sure if Blitz3d has a function to get the current time of a music/sound in milliseconds... There is nothing that can do that with the natives blitz commands. But all you need to do is start a timer when you start the music. Then just check the current time (MilliSecs()) and substract to the start time. You'll get the current track time. |
| ||
| But all you need to do is start a timer when you start the music. Then just check the current time (MilliSecs()) and substract to the start time. You'll get the current track time. good idea, thanks. I will take a look at the bass.dll, second time it is mentioned to help me with that. |
| ||
| Here's something to access samples value from a .wav file : http://www.blitzbasic.com/codearcs/codearcs.php?code=3287 Only works with 16bits stereo PCM .wav for the moment. |
| ||
| finally, i have chosen another weak but good enough approach : i have coded a system where i can play the music and track the mstime and i can press/hold/release some keys depending on the different kinds of sounds played in a music (beat, clap, bass, melody, voice) and depending on my input this will store these input states in arrays (using the mstime), and then when i play the music, i can use access these input states (using the mstime) to trigger specific graphics effects associated to the specific sounds played... the advantage is that i can have different graphics effects triggered depending on different kinds of sounds (not only beats) macgyver style, i know :P |
   |