Forums > Technical Discussion > MIDI Sequenced POI - Multimedia POI Theater
Login/Join to Participatemember
3 posts
Location: Ft. Lauderdale
3 posts
Location: Ft. Lauderdale
Posted:
I am just starting research on a new art project I would like to bring to this years burn. My basic idea is this: A multimedia MIDI sequenced POI theater. Fully interactive for all the POI heads out there. Ultimately I would like to have a dome set up with projectors
Illuminating the outer surface of the dome. The performance space will be outlined inside the dome with cameras rolling real time video effects controlled by the midi data generated by 3D sensors on the ends of the POI. Chroma key, Kaleidoscopic, crazed Avis..all sequenced by the movement of the POI. Setup an audio track with a consistent tribal bass line. The Midi data from the POI playing samples over the bass line.. melody like.. ala a sampler. I have allot of help in the local area for building the dome and A/V supplies. I could use some help building the Midi interface... This is
what I got so far:
As for the MIDI POI I see three options for POI motion capture. (Video to MIDI) (3D serial data to MIDI) (Flex to MIDI)
One. (Video to MIDI) Use three infrared cameras, one for each axes XYX, and use
some type of capture card and real time video analysis. This option world be
kind of cool in that you could use infrared glow sticks for the motion
capture. Which would make the thing really interactive. You don't need to be
good with POI to get it to do something. My boss was a Russian engineer one
day long ago. He helped design some missile guidance systems that had allot
of frame / pixel analysis. He would be able to help me design the motion
capture side of things. I just don't know too much about programming
(besides BASIC) so new ground for me. Pros: Highly interactive, Help from
the Russian, cost effective sorta (Surplus IR Cameras) Cons: I'm not big on
programming so boocoo time added on the development side (probably slow
development)
Two. (3D XYZ to MIDI) Premade 3D motion capture systems ala
https://polhemus.com/
ISOTRAK II or Ascension Technology
https://www.ascension-tech.com/
"Flock of Birds" (FOB). Both are fairly
expensive systems in the 2-3k range. The FOB is a more robust system for
real time 3D
https://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/ai-repository/ai/html/faqs/ai
/robotics/part4/faq.html I got some good feedback from Polhemus. They mite
be willing to sell me a refurbished system on the cheep or work out a
sponshership deal. Both devices output XYZ data in serial standard RS-232c
or RS-422/485. Eater way converting that positional data Via a Atom
Microcontroler should pose little difficulty. Pros: Easy interface between
Serial & MIDI Via a Microcontroler I already have (fast development) , Real
time low latency 3D data, Cons: Expensive
Three. (Flex to MIDI) Using multiple flex sensors at 90 deg to each other
attached to the POI handholds. I just came up with this idea today so bear
with me. I need to do some research into exactly how do the POI move in
relation to hands. But I believe I can put an prototype with the hardware I
already have. https://www.imagesco.com
sells cheap 10$ flex sensors. I am
going to wait till Monday and talk with the sales guy about the robustness /
if they can be flexed in ways that would make the sensor break. Ill pick up
a couple of these next week if they can meet the needs of the project. Pros:
Super CHEEP, Use the Atom microcontroler I already have to convert the Flex
data to MIDI Cons: Resolution? I mean how well will these track? unknowns
abound... but this mite be THE keep is simple solution. Ill know more on
Monday.
My project bucket is officially full.
Any Ideas / Comments are greatly appreciated
I am just starting research on a new art project I would like to bring to this years burn. My basic idea is this: A multimedia MIDI sequenced POI theater. Fully interactive for all the POI heads out there. Ultimately I would like to have a dome set up with projectors
Illuminating the outer surface of the dome. The performance space will be outlined inside the dome with cameras rolling real time video effects controlled by the midi data generated by 3D sensors on the ends of the POI. Chroma key, Kaleidoscopic, crazed Avis..all sequenced by the movement of the POI. Setup an audio track with a consistent tribal bass line. The Midi data from the POI playing samples over the bass line.. melody like.. ala a sampler. I have allot of help in the local area for building the dome and A/V supplies. I could use some help building the Midi interface... This is
what I got so far:
As for the MIDI POI I see three options for POI motion capture. (Video to MIDI) (3D serial data to MIDI) (Flex to MIDI)
One. (Video to MIDI) Use three infrared cameras, one for each axes XYX, and use
some type of capture card and real time video analysis. This option world be
kind of cool in that you could use infrared glow sticks for the motion
capture. Which would make the thing really interactive. You don't need to be
good with POI to get it to do something. My boss was a Russian engineer one
day long ago. He helped design some missile guidance systems that had allot
of frame / pixel analysis. He would be able to help me design the motion
capture side of things. I just don't know too much about programming
(besides BASIC) so new ground for me. Pros: Highly interactive, Help from
the Russian, cost effective sorta (Surplus IR Cameras) Cons: I'm not big on
programming so boocoo time added on the development side (probably slow
development)
Two. (3D XYZ to MIDI) Premade 3D motion capture systems ala
https://polhemus.com/
ISOTRAK II or Ascension Technology
https://www.ascension-tech.com/
"Flock of Birds" (FOB). Both are fairly
expensive systems in the 2-3k range. The FOB is a more robust system for
real time 3D
https://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/ai-repository/ai/html/faqs/ai
/robotics/part4/faq.html I got some good feedback from Polhemus. They mite
be willing to sell me a refurbished system on the cheep or work out a
sponshership deal. Both devices output XYZ data in serial standard RS-232c
or RS-422/485. Eater way converting that positional data Via a Atom
Microcontroler should pose little difficulty. Pros: Easy interface between
Serial & MIDI Via a Microcontroler I already have (fast development) , Real
time low latency 3D data, Cons: Expensive
Three. (Flex to MIDI) Using multiple flex sensors at 90 deg to each other
attached to the POI handholds. I just came up with this idea today so bear
with me. I need to do some research into exactly how do the POI move in
relation to hands. But I believe I can put an prototype with the hardware I
already have. https://www.imagesco.com
sells cheap 10$ flex sensors. I am
going to wait till Monday and talk with the sales guy about the robustness /
if they can be flexed in ways that would make the sensor break. Ill pick up
a couple of these next week if they can meet the needs of the project. Pros:
Super CHEEP, Use the Atom microcontroler I already have to convert the Flex
data to MIDI Cons: Resolution? I mean how well will these track? unknowns
abound... but this mite be THE keep is simple solution. Ill know more on
Monday.
My project bucket is officially full.
Any Ideas / Comments are greatly appreciated
I Love Cheese
GOLD Member
Pooh-Bah
2,109 posts
Location: Philadelphia, PA - USA
Pooh-Bah
2,109 posts
Location: Philadelphia, PA - USA
Posted:
Sounds like a cool idea but unfortunately I dont uderstand most of what you said hehe. Only thing I got from it is that you want to basically make music with poi using motion capture type technology. Seems like it would be a LOT of work for just a little novelty... But then again, I know nothing about that sort of stuff so maybe Im wrong.
Sounds like a cool idea but unfortunately I dont uderstand most of what you said hehe. Only thing I got from it is that you want to basically make music with poi using motion capture type technology. Seems like it would be a LOT of work for just a little novelty... But then again, I know nothing about that sort of stuff so maybe Im wrong.
Let's turn those old bridges we crossed into ashes.
We'll blaze a new trail,
and torch the rough patches.
-Me
PLATINUM Member
Hyperloops suck
595 posts
Location: USA
Hyperloops suck
595 posts
Location: USA
Posted:
If your interpretation of that technical jargon I didn't fully understand is correct, in that the idea is to make music from poi spinning patterns (Which is what I gathered as well) I believe it to be much more than a novelty. Similar projects have been done with numerous "instruments" being the body. I saw on TV an entire exhibit filled with such instruments... you'd put your hands between two metallic bars and do different movements and it'd play different notes. Others would envolve you standing in a room, which is much closer to what you're talking about, and pretty much anything you did triggered sounds. I'm interested in how serious you are about this... it's one thing to work out the details, but it's something completely different to actually start building and writing the software.
If your interpretation of that technical jargon I didn't fully understand is correct, in that the idea is to make music from poi spinning patterns (Which is what I gathered as well) I believe it to be much more than a novelty. Similar projects have been done with numerous "instruments" being the body. I saw on TV an entire exhibit filled with such instruments... you'd put your hands between two metallic bars and do different movements and it'd play different notes. Others would envolve you standing in a room, which is much closer to what you're talking about, and pretty much anything you did triggered sounds. I'm interested in how serious you are about this... it's one thing to work out the details, but it's something completely different to actually start building and writing the software.
GOLD Member
Pooh-Bah
2,109 posts
Location: Philadelphia, PA - USA
Pooh-Bah
2,109 posts
Location: Philadelphia, PA - USA
Posted:
When I said novelty, I just meant that it has no real practical use. Sure, it may be neat to try that out but it wont be any kind of revolution in music or poi. Its also too big and complex to be sold to the average consumer.
When I said novelty, I just meant that it has no real practical use. Sure, it may be neat to try that out but it wont be any kind of revolution in music or poi. Its also too big and complex to be sold to the average consumer.
Let's turn those old bridges we crossed into ashes.
We'll blaze a new trail,
and torch the rough patches.
-Me
PLATINUM Member
Hyperloops suck
595 posts
Location: USA
Hyperloops suck
595 posts
Location: USA
Posted:
Oh, I most definitely disagree - I would consider it a huge revolution. Instead of dancing to the music, the music is conforming to the dance. It'd be an extremely interesting thing to watch as far as a show is concerned, and poi as a musical instrument is quite an intruiging idea in the first place.
Oh, I most definitely disagree - I would consider it a huge revolution. Instead of dancing to the music, the music is conforming to the dance. It'd be an extremely interesting thing to watch as far as a show is concerned, and poi as a musical instrument is quite an intruiging idea in the first place.
Pooh-Bah
2,364 posts
Location: austin,tx
2,364 posts
Location: austin,tx
Posted:
grrrr. now i know i'm just too busy, i was hoping to get the jump on this before anybody else thought of it
oh well.
i've been practicing poi spinning for a while now with midi sequencing/drumming in mind. plus i already have a lot of the engineering done and figured out, actually more functional and simplistic than what i see above... but no funding on a pro dancer's salary. actually this is just one of my truly incredible ideas
if you are really actually serious, and you have funding for this, pm me.
grrrr. now i know i'm just too busy, i was hoping to get the jump on this before anybody else thought of it
oh well.i've been practicing poi spinning for a while now with midi sequencing/drumming in mind. plus i already have a lot of the engineering done and figured out, actually more functional and simplistic than what i see above... but no funding on a pro dancer's salary. actually this is just one of my truly incredible ideas
if you are really actually serious, and you have funding for this, pm me. -Such a price the gods exact for song: to become what we sing
-Seek freedom and become captive of your desires. Seek discipline and find your liberty.
-When the center of the storm does not move, you are in its path.
member
1,591 posts
Location: Nova Scotia, Canada.
1,591 posts
Location: Nova Scotia, Canada.
Posted:
I wish this (and all variations/similar ideas of this, past, present and to come in the futre) would catch on and truely become a revolution. Because honestly we need a revolution in music badly, No one invents new instruments, and the closest thing we have is synth and sample based electronic music, which is really starting to stiffle it's self hardcore with all it's genre nonsense, and the fact that the laziest/simpliest/crapiest electronic music seems to be the most popular.
BTW I thought of something like this, only using an electrical field like that crazy russian instrument that dude invented (you put your hand in the magnetic/radio wave field (I can't remember which) and depending how you position and move your hand it makes freaky noises (used for the beach boys song good vibrations)
I was thinking if you had metal poi heads, it would react strongly to the field so your spinning would be translated to sound. But I figure makeing the instrument large enough so you can stand inside of it, would probably just result in a lot of unpleasant noise.
really I know nothing about it though, which is obvious.
I wish this (and all variations/similar ideas of this, past, present and to come in the futre) would catch on and truely become a revolution. Because honestly we need a revolution in music badly, No one invents new instruments, and the closest thing we have is synth and sample based electronic music, which is really starting to stiffle it's self hardcore with all it's genre nonsense, and the fact that the laziest/simpliest/crapiest electronic music seems to be the most popular.
BTW I thought of something like this, only using an electrical field like that crazy russian instrument that dude invented (you put your hand in the magnetic/radio wave field (I can't remember which) and depending how you position and move your hand it makes freaky noises (used for the beach boys song good vibrations)
I was thinking if you had metal poi heads, it would react strongly to the field so your spinning would be translated to sound. But I figure makeing the instrument large enough so you can stand inside of it, would probably just result in a lot of unpleasant noise.
really I know nothing about it though, which is obvious.
PLATINUM Member
Flying Water Muppet
5,276 posts
Location: Edin-borrow., United Kingdom
Flying Water Muppet
5,276 posts
Location: Edin-borrow., United Kingdom
Posted:
That's sounds like a great idea. Not just the music conforming to the poi, but also the poi creating patterns on the video wall. It would rock! Plus it would be excellent poi practise to get the timing just right so you create a nice steady beat.
Hey runnerr0, if you need any help with the coding, I'm doing an Artificial Intelligence and Computer science degree (third year now) And would love to help. Having fun with moving objects is a big part of AI. (well mostly it's trying to recognise moving objects/faces, plot where their going and the like...)
That's sounds like a great idea. Not just the music conforming to the poi, but also the poi creating patterns on the video wall. It would rock! Plus it would be excellent poi practise to get the timing just right so you create a nice steady beat.
Hey runnerr0, if you need any help with the coding, I'm doing an Artificial Intelligence and Computer science degree (third year now) And would love to help. Having fun with moving objects is a big part of AI. (well mostly it's trying to recognise moving objects/faces, plot where their going and the like...)
"the now legendary" - Kaskade
"the still legendary" - Kaskade
I spunked in my friend's aquarium and the fish ate it. I love all fish. Especially the pink ones. They are my bitches. - Anon.
member
52 posts
Location: England
52 posts
Location: England
Posted:
fascinated
please please please keep me informed
fascinated
please please please keep me informed
member
1,591 posts
Location: Nova Scotia, Canada.
1,591 posts
Location: Nova Scotia, Canada.
Posted:
Does anyone else think this will end up on david lettermen or something if it turns out the way it's envisioned? oh yeah and another thing I thought of is you could put a g force sensor in the poi to determine speed, although it may be redundent, I think it would probably give you a more accurate reading for the speed then motion capture for all angles of faceing the camera/spinning the poi.
Does anyone else think this will end up on david lettermen or something if it turns out the way it's envisioned? oh yeah and another thing I thought of is you could put a g force sensor in the poi to determine speed, although it may be redundent, I think it would probably give you a more accurate reading for the speed then motion capture for all angles of faceing the camera/spinning the poi.
newbie
33 posts
Location: London, UK
33 posts
Location: London, UK
Posted:
I wrote a video to audio thingy ages back.
It basically works by tracking movement in the frame to work out where you are and then using that as an input to a sound generator. Was really fun to play with, kind of like a theremin, except with three axes of control. But it only worked in pretty controlled situations, with a plain unmoving background behind the performer.
The reason this is pretty difficult in the situation you're envisaging is that the lights you're controlling will do screwy things to the motion capture & tracking, even if you're using IR. I'd reckon it'd be easier to do it using some kind of sensors in the poi themselves.
The other thing that's difficult is converting it into a musical form. It's easy to make theremin style whistles and squeaks, but it isn't easy to generate music completely using this sort of thing.
One way would be to use the input as a source to some kind of generative music system, like Koan.
Personally I don't think this is going to create a new type of music as the generation process will inevitably be influenced by the musical tastes of the person programming it.
The other cautionary thing I'd say is that everyone who does something like this thinks that it'll mean that anyone can perform on it. Mr Theremin and the makers of various video->audio conversion things have all thought this. It's always turned out that it actually requires a lot of skill to play the new 'instrument' well, in a similar way to any musical instrument.
Joe
I wrote a video to audio thingy ages back.
It basically works by tracking movement in the frame to work out where you are and then using that as an input to a sound generator. Was really fun to play with, kind of like a theremin, except with three axes of control. But it only worked in pretty controlled situations, with a plain unmoving background behind the performer.
The reason this is pretty difficult in the situation you're envisaging is that the lights you're controlling will do screwy things to the motion capture & tracking, even if you're using IR. I'd reckon it'd be easier to do it using some kind of sensors in the poi themselves.
The other thing that's difficult is converting it into a musical form. It's easy to make theremin style whistles and squeaks, but it isn't easy to generate music completely using this sort of thing.
One way would be to use the input as a source to some kind of generative music system, like Koan.
Personally I don't think this is going to create a new type of music as the generation process will inevitably be influenced by the musical tastes of the person programming it.
The other cautionary thing I'd say is that everyone who does something like this thinks that it'll mean that anyone can perform on it. Mr Theremin and the makers of various video->audio conversion things have all thought this. It's always turned out that it actually requires a lot of skill to play the new 'instrument' well, in a similar way to any musical instrument.
Joe
© 1998 - 2024 Home of Poi
All prices converted to United States dollars using XE.com