Skip to main content

MInority Report like UI on LG3D became reality...

12 replies [Last post]
hideya
Offline
Joined: 2003-07-22

Hello LG3Ders,

It became reality earlier than I expected...

http://www.oliverjones.org/Sun_GoMonkey_demo.avi (~9MB, DivX/PCM)

Enjoy...

hideya

P.S. A big thanks to you, Alan Coopersmith, for the forwarding me the following email!

-------- Original Message --------
Subject: RE: [solarisx86] Cool Project Looking Glass at CeBIT 2006
Date: Mon, 13 Mar 2006 11:18:36 +0100
From: Oliver Jones
Reply-To: solarisx86@yahoogroups.com
To: solarisx86@yahoogroups.com

Thanks for the heads-up, Ivan. :)

I visited the booth and snapped a video of it in action, for those here who
couldn't attend.

http://www.oliverjones.org/Sun_GoMonkey_demo.avi (~9MB, DivX/PCM)

Kindest regards,

Oliver.

________________________________

From: solarisx86@yahoogroups.com [mailto:solarisx86@yahoogroups.com] On
Behalf Of Ivan Wang
Sent: 10 March 2006 17:34
To: solarisx86@yahoogroups.com
Subject: [solarisx86] Cool Project Looking Glass at CeBIT 2006

Hi all,

Too excited to not post this, I just visited Sun booth at CeBIT.
Among all cool stuffs they have there, one of the most eye-catching
demo is a demo called GoMonkey or something. It basically shows an
interactive desktop environment based on Project Looking Glass. The
system interacts with user with some kind of motion capturing
mechanism, so you can move aroung within the Looking Glass desktop
environment simply by waving your hand, including zooming in on a
purticular window, dropping it aside, and so on.

Its coolness factor is 10 of 10 I think :) Though I don't know how
this can translate into real $$ though ;)

The system is powered by an Ultra 40, if you are interested.

Ivan.

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
ekerazha
Offline
Joined: 2006-03-02

> Ha, and I thought it was just cool movie tech.
> One of the commenters on there noted that he had some
> experience using
> the system, and his arms got tired quickly ;-)

Maybe you can put down the elbow and only move the hand ;)

However... the "led" thing seems pretty nice.

u1jd
Offline
Joined: 2005-09-01

Has anyone seen or used one of these gloves before

http://www.vrealities.com/P5.html

It seems to have the possibility, dont know how much work would be involved in making it work with LG3D, its something i would be interested in looking into, has anyone had any further thoughts on this

Mauricio Henriquez

Hey, is not necesary somekind of LED, I have a "home-made" dataglove based
on an old nintendo dataglove (very ship), and the dataglove is conected to
the PC with the serial port, the dataglove have a "acelerometer" to measure
the pith and roll of the hand, and flex sensor for the finger, also you can
add a giroscope like the gyration mouse (http://www.gyration.com/), with the
gyroscope you can read the position of the hand in the XYZ coordinates.

Here http://domotica.pelluco.uach.cl/ you can see in the "Galeria de
Imagenes" section, some pictures of the glove and some pictures of one
domotic aplication and some videos (the 3D scenario is in VB .NET with
opengl)

is this some help?, or ideas??

bye.

Mauricio
----- Original Message -----
From: "Jeff"
To:
Sent: Tuesday, April 25, 2006 1:58 PM
Subject: Re: MInority Report like UI on LG3D became reality...

>I had spent a little time thinking about this actually. Unfortunately, I'm
>always thinking of devices to make it work rather than better
>image-recognition stuff.
>
> My latest idea on it involved making the three-fingered glove you see in
> the movie, and putting infrared (or visible) LEDs on the end for easier
> designation of where your hand is.
>
> Well, I was going to post a screen-cap from the movie with that glove, but
> a little Google search-fu found me this:
> http://www.engadget.com/2005/04/15/raytheons-knockoff-of-the-minority-re...
> Ha, and I thought it was just cool movie tech.
> One of the commenters on there noted that he had some experience using the
> system, and his arms got tired quickly ;-)
>
> Always watching from the shadows-
> Jeff
>
> lg3d@javadesktop.org wrote:
>> I have absolutely no idea... there should be a nice AI algorithm to
>> interpret the moves (ex. if I put the hand ahead, then the hand is seen
>> bigger = the pink area of my hand is bigger).
>>
>> Maybe it could be something like this...
>> [Message sent by forum member 'ekerazha' (ekerazha)]
>>
>> http://forums.java.net/jive/thread.jspa?messageID=107672
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: interest-unsubscribe@lg3d.dev.java.net
>> For additional commands, e-mail: interest-help@lg3d.dev.java.net
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: interest-unsubscribe@lg3d.dev.java.net
> For additional commands, e-mail: interest-help@lg3d.dev.java.net
>

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@lg3d.dev.java.net
For additional commands, e-mail: interest-help@lg3d.dev.java.net

constantin_g
Offline
Joined: 2006-03-24

Hi,

here's a short overview on the GoMonkey system:

GoMonkey is an interactive multimedia system developed by the Austria based company GoMonkey IT GmbH (http://www.gomonkey.at/).

It uses a pair of firewire cameras to recognize a user in 3D space and a defined set of hand gestures that she can use. Using gestures such as "wave left", "wave right", "push", the user can control an interactive multimedia presentation on a big screen.

The GoMonkey system was recently ported to Project Looking Glass 3D and was shown during CeBIT 2006 in the Sun booth, running on Solaris and Sun's Ultra 20 and Ultra 40 workstations.

The system attracted quite a lot of visitors and the media. Here are some media sightings:

- The GoMonkey Website has a flash video. Please note that it currently only shows the GoMonkey system still running on Windows and without Looking Glass 3D.

- Oliver Jones came by the CeBIT 2006 Sun booth and recorded a video clip of the system in action which is now linked from the lg3d-core news page: https://lg3d-core.dev.java.net/files/documents/1834/31064/Sun_GoMonkey_d...

- Germany's second-largest public TV station ZDF featured the GoMonkey demo at the CeBIT 2006 Sun booth in their major evening news show "Heute" on March 14th, 2006.

- Another public broadcaster called "3Sat" featured the GoMonkey system in a CeBIT related tech show called "neues". Here's the video stream featuring GoMonkey after the 21st minute: http://rstreaming.zdf.de/3sat/300/neues_060311_cum.ram
The funny thing here is that the TV anchor lady waved her hand one too many times, causing the system to play a Sun related commercial reel which was a nice, involuntary, free TV commercial for Sun :).

- Here are some photos of the GoMonkey setup at Sun's CeBIT booth: http://www.flickr.com/photos/constanting/sets/72057594083354521/

The GoMonkey system is split into 2 parts:

- The capture machine uses 2 firewire cameras to capture a stereoscopic image of the user. The software is capable of extracting 3D data out of the 2 cameras and can therefore determine the frontmost person and her gestures. Many different gestures can be extracted, but studies show that a KISS approach is really best here. For CeBIT, we only used three gestures: "Wave left with the left hand", "Wave right with the right hand", "Push towards the screen". The capture machine sends gestures as events to the display machine.

- The display machine generates the multimedia presentation which is controlled through the events fed from the capture machine. In our case, the multimedia presentation runs as a Looking Glass 3D application on Solaris. LG3D is used here as a 3D engine to render the multimedia content in. A content management system makes it easy to feed content such as images, sound, videos, 3D objects and other content into the system.

Hope this helps,
Constantin

hideya
Offline
Joined: 2003-07-22

Wow, thanks, Constantin, you've answered lots of questions I had regarding the system. Very cool! This is too good to just use once. Hope we can share the demo with many more people during JavaOne this year and other shows. It's going to be fun fun fun! :)

hideya

hideya
Offline
Joined: 2003-07-22

I've put links to a new GoMonkey demo on the LG3D front page.

https://lg3d-core.dev.java.net/#WhatsNew

This one has more marketing-ish flavor, but thought you might enjoy watching how people are reacting with this unusual interface :-)

hideya

ekerazha
Offline
Joined: 2006-03-02

This is the coolest thing I ever seen.

A *VERY STUPID* question: with 1 webcam it would be possible a similar result (maybe less precise, because it can't have a stereoscopic view)?

hideya
Offline
Joined: 2003-07-22

Hi ekerazha,

> A *VERY STUPID* question: with 1 webcam it would be possible a similar result (maybe less precise, because it can't have a stereoscopic view)?

Yeah, it would be great if it is possible. Do you have any idea? There was one guy who was saying to develop such a thing, but he disappeared quietly...

hideya

ekerazha
Offline
Joined: 2006-03-02

I have absolutely no idea... there should be a nice AI algorithm to interpret the moves (ex. if I put the hand ahead, then the hand is seen bigger = the pink area of my hand is bigger).

Maybe it could be something like this...

Jeff

I had spent a little time thinking about this actually. Unfortunately,
I'm always thinking of devices to make it work rather than better
image-recognition stuff.

My latest idea on it involved making the three-fingered glove you see
in the movie, and putting infrared (or visible) LEDs on the end for
easier designation of where your hand is.

Well, I was going to post a screen-cap from the movie with that glove,
but a little Google search-fu found me this:
http://www.engadget.com/2005/04/15/raytheons-knockoff-of-the-minority-re...
Ha, and I thought it was just cool movie tech.
One of the commenters on there noted that he had some experience using
the system, and his arms got tired quickly ;-)

Always watching from the shadows-
Jeff

lg3d@javadesktop.org wrote:
> I have absolutely no idea... there should be a nice AI algorithm to interpret the moves (ex. if I put the hand ahead, then the hand is seen bigger = the pink area of my hand is bigger).
>
> Maybe it could be something like this...
> [Message sent by forum member 'ekerazha' (ekerazha)]
>
> http://forums.java.net/jive/thread.jspa?messageID=107672
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: interest-unsubscribe@lg3d.dev.java.net
> For additional commands, e-mail: interest-help@lg3d.dev.java.net
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@lg3d.dev.java.net
For additional commands, e-mail: interest-help@lg3d.dev.java.net

Krishna Gadepalli

There was also a reference to this funny take on the movie's UI

http://www.ok-cancel.com/comic/3.html

-krishna

Jeff said the following on 04/25/2006 10:58 AM:
> I had spent a little time thinking about this actually.
> Unfortunately, I'm always thinking of devices to make it work rather
> than better image-recognition stuff.
>
> My latest idea on it involved making the three-fingered glove you
> see in the movie, and putting infrared (or visible) LEDs on the end
> for easier designation of where your hand is.
>
> Well, I was going to post a screen-cap from the movie with that glove,
> but a little Google search-fu found me this:
> http://www.engadget.com/2005/04/15/raytheons-knockoff-of-the-minority-re...
>
> Ha, and I thought it was just cool movie tech.
> One of the commenters on there noted that he had some experience using
> the system, and his arms got tired quickly ;-)
>
> Always watching from the shadows-
> Jeff
>
> lg3d@javadesktop.org wrote:
>> I have absolutely no idea... there should be a nice AI algorithm to
>> interpret the moves (ex. if I put the hand ahead, then the hand is
>> seen bigger = the pink area of my hand is bigger).
>>
>> Maybe it could be something like this...
>> [Message sent by forum member 'ekerazha' (ekerazha)]
>>
>> http://forums.java.net/jive/thread.jspa?messageID=107672
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: interest-unsubscribe@lg3d.dev.java.net
>> For additional commands, e-mail: interest-help@lg3d.dev.java.net
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: interest-unsubscribe@lg3d.dev.java.net
> For additional commands, e-mail: interest-help@lg3d.dev.java.net
>

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@lg3d.dev.java.net
For additional commands, e-mail: interest-help@lg3d.dev.java.net

hideya
Offline
Joined: 2003-07-22

Because of popular demand, the movie clip has been moved to a place that can support more downloads:

Oliver Jones wrote On 03/14/06 06:17,:
> Hi all,
>
> Al Hopper came to the rescue (thanks, Al!) and has hosted the video on
> genunix.org. It can now be found here:
>
> http://www.genunix.org/gen/gomonkey/Sun_GoMonkey_demo_by_Oliver_Jones.avi
>
> Kindest regards,
>
> Oliver.
> ________________________________
>
> From: solarisx86@yahoogroups.com [mailto:solarisx86@yahoogroups.com] On
> Behalf Of Oliver Jones
> Sent: 14 March 2006 09:19
> To: solarisx86@yahoogroups.com
> Subject: RE: [solarisx86] Cool Project Looking Glass at CeBIT 2006
>
>
> Hi all,
>
> Unfortunately, due to excessive demand (and my entire Web host's bandwidth
> allocation being used up in one day!) I've had to remove this from my Web
> site. If someone knows of a decent place to put it (BitTorrent sounds like a
> good idea, too), I'm all ears - it seems to be quite popular!
>
> I might have to send a mail to The Inquirer / The Register, and ask them if
> they want to host a news story.
>
> Kindest regards,
>
> Oliver.

> Hello LG3Ders,
>
> It became reality earlier than I expected...
>
> http://www.oliverjones.org/Sun_GoMonkey_demo.avi
> vi (~9MB, DivX/PCM)
>
> Enjoy...
>
> hideya
>
> P.S. A big thanks to you, Alan Coopersmith, for the
> forwarding me the following email!
>
> -------- Original Message --------
> Subject: RE: [solarisx86] Cool Project Looking Glass
> at CeBIT 2006
> Date: Mon, 13 Mar 2006 11:18:36 +0100
> From: Oliver Jones
> Reply-To: solarisx86@yahoogroups.com
> To: solarisx86@yahoogroups.com
>
> Thanks for the heads-up, Ivan. :)
>
> I visited the booth and snapped a video of it in
> action, for those here who
> couldn't attend.
>
> http://www.oliverjones.org/Sun_GoMonkey_demo.avi
> (~9MB, DivX/PCM)
>
> Kindest regards,
>
> Oliver.
>
> ________________________________
>
> From: solarisx86@yahoogroups.com
> [mailto:solarisx86@yahoogroups.com] On
> Behalf Of Ivan Wang
> Sent: 10 March 2006 17:34
> To: solarisx86@yahoogroups.com
> Subject: [solarisx86] Cool Project Looking Glass at
> CeBIT 2006
>
>
>
> Hi all,
>
> Too excited to not post this, I just visited Sun
> un booth at CeBIT.
> Among all cool stuffs they have there, one of the
> most eye-catching
> demo is a demo called GoMonkey or something. It
> basically shows an
> interactive desktop environment based on Project
> Looking Glass. The
> system interacts with user with some kind of motion
> capturing
> mechanism, so you can move aroung within the Looking
> Glass desktop
> environment simply by waving your hand, including
> zooming in on a
> purticular window, dropping it aside, and so on.
>
> Its coolness factor is 10 of 10 I think :) Though I
> I don't know how
> this can translate into real $$ though ;)
>
> The system is powered by an Ultra 40, if you are
> re interested.
>
> Ivan.