Integral Design

This experiment leaves much to the imagination, but it does demonstrate how wave forms of the human brain can determine events on a device with an embedded microcontroller.  This code scenario is submitted for use in the Open Health and Fitness Data Aggregator project headed by Ira Laefsky. The Neurosky Mind Wave sensor is an economical electroencephalograph designed for extensibility with other applications.  For less than $100 the device includes the Think Gear Connector, which runs on the Windows or Mac operating systems and supports an API.  With two lines of code you can initialize the Think Gear Connector to access the output of the Neurosky electroencephalograph (C# requires the interop thinkgear.dll in the folder with the executable).

    int connectId = ThinkGear.TG_GetNewConnectionId();
    int succeeded = ThinkGear.TG_Connect(0, "COM7",
                    ThinkGear.BAUD_57600, ThinkGear.STREAM_PACKETS);

    Console.WriteLine("Any key to continue... .");
    Console.ReadKey();

The connector sorts the data into the following wave-form categories – from the developer’s guide:

eSense:  A container for the eSense™ attributes, which are integer values between 0 and 100.  Zero is perceived as a lack of that attribute and 100 is an excess of that attribute.

– attention. The eSense Attention value.

– meditation. The eSense Meditation value.

eegPower:  A container for the EEG powers. These may be either integer or foating-point values.

– delta. The “delta” band of EEG.

– theta. The “theta” band of EEG.

– lowAlpha. The “low alpha” band of EEG.

– highAlpha. The “high alpha” band of EEG.

– lowBeta. The “low beta” band of EEG.

– highBeta. The “high beta” band of EEG.

– lowGamma. The “low gamma” band of EEG.

– highGamma. The “high gamma” band of EEG.

rawEeg: The raw data reading off the forehead sensor. This may be either an integer or a foating-point value.

blinkStrength: The strength of a detected blink. This is an integer in the range of 0-255.

Using the Neurosky Think Gear Connector API with C#

When a C# application has a connection to the Think Gear Connector, it can read the output of the Neurosky electroencephalograph without code that parses raw data into JSON script elements.
The following code block gets an Attention wave-form datum from the connection, writes the value to the console, an appends the result to a StringBuilder object that will be saved to a file.

    if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_ATTENTION) != 0)
    {
        float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_ATTENTION);
        Console.WriteLine("Attention: \t{0:n}", result);
        sb.AppendLine("Attention: " + result.ToString());
    }

Using code on this model you can capture data sorted into the various wave-forms supported by the Neurosky sensor.  The output to the command window is shown in the following illustration (click to enlarge).

Neurosky Mind Gear Connector Console Output

Control the .NET Gadgeteer Device with Brain Waves

The sorted wave-form data from the Mind Gear Connector could be sent to a .NET Gadgeteer device in several ways.  This example sends data as an http request to a Web service running on a .NET Gadgeteer device.  A WiFi connection enables the connection by using the WiFi RS21 Module available from GHI Electronics.

The following illustration shows the WiFi RS21 module, the GHI Electronics FEZ Spider mainboard, and four Smart Multicolor LED Modules connected by the daisy-link method, as described in the .NET Gadgeteer Module Builder’s Guide version 1.8.

.NET Gadgeteer Modules

We can set up and start a Web service on a .NET Gadgeteer device in three lines of code, as shown in the following NetworkUp event delegate.   The first three lines of this example only notify the user that the network is ready and print out the IP address of the Web service; this will be needed by the client application that will send wave-form data to the service.

    void wifi_NetworkUp(GTM.Module.NetworkModule sender, GTM.Module.NetworkModule.NetworkState state)
    {
        led.TurnGreen();

        Debug.Print("Connected to network.");
        Debug.Print("IP address: " + wifi.NetworkSettings.IPAddress);
        WebServer.StartLocalServer(wifi.NetworkSettings.IPAddress, 80);

        waveEventHost = WebServer.SetupWebEvent("wave");
        waveEventHost.WebEventReceived +=
             new WebEvent.ReceivedWebEventHandler(waveEventHost_WebEventReceived);
    }

Now we have described the basic the scenario, we can add a feature to the console application that gets data from the Mind Gear Connector.  A Web request can be added to the code that gets the wave-form data that sends various wave-form measurements to the Web service running on the .NET Gadgeteer device.  The following code segment appends an if statement that runs when levels reach an arbitrary value.  The if statement contains a Web request that sends measurements obtained by the ThinkGear.GetValueStatus method of the type defined as Attention.

    if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_ATTENTION) != 0)
    {
        float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_ATTENTION);
        Console.WriteLine("Attention: \t{0:n}", result);
        sb.AppendLine("Attention: " + result.ToString());
        if (result > 50)
        {
            HttpWebRequest wreq =
                HttpWebRequest.Create("http://192.168.1.6/wave?lednum=1") as HttpWebRequest;
            wreq.BeginGetResponse(null, null);
        }
    }

Note that the code uses an asynchronous request, BeginGetResponse, without an EndGetResponse method.  This sends the data to the Web service without waiting for the request to return an ok value.  Waiting would would block the application, which is receiving 512 packets per second.

To understand the URL parameters of the Web request, you have to see the rest of the code on both sides of this scenario.  The Web service on the .NET Gadgeteer device processes the requests using integer values sent from the console application, in the previous code segment.  The following WebEventReceived delegate contains a long switch statement that simply turns on LEDs of various colors according to the wave-forms that the integers in the URL parameters specify.  Turning on a LED is trivial, but this method could start any kind of electrical circuit or mechanical actuator.  The following code block shows the complete .NET Gadgeteer Program.cs file.

using Microsoft.SPOT;
using Gadgeteer.Networking;
using GT = Gadgeteer;
using GTM = Gadgeteer.Modules;
using Gadgeteer.Modules.GHIElectronics;

namespace MindLEDs
{
    public partial class Program
    {
        GT.Networking.WebEvent waveEventHost;

        void ProgramStarted()
        {
            wifi.UseDHCP();
            Gadgeteer.Modules.GHIElectronics.WiFi_RS21.WiFiNetworkInfo info =
                                                    new WiFi_RS21.WiFiNetworkInfo();
            info.SSID = "WWAT5";
            info.SecMode = WiFi_RS21.SecurityMode.WEP;
            info.networkType = WiFi_RS21.NetworkType.AccessPoint;
            wifi.Join(info, "EAEFB03BF6");

            wifi.NetworkUp += new GTM.Module.NetworkModule.NetworkEventHandler(wifi_NetworkUp);
            led.TurnRed();

            Debug.Print("Program Started");
        }

        void wifi_NetworkUp(GTM.Module.NetworkModule sender, GTM.Module.NetworkModule.NetworkState state)
        {
            led.TurnGreen();

            Debug.Print("Connected to network.");
            Debug.Print("IP address: " + wifi.NetworkSettings.IPAddress);
            WebServer.StartLocalServer(wifi.NetworkSettings.IPAddress, 80);

            waveEventHost = WebServer.SetupWebEvent("wave");
            waveEventHost.WebEventReceived +=
                new WebEvent.ReceivedWebEventHandler(waveEventHost_WebEventReceived);
        }

        void waveEventHost_WebEventReceived(string path, WebServer.HttpMethod method, Responder responder)
        {
            if (responder.UrlParameters.Contains("lednum"))
            {
                int number = int.Parse(responder.UrlParameters["lednum"].ToString());

                switch (number)
                {
                    case 0:
                        led.TurnRed();
                        break;
                    case 1:
                        led.BlinkOnce(GT.Color.Blue);
                        break;
                    case 2:
                        led1.BlinkOnce(GT.Color.Green);
                        break;
                    case 3:
                        led2.BlinkOnce(GT.Color.Red);
                        break;
                    case 4:
                        led3.BlinkOnce(GT.Color.Yellow);
                        break;
                    case 5:
                        led.BlinkOnce(GT.Color.Yellow);
                        break;
                    case 6:
                        led1.BlinkOnce(GT.Color.Red);
                        break;
                    case 7:
                        led2.BlinkOnce(GT.Color.Green);
                        break;
                    case 8:
                        led3.BlinkOnce(GT.Color.Blue);
                        break;
                    case 9:
                        led.BlinkOnce(GT.Color.Red);
                        break;
                    case 10:
                        led1.BlinkOnce(GT.Color.Yellow);
                        break;
                    default:
                        break;
                }
            }
            responder.Respond("ok");
        }
    }
}

Having seen the Web service switch statement, you can decypher the complete application code that interfaces with the Mind Gear Connector, the console application that runs on Windows or Mac operating systems and relays wave-form data to the .NET Gadgeteer device.

The user wearing the Neurosky EEG sensor starts sending data.  The command window prompts the user and waits for connection to the Mind Gear Connector to initialize.  When it continues, it will read packets of data from the connection according to the defined wave-forms.  When various wave-forms reach an arbitrary value, the wave-form are assigned an integer value that is sent to the .NET Gadgeteer device .   Sending all of the data is unnecessary for this experiment, so the threshold is set high enough to limit the amount of data sent to the Web service.  Probably the middle range of each wave-form values could be disregarded and the low and high ranges sent.

using System;
using System.Text;
using System.IO;
using System.Media;
using System.Net;

namespace MindGearConsoleClient
{
    class Program
    {
        static void Main(string[] args)
        {
            System.Timers.Timer timer;
            timer = new System.Timers.Timer(60000);
            timer.Elapsed += new System.Timers.ElapsedEventHandler(timer_Elapsed);
            timer.Enabled = true;

            Console.WriteLine("Any key to continue... .");
            Console.ReadKey();

            int connectId = ThinkGear.TG_GetNewConnectionId();
            int succeeded = ThinkGear.TG_Connect(0, "COM7",
                            ThinkGear.BAUD_57600, ThinkGear.STREAM_PACKETS);

            Console.WriteLine("Any key to continue... .");
            Console.ReadKey();

            StringBuilder sb = new StringBuilder();

            int packetsRead = 0;
            while (timer.Enabled && connectId != -1)
            {
                packetsRead = ThinkGear.TG_ReadPackets(connectId, 1);

                if (packetsRead > 0)
                {
                    try
                    {
                        if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_ATTENTION) != 0)
                        {
                            float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_ATTENTION);
                            Console.WriteLine("Attention: \t{0:n}", result);
                            sb.AppendLine("Attention: " + result.ToString());
                            if (result > 50)
                            {
                                HttpWebRequest wreq =
                                    HttpWebRequest.Create("http://192.168.1.6/wave?lednum=1") as HttpWebRequest;
                                wreq.BeginGetResponse(null, null);
                            }
                        }

                        if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_ALPHA1) != 0)
                        {
                            float result =  ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_ALPHA1);
                            Console.WriteLine("Alpha 1: \t{0:n}", result);
                            sb.AppendLine("Alpha 1: " + result.ToString());
                            if (result > 2500)
                            {
                                HttpWebRequest wreq =
                                    HttpWebRequest.Create("http://192.168.1.6/wave?lednum=2") as HttpWebRequest;
                                wreq.BeginGetResponse(null, null);
                            }
                        }

                        if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_ALPHA2) != 0)
                        {
                            float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_ALPHA2);
                            Console.WriteLine("Alpha 2: \t{0:n}", result);
                            sb.AppendLine("Alpha 2: " + result.ToString());
                            if (result > 2500)
                            {
                                HttpWebRequest wreq =
                                    HttpWebRequest.Create("http://192.168.1.6/wave?lednum=3") as HttpWebRequest;
                                wreq.BeginGetResponse(null, null);
                            }
                        }

                        if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_BATTERY) != 0)
                        {
                            float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_BATTERY);
                            Console.WriteLine("Data Battery: \t{0:n}", result);
                            sb.AppendLine("Data Battery: " + result.ToString());

                            HttpWebRequest wreq =
                                HttpWebRequest.Create("http://192.168.1.6/wave?lednum=0") as HttpWebRequest;
                            wreq.BeginGetResponse(null, null);

                        }

                        if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_BETA1) != 0)
                        {
                            float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_BETA1);
                            Console.WriteLine("Beta 1: \t{0:n}", result);
                            sb.AppendLine("Beta 1: " + result.ToString());
                            if (result > 15000)
                            {
                                HttpWebRequest wreq =
                                    HttpWebRequest.Create("http://192.168.1.6/wave?lednum=4") as HttpWebRequest;
                                wreq.BeginGetResponse(null, null);
                            }
                        }

                        if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_BETA2) != 0)
                        {
                            float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_BETA2);
                            Console.WriteLine("Beta 2: \t{0:n}", result);
                            sb.AppendLine("Beta 2: " + result.ToString());
                            if (result > 2500)
                            {
                                HttpWebRequest wreq =
                                    HttpWebRequest.Create("http://192.168.1.6/wave?lednum=5") as HttpWebRequest;
                                wreq.BeginGetResponse(null, null);
                            }
                        }

                        if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_DELTA) != 0)
                        {
                            float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_DELTA);
                            Console.WriteLine("Delta: \t{0:n}", result);
                            sb.AppendLine("Delta: " + result.ToString());
                            if (result > 100000)
                            {
                                HttpWebRequest wreq =
                                    HttpWebRequest.Create("http://192.168.1.6/wave?lednum=6") as HttpWebRequest;
                                wreq.BeginGetResponse(null, null);
                            }
                        }

                        if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_GAMMA1) != 0)
                        {
                            float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_GAMMA1);
                            Console.WriteLine("Gamma 1: \t{0:n}", result);
                            sb.AppendLine("Gamma 1: " + result.ToString());
                            if (result > 5000)
                            {
                                HttpWebRequest wreq =
                                    HttpWebRequest.Create("http://192.168.1.6/wave?lednum=7") as HttpWebRequest;
                                wreq.BeginGetResponse(null, null);
                            }
                        }

                        if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_GAMMA2) != 0)
                        {
                            float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_GAMMA2);
                            Console.WriteLine("Gamma 2: \t{0:n}", result);
                            sb.AppendLine("Gamma 2: " + result.ToString());
                            if (result > 5000)
                            {
                                HttpWebRequest wreq =
                                    HttpWebRequest.Create("http://192.168.1.6/wave?lednum=8") as HttpWebRequest;
                                wreq.BeginGetResponse(null, null);
                            }
                        }

                        if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_MEDITATION) != 0)
                        {
                            float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_MEDITATION);
                            Console.WriteLine("Meditation: \t{0:n}", result);
                            sb.AppendLine("Meditation: " + result.ToString());
                            if (result > 30)
                            {
                                HttpWebRequest wreq =
                                    HttpWebRequest.Create("http://192.168.1.6/wave?lednum=9") as HttpWebRequest;
                                wreq.BeginGetResponse(null, null);
                            }
                        }

                        if (ThinkGear.TG_GetValueStatus(connectId, ThinkGear.DATA_THETA) != 0)
                        {
                            float result = ThinkGear.TG_GetValue(connectId, ThinkGear.DATA_THETA);
                            Console.WriteLine("Theta: \t{0:n}", result);
                            sb.AppendLine("Theta: " + result.ToString());
                            if (result > 300000)
                            {
                                HttpWebRequest wreq =
                                    HttpWebRequest.Create("http://192.168.1.6/wave?lednum=10") as HttpWebRequest;
                                wreq.BeginGetResponse(null, null);
                            }
                        }
                    }
                    catch (Exception ex)
                    {
                        Console.WriteLine(ex);

                    }
                }
            }

            File.WriteAllText("results.txt", sb.ToString());

            Console.WriteLine("Exit... .");

            ThinkGear.TG_Disconnect(connectId);
            ThinkGear.TG_FreeConnection(connectId);
            Console.ReadKey();
        }

        static void timer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
        {
            ((System.Timers.Timer)sender).Stop();
            ((System.Timers.Timer)sender).Enabled = false;
        }
    }
}

The following video shows the running .NET Gadgeteer device.