|
SparshUI_Device_Adapter
Tutorial to write an Input Device Adapter for Sparsh-UI.
Featured IntroductionThis page is a tutorial on writing an Input Device Adapter for Sparsh-UI. Sparsh-UI requires the device input data to be in a specific format. Since the native data from the device driver that you use may not be in the format required by Sparsh-UI, it must be converted into the required format before sending to the Sparsh-UI Gesture Server. Protocol DetailsImportant: All the data sent over the socket to the Sparsh-UI Gesture Server needs to be in Network-endian (Big-endian) byte order. If you don't know what this means, refer to this wikipedia page. If you are using Java, you don't need to do anything. If you're using a system-dependent language, make sure you convert your data types correctly, as shown in the example adapter below. The Sparsh-UI development team is not liable for user-inflicted damage to self or equipment incurred from developer frustration. Do not forget this. 1) Input Device ProtocolThis is the protocol between an input device and the Sparsh-UI Gesture Server. The Input Device Adapter (described earlier) is responsible for providing the information in the format described in this section. Touch Point StatesIn nearly all multitouch systems, a touch point is associated with three states, namely:
Touch Point data structureThe touch point data structure consists of the following fields in the given order:
Touch point ID -> A unique integer identifier that identifies the touch point. The id is generated at touch point birth (touch point down) and is retained through its transitions until touch death (touch point up). X coordinate -> this is the normalized coordinate in the x-direction. The normalized x coordinate is a value between 0 and 1, and is obtained by dividing the original x coordinate with the width. For example, if the range of coordinates provided by the touch device in x-direction is 0-1023, and the x coordinate of the detected touch point is 40, then the normalized point will be 40/1024 = 0.0390625. Normalized coordinates are needed because some multi-touch input devices would provide a range of coordinates which are different from the screen resolution. Y coordinate -> This is the normalized coordinate in the y-direction. Similar to the x coordinate, the normalized Y coordinate is obtained by dividing the coordinate with the height. For example if the range of coordinates provided by the touch device in the y-direction is 0-767,and the y coordinate detected by the touch driver is 30, then the normalized y coordinate would be 30/768 = 0.0390625. Touch point state -> This denotes the state of the touch point. It should be set equal to one of the values in the enum shown below. The touch point state is an enumerator with the following fields: enum Touch_Point_State POINT_BIRTH, POINT_DEATH, POINT_MOVE end enum A Sparsh-UI Input Device Adapter communicates with the Sparsh-UI Gesture Server over a TCP/IP socket. The Gesture Server listens for incoming connectiosn on port 5945. On connection, the Input Device Adapter shall send a single byte with value set to 1, indicating to the Gesture Server that it is an input device. After this, the Input Device Adapter can start sending the data. The payload format is given below:
Note: The number of Touch Point data structures set in each message should be equal to the number of Touch Points sent in the first field of each message. Example Sparsh-UI Input Device AdapterMost multitouch device drivers provide a callback function which is called whenever there is a touch. Some multitouch drivers provide several touch points in a single frame in the call back and others will call the callback for every touch point. It is more efficient to send all the points in a frame at once, so to accommodate this, the Sparsh-UI Input Device Protocol has the provision for sending multiple points at the same time. Let us look at a example for the case where there is a callback for each touch point detected.
#include <DRIVER SPECIFIC HEADERS>
#include <OS HEADERS FOR TCP IP COMMUNICATION>
//Sparsh touch point structure
//This stores the touch point information
//*Note: Sparsh UI uses transmits all data in network endian order*
struct sparsh_touchPoint
{
int _id; //Integer id which uniquely represent a touch point. *NETWORK ENDIAN *
float _x; //normalized value of x-co-ordinate *NETWORK ENDIAN *
float _y; //normalized value of y-co-ordinate *NETWORK ENDIAN *
char _type; //this is the touch point state
};
//use this function to swap the endian-ness of a float
float swapFloatEndian( float x)
{
union u {float f; char temp[4];};
union u un,vn;
un.f = x;
vn.temp[0] = un.temp[3];
vn.temp[1] = un.temp[2];
vn.temp[2] = un.temp[1];
vn.temp[3] = un.temp[0];
return vn.f;
}
//use this function to swap the endian-ness of an int.
int swapIntEndian( int x)
{
union u {int f; char temp[4];};
union u un,vn;
un.f = x;
vn.temp[0] = un.temp[3];
vn.temp[1] = un.temp[2];
vn.temp[2] = un.temp[1];
vn.temp[3] = un.temp[0];
return vn.f;
}
/* Notify that a finger has just been made active.
This function will be called by the driver API whenever
the finger is placed on the multi-touch device. The definition and argument will
vary depending upon the driver that you have */
/* NOTE:
* 1) Sparsh UI Requires all data to be in NETWORK ENDIAN
* DO THE CONVERSION IF YOUR DATA IS IN HOST ENDIAN, THERE IS NO NEED TO DO THE
* CONVERSION IF IT IS ALREADY IN NETWORK ENDIAN.
* 2) DO THE NORMALIZATION IF YOUR CO-ORDINATE DATA IS NOT NORMALIZED. DO NOT DO IT
* IF ITS ALREADY NORMALIZED
* 3) MOST DRIVERS DO PROVIDE A UNIQUE ID TO EACH TOUCH POINT ,
(ex: touchlib , stantum multi-touch tablet drivers etc, Some may not provide
this.In that case you need to assign an Unique ID to each touch point.
*/
/*assume this is the callback when finger is pressed
*TouchData is the structure provided by the driver , parameters and name would be
*different based on your implementation.
*/
void fingerDown(TouchData data)
{
sparsh_touchpoint tp;
tp._id = swapIntEndian(data.id); // change the endianness
tp._x = swapFloatEndian((data.X)/(X_WIDTH)); //normalize and change endianness
tp._y = swapFloatEndian((data.Y)/(Y_LENGTH)); //normalize and change endianness
tp._data = POINT_BIRTH; //POINT STATE , single byte , no need to change endianness
//now send the point over a network
sendPoint(&tp);
}
//! Notify that a finger has moved
void fingerUpdate(TouchData data) //assume this is the callback when finger is moved
{
sparsh_touchpoint tp;
tp._id = swapIntEndian(data.id);
tp._x = swapFloatEndian(data.X);
tp._y = swapFloatEndian(data.Y);
tp._data = POINT_MOVE;
//now send the point over a network
sendPoint(&tp);
}
//! A finger is no longer active..
void fingerUp(TouchData data) //assume this is the callback when finger is lifted
{
sparsh_touchpoint tp;
tp._id = swapIntEndian(data.id);
tp._x = swapFloatEndian(data.X);
tp._y = swapFloatEndian(data.Y);
tp._data = POINT_DEATH;
//now send the point over a network
sendPoint(&tp);
}
/*This function sends the touchpoint structure over socket to sparsh UI server
*/
void sendPoint(sparsh_touchpint *tp)
{
int struct_size = sizeof(int) + 2 * sizeof(float) + sizeof(char);
int buffersize = struct_size + sizeof(int); //the second term is for length parameter
char * buffer = (char*) malloc(buffersize); //allocate memory
char *bufferptr = buffer;
//first copy the size information
//remember to convert to network endian before copying
int network_endian_size = swapIntEndian(buffersize);
memcpy(bufferptr,&network_endian_size,sizeof(int));
bufferptr += sizeof(int);
//now copy the id
memcpy(bufferptr,&(tp->_id),sizeof(int));
bufferptr+=sizeof(int);
//now copy the X co-ordinate
memcpy(bufferptr,&(tp->_x),sizeof(float));
bufferptr+=sizeof(float);
//now copy the Y co-ordinate
memcpy(bufferptr,&(tp->_y),sizeof(float));
bufferptr+=sizeof(float);
//now copy the STATE
memcpy(bufferptr,&(tp->_type),sizeof(char));
bufferptr+=sizeof(char);
//Your buffer is now ready to be sent over the network
// Sending the buffer over network
int sent = send(sockfd, buffer, buffersize , 0);
free(buffer);
}
/*This function initializes the connection to sparsh UI.
*upon successful connection send one byte set to 1.
*publishPort has the value 5945 , SparshUI service runs on this port
*/
void init(int publishPort)
{
sockaddr_in inetAddress;
int sockfd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); //use SOCKET if its windows
if(sockfd==-1)
{
printf("invalid socket\n");
return ;
}
inetAddress.sin_family = AF_INET;
inetAddress.sin_port = htons((u_short)publishPort);
unsigned long addr = inet_addr("127.0.0.1");
hostent *host = gethostbyaddr((char*)&addr,sizeof(addr),AF_INET);
inetAddress.sin_addr.s_addr = *((unsigned long*)host->h_addr);
if(connect(sockfd, (struct sockaddr *)&inetAddress,sizeof(sockaddr)) != 0)
{
printf("connect error\n");
return;
}
else
{
const char one = 1;
int sent = send(sockfd, &one, sizeof(char), 0);
cout<<"sent "<<sent<<"bytes"<<endl;
}
}
void main()
{
/* do initialization*/
init(5945);
//register for callbacks from your driver etc..
this portion would depend on your driver//
}
What if my device gives more than one point at the same time?Some multi-touch devices give more than one point at the same time. As discussed above, Sparsh-UI accomodates this functionality. The code for the device adapter is similar except that you send more than one point at the same time over the socket. For example, let's assume two points are generated in a single capture (1,30,40,POINT_BIRTH),(2,80,90,POINT_BIRTH). The packet being sent should have the following format
Remember that all the data needs to be in network (big) endian type! If you fail to remember this, your personal behavior while debugging your code is undefined. The Sparsh-UI development team takes no responsibility for damage to self or equipment incurred from such failure to remember this important point. I am lost! What do I do if I cannot figure this out on my own?Request to join our google group at http://groups.google.com/group/sparsh-ui Post a discussion with your issue and we'll gladly help you out. | ||||||||||||||||||||
► Sign in to add a comment