Creating software components

Here we explain how to create software components with the ES operating system.

Component software

When creating operating systems and applications, rather than building a single large-scale program, we combine various parts - software components - resulting in the basic concept of component software.

The method of building applications by linking various libraries introduces the problem of naming collisions and of various contributors using different naming conventions and coding styles, making combining libraries from various vendors difficult. A more practical approach would be to use libraries containing the essential elements provided by a single vendor.

At present, the concept of open source is spreading with the internet making it easy to obtain an abundance of software created by people around the world. Most of the software released in this way is still implemented as individual applications but if we could use those as parts of software, there is the chance that we could combine the creations of others to freely and efficiently build systems as required.

In order to make this a reality, it is important to draw up a collection of interfaces that various software components can use. These interfaces would be entirely separate from so-called software development kit (SDK) APIs, operating system system calls or standard library functions. These interfaces would be conventions that all software components would follow, for example any component would support a list method that gets a list of its elements, which would return an interface for an iterator object. This is similar in concept to the example of the file menu being at the far left of the menu bar in window system user interfaces, a rule that is followed by nearly all applications that have a file menu.

Interface Definition Language (IDL)

The ES operating system, like other component systems, defines various interfaces using Interface Definition Language (IDL). C++ programmers should notice that ES's IDL is very similar to class declarations in C++.

The code below is taken from the IDL file for ES's Binding interface (IBinding.idl).

/** This interface represents a name-to-object binding.
*/
interface Binding : Interface
{
	#pragma ID Binding = "DCE:33f5e13e-25dc-11db-9c02-0009bf000001";
	/** The object bound to this binding.
	*/
	attribute Interface object;
	/** The string representation of this binding.
	*/
	readonly attribute string name;
};

The Binding interface inherits from the Interface interface. This stipulates the object and name properties. The next section shows the C++ header files generated by this IDL. The C++ methods getObject, setObject and getName are generated from these properties.

Please refer to the specifications for esidl, the IDL compiler for details of the IDL syntax.

The interface ID 33f5e13e-25dc-11db-9c02-0009bf000001 is allocated to the Binding interface. In this way, all interfaces in ES are allocated a unique UUID. The interface ID, rather than the Binding string representation, is used to identify the interface in use when calling software component methods. Wherever, whenever and by whomever an interface is created, it is given a different UUID, thus avoiding collisions that could occur if strings were used to name interfaces.

Because the interface semantics are defined by IDL, you can use them in the same process or in different processes (ignoring the difference in their calling times) without being aware of their differences.

esidl, the IDL compiler

The IDL compiler, esidl, generates the C++ header files and interface reflection data files essential to implement the software component that provides the interface described in IDL. esidl generates IBinding.h and IBinding.ird from IBinding.idl. IBinding.h is a C++ header file and IBinding.ird is where reflection data needed for the ES operating system to introduce the method calls between components is saved in binary format. (esidl is different to other component systems in that it cannot create stub code to marshal parameters per interface. The ES operating system kernel automatically returns reflection data in many marshalling processes.)

When IBinding.idl is compiled with esidl, IBinding.h is generated as below:

/** This interface represents a name-to-object binding.
*/
class Binding : public Interface
{
	public:
	/** The object bound to this binding.
	*/
	virtual Interface* getObject() = 0;
	virtual void setObject(Interface* object) = 0;
	/** The string representation of this binding.
	*/
	virtual int getName(char* name, int nameLength) = 0;
	static const Guid& iid()
	{
		static const Guid iid =
		{
		 	0x33f5e13e, 0x25dc, 0x11db, { 0x9c, 0x02, 0x00, 0x09, 0xbf, 0x00, 0x00, 0x01 }
		};
		return iid;
	}
};

The getObject, setObject and getName methods that handle the object and name properties defined in IDL are generated. The interface ID can be obtained with Binding::iid().

In ES, the interface and method descriptions are contained in IDL files in Javadoc format. The esidl compiler writes out the Javadoc comments in the IDL files to the C++ header files. The ES HTML reference manual is automatically created from these C++ header files using the open source CppDoc.

Software component implementation - Server programs

Here we introduce programs implementing software components in ES. binder.cpp is the simple source file of a software component that provides an object that prepares the Binding interface.

The actual implementation of the object's class that prepares the Binding interface is class Binder. This contains addRef, release and queryInterface which are methods stipulated in the Interface interface which the Binding interface inherits. (Interface has its roots in IUnknown from Microsoft's Component Object Model (COM).) addRef and release govern object lifetimes on the basis of reference counts. queryInterface returns the interface pointer of an object specified by an interface ID. (We have already explained the use of interface IDs rather than symbols to identify software component interfaces.)

In ES, all software component objects have to implement the Interface interface. Fortunately, however, as you can see in the following code, it is usually not necessary to be directly aware of these methods when using objects.

Here is a look at the main function in binder.cpp:

int main(int argc, char* argv[])
{
	esReport("This is the Binder server process.\n");
	System()->trace(true);
	Handle<Context> nameSpace = System()->getRoot();
	Handle<ClassStore> classStore = nameSpace->lookup("class");
	TEST(classStore);
	// Register Binder factory.
	Handle<ClassFactory> binderFactory(new(ClassFactory<Binder>));
	classStore->add(CLSID_Binder, binderFactory);
	// Create a client process.
	Handle<Process> client;
	client = reinterpret_cast<Process*>(
		classStore->createInstance(CLSID_Process, client->iid()));
	TEST(client);
	// Start the client process.
	Handle<File> file = nameSpace->lookup("file/binderClient.elf");
	TEST(file);
	client->start(file);
	// Wait for the client to exit
	client->wait();
	// Unregister Binder factory.
	classStore->remove(CLSID_Binder);
	System()->trace(false);
}

esReport("This is the Binder server process.\n");

esReport is ES's library function equivalent of printf. At least for the time being, C's stdio is not supported in ES.

System()->trace(true);

System() is a library function that returns the CurrentProcess interface pointer (defined in libes++.a). CurrentProcess is the only interface pointer that ES user processes can use right from the start. When CurrentProcess is used, you can retrieve the standard input/output stream's interface pointer allocated to the current process.

Here, we show how to output debug information for the system calls and remote procedure calls running the current process using CurrentProcess's trace method.

Handle<Context> nameSpace = System()->getRoot();

Here, we get the root namespace's Context interface. In dealing with raw interface pointers, because of tricky cases such as administrating reference counts, we write Handle<Context> in ES instead of writing Context*, using the smart pointer class Handle defined in handle.h. Handle<Context> internally processes addRef, release and queryInterface calls automatically.

Handle<ClassStore> classStore = nameSpace->lookup("class");

Next, the class store's ClassStore interface is retrieved from the namespace. The class store is where the factory classes for software components are stored. Although there's no need to be aware of this, the return value type of the Context interface's lookup method is Interface*. When getting the ClassStore* type pointer used in classStore, calling the queryInterface method and getting a new interface pointer means we need to adjust reference counts, so onerous processes such as these are processed internally by the Handle template class.

// Register Binder factory.
Handle<ClassFactory> binderFactory(new(ClassFactory<Binder>));

The only things we can register in the class store are factory class objects prepared by the ClassFactory interface. Here, we create a Binder class factory class object using the ClassFactory template class. If the default constructor is defined in a particular class, then we can easily create an instance of the factory class for that class using the ClassFactory template class.

classStore->add(CLSID_Binder, binderFactory);

Here we're registering binderFactory in the class store. When registering a factory class in the class store, a unique class ID is used instead of a name. Using a class ID instead of a name avoids accidental name collisions with factory classes created by a variety of people.

When a factory class is registered in the class store, ES operating system processes start a server process that enables the creation and use of instances by other processes.

binder.cpp starts a client process that then uses Binder.

// Create a client process.
Handle<Process> client;
client = reinterpret_cast<Process*>(classStore->createInstance(CLSID_Process, client->iid()));

Using the process's class ID, the creation of a new process in the class store is requested and the Process interface pointer for the client process is received. The process is a kernel object registered beforehand by the ES kernel in the class store.

// Start the client process.
Handle<File> file = nameSpace->lookup("file/binderClient.elf");

Next, the executable file running the client process is found in the namespace and the File interface pointer is retrieved.

client->start(file);

The start method is called using this File interface pointer as an argument and the client process starts running the specified program.

// Wait for the client to exit
client->wait();

The binder.cpp now waits for the client process to finish.

// Unregister Binder factory.
classStore->remove(CLSID_Binder);

When the client process has finished, the Binder class factory is deleted from the class store and binder.cpp itself stops tracing and finishes.

System()->trace(false);

Client programs

binderClient.cpp contains the source code for the client-side program that starts binder.cpp. Whereas the CLSID_Process process was created in binder.cpp's createInstance method, here Binder is created as CLSID_Binder. We also call the getName method to display the Binder name.

int main(int argc, char* argv[])
{
	esReport("This is the Binder client process.\n");
	System()->trace(true);
	Handle<Context> nameSpace = System()->getRoot();
	Handle<ClassStore> classStore = nameSpace->lookup("class");
	// Create binder objects.
	Handle<Binding> binder[2];
	for (int i(0); i < 2; ++i)
	{
		binder[i] = reinterpret_cast<Binding*>(
			classStore->createInstance(CLSID_Binder,
				binder[i]->iid()));
		char name[14];
		binder[i]->getName(name, sizeof(name));
		esReport("%s\n", name);
	}
	System()->trace(false);
}

The interface used when a new process is created and when a Binder object is created is the same. Also, whether the methods of an object executed by another process are called or whether the methods of an object implemented inside the kernel are called, from the client's point of view they can both be written as pure virtual function calls using interface pointers.

As we have seen up to now, in the ES operating system there are no independent system calls that do not use interface pointers. The operating system itself is being created as a collection of software components.

Execution example

Let's try and run binder. Running the cmd/testsuite/binder script in the build tree starts the ES kernel and runs binder.

% cmd/testsuite/binder

If trace is not set to on:

This is the Binder server process.
This is the Binder client process.
id1
id2
done.

should be displayed. However, kernel tracing is enabled so you can see the actual conversation taking place between the operating system's kernel and server and client processes.

There may be people who have read this far who feel that the server process was broken with wait(). There may also be people who are wondering how new() and getName() in Binder were run by a server process.

Each time a client process calls getName, the following trace is displayed:

system call[9:8011B820]: Binding::getName(name, len);
upcall: Binding::getName(name, len);
return from upcall: Binding::getName(name, len);

system call[9:8011B820]: Binding::getName(name, len);

This trace shows that the getName method call:

binder[i]->getName(name, sizeof(name));

from the client passes some control to the kernel. If Binder is an object instantiated in the kernel like a process, the kernel will process the request and complete the system call. However, Binder is actually an object implemented in a separate process.

upcall: Binding::getName(name, len);

This trace shows the server's getName method is being called instead of the client's. In the ES kernel, the kernel thread moves from a client process to a server process and upcalls getName. As a result, the getName method in the server process is able to run even if the main thread dies waiting for the client process to finish.

return from upcall: Binding::getName(name, len);

This trace shows that the server's getName method has finished processing and is returning to the client. At this time, the kernel copies the string "id1" created in the server process to the array "name14" in the client process and after this line:

binder[i]->getName(name, sizeof(name));

re-opens the client process.

Extending the interface

In the binder example, we showed how to create a server program that provides a Binding interface that is registered in the ES kernel right from the start. In the ES operating system, it is also possible to provide interfaces that are not initially registered in the kernel to other applications defined in individual components.

location.cpp is a program that defines a new interface called Location and provides the accompanying service. When the ILocation.idl IDL file that defines Location is compiled by the esidl compiler, it simultaneously generates an ILocation.h header file and an ILocation.ird file that contains the interface's reflection data. By registering this newly-created reflection data in the kernel, the new Location interface can be used by other programs in addition to location.cpp.

To register new reflection data in the kernel, an InterfaceStore interface pointer is obtained from the namespace and the add method is called as below:

// Register Location interface.
Handle<InterfaceStore> interfaceStore = nameSpace->lookup("interface");
interfaceStore->add(LocationInfo, LocationInfoSize);

Running the cmd/testsuite/location script in the build tree starts the ES kernel and runs the location server. You can probably see that the position and name of the location server's Location class object is naturally handled as a local object by the external locationClient.cpp program.

Summary

We've tried to explain how to create software components in the ES operating system. We showed that if you define an object's class, just by registering its factory class in the class store the process is available as a software component. In ES, local remote method calls are no more than a combination of system calls and upcalls. When compared to the way that many other operating systems transmit the data of user program stub code marshalling arguments and return values by system call messages and instantiate remote procedure calls, implementing user programs and IDL compilers in ES is extremely simple.

The concept of addRef, release and queryInterface specified in ES's Interface interface is derived from COM's IUnknown interface established by Microsoft. This is an extremely basic concept that is not just confined to Microsoft's COM but is widely applied in frameworks such as the Mozilla Foundation's XPCOM, QUALCOMM's BREW and CERN's Gaudi. We recommend the following book for an explanation of COM technology:

Don Box, "Essential COM", Addison-Wesley, 1999

COM technology was established several years before ISO C++ standardization was initially completed in 1998 and at that time, the use of smart pointers with templates had not been fully developed. Consequently, you may get the impression that it is an even more complex subject than we have covered here, however we believe "Essential COM" leads to an understanding of the basic thought process behind software components and beyond.