Touch Framework is an underlying framework built around multi-modal based applications. This framework will allow others to develop multi-modal applications that will both comply with a standard TUIO (Tangible User Interface Objects) protocol and interface with a multitude of current and future multi-modal devices. The Touch Framework consists of different levels that allow users of multi-modal devices to take tracking information from their device, and create both low and high level interactions as well as a final GUI. The idea is to create a standard framework that will function across different platforms and programming languages. This universal framework will allow standardization when creating multi-modal applications.