My favorites | Sign in
Project Home Downloads Wiki Issues Source
Search
for
Vtocc  
vtocc implementation details
Updated Mar 16, 2012 by sougou@google.com

We've taken great care to keep the code simple and readable, and we intend to keep it this way. So, this document is intended to act as more of a roadmap to point you to the right places in the code.

Dependency diagram

The image below shows the dependency diagram for vtocc:

Legend

  • Normal boxes represent files. You will find these files in the source under vitess/go/vt/tabletserver.
  • Double-lined boxes represent packages external to tabletserver.
  • Cloudy boxes represent native go packages.

Difference between vtocc and tabletserver: vtocc is the name of the main program and binary. Tabletserver is a go package that implements most of vtocc's functionality.

Details

vtocc has the following broad areas:

  • Control: Launches and sets up the tabletserver to serve queries.
  • Communication: Listens for messages, decodes requests, and calls corresponding rpc methods in sqlquery.
  • Query Analysis: Loads the schema, parses queries, and builds plans.
  • Connection pools: Supply the necessary connection to MySQL for execution.
  • Execution: Use the connection to execute commands in MySQL.

If you look at the diagram above, a query starts at the top, and goes in a counter-clockwise direction to the point where it's executed.

One thing that is not shown is the query cache: Every query that contains bind variables will go into a cache. If a query is received that is already in the cache, then we reuse the plan in the cache, thereby skipping the entire query analysis phase.

Other than the broad areas, you will notice two stray boxes:

  • comments: Strips trailing comments for improved query cache hit.
  • consolidator: Before sending a query to MySQL, tabletserver checks with the consolidator to see if an identical query is running. If it is, then the request just waits for the other query to finish, and returns the same result.

Connection Pools

The way tabletserver uses connection pools could get confusing. Here's some explanation:

There are actually three main connection pools in tabletserver. However, the ReservedPool is reserved for future use:

  • ConnPool: Connections are pulled out of this pool for sessionless reads, and returned there once the query has executed.
  • TxPool: Connections are pulled out of this pool when tabletserver receives a begin, and returned on commit or rollback.
  • ReservedPool: Allows a client to reserve a connection for reuse.
  • ActiveTxPool: All connections that are in a transaction are moved into this pool. These may come from TxPool or ReservedPool. A goroutine watches over this pool and aborts transactions that are open for too long.
  • ActivePool: Any connection that is currently executing a query with MySQL goes into this pool. A goroutine watches over this pool and aborts queries that run for too long. In reality, this is not actually a pool. It just tracks queries and connections ids to send kill messages to MySQL.

Comment by naushe...@gmail.com, Apr 22, 2012

What does the Row Cache sub-system do ?

Comment by project member sougou@google.com, Apr 22, 2012

The row cache should dramatically increase the throughput of your mysql box. It's currently not enabled because we're still testing & tuning it. We'll send out an official announcement to vitess@googlecode.com when it's ready.

Comment by albertwz...@gmail.com, Dec 29, 2012

What does OCC stands for here?

Comment by project member sougou@google.com, Dec 29, 2012

Object Connection Cache. The first version of vtocc was launched as a connection pooling mechanism. It now does a lot more.


Sign in to add a comment
Powered by Google Project Hosting