My favorites | Sign in
Project Home
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 288: VMware VMSS file format support as a new address space - code submission
6 people starred this issue and may be notified of changes. Back to list
Status:  Done
Owner:  ----
Closed:  Jan 2013


Sign in to add a comment
 
Reported by hmi...@gmail.com, Jul 4, 2012
in the past few weeks (well, i may have been postponing this more then a few weeks :) )
I wrote an extension to Volatility's Address space layouts for VMWare's VMSS/VMSN file format, used in newer VMWare products to store the virtual machine's state when suspending or taking a snapshot.

the code I came up with is split in two:
VMSSParser.py is in charge of parsing the VMSS files and providing a generic access to the file's internal data types. I used a different file because i'm unfamiliar with volatility's structure parsing code but mostly because i would like the VMSSParser to be independent and used outside Volatility.
vmware.py implements the VMWareSnapshotFile address space. it just reads the VMSS file using VMSSParser to extract the memory regions for the virtual machine and provide an interface to the VMSSParser's methods, and then reads the CR0 register of the first core and sets the DTB for upper address spaces to take it from there.

I've used these for some time now and can't find any additional bugs, but it's important to remember VMSSParser was implemented without documentation and thus might not work for some VMSS files.
I used it against the Volatility 2.0 tree as well as the Volatility 2.1 alpha tree (dated to May 24) and it seems to work without any glitches.
I tried to write it as commented as possible.
VMSNParser.py
16.1 KB   View   Download
vmware.py
6.5 KB   View   Download
Jul 4, 2012
Project Member #1 mike.auty@gmail.com
Thanks very much, these look awesome!!!

I'll try to dig up a vmss file to run some tests, and give you some feedback.  It's probably a little too late to make it into 2.1, but hopefully they'll make their debut in 2.2!  5:)
Jul 4, 2012
Project Member #2 mike.auty@gmail.com
(No comment was entered for this change.)
Labels: Milestone-2.2.x
Jul 4, 2012
#3 Nir...@gmail.com
sounds great, i hope they'd be helpful for someone.

i'd love any feedback i can get my hands on, especially from an official developer :)
Jul 4, 2012
Project Member #4 michael.hale@gmail.com
Yes very cool, thank you for the contribution. Coincidentally there was some discussion about parsing vmss files with volatility earlier this week: http://lists.volatilesystems.com/pipermail/vol-users/2012-July/000470.html


Jul 6, 2012
#5 scude...@gmail.com
It looks to me like the address space is not implementing zread()
properly (or at all). Can you please make sure that you are
implementing zread() in such a way that when you read outside a valid
or mapped region you will receive a null padded buffer rather than
None?

Some more comments about the address space VMWareSnapshotFile:
   - Please do not use inner classes. There is no need to have a class
defined in such a way - just place the class at the module level. Also be consistent regarding class names (prefer CamelCase for classes).

   - Minor style issues - long lines should be wrapped at 80 chars,
commented out lines should be removed.

   - Do no use double underscore member variable names (they mean
something specific (non inheritable) e.g. self.__hasseek). Since the seek/read interface is totally different to the read(offset, length) interface I would not try to support both in the same class. I would just have 2 separate classes (one can inherit from the other).

   - It would also be nicer if we used the volatility object system
rather than struct module directly for parsing these things - it would
make the file formats more readable and simplify the code a lot. I can help you with that if you like to send me the documentation of the file format, or we can refactor it later.
Aug 26, 2012
Project Member #6 michael.hale@gmail.com
(No comment was entered for this change.)
Labels: -Milestone-2.2.x Milestone-2.3.x
Sep 18, 2012
#7 Nir...@gmail.com
I opened a google code project for the vmsn parser, you can find it at https://code.google.com/p/vmsnparser/
hopefully sometime soon i'll start writing the actual format specs in the wiki but i'll have to make my notes more readable first :-)

I've also made a few fix-ups in the code, mostly to the volatility interface.
it should now support all the required methods according to the interface changes.
to do that i used two standard ASes from scudette's code branch. Mike Auty told me on IRC that including them won't be a problem.
These ASes are: RunBasedAddressSpace, PagedReader from branches/scudette/volatility/addrspace.py
to be able to use these you need to copy these classes from scudette's branch to the matching file in working copy.
I got to test it a bit using the single VMSN file i had on my laptop, but i can only test it properly at work (which is actually in a few hours :) )

As for scudette's comments:
 - Inner class was removed, regions are now implemented as they should in the runs member inherited from RunBasedAddressSpace.

 - haven't done this yet, but I also have other small code changes to make (most notably replace "BLAH" with actual error messages for as_assert()s i never got)

 - removed. i now directly use the file handler from the base AS (I assume it's a FileAddressSpace) but actually i'm not sure that's the best way to go...

 - As i said before, i'd like the file parser to be used outside the volatility framework and I don't want to have to maintain both code pieces. I don't see any way to make that possible using the volatility object system. I believe there won't be many changes once the parser is more mature, so we could probably work something out later.

We might want to move this discussion to the vmsnparser project, but I don't really mind continuing it here if you guys prefer.

 - Nir
Oct 8, 2012
Project Member #8 michael.hale@gmail.com
Hey Nir, 

Thanks a bunch. I starred your project ;-) Also, this is on the roadmap for 2.3 (next volatility version) so we may be reaching out to you for some questions or testing as we work to implement your parser. Talk to you soon!
Jan 31, 2013
Project Member #9 michael.hale@gmail.com
Hey Nir, 

Your VMware address space has made it into the volatility core, in time for the 2.3 release: 

https://code.google.com/p/volatility/source/browse/trunk/volatility/plugins/addrspaces/vmware.py

So look out for proper credits in the release notes, wiki, blog, etc. Also, everything below the line "FIXME: everything below can be abstract - shared with virtualbox" will end up being removed before the release. 

Thanks again, great work!
Status: Done
Feb 1, 2013
#10 Nir...@gmail.com
Cool!

I already noticed the FIXME and was wondering how long it's gonna last there :P

please let me know if there's anything i can further assist with in the future.
bugs/new versions etc...

I hope it'll be put to some good use :)
Feb 1, 2013
#11 Nir...@gmail.com
It looks good btw, waiting until i can try it for the first time :)
Sign in to add a comment

Powered by Google Project Hosting