|
MacOSMetadata
Content related to metadata under the Mac OS
Mac OS X Archival ExperimentIt seems that not all metadata is archived by every archiving tool available for Mac OS X, and tools that claim to be able to archive a piece of metadata may not do so with full fidelity. What Does It Involve?
Caveats and Historical IssuesNote: Some of these problems have been fixed in later versions, or have been worked around in some manner.
Mac OS (X) File Type and Creator CodesBrief History
Implementation
com.apple.FinderInfo: 0000 57 58 42 4E 4D 53 57 44 00 00 00 00 00 00 00 00 WXBNMSWD........ 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ # Finder Info fdType = 0x5758424e (WXBN) fdCreator = 0x4d535744 (MSWD) fdFlags = 0000000000000000 fdLocation = (v = 0, h = 0) opaque = 0 NTFS Streams and MS-DOS/Windows Flags on HFS+ (via Samba)
:SUMMARYINFORMATION:$DATA:
0000 FE FF 00 00 05 01 02 00 00 00 00 00 00 00 00 00 ................
0010 00 00 00 00 00 00 00 00 01 00 00 00 E0 85 9F F2 ................
0020 F9 4F 68 10 AB 91 08 00 2B 27 B3 D9 30 00 00 00 .Oh.....+'..0...
0030 28 00 00 00 02 00 00 00 01 00 00 00 18 00 00 00 (...............
0040 00 00 00 80 20 00 00 00 02 00 00 00 E4 04 00 00 .... ...........
0050 13 00 00 00 09 08 00 00 ........
:{4C8CC155-6C1E-11D1-8E41-00C04FB9386D}:$DATA:
0000 00 .
:DOCUMENTSUMMARYINFORMATION:$DATA: 0000 FE FF 00 00 05 01 02 00 00 00 00 00 00 00 00 00 ................ 0010 00 00 00 00 00 00 00 00 01 00 00 00 02 D5 CD D5 ................ 0020 9C 2E 1B 10 93 97 08 00 2B 2C F9 AE 30 00 00 00 ........+,..0... 0030 4C 00 00 00 03 00 00 00 01 00 00 00 28 00 00 00 L...........(... 0040 00 00 00 80 30 00 00 00 02 00 00 00 38 00 00 00 ....0.......8... 0050 00 00 00 00 00 00 00 00 02 00 00 00 E4 04 00 00 ................ 0060 13 00 00 00 09 08 00 00 1E 00 00 00 09 00 00 00 ................ 0070 43 61 74 65 67 6F 72 79 00 00 00 00 Category.... Samba Metadata Options
Cygwin File Attributes (NTEA)
user..UNIXATTR: 0000 FF 01 00 00 .... Access Control Lists (ACLs)
# Catalog File Record
type = file
file ID = 3578569
flags = 0000000000001110
. File has a thread record in the catalog.
. File has extended attributes.
. File has security data (ACLs).
reserved1 = 0
createDate = Mon Dec 28 17:22:31 2009
contentModDate = Mon Dec 28 17:25:17 2009
attributeModDate = Wed Dec 30 00:34:07 2009
accessDate = Wed Dec 30 00:33:12 2009
backupDate = 0
# BSD Info
ownerID = 501 (tyson)
groupID = 20 (staff)
adminFlags = 00000000
ownerFlags = 00000000
fileMode = ----------
linkCount = 1
textEncoding = 0
attrBlocks = 0
# Attribute Key
keyLength = 62
pad = 0
fileID = 3578569
startBlock = 0
attrNameLen = 25
attrName = com.apple.system.Security
# Inline Data
recordType = 0x10
reserved[0] = 0
reserved[1] = 0
attrSize = 140 bytes
attrData = 01 2c c1 6d 00 00 00 00 00 00 00 00 00 00 00 00
, m
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 04 00 00 00 00 47 ec b3 f3
G
76 19 4d 40 b7 0d 37 14 17 d1 81 db 00 00 00 01
v M @ 7
00 00 0f a6 ab cd ef ab cd ef ab cd ef ab cd ef
00 00 00 14 00 00 00 01 00 00 0a 82 ab cd ef ab
cd ef ab cd ef ab cd ef 00 00 00 0c 00 00 00 01
00 00 0a 82 ab cd ef ab cd ef ab cd ef ab cd ef
00 00 00 01 00 00 00 01 00 00 3f fe
?
# File Security Information
fsec_magic = 0x12cc16d
fsec_owner = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
fsec_group = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
# ACL Record
acl_entrycount = 4
acl_flags = 0
# ACL Entry
ace_applicable = 47 ec b3 f3 76 19 4d 40 b7 d 37 14 17 d1 81 db
user = tyson
uid = 501
group = *
ace_flags = 00000000000000000000000000000001 (0x000001)
. KAUTH_ACE_PERMIT
ace_rights = 00000000000000000000111110100110 (0x000fa6)
. KAUTH_VNODE_READ_DATA
. KAUTH_VNODE_WRITE_DATA
. KAUTH_VNODE_APPEND_DATA
. KAUTH_VNODE_READ_ATTRIBUTES
. KAUTH_VNODE_WRITE_ATTRIBUTES
. KAUTH_VNODE_READ_EXTATTRIBUTES
. KAUTH_VNODE_WRITE_EXTATTRIBUTES
. KAUTH_VNODE_READ_SECURITY
# ACL Entry
ace_applicable = ab cd ef ab cd ef ab cd ef ab cd ef 0 0 0 14
user = *
group = staff
gid = 20
ace_flags = 00000000000000000000000000000001 (0x000001)
. KAUTH_ACE_PERMIT
ace_rights = 00000000000000000000101010000010 (0x000a82)
. KAUTH_VNODE_READ_DATA
. KAUTH_VNODE_READ_ATTRIBUTES
. KAUTH_VNODE_READ_EXTATTRIBUTES
. KAUTH_VNODE_READ_SECURITY
# ACL Entry
ace_applicable = ab cd ef ab cd ef ab cd ef ab cd ef 0 0 0 c
user = *
group = everyone
gid = 12
ace_flags = 00000000000000000000000000000001 (0x000001)
. KAUTH_ACE_PERMIT
ace_rights = 00000000000000000000101010000010 (0x000a82)
. KAUTH_VNODE_READ_DATA
. KAUTH_VNODE_READ_ATTRIBUTES
. KAUTH_VNODE_READ_EXTATTRIBUTES
. KAUTH_VNODE_READ_SECURITY
# ACL Entry
ace_applicable = ab cd ef ab cd ef ab cd ef ab cd ef 0 0 0 1
user = daemon
uid = 1
group = daemon
gid = 1
ace_flags = 00000000000000000000000000000001 (0x000001)
. KAUTH_ACE_PERMIT
ace_rights = 00000000000000000011111111111110 (0x003ffe)
. KAUTH_VNODE_READ_DATA
. KAUTH_VNODE_WRITE_DATA
. KAUTH_VNODE_EXECUTE
. KAUTH_VNODE_DELETE
. KAUTH_VNODE_APPEND_DATA
. KAUTH_VNODE_DELETE_CHILD
. KAUTH_VNODE_READ_ATTRIBUTES
. KAUTH_VNODE_WRITE_ATTRIBUTES
. KAUTH_VNODE_READ_EXTATTRIBUTES
. KAUTH_VNODE_WRITE_EXTATTRIBUTES
. KAUTH_VNODE_READ_SECURITY
. KAUTH_VNODE_WRITE_SECURITY
. KAUTH_VNODE_TAKE_OWNERSHIP
cacls "Y:\New Text Document.txt"
Y:\New Text Document.txt <Account Domain not found>(special access:)
READ_CONTROL
SYNCHRONIZE
FILE_GENERIC_READ
FILE_GENERIC_WRITE
FILE_READ_DATA
FILE_WRITE_DATA
FILE_APPEND_DATA
FILE_READ_EA
FILE_WRITE_EA
FILE_READ_ATTRIBUTES
FILE_WRITE_ATTRIBUTES
BUILTIN\Users:(special access:)
READ_CONTROL
SYNCHRONIZE
FILE_GENERIC_READ
FILE_READ_DATA
FILE_READ_EA
FILE_READ_ATTRIBUTES
Everyone:(special access:)
READ_CONTROL
SYNCHRONIZE
FILE_GENERIC_READ
FILE_READ_DATA
FILE_READ_EA
FILE_READ_ATTRIBUTES
NT AUTHORITY\SYSTEM:FMac OS X Smart Folders
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>CompatibleVersion</key> <integer>1</integer> <key>RawQuery</key> <string>((true) && (((* = "test*"cdw || kMDItemTextContent = "test*"cdw)))) && ((* = "test*"cdw || kMDItemTextContent = "test*"cdw))</string> <key>RawQueryDict</key> <dict> <key>FinderFilesOnly</key> <true/> <key>RawQuery</key> <string>((true) && (((* = "test*"cdw || kMDItemTextContent = "test*"cdw)))) && ((* = "test*"cdw || kMDItemTextContent = "test*"cdw))</string> <key>SearchScopes</key> <array> <string>kMDQueryScopeComputer</string> </array> <key>UserFilesOnly</key> <true/> </dict> <key>SearchCriteria</key> <dict> <key>AnyAttributeContains</key> <string>test</string> <key>CurrentFolderPath</key> <array> <string>/Users/tysonkey</string> </array> <key>FXCriteriaSlices</key> <array> <dict> <key>criteria</key> <array> <string>com_apple_UserSearchStringAttribute</string> <integer>104</integer> </array> <key>displayValues</key> <array> <string>Items matching text</string> <string>test</string> </array> <key>rowType</key> <integer>0</integer> <key>subrows</key> <array/> </dict> </array> <key>FXScope</key> <integer>1396925814</integer> <key>FXScopeArrayOfPaths</key> <array> <string>kMDQueryScopeComputer</string> </array> </dict> </dict> </plist> Mac OS X 10.5 Finder/Spotlight Comments
com.apple.metadata:kMDItemFinderComment: 0000 62 70 6C 69 73 74 30 30 5F 10 11 53 70 6F 74 6C bplist00_..Spotl 0010 69 67 68 74 20 43 6F 6D 6D 65 6E 74 08 00 00 00 ight Comment.... 0020 00 00 00 01 01 00 00 00 00 00 00 00 01 00 00 00 ................ 0030 00 00 00 00 00 00 00 00 00 00 00 00 1C ............. <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <string>Spotlight Comment</string> </plist>
Experiment, Part 1
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <string>Hand-crafted Comment</string> </plist>
Experiment, Part 2
HexedComment.plist: XML parser error: Unexpected character b at line 1 Old-style plist parser error: Unexpected ';' or '=' after key at line 1 Mac OS X Finder Colour LabelsRed
com.apple.FinderInfo: 0000 00 00 00 00 00 00 00 00 00 0C 00 00 00 00 00 00 ................ 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ # Finder Info
fdType = 0
fdCreator = 0
fdFlags = 0000000000001100
. Color label = red
fdLocation = (v = 0, h = 0)
opaque = 0Orange
com.apple.FinderInfo: 0000 00 00 00 00 00 00 00 00 00 0E 00 00 00 00 00 00 ................ 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ # Finder Info
fdType = 0
fdCreator = 0
fdFlags = 0000000000001110
. Color label = orange
fdLocation = (v = 0, h = 0)
opaque = 0Yellow
com.apple.FinderInfo: 0000 00 00 00 00 00 00 00 00 00 0A 00 00 00 00 00 00 ................ 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ # Finder Info
fdType = 0
fdCreator = 0
fdFlags = 0000000000001010
. Color label = yellow
fdLocation = (v = 0, h = 0)
opaque = 0Green
com.apple.FinderInfo: 0000 00 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00 ................ 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ # Finder Info
fdType = 0
fdCreator = 0
fdFlags = 0000000000000100
. Color label = green
fdLocation = (v = 0, h = 0)
opaque = 0Internet-Originated Files MetadataQuarantined Files
com.apple.quarantine: 0000;4b392bb2;Firefox;|org.mozilla.firefox com.apple.quarantine: 0000;4b38d820;Firefox;|org.mozilla.firefox Opaque Quarantine IDsNOTE: This information may not be accurate, and is currently considered to be of draft quality.
com.apple.quarantine: 0000;4b3a40d0;Safari;|com.apple.Safari com.apple.quarantine: 0000;4b3a4107;Safari;|com.apple.Safari Safari File Origin Info
com.apple.metadata:kMDItemWhereFroms: 0000 62 70 6C 69 73 74 30 30 A1 01 5F 10 1F 68 74 74 bplist00.._..htt 0010 70 3A 2F 2F 77 77 77 2E 61 70 70 6C 65 2E 63 6F p://www.apple.co 0020 6D 2F 73 74 61 72 74 70 61 67 65 2F 08 0A 00 00 m/startpage/.... 0030 00 00 00 00 01 01 00 00 00 00 00 00 00 02 00 00 ................ 0040 00 00 00 00 00 00 00 00 00 00 00 00 00 2C ............., <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <array> <string>http://www.apple.com/startpage/</string> </array> </plist> com.apple.metadata:kMDItemWhereFroms: 0000 62 70 6C 69 73 74 30 30 A1 01 5F 10 49 68 74 74 bplist00.._.Ihtt 0010 70 3A 2F 2F 69 6D 61 67 65 73 2E 61 70 70 6C 65 p://images.apple 0020 2E 63 6F 6D 2F 73 74 61 72 74 70 61 67 65 2F 69 .com/startpage/i 0030 6D 61 67 65 73 2F 32 30 30 39 2F 30 39 2F 70 72 mages/2009/09/pr 0040 6F 6D 6F 2D 74 6F 75 63 68 2D 32 30 30 39 30 39 omo-touch-200909 0050 30 39 2E 6A 70 67 08 0A 00 00 00 00 00 00 01 01 09.jpg.......... 0060 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 ................ 0070 00 00 00 00 00 00 00 56 .......V A Mac OS 7.0.1 Alias
Mac OS 7.0.1 User, Group, and Member filesMac OS X 10.6 BookmarksUniversal Disk FormatExtended AttributesFIFOs, Block Nodes and Character NodesFinder Aliases/BookmarksWhiteout Nodes
Web Internet Location files (Mac OS X 10.6)
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>URL</key> <string>http://www.google.co.uk/</string> </dict> </plist> 00000000 00 00 01 00 00 00 01 6C 00 00 00 6C 00 00 00 5A |.......l...l...Z| 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000000A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000000B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000000C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000000D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000000E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000000F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000100 00 00 00 30 00 00 00 01 00 00 00 00 00 00 00 00 |...0............| 00000110 00 00 00 02 54 45 58 54 00 00 01 00 00 00 00 00 |....TEXT........| 00000120 00 00 00 00 75 72 6C 20 00 00 01 00 00 00 00 00 |....url ........| 00000130 00 00 00 00 00 00 00 18 68 74 74 70 3A 2F 2F 77 |........http://w| 00000140 77 77 2E 67 6F 6F 67 6C 65 2E 63 6F 2E 75 6B 2F |ww.google.co.uk/| 00000150 00 00 00 18 68 74 74 70 3A 2F 2F 77 77 77 2E 67 |....http://www.g| 00000160 6F 6F 67 6C 65 2E 63 6F 2E 75 6B 2F 00 00 01 00 |oogle.co.uk/....| 00000170 00 00 01 6C 00 00 00 6C 00 00 00 5A 00 00 00 00 |...l...l...Z....| 00000180 E0 00 00 00 00 1C 00 5A 00 02 64 72 61 67 00 00 |.......Z..drag..| 00000190 00 1A 54 45 58 54 00 00 00 26 75 72 6C 20 00 00 |..TEXT...&url ..| 000001A0 00 32 00 80 FF FF 00 00 00 00 84 34 00 08 01 00 |.2.........4....| 000001B0 FF FF 00 00 00 34 8C 34 00 08 01 00 FF FF 00 00 |.....4.4........| 000001C0 00 50 90 34 00 08 |.P.4..| 000001c6 |
► Sign in to add a comment
for mac os x files with resourcefork (seen in xttr of file), in the Terminal application enter "strings 'theFileName'/rsrc | grep 'alis'" to confirm that the reg file contains an alias link. ralf.
com.apple.metadata:kMDItemWhereFroms can easily be shown in a human readable form using:
Storing a new value using plain text (rather than the binary plist format) works well too. To see that: download some file, and then run:
Next, open the file in Finder and see that the source URL as shown in the warning has indeed changed. Note that the output format for xattr -p com.apple.metadata:kMDItemWhereFroms mydownload.gz and mdls -name kMDItemWhereFroms mydownload.gz will then be different, but that doesn't seem to break things.
For downloads that start automatically (but, in my Safari, not for downloads initiated by Ctrl-click, Download Linked File As...), kMDItemWhereFroms can actually be an array of both the file origin and its referrer. In Get Info that is shown as a comma-separated list. Using the xattr -w like I commented above will not create an array (and hence mdls will not show the parentheses like it does for a Safari-generated attribute).
For more information on the Alias structure, see http://xhelmboyx.tripod.com/formats/alias-layout.txt That page describes the basic structure, even though a few new fields have been added since. I've used this into to parse OSX's Alias records to get all the info from it that even OSX's FileMgr? API doesn't offer.
The following statement is technically inaccurate on two major points:
1. How POSIX Permissions are Assigned to Newly-Created Items
Saying that "all privileges" are "granted" or "inherited" glosses things over way too much. In fact, only the POSIX group assignment (not its permission bits) are inherited.
POSIX permissions consist of an owner, a group, and an everyone else field, and each field has a set of bits (read, write, execute). They also include a special permissions field with bits for set-UID, set-GID, and sticky. Without going into the special permissions, the following is the correct behavior for a newly-created file or directory:
The POSIX owner shall be set to the action-bearing user. The POSIX owner must be assigned to a user account, not a group. Generally speaking, the action-bearing user is the user who creates the file or directory. This value is NOT inherited from the parent folder where the new item is being created, although it's possible for the owners of the new child and its parent to match, in cases where the action-bearing user is the owner of the parent. This assignment is in name-only; the POSIX permission bits associated with this owner are set as per the umask (see two points below).
The POSIX group is set to the POSIX group assigned to the parent of the newly-created item. As with the POSIX owner, the actual permission bits for the group are set by the umask (see next point).
The POSIX permission bits assigned to the owner, group, and everyone else (other) fields are assigned by the umask variable. By default, Mac OS X uses a umask of 0022, which means that newly-created directories receive POSIX 0755 permissions, and new files receive 0644 ones.
Example: Even if the parent folder's POSIX permissions are different from what would otherwise be assigned by the umask, the umask applies to newly-created children within that folder. To prove this to yourself, create a new folder or save something in the Library folder in your home. The Library folder has POSIX permissions of 0700 (default), but you'll see that newly-created items will be readable by the POSIX group and others (0755 or 0644). In this case, I'm choosing the Library folder as an easy example, not because there's something special about it. You'd also notice similar behavior when creating new items in /private/tmp. That folder has POSIX permissions of 1777 (everyone read and write with the sticky bit set), but new items created there will receive POSIX bits from the umask.
Note again that the above behavior is for newly-created items; POSIX permission bits are preserved when making a copy of the original item, even though the owner and group assignment change in a similar manner.
2. ACL Entries May be Set on a Newly-Created Item
Saying that no ACL is present on a new item isn't always the case, although some applications don't support ACLs, so this behavior can be observed. Also, not all filesystems support ACLs; when the filesystem doesn't meet this necessary condition, ACLs can't be created anyway.
If an item is created within a parent folder and that parent folder has one or more ACL entries set that specify inheritance (via the file_inherit or directory_inherit controls), then that ACL entry will be propagated to the newly-created child file or directory (or both), according to the inheritance specified in the entry on the parent. This propagated entry will be marked as an "inherited" one, because it wasn't explicitly set.
When copying items, inherited ACL entries are not preserved; however, new inheritable entries will be pushed to the copy from its new parent. Explicitly-set ACL entries are preserved on a copied item.
--Gerrit, ACSA
The new Bookmarks stuff in Mac OS X 10.6.x might be worth investigation. It appears as if the file format has significantly changed since 10.5.x...
A note from your wiki administrator:
Thanks for the interest in the project, and for the additional information in the comments section - it's very much appreciated on my part.
If anyone feels like updating the page content to reflect their discoveries and additional information, please feel free to request commit access using the e-mail address provided on the homepage of the project.
I also intend to update this page with additional information, and fix various typographical errors, when I've got time spare.
Tyson.
Regarding quarantine events:
There is also some documentation of the .DS_Store fies at http://search.cpan.org/~wiml/Mac-Finder-DSStore/DSStoreFormat.pod
In order to successfully set the spotlight's comment as shown in experiment part 1 and 2:
xattr -wx "com.apple.metadata:kMDItemFinderComment" "`xxd -ps CustomComment.plist`" NoComment
For com.apple.quarantine: Mac OS X Lion and later, UUID refers to ~/Library/Preferences/com.apple.LaunchServices?.QuarantineEventsV2?
Hi Tyson,
I noticed your section on quarantine for files. The second "block" of data is hex. Convert it to decimal, and then convert the decimal to Safari/Webkit epoch and you get the exact time of download!
Best wishes,
Ryan K
Ryan R. Kubasiak rkubasiak@AppleExaminer?.com BlackBag? Technologies Forensic Analyst and Instructor http://www.AppleExaminer.com @appleexaminer (Twitter)