My favorites | Sign in
Project Home Issues Source
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 138: Quod Libet doesn't match FreeDesktop directories specs
19 people starred this issue and may be notified of changes. Back to list
Status:  Open
Owner:  ----


Sign in to add a comment
 
Reported by thibaut....@gmail.com, Feb 24, 2009
I've tried Quod Libet 2.0 on Ubuntu 9.04 alpha 4
and it seems that Quod Libet places its configuration files in
/home/.quodlibet directory which doesn't match FreeDesktop directories specs :

The default for $XDG_CONFIG_HOME is $HOME/.config, the default for
$XDG_DATA_HOME is $HOME/.local/share. So all applications should look for
those environment variables and use those default values if the variables
are not set.

This is quite important since it's not possible to easily backup config
& data files if these files are not stored at the right place

See http://www.freedesktop.org/wiki/Specifications/basedir-spec
See also
http://ploum.frimouvy.org/?184-cleaning-user-preferences-keeping-user-data
(main post and comment#8)
Mar 7, 2009
#1 fverschelde
Hello,

First I would like to add support for this request.

Implementing the Freedesktop specs for this means that two problems must be solved:
1. Discerning what is "user-specific data files" and "user-specific configuration".
2. Implementing the standard while remaining backwards compatible (i.e.: what happens
if a user upgrades from 2.0 to say 2.1, where 2.1 follows the Freedesktop XDG Base
Directory Specification).

FIRST ISSUE: WHAT GOES WHERE?

Quod Libet would need to put "user-specific data files" in $XDG_DATA_HOME/quodlibet/
or, if $XDG_DATA_HOME is not defined, in $HOME/.local/share/quodlibet/.
It would need to put "user-specific configuration" in $XDG_CONFIG_HOME/quodlibet/ or,
if $XDG_CONFIG_HOME is not defined, in $HOME/.config/quodlibet/.

But what is "user-specific data files" and what is "user-specific configuration"?
Configuration is anything that is defined by the user but for which there is a
predefined sensible default in the application. If it is lost, the user might to
configure a few things to get their previous setup back, but nothing of value will be
lost.
Data is anything that is produced by the activity of the user. For example: song
ratings, number of plays (if Quod Libet tracks those), playlists.

SECOND ISSUE: BACKWARD COMPATIBILITY

My take on this is that most applications that want to switch (from storing files in
$HOME/.appname/ to storing them in
$XDG_CONFIG_HOME/.appname/+$XDG_DATA_HOME/.appname) should do it like this: when
looking for data, it should first check if there is a $HOME/.appname/ directory (and
relevant files maybe). If it's there, it could either copy it to the new location, or
continue using it. If it's not there, it would then use the Freedesktop-compliant
locations.

It basically means that users upgrading won't lose configuration or data, and for new
users Quod Libet will follow the XDG Base Directory Specification.

(I have no opinion on whether the application, if the $HOME/.appname/ directory
exists, should keep using it, or copy its content to new locations and use those, or
move its content to new locations and use them. First solution seems simpler to
implement, the third one is maybe cleaner.)

Mar 7, 2009
#2 fverschelde
Two more things:

1. There is a move towards supporting this Freedesktop specification. (Which is
logical if application developers agree that Freedesktop specs are "a good thing" and
strive to adopt them. I'm not sure if there's a very strong consensus towards
Freedesktop specs, though.) Right now most applications still use a $HOME/.appname
directory, or worse (VMWare uses a $HOME/vmware directory, annoying as hell). But
things are on the move. Applications that support this spec since at least
summer/fall 2008 include VLC, Totem, Sound Juicer, Transmission and others.

2. I played with the Quod Libet source (from a SVN checkout). From my experiments,
very basic support for the Freedesktop spec implies:
- removing the USERDIR constant defined in quodlibet/const.py;
- replacing it with two constants, for instance USERCONFDIR and USERDATADIR;
- replacing every occurrence of USERDIR in quodlibet source with either USERCONFDIR
and USERDATADIR.

I did just that (around 25 occurrences of USERDIR to replace), using those constants:

USERCONFDIR = os.path.join(HOME, ".config", "quodlibet")
USERDATADIR = os.path.join(HOME, ".local", "share", "quodlibet")

Which means I'm not checking whether there is a $HOME/.quodlibet directory already,
or doing fancier stuff (I'm not a developer, I'm just learning a bit of Python but
mostly for web development with Django). Well, I tried this and it worked fine.

With my limited knowledge of both Python and Quod Libet, I should be able to submit a
patch for implementing the first solution I suggested (checking if $HOME/.quodlibet
exists and use it if it does, and use the standard directories otherwise). I could
also ask for some information from people who understand the Freedesktop spec better
than I do, so that I may know where different data should go.

Tell me what you think. :)
Mar 9, 2009
Project Member #3 reiter.christoph@gmail.com
Imho, before any coding someone should decide which files
go in which directory.

GNOME wiki quote:
"if you remove the .cache folder, the user will not notice it except maybe
for performance (might be huge performance). If you remove the .config folder,
the user will see all the preferences reset to the default but without any
data loss. In fact, it would not be a problem if the user didn't customize
his desktop."

The simplest, less error prone migration would be: only use the new schema
if there is no %HOME/.quodlibet or if there is a $XDG_CONFIG_HOME/.quodlibet
directory.

Here is a list of all used files in QL:
01) All plugin files in XXX/plugins/(playorder|songsmenu|editing|events)
02) Dump/MiniDump files
03) accels file
04) queue file
05) songinfo file
06) control file
07) config file
08) current file
09) songs file
10) logs folder (*.log files in XXX/logs/)
11) list folder:
    tagpatterns / tagpatters.saved
    renamepatterns / renamepatterns.saved
    queries /queries.saved
12) stations file
13) playlists file
14) browsers folder (containes py/pyc files)
15) feeds file
16) devices file
17) cache file

The directories could be different if he environmen variable is set...
I guess this is right:

DATADIR = os.path.join(os.getenv("XDG_DATA_HOME", HOME, ".local", "share"), 'quolibet')
CONFIDIR = os.path.join(os.getenv("XDG_CONFIG_HOME", HOME, ".config")), 'quolibet')
CACHEDIR = os.path.join(os.getenv("XDG_CACHE_HOME", HOME, ".cache")), 'quolibet')

Mar 9, 2009
Project Member #4 reiter.christoph@gmail.com
Found some more.. here is a first attempt.
Feedback please.

01) All plugin files in XXX/plugins/(playorder|songsmenu|editing|events) -> data
02) Dump/MiniDump files -> data
03) accels file -> config
04) queue file -> data
05) songinfo file -> data
06) control file -> data
07) config file -> config
08) current file -> data
09) songs file -> data
10) logs folder (*.log files in XXX/logs/) -> data/logs
11) list folder:
    tagpatterns / tagpatters.saved
    renamepatterns / renamepatterns.saved
    queries /queries.saved -> config/list
12) stations file -> data
13) playlists file -> data
14) browsers folder (containes py/pyc files) -> data/browsers
15) feeds file -> data
16) devices file -> data
17) cache file -> cache
18) cover.png -> data (notify plugin)
19) album_pattern -> config (album browser)
20) scrobbler_cache -> cache (qlscrobbler plugin)

Apr 5, 2009
#5 fverschelde
Christoph, thanks for that work. I had missed the “cache” part.

Ploum, cited in Thibault's opening message, has a follow-up post which presents the
distinction between user data, user configuration and cache in plain English:
http://ploum.frimouvy.org/?207-modify-your-application-to-use-xdg-folders

Here is some feedback for your list of Quod Libet data. I must say i'm not a
developer per se, and don't know a lot about Quod Libet, though. ;)

Note 1: when my choice differs from yours, i indicate yours in parenthesis.
Note 2: sometimes i'm not sure what a file is about, so i only indicate your choice.
I'm not on a Linux computer right now so i can't check those files.
Note 3: i don't write subdirectories in my feedback. Of course, once a category (user
data/user config/cache) is chosen, the files can go in subdirectories like "list" or
whatever's needed.

01) All plugin files in XXX/plugins/(playorder|songsmenu|editing|events) -> CONFIG?
(data)
02) Dump/MiniDump files -> not sure what it is (data)
03) accels file -> not sure what it is (config)
04) queue file -> CONFIG (data), whereas default setting is "empty queue"
05) songinfo file -> not sure what it is (data)
06) control file -> not sure what it is (data)
07) config file -> CONFIG, obviously
08) current file -> CACHE (data)
09) songs file -> DATA, could have been CACHE if it did not contain metadata that
cannot be retrieved by refreshing the library (like song ratings, number of plays, etc.)
10) logs folder (*.log files in XXX/logs/) -> CACHE? (data/logs) since it's not user
data per se
11) list folder:
    tagpatterns / tagpatters.saved -> CONFIG
    renamepatterns / renamepatterns.saved -> CONFIG
    queries /queries.saved -> CONFIG
12) stations file -> DATA, actually could be config as well but if you only want to
backup your user data you'll probably want to backup this
13) playlists file -> DATA, actually could be config as well but if you only want to
backup your user data you'll probably want to backup this
14) browsers folder (containes py/pyc files) -> not sure (data/browsers)
15) feeds file -> not sure (data)
16) devices file -> CONFIG? (data)
17) cache file -> CACHE
18) cover.png (notify plugin) -> CACHE (data), like current file that's volatile data
19) album_pattern (album browser) -> CONFIG
20) scrobbler_cache (qlscrobbler plugin) -> CACHE

I'll have to revise/complete this.

ON A SIDE NOTE: i guess a quick 'n dirty solution would be to move HOME/.quodlibet to
HOME/.config/quodlibet, and not to sort between user data, user preference and cache.
I can see this could be acceptable (though non conforming) for the following reasons:

- Quod Libet doesn't generate lots of user data. The only data that's clearly uer
data is playlists, and part of the songs database IF the user stores information in
the songs database that can't be retrieved from the files themselves. The main user
data quodlibet uses and even writes (through Ex Falso) is music files, and those are
managed by the user in a non-hidden directory or other places.
- Quod Libet and its plugins use little cache (not sure how the cache file itself is,
but i think it's quite small), so going to the extent of using a specific cache
folder might be overkill.
Jun 18, 2009
#6 steven.strobe.cc@gmail.com
Since QL won't be switching names, this change should not be rushed; I'd prefer to
wait on this one until after the rather large list of changes in 2.1 have been tested
and released.
Labels: Milestone-longterm
Jun 18, 2009
#7 fverschelde
Long term sounds good if that means work on this issue is scheduled and it gets done
eventually. :)
Nov 23, 2010
#8 jason5...@gmail.com
I'd really like to see this change. this issue exist for almost 2 years but not implement...

I don't think the config files are that important that application can not switch the restore place.

This can be done in a major version change. just a warning window would be good enough. if there is any problem, user can easily solve them by google it. 

Acturally i think change like this should implement as soon as possible, cuz this application is more and more widely used.

And apps like AWN, cairo-dock, rhythmbox didn't wait that long.
Nov 23, 2010
#9 jason5...@gmail.com
Patch: Simply make ~/.quodlibet to $XDG_CONFIG_HOME. use glib way to do this job will be much easier.
xdg.patch
676 bytes   View   Download
Nov 23, 2010
Project Member #10 joe.wreschnig@gmail.com
That patch makes it impossible for me to have ~/.quodlibet, which is what I want.
Nov 23, 2010
#11 jason5...@gmail.com
Is there a big different to access ~/.quodlibet than ~/.config/quodlibet? if you want you can make soft link.

Do you like to see a huge number of hiden files when enable show hiden files in filemanager or use tab to complete command? It's in your $HOME dir so you have to see them a lot no matter you want to or not.
Nov 23, 2010
Project Member #12 joe.wreschnig@gmail.com
I don't see the value in having half my files in ~/.config and half my files in ~/. I don't find value in change for the sake of change. I don't find any arguments for ~/.config persuasive. (Is the only argument "We can crappy up this folder instead of that folder?")

I've been using Unix for 18 years now and I've never been bothered by having rc files in my home directory, at least not in any way that moving them all to another directory would help, and certainly not in a sense that moving some random subset to another directory would help.
Nov 23, 2010
#13 thibaut....@gmail.com
One of the advantage of move is to allow users to easily backup their system choosing to restore only their data or their config or both easily.
It can't be done easily at the moment.
It will be very easy to create backup tools that will provide that kind of choice then.
Actually, it is a move to allow users to better handle its files to my mind.
it is definitely worth it.
Nov 23, 2010
#14 jason5...@gmail.com
example: you can easily clean up cache files by delete ~/.cache dir, if every apps match XDG standards, or it would be really hard for a user to do taht.

besides, there is no advantage to put all files into $HOME, either. so why not just match XDG standards?

@thibaut.bethune, actually it woundn't be done easily at any moments, and if more and more user using this, it would be harder and harder to handle all the situation.
Nov 24, 2010
Project Member #15 joe.wreschnig@gmail.com
Why is there this arbitrary distinction between "config" and "data"? When am I going to be in a situation where I don't want to back up both? It seems like a very un-Unix split also - what's my ~/.emacs? Surely it's a configuration file, but it contains more code than many programs!

Why is there no way to configure all XDG-aware apps to behave as applications have for decades? (There's trivial ways to do this, like using XDG_CONFIG_ROOT as a string prefix rather than a parent directory, but the standard forbids them.)

Why did XDG not standardize on existing practice, as was the common way for e.g. the LSB and other open and closed standards bodies to work?

Changes like this contribute to nothing but bitrot of otherwise stable software and unnecessary code churn for active ones. This whole thing reminds me of how GConf was going to be the be-all-end-all solution for configuration storage some years ago, and it was vitally important for application authors to support that.
Nov 24, 2010
#16 thibaut....@gmail.com
Traditionally programs don't make a clear distinction between config and data (and cache) files therefore the move can't be automated.
It would have been possible if there still were /.config and /.data subfolders in any ~/.program folder but it is not the case.
The spec asks developpers to make that distinction obvious.
That lloks like a progress to me since a better classification will allow new developpements that weren't possible before.
Nov 24, 2010
Project Member #17 joe.wreschnig@gmail.com
No, traditionally programs don't make a clear distinction between config and data because they're not different, and to pretend they are different is to pretend programmable, extensible, really configurable applications don't exist.

Traditionally, programs don't write cache files, because cache files only matter for the small set of programs that download large resources over slow links. (I think the only thing QL caches are thumbnails - which there's a separate fd.o spec for, and it doesn't use ~/.cache either. Great thing about standards, etc.) (The scrobbler cache file is identified as a cache above, but it's not, it's data.)

And there's no place in the XDG spec for QL's control, songinfo, cover.png, and other interface files. Their lifetime would be appropriate for a cache folder, but they're not cache files.

This is change, not progress.
Nov 24, 2010
#18 thibaut....@gmail.com
If my memory is correct, Rhythmbox treats covers as cache since the application automatically downloads them from the Internet.
Therefore if you delete them, they won't miss you since the application will bring them back. It will only take some more time : like caching. No data will be lost from that point of view.
Nov 24, 2010
Project Member #19 joe.wreschnig@gmail.com
Quod Libet writes cover files alongside the audio files, where they should be. (Or in metadata tags.) The cover cache is to avoid loading sometimes extremely large files off slow media and rescaling. That's identical in purpose to the thumbnail cache, a standard for which already exists, and is not in ~/.cache. I don't understand why you brought that up though.
Nov 24, 2010
#20 thibaut....@gmail.com
There is indeed a conflit between the 2 specs : Thumbnail Managing Standard and FreeDesktop directories.

I only wanted to help by answering "there's no place in the XDG spec for QL's control, songinfo, cover.png, and other interface files. Their lifetime would be appropriate for a cache folder, but they're not cache files."

If you don't think that this spec can give the user the ability to control its configuration then i guess you should mark this bug as wontfix.
Since you are the developer, you decide : no offense :-)



Sign in to add a comment

Powered by Google Project Hosting