From keirangtp on April 19, 2011 05:17:13

As a workaround try using "Tools / Rescan whole library" - this will probably remove the duplicates but the process may take some time especially if you have a huge library.

Restarting Clementine doesn't help - the duplicates are still there?

Summary: Clementine creates duplicates in library

From apgrant2 on April 19, 2011 10:46:08

I've tried that in the past and it hasn't worked but I'm hoping it does this time and unfortunately restarting it doesn't help either.

From keirangtp on April 20, 2011 00:58:40

When did you try it? 0.7 has a whole library rescan feature, 0.6 had only "refresh changed folders" feature and that one wouldn't fix it. http://code.google.com/p/clementine-player/wiki/LibraryScanning

From apgrant2 on April 20, 2011 09:39:56

I tried it recently with version 0.7's library rescan feature and no joy. I've also restarted Clementine a bunch and have been regularly updating the version from my update manager as they arise but alas I still have dupes all over the place. It's very strange.

From chuck.smith08 on April 20, 2011 13:00:48

Just saw this issue.....

I reported as issue 1790 and am having the same issue, Re-scan does not work....

Probably should merge these issues....

From davidsansome on April 20, 2011 13:17:02

Issue 1790 has been merged into this issue.

From chuck.smith08 on April 20, 2011 17:50:06

I tracked down a problem that I believe is where the duplicates are coming from. My music folder is:

/media/storage/Music

Inside that folder there is a link to /media/storage/Music, etc. The resulting directory structure is:

/media/storage/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music

I can only assume clementine created but I do not know how or when..

From apgrant2 on April 22, 2011 02:50:09

Hmm.. that does sound plausible considering that when I choose the option to delete the dupes from disc in Clementine it drops an error regarding being able to delete the songs. I assume it's because the songs aren't actually there but are rather somehow linked to the original song. Also however, how or when or why I have no idea.

From apgrant2 on April 22, 2011 03:02:26

A rather inelegant solution depending on how large your collection is to choose the "Organize Files" option in Clementine with "Overwrite existing files" clicked and "Delete the original Files" selected in the "After Copying" section located at the top of the menu. Take note however that I have not tested for unforeseen after effects. So, use at your own risk.

From schizosfera on June 03, 2011 07:26:26

i had a similar problem when duplicates appeared in my library. usually one of the duplicate entries would have an invalid path. that was because of some reason, clementine stored escaped file url's in the database (spaces were escaped as %20). doing library rescans didn't remove any of the invalid entries (which displayed as duplicates). manually deleting the entries from the db fixed my problem.

From schizosfera on June 04, 2011 03:24:26

i just noticed that (mp3, cue) pairs create duplicates in my library (probably when rescanning the library). i'm not sure if it's actually the same issue, but it definitely looks like the same symptom.

Attachment: mp3 with cue duplicates.png

From adam.chrzascik on June 06, 2011 05:36:45

To reproduce the bug(?) on Win7 x64:

Install r3341 version over (not clean install) existing stable 0.7.1 release with library in place. Now rescan the library. For every song you will have two entries in library view, with paths in properties shown differently :

D:\Music\U2\How To Dismantle An Atomic Bomb\01 - Vertigo.mp3
\d\Music\U2\How To Dismantle An Atomic Bomb\01 - Vertigo.mp3

Restart computer. Now paths look like this:

file:\D:\Music\U2\How%20To%20Dismantle%20An%20Atomic%20Bomb\01%20-%20Vertigo.mp3
file:\D:\Music\U2\How To Dismantle An Atomic Bomb\01 - Vertigo.mp3

Now run stable installer again, rescan the library, and you will get third entry with correct path:

D:\Music\U2\How To Dismantle An Atomic Bomb\01 - Vertigo.mp3
file:\D:\Music\U2\How%20To%20Dismantle%20An%20Atomic%20Bomb\01%20-%20Vertigo.mp3
file:\D:\Music\U2\How To Dismantle An Atomic Bomb\01 - Vertigo.mp3

Cheers,

From schizosfera on June 06, 2011 05:51:08

it seems like the escaped file URIs (those containing the %20's )are the new standard entries. a migrate for the other "old" DB is not available yet, but is planned (so i heard) - probably with the next release.

@adam.chrzascik: the entries starting with file:/// are probably the working/correct ones. strange thing that on windows they use backslashes instead of slashes in URIs...

From keirangtp on June 11, 2011 12:39:13

Labels: Milestone-1.0

From ColdAssassin008 on June 15, 2011 00:44:13

Im also having this problem. It seems that somehow it is getting the wrong directory path.

C:\Users\ColdAssassin\Music\Jap\M-Flo\m-flo - inside-WORKS BESTII-\disk1\04 I Know.mp3
This is my real path
C:\Users\ColdAssassin\Music\Jap\m-flo\m-flo - inside-WORKS BESTII-\disk1\04 I Know.mp3
and this is the duplicate path name. Oh I figured out what the problem was. somehow the dash was causing the duplicates so i just remove "M-Flo" and made it "MFlo" and the duplicates were removed.

From nalle.karhulitos on August 03, 2011 10:56:17

I got rid of my duplicates by
$ cd .config/Clementine
$ cp clementine.db clementine.db.bk
$ sqlite3 clementine.db
sqlite> delete from songs where filename like 'file:///%';
sqlite> .quit

From graeme@thelittlehippocompany.co.uk on August 04, 2011 13:14:11

I had an issue earlier that I think belongs to this ticket: I used the organise files function to "move" an album (after fixing the artist tag) from "Vnv Nation" to "VNV Nation". And while my OS is case-sensitive (OSX), I guess sqlite might not be and so I had duplicate records for each track in the album (both the old and new artist name).

A full library scan would probably have fixed it but instead I "cheated" and renamed the artist folder to something else (not using clementine), waited for auto-update to remove the songs and then changed it back so after the second auto-update it only found the one copy of each track.

From keirangtp on August 18, 2011 11:20:22

Issue 2086 has been merged into this issue.

From davidsansome on September 24, 2011 04:24:06

Issue 2248 has been merged into this issue.

From davidsansome on November 12, 2011 18:01:53

Labels: -Milestone-1.0

From john.maguire on January 05, 2012 03:24:34

Issue 2566 has been merged into this issue.

From spackmat on February 06, 2012 03:42:27

Having the same trouble on all my 3 installations: Seems as if neraly every startup of Clementine rescans the library and adds another duplicate entry for every track. In all cases the music is stored on a network share and the library base is given as a UNC-path (\homeserver\music), the computers are running Win7 x86 and x64. The rescan takes much time just like a complete new scan.

I rebuilt the whole library several times, but after promising restarts with a quick (a few seconds) library scan without creating duplicates it always fell back into the behaviour described above rendering the library-feature completely useless.

From jen.mclean8 on March 14, 2012 04:00:47

Exactly the same issue. Library on network drive, duplicates on restart. Remove and re-add library, fine until next restart.

From ajgallego on March 24, 2012 09:50:23

For me the solution was the one at the comment #16

From ibefraggle on May 08, 2012 10:30:33

Confirming #22/23. Win7 x64, some music on network share, some on local hard drive. Anytime the library updates (folder contents change, manually from Tools menu, on startup), new duplicate library entries of every song in the network share are created. These library entries have no apparent way of being removed, short of removing the network share from the library folder list in Preferences. Deleting the files does not help.

Wondering if this is a seperate issue to 1785.

From sebastiangoncalves on July 13, 2012 10:42:31

Got the same issue after upgrading from 10.10 to 11.04 (yesterday).

I solved in a simple way: Tools -> Preferences -> Music Library, then
select folder with mouse, press
and finally selecting the original folder

From ayatzid on December 11, 2012 10:34:28

and my saved playlists...gone with the wind...

From ivan.gojkovic on December 13, 2012 13:42:14

Sorry, people. This issue is too long with no reply. My clementine is going to trash for good. RhitmBox seams to be best for linux, and Aimp for Win. Ciao...

From spackmat on December 13, 2012 14:28:29

Same here, I love the featureset of Clementine, but with music on a windows network share it is really unstable, not just with the persisting problem of the many duplicate entries in the library. It starts relly slow and hangs constantly for several seconds with no response. Sad, but I have to look after a new player again.

From dtverder on January 02, 2013 23:50:55

I have started using Clementine 1.1 on Windows 7 (previously I mainly used it on Fedora) and I am also starting to get lots of duplicates on large playlists, e.g. 'all tracks' or 'all unplayed'. The tracks it happens to seem to be the ones not properly tagged and with basic file names such 'Track01.mp3'. The duplicates seem to occur in multiples of five, usually it duplicates the track ten times but sometimes it is five or fifteen.

Hope this screenshots makes it clearer. When I go to "Show in File Browser" for each of the double entries, it opens the same folder, where there are no duplicates. This affects maybe 5% of the library. Most folders look OK.

I can also confirm using 1.3.1-268-g789c492 that drag n drop a file from file browser adds the file(s) twice (double entries).

Also a strange thing: if I select "Remove duplicates from playlist" not only the duplicates get removed, some more ;(

@crissimaster that would be two more different bugs I suppose. Try to make repeatable cases for each of them and file separately?

I have the same problem Here. I have no clue how they appear but this is how I check they are there:

select count(c) from (
    select count(ROWID) as c from songs group by filename
);

This gives me 66948 and if I run:

select count(ROWID) from songs;

I get 133238. Dividing the two numbers gives 1.990171 this means that almost every song is duplicated on the database with IDENTICAL PATHS. Running another query:

select count(ROWID) as c, cast(filename as TEXT), *
from songs group by filename order by c desc;

The counts c I get for the first 3 rows are 44, 38, 28 and 2 for all the others.

I can't believe this bug did not get more attention since 2013. I kind of got used to append an album to the playlist and then use the "remove duplicates from playlist" to not having to listen to every song twice.

@falkartis can you try downloading the free album in this issue I created? #5654

Yes, I'm downloading it. How I add it to Clementine? do I copy it to my music folder? or do I drag and drop it from the downloads? or some other way?

@falkartis I just put it in the music folder, Clementine watches it and adds it to the database.

Edit: don't use my procedure, it's inefficient, use this one: #1785 (comment)

Ok, done. It adds it only once. Nice music by the way.

I'm currently working on a sql query I use to remove all the duplicates.

delete from songs where filename in (
select filename from songs group by filename having count(ROWID) > 1 limit 1
) limit 1;

But the problem is that in this way only one entry can be deleted at once. The solution is running this sql thousands of times. To do so I've put this sql on a file and runed it repeatedly with watch -n 0.01. I also copied the database to a tmpfs (a RAM filesystem) to avoid reading and writing to the HDD.
on the sql file I also added this 3 lines to monitor the progress:

select count(c) from(select count(ROWID) as c from songs group by filename having count(ROWID) > 1);
select count(c) from(select count(ROWID) as c from songs group by filename);
select cast(filename as TEXT) from songs group by filename having count(ROWID) > 1 limit 1;

when the first number reaches 0 I'll stop the watch and copy the database file back to it's location replacing the old one.

WARNING: this procedure is slow and destructive, run it only if you understand what all this means, and backup your database before.

I suppose this bug and possibly #3657 and #5908 are related. I found a workaround: Running the "organize files" tool (to make the id3 tags match exactly the folder structure) seems to remove the double or triple entries in the library.

What seems to have sorted this for me was having just one folder in the music library.

2018 and this is still an issue....just did a fresh library new scan and duplicates. duplicates everywhere!

BUMP. Confirmed to appear in 1.3.1 Win10. I even managed to get triplicate entries in the database :(

I ran into this problem a couple weeks ago. In case it's helpful to someone else, here's the sql I used to examine the issue and then clean it up.

-- examine duplicates
select s1.rowid
from songs s1
left join songs s2 on s1.filename = s2.filename and s1.rowid != s2.rowid
where s1.rowid > s2.rowid
and s2.rowid is not null
and s2.rowid != s1.rowid
order by s1.filename, s1.rowid;
-- look for duplicates in playlists
select p.name, s.album, s.title, s.filename
from playlist_items pi
left join playlists p on pi.playlist = p.rowid
left join songs s on pi.library_id = s.rowid
where library_id in (
    select s1.rowid
    from songs s1
    left join songs s2 on s1.filename = s2.filename and s1.rowid != s2.rowid
    where s1.rowid > s2.rowid
    and s2.rowid is not null
    and s2.rowid != s1.rowid
);
-- delete duplicates from playlists
delete from playlist_items where library_id in (
    select s1.rowid from songs s1
    left join songs s2 on s1.filename = s2.filename and s1.rowid != s2.rowid
    where s1.rowid > s2.rowid
    and s2.rowid is not null
    and s2.rowid != s1.rowid
);
-- delete duplicates
delete from songs where row_id in (
    select s1.rowid
    from songs s1
    left join songs s2 on s1.filename = s2.filename and s1.rowid != s2.rowid
    where s1.rowid > s2.rowid
    and s2.rowid is not null
    and s2.rowid != s1.rowid
);

The advantage over the solution from falkartis is that I only had to run this once. This worked for me because most of my duplicate songs had a higher row_id than the entry I wanted to keep. I first manually removed the bad/duplicate entries with lower id.

Yeha! this solution looks much better.
I can't try it because (luckily) there are no new duplicated entries in my database.

@lcarlsen Thank you so much for this! This issue has been a thorn in my side for many years, leaving me between a rock and a hard place, with Clementine being a pain to use and Amarok being awful since the redesign.

I had to replace row_id with rowid, however, for the delete duplicates to work:

…
-- delete duplicates
delete from songs where rowid in (
    select s1.rowid
    from songs s1
    left join songs s2 on s1.filename = s2.filename and s1.rowid != s2.rowid
    where s1.rowid > s2.rowid
    and s2.rowid is not null
    and s2.rowid != s1.rowid
);
…

I still don't understand why this issue, six and a half years since it was first reported, still hasn't been fixed.

It should not require a workaround that is well and truly beyond the capabilities or understanding of ordinary users.

I, for one, have no knowledge whatsoever of SQLite, and had to do a substantial amount of reading up on SQLite in order to fix the "Error: no such column: row_id" problem I encountered while trying to execute these commands, or even on how to use SQLite at all and open the database in the first place. And I'm far from a novice computer user.

Without your post I would never have been able to fix this extremely annoying problem.

Hello.

I'm using Clementine 1.4.0~rc1+dfsg-1 (the version available on my Linux Mint). And I have a bazillion of duplicates, triplicates, quadriplicate (if that's a word) on my playlists. I tried the series of Sqlite3 statements above using DB Browser for Sqlite 3.11.2 with no success. The two first SELECT statements return no results, as if the issue was not in the DB itself.

Frankly, I'm puzzled. Even the track count is the triple+ of what it should be (45k+ instead of 12k).

When I try to remove the duplicates via the menu in Clementine, I see all the memory of the computer, then the swap being eaten up until the systems kills Clementine.

Anyone experiencing the same issues ?

Best regards,

Brinbrin

I think clementine is more or less abandoned. I moved to Strawberry and have no more such issues.

I tried Strawberry, but with no smart playlist, and no ratings, it's of no use for me. Sorry to hear this about clementine. I already saw Banshee die. A music player is such an central software that it's like a friend.
As a workaround for my issue, I wiped out the playlists (Ctrl-K) and re-asked for them, and all the duplicated have disappeared.
But each time I add some new music, it goes havoc.
I guess I'll make do.

First RIP Amarok (technically still going, but the UI change in Amarok 2 simply sucks), now RIP Clementine. I gave up on Clemetine months ago. Issues remain unresolved for years. Importing my Amarok ratings never worked. A shame, cos it's still one of the best music players on Linux.

I've now switched to a mix of Cantata + MPD and Logitech Media Server + several Pi's running piCorePlayer.

So long, Clementine, thanks for the good times. Hopefully someone will revive it and fix the problems. It'd be a shame to see it disappear, even if I've stopped using it.

And Rhythmbox UI is ugly AF. Quod Libet is even worse. Audacious is too simple.

What is the repository for cantata ? ubuntuhandbook1/cantata do not support the latest linux Mint version...

And Rhythmbox UI is ugly AF. Quod Libet is even worse. Audacious is too simple.

Really, try Cantata + MPD. I had been avoiding an MPD based setup because of perceived difficulty, plus the habit of using full blown clients like Amarok, Clementine, and n my Wijdows days MediaMonkey.

Should have tried MPD years earlier. Oh, the benefit of hindsight.

It was there : https://community.linuxmint.com/software/view/cantata
I'll switch to Cantata. It has smart playlists and supports ratings, even if none of the ratings from clementine appear.
I'll do with it. thanks a lot for the Tip.

I, for one, have no knowledge whatsoever of SQLite, and had to do a substantial amount of reading up on SQLite in order to fix the "Error: no such column: row_id" problem I encountered while trying to execute these commands, or even on how to use SQLite at all and open the database in the first place. And I'm far from a novice computer user.

Without your post I would never have been able to fix this extremely annoying problem.

Thanks a lot @lcarlsen and @lord-aerion , your fix worked like a charm. All duplicates gone, no file statistics lost.

I had this issue too, where all my tracks were duplicated. When I tried a full library rescan, a new copy appeared for each track!
So, instead, I have removed the directory where I store my music from the library, clicked on "Apply", re-added it, clicked on "Apply" and voilà! It didn't even do a full rescan, it worked instantly. Hope it helps.

Thx to @TheBlackBeans !! This was the tip of the year - don't understand why it works - but it does; all other (including full rescan) failed, but now my duplicates have vanished (now).