Please fill in this bug report template:
Lexicon version: 1.1.24
Operating system: Windows
Bug description: I have a Macbook with Serato and all my music is on it. I exported them to the 1TB External SSD via Serato. I plugged it into my desktop PC, opened Lexicon and imported the Serato library on the External SSD into it. All Crates appeared, however, songs that have any special characters like “ç, ã, á, è, â, etc…” are marked as “missed” with an exclamation mark in front. I tried to relocate lost files, but it didn’t work. The only way to solve it is going one by one and pointing in the folder to the file in question. However, this is impractical, as I have thousands of files that have at least one special character. In my country (Brazil) we always use these characters.
Step by step to reproduce: Import from Serato and other programs, folders and music that have special characters.
Screenshot:
Note that all files that have special characters in their filename have an exclamation mark in front of them. I’ve tried to relocate and everything, but it didn’t work. It only works if I manually relocate it one by one, which is impractical since I have thousands of songs that have special characters:
Only folders and files that do not have any special characters present in the UTF-8 table will work. It seems that Lexicon only “understands” characters present in the ASCII table but not in Unicode.
I need the software to accept special characters otherwise the program is useless to me!
Waiting for a solution.
Accents and unicode are always a problem, especially with Serato that has its own system to handle those.
Can you zip and upload a few of these files that have an accent in the filename? If I can reproduce it here, then I can fix it.
Zipping them is important as it preserves the correct unicode characters.
Upload link
1 Like
Christiaan, I did an experiment here:
- On my Windows desktop PC, I opened SERATO and imported into it several songs from my personal library that have some Unicode accents (ç, â, õ, ü…), but that have nothing to do with what was on the external SSD. I created some Crates too.
- I closed it and opened the Lexicon
- I imported Serato’s library into it. Yes, all the songs were imported, without any errors.
- I exported it to Rekordbox, Traktor, Virtual DJ and Engine. Everything perfect.
In this process all the songs worked, without any errors and they all have some kind of Unicode accent that the songs that are on my external SSD.
I’m thinking that there is a problem when I migrated from Serato that is on my Macbook to the external SSD (GPT/ExFAT). Maybe Serato caused some corruption in the database and when I imported to Lexicon directly from this external SSD on my PC, the bug occurred.
Tonight I’m going to do the import process from Serato to Lexicon directly on my Macbook to see what happens. Then I’ll migrate everything to Lexicon from my Windows PC and see what happens.
I’ll keep you informed.
Thanks!
Alright, thanks for the update. It might have to do with unicode normalization. Two characters can be the same as one different unicode character visually, but different on the file system. Changing character set may introduce that problem.
That is a problem that I run into sometimes and was my first thought here.
1 Like
Do you think then that, due to differences in operating system and file structure between macOS and Windows, when I bring my music + Lexicon Mac Database to Lexicon Windows (using utilites “find lost files”), the same error will occur?
I find it a bit strange, as both systems currently share Unicode tables, both support virtually every language in the world. Maybe the problem really appeared due to the export I made from Serato to the external SSD without going through Lexicon first.
Serato doesn’t use real unicode, but stores it in a custom numbering system so maybe something went wrong there, I don’t really know because this is quite an edge case that I’ve never encountered.
The Find Lost Tracks utility should work with all filenames and folders, although I will admit accents are always a problem so maybe that is something it’s not quite catching properly yet. And if Serato has different unicode accents than the filesystem, then Lexicon might not match those.
1 Like
Yeah, it could be. Serato is very “closed” when it comes to library management. In fact, it is the only one who uses Crates. Others use playlists in M3U or PLS format (Traktor uses a similar NML, but it’s almost the same thing).
Well, initially, I exported my songs and Crates to Lexicon inside my Macbook and everything is normal for now.
The only thing I noticed is that in some songs, it didn’t migrate their colors. I found it strange because this fact only occurred in a certain Crate. The others, Lexicon imported everything correctly, without failures… But as this Crate has few, I can fix it manually.
Later I will try the migration to my Windows PC and see what happens.
I’ll let you know.
Grateful.
1 Like
Good morning, @Christiaan
Yesterday I did the following test and found that we have a big problem with Unicode characters in Mac environments. Everything I’m going to report here I did on my Macbook Pro, OS Catalina, APFS, configured for Portuguese-Brazil language:
- I opened Lexicon and imported my Serato library into it. Until then, no problem.
- Some songs did not import the corresponding colors in Serato. But so far, no problems.
- I made some changes to the library like looking for duplicate files, checking file integrity, etc.
- Exported to Serato again.
- I opened Serato and, at first, everything was there, correct and without any problems. I didn’t do further testing to see if something would misbehave. But I intend to do it today.
- I went back to Lexicon and exported the library to Rekordbox and Traktor.
- I opened Rekordbox. The entire library was there, no problems so far.
- I opened Traktor. The entire library and playlists were there, no problem at all.
- However, in both softwares, I noticed a bad behavior with the songs that have Unicode characters in their filenames: for some reason, when I loaded these songs in any Deck, the software re-analyzed them and didn’t mark them as loaded, as if they were “read-only”. Both in RB and Traktor, the behavior is the same. With songs that don’t have any Unicode characters (ç, ã, á, à, among others) everything worked as expected. Remembering that I also have folders with Unicode characters!
This only happens in a Mac environment. Windows, no!
As I informed you earlier, when I did the same test, but in Windows environment, everything worked perfect!
So I believe it is something related to Serato > Lexicon in macOS environment! Maybe that’s a point you can focus on to try to figure out the problem.
Soon, I’ll send you some songs for you to test there in your environment.
Interesting, great tests! If you zip a few files and upload them, I’ll do the same test here
1 Like
Accents and unicode are always a problem, especially with Serato that has its own system to handle those.
Can you zip and upload a few of these files that have an accent in the filename? If I can reproduce it here, then I can fix it.
Zipping them is important as it preserves the correct unicode characters.
I just uploaded some songs. Take a look please.
@Christiaan
Any news? Did you receive my files?
I got your files, I’ll test it next week
1 Like
Yesterday I did a test migration from Mac to Windows. Did not work 100%!
Process 1 (Macbook):
- I imported my entire Serato library into Lexicon.
- I backed up the database.
- I copied the database and my folder where my songs are (Music Files) to the External SSD. Path where the folder was: Macintosh SSD/Users/rodrigovolta/Music
Process 2 (Windows):
- I connected the External SSD to the PC.
- I copied the folder where my songs are (Music Files folder) to Windows. Location: C:\Users\rodrigo\Music
- I opened Lexicon and imported the database that I had backed up my Mac.
- All songs had an exclamation mark (OK, so far so good).
- I opened the utility to find lost files, manual mode.
- Originally, it pointed to the old path (Macintosh SSD/Users/rodrigovolta/Music) in the Music Files folder and the other subfolders. I left it with the most absolute path possible: “Macintosh SSD/Users/rodrigovolta/Music/Music Files” which is the folder where the subfolders of my songs are (each subfolder is a genre).
- In Destination, I pointed to the path C:\Users\rodrigo\Music\Music Files.
- I clicked the green button and let him do it.
- Finished. However, I noticed that some songs still had an exclamation mark on the front. Coincidentally, it’s the songs that have some Unicode character in their filename! The funniest thing is that these songs had an exclamation mark on the front, but at the same time, as if the Lexicon had detected it! Then I clicked one by one and pointed to the MP3 file where it should be pointed and then it worked! But as I have more than 3000 files like this in Lexicon, it’s totally unfeasible to do it manually.
Bottom line: I strongly believe that the problem lies with Lexicon (Mac version) in interpreting Unicode characters when it comes to a Serato import (I haven’t tested importing from Traktor, VDJ, Rekordbox). As I had previously informed, in a purely Windows environment, this does not happen. Everything works perfect.
I don’t know if it helps, but one thing I noticed is that Serato “doesn’t understand” Unicode characters when we try to edit the Tag through its playlist. Like this: I have the music list open. If I click on any field in the tag, it allows me to edit the field. If I try to put any Unicode character in it like “ç, â, á, ê” it won’t understand and will put it like “c, a, a, e” for example. And this only happens in Mac environment. Perhaps this is a relevant factor to consider. Mac has never been friendly when it comes to languages outside of English, that much I know. Windows is much friendlier in this regard.
Just ran a test here:
- Added tracks to Serato
- Imported to Lexicon, everything works
- Sync to Traktor, everything works
What exactly is the problem in Traktor? I’m not noticing anything strange or “read only” like you said.
I’m not getting broken tracks in Lexicon either (like in your first post).
Good Morning @Christiaan .
1- Did you do these tests on Mac or PC?
2- When you imported from Serato to Lexicon, was everything ok?
3- From Lexicon to Traktor OK too?
4- Have you analyzed all the tracks by Traktor?
5- When dragging a track to deck A, for example, the letter “A” was lit next to the track, in the playlist? And when dragging to Deck B, the letter “B” was lit next to the track?
6- After playing for a while, was it marked as “played”?
7- Did you do this test with ALL THE MUSIC I sent you?
8- Right-clicked on the track, chose EDIT and checked which path/name Traktor assumed, if all the letters are correct?
Finally, did you do the same test on Rekordbox and VirtualDJ?
In my first post, I had exported my music via Serato on Macbook to an external SSD.
Note that in other posts I did other tests that also pointed out problems with Unicode when importing from Serato to Lexicon, but only on the Mac platform.
If you want to test what I did in the first post, do the following:
On Macbook:
1)- Copy the songs to any folder on the Mac’s internal HD/SSD. Let’s base ourselves on the “Music > Music Files” folder.
2)- Open Serato and import these songs into it, create Crates, etc.
3)- Analyze these tracks by Serato, BPM, Key, etc and then close it.
4)- Copy this Music Files folder to an external HD or pendrive.
5)- Move the folder _ SERATO _ which is in the Music folder to the external HD.
6)- Open Serato again, with the external hard drive still connected to your Mac.
7)- Serato will recognize the songs, now on the External HD. Click on “Files” and then on “Rescan ID3 Tags”. Your songs will now turn yellow.
8)- Click on the Music Files folder on the External HD and drag it to the “Realocate Lost Files” button and wait!
9)- Finished, do a test with the songs dragging to the Deck. Everything should work correctly.
10)- Close Serato. At this point, Serato exported the database and songs to the external hard drive.
On Windows PC:
1)- Delete or rename any _ SERATO _ folder you have in your PC’s Music folder so it doesn’t interfere with the process!
2)- Connect the External HD.
3)- Open Serato. If all went well, the songs will be there and working. Drag any songs to the Deck and see if they are OK.
4)- Close Serato and open Lexicon.
5)- Import from Serato.
6)- Check that after importing, the songs are perfect!
For me, all with Unicode characters got an exclamation mark in front!
Remembering that both my Windows and Mac are configured for Portuguese-Brazilian language and ABNT/Brazilian keyboard layout.
Thanks, I was able to recreate the problem in Traktor and fixed it. I’ll look into Rekordbox separately.
Should also be fixed in Rekordbox. I’ll send you a test version later.
OK.
If you need to remotely access my computer to see how the problem occurs, we can schedule it. We use Anydesk if you prefer.
I am available for future tests and improvements in the software. I was a beta tester for Mixed In Key and helped them fix some Platinum Notes bugs.
Grateful!
