.m4a filetypes (not from iTunes) beat grid shift

[sadboolean]

Please fill in this bug report template:

Lexicon version: 1.2.2
Operating system: Mac

Bug description: So I’ve been subscribed for a couple of months to test out functionality, but I’m realizing I won’t be able to migrate the majority of my library from Serato to Rekordbox given almost all of my files are .m4a (not from iTunes).

Read this thread https://discuss.lexicondj.com/t/beat-grid-shifted-when-importing-to-rekordbox6-from-serato-probably-m4a-files/403 and reproduced the beat grid shift with my .m4a files files vs the iTunes .m4a Christiaan provided, along with some .mp3s—no issue there for the iTunes .m4a or .mp3 files.

With that said, I was wondering if functionality for these files types will ever be supported (if possible), as I am likely going to have to end my subscription and manually transfer everything over…not ideal, but glad I at least gave it a shot.

Any insight into this or any workarounds would be greatly appreciated

[Christiaan]

If you’re sure that all your M4A are different from the iTunes M4A (or if you can determine which are different, maybe in a certain folder?) then you can apply a manual cue/grid shift before syncing.

You can check what shift is needed by going into your Rekordbox and loading a shifted track. Then set 2 memory cues on the current grid and where it should be. The difference there is the shift you need to apply in Lexicon.

You can apply the shifted in Lexicon by right click tracks → Recipes → Shift cues/grid and then entering that value in milliseconds. So if the difference in RB was 0.020 then enter +20 or -20 (plus makes cues go to the right, minus to the left).

[sadboolean]

Apologies for the delay (was traveling)—I’m certain given I went ahead and purchased one song directly from iTunes and it accurately transferred cues created in Lexicon into Serato and rekordbox…just frustrating because I have over 3000+ tracks in this non-iTunes .m4a format.

“Me Rehuso (iTunes)” is purchased directly through iTunes and one copy of “Dos Mil 16" was not, and even though both say “Apple MPEG-4 audio” file type in properties (including both as attachments), still get the shift on the non-iTunes version.

Ran the script as you suggested, just afraid it may not be efficient enough for every scenario or to use Lexicon as primary music management tool (which is my main goal)—for example, when I manually cue’d non-iTunes “Dos Mil 16" inside Lexicon and imported into Serato it was shifted, so to get the right cues I would have to isolate these .m4a files in Lexicon, cue, apply the recipe, and then sync…but this leaves all of my cues shifted inside of lexicon which can get complicated, especially once I factor in other file types or syncing to Rekordbox.

For example, workflow of going back and forth between Serato, Lexicon, and Rekordbox with non-tunes .m4a files looks like the following:

1. Cue non iTunes .m4a files inside Lexicon (“Dos Mil 16”)
2. Apply cue shift recipe (~-80ms) and export to Serato (cues now shifted in Lexicon but accurate in Serato)
3. Use newly cue’d music inside of Serato (main software) and apply / change some cues on the fly (let’s say I accidentally delete an existing cue and set it back along with a new cue)
4. Import Serato back into Lexicon and all cues still shifted same amount as before (checks out)
5. Sync Lexicon library to Rekordbox (which one would assume has the same or similar shift issue)
6. Discover Rekordbox cues are shifted incorrectly to the left of down beat (as they were in Lexicon after re-importing Serato in step 5)
7. Apply cue shift recipe (~80ms) to correctly set cues in lexicon and sync to rekordbox
8. Open rekordbox and discover cues are now correctly synced

In summary, Lexicon has no issue syncing these .m4a files to Rekordbox accurately, but when it comes to Serato, there is an ~-80ms shift. Not sure what happens under the hood when cues transfer with non-iTunes .m4a files, but I did notice that for “Dos Mil 16” both Lexicon and Rekordbox assigned the same BPM 129.99, where as Serato assigned 129.0 (which is wrong, which is clear when you beat jump). Tested against same track but from iTunes “Dos Mil 16 (iTunes)” and bpms match across all three platforms.

Thought maybe the issue had to do with analyzing in Serato first, so I went ahead and added an unanalyzed, .m4a file (not from iTunes) to a Serato crate, imported to Lexicon and analyzed and cued there, and then exported to Serato and Rekordbox…this led to same bpm across all 3 platforms and no issues with beat jumping, but still shifted cues in Serato ~80ms (to the right)—Rekordbox cues were identical to Lexicon.

Seems to be an issue with how cue point transfers are handled between Serato and Lexicon when using non-iTunes .m4a files, signaling that this is likely an isolated issue with Serato.

Any insights to this would be greatly appreciated, and apologies for the lengthy response—took some time to test all of this and wanted to make sure write up was detailed so that troubleshooting could be as helpful as possible.

Thanks again for giving this some attention—really want to be able to use Lexicon as main organization tool, it truly is wonderful!

tried to upload test files in reply below, let me know if I should upload them another way

[sadboolean]

Dos Mil 16
Me Rehuso (iTunes)
Dos Mil 16 (iTunes)

[Christiaan]

Thanks for that detailed response!
I’m going to try and see if there is any way to detect the difference in these m4a files. No promises though! This is pretty much guess work and trial and error, so not something I’m too confident in.

[Christiaan]

ffmpeg - what is the default format for .m4a output?

This is the same output I’m getting from ffprobe

iTunes
  Metadata:
    major_brand     : M4A
    minor_version   : 0
    compatible_brands: M4A mp42isom

Other
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41

I’ve made a few test m4a’s with ffmpeg, same in the stackoverflow topic and those tracks are offset too. Both the isom version and the M4A version.

This is what my ffmpeg v4.3.2 gives me with the -f mp4:

  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41

This is what my ffmpeg v4.3.2 gives me without the -f mp4 or with the -f ipod flag:

  Metadata:
    major_brand     : M4V
    minor_version   : 512
    compatible_brands: isomiso2avc1

So both have the same shift.

What I’m wondering is how I can re-create a major_brand: M4A file. I can’t seem to do it with ffmpeg, or maybe I don’t know the right flags. Or maybe a different tool (something Apple internal?) creates it. If I can recreate that and confirm it has no shift, maybe it is safe to say that major_brand: M4A never has a shift.

– Edit
Okay so using Apple Music (iTunes) to convert a track from MP3 to AAC creates a .M4A file with the major_brand we’re looking for. Tested it in Lexicon and appears to have no shift to Serato.

So I think it is safe to say that major_brand : M4A means there is no shift and other M4A’s do have a shift. Might not be 100% accurate because there can be many different encoders out there, but I think it’s a good step in the right direction.

Now I just have to add this information to the beatshift scanner.
I’m doing a huge rework of the music player right now for the next beta and I think that’s a good place to add this fix too. So keep an eye out for the beta! Maybe next week or the week after that, depends on how well things progress.

[sadboolean]

@Christiaan

s a good step in

@Christiaan

s a good step in

Hey Christiaan—sincerely appreciate the breakdown and all the attention you’re giving this! Will hang tight and keep and eye out for the beta!

Let me know if I can be of any help!

[sadboolean]

Hey @Christiaan downloaded beta and looks sharp! Did try the beatshift scanner but no luck with the shifted non-tunes .m4a files—did the above make it into this beta?

Thanks!

[Christiaan]

Not yet, working on it right now. Got it (mostly) fixed for Traktor, now doing the other DJ apps. Next beta update will have it

[Christiaan]

Beta update just released has the M4A beatshift scan.

All you have to do is run the beatshift scanner and then sync to Serato.

[sadboolean]

Hey @Christiaan—really appreciate all of the work you’ve been putting towards this!

I’ve been able to test for the last week or so, and can confirm that cues for the most part are where they should be. There are still some tracks (all from the same .m4a source) that are still exhibiting the shift (attaching an example).

Not sure why, I know the beathshift scanner is gone from utility and done automatically (which is amazing!), so I went ahead and re-imported my library from Serato, but seems like some tracks are affected and others are not.

For now, I’ve contemplated creating a copy of my library through the Music app by converting to AAC (running some scripts to automate the process), as this produces no shift alike we tested before, but does so at the expense of audio quality degradation, and of course having to re-add meta data, cues etc.)

Just wanted to point out (currently on latest beta .26), if there’s anything I can do on my end please let me know!

File link: iCloud)(Explicit)

[Christiaan]

It’s probably a false positive, I had one with the original M4A’s we used too. I couldn’t see any other difference between that one and the iTunes M4A’s so I don’t know why they are false positives.

So I don’t think this is fully solved yet but a lot better than how it was.

[sadboolean]

Absolutely, certainly feel more comfortable switching between the two now—just contemplating copying my entire library to AAC at this point to avoid (huge undertaking), or perhaps importing from Serato and then correcting those tracks myself and sticking to Rekordbox for the time being.

Again, truly do appreciate you even looking into this, know you have a million things on your plate—all of the updates have truly been spectacular!

[Christiaan]

No problem, glad we got this for iTunes tracks at least