manage-music

One entry point for new music. For every new album in the download directories it runs the whole ingest:

1. beet import --move (or --copy for sources that are still seeding), attended so the usual beets prompts still work: tags, file names, album art.
2. The MusicBrainz check. Lidarr only knows albums by their MusicBrainz ids, and a Discogs match leaves Discogs numbers in those tags. For an album without real ids, MusicBrainz is searched for a release with the same artist and title, the same number of tracks, and each track the same title and length (within 3 s). If there is one (several: the one Lidarr prefers), you are asked whether to retag from it (mbmatch.py: only ids and metadata change, not genre, art or replaygain). --quiet only lists it at the end. An album with none is kept as it is, and listed: Lidarr won't see it.
3. beet replaygain on the new albums (already analysed ones are skipped).
4. The sort name check. albumartist_sort decides the initial directory, so a wrong one files "The Who" under T. If the artist's other albums already use the same sort name, it is trusted. Otherwise you are asked when there is something to decide: the artist's other albums use another sort name, the name starts with an article, or a new multi-word artist has no sort name. A fix runs beet modify, which moves the files, and can be applied to the artist's other albums too so they stay together. Every imported album is also checked for a cover that decodes (magick -regard-warnings): if beets has none (an import that stopped before fetchart ran, say), or its cover is broken (a download cut short, which players show as a generic cover), beet fetchart is run once, then beet embedart (embedart.auto skips a cover.jpg fetchart finds already in the album folder), and albums still without a good one are listed at the end. A broken cover is first moved to ~/.local/state/manage-music/broken-covers/<album id>-<name>.
5. Cleans up the source directory of a moved download, with one question per directory. With beets' convert plugin on, the FLAC originals of the m4a files it imported stay behind; a FLAC counts as converted only if exactly one imported track has its disc and track number and length (within 2 s) and is not a FLAC itself. If nothing but converted FLACs and leftovers (nfo, sfv, m3u, cover...) remain, it offers to delete the whole directory; if other audio remains (tracks beets skipped or couldn't read), only the converted FLACs. A --copy source (still seeding), --quiet and -n delete nothing. An album that went in as FLAC although convert is on gets a warning: the conversion failed.
6. Updates each user link farm on the NAS: ssh -t there and run manage-linkfarm --only <Artist> (attended y/N/q prompt, rejections remembered). A link left dangling by a sort fix is re-pointed.
7. Runs a targeted Navidrome scan of the album in the farm that Navidrome reads.
8. Asks Lidarr, which reads the library itself (lidarr.music_root), to rescan the album's artist folder, so the albums it downloaded show as owned. The artist (found by MusicBrainz id) is first pointed at that folder if Lidarr has another path for it, such as the flat one it gives artists it added. Compilations and Non-Album albums are rescanned without touching any artist. --no-scan skips this too.

A multi-disc album downloaded as one directory per disc (CD1 and Disc 2 side by side, say) is ingested as one album: directories of a source with the same album and album artist tags, one disc each, are merged into <Album artist> - <Album>/Disc N first. A move source has the disc directories moved there; a copy source gets a hidden .manage-music-merge directory of hard links, removed after the import. A disc directory of the album that wasn't picked or given is pulled in too (also one skipped before, when it came on its own), and while one is still being written to, the whole album waits for it (unless picked or given). Two directories with the same disc are ingested separately.

An album that comes as one FLAC and a cue sheet is split first. The download itself isn't touched: beets imports a hidden .manage-music-split/<album> next to it, with hard links to everything but the image and the tracks shntool split out of it, tagged by cuetag (with the artist for tracks that have none of their own, album artist, date, genre and disc from the cue sheet added). That directory is removed after the import, so the image never reaches beets. Once all its tracks are in, the image counts as a converted original for the cleanup question and for --cleanup. A cue sheet left beside split tracks is ignored, an image next to as many tracks as it has is left out without splitting, and one next to fewer other audio files ends as failed, as does a failed split or a missing tool. Several images with the same title in one directory become one album with a Disc N per image. Only FLAC images are split; two source directories holding one untagged image each aren't merged into one album.

Handled directories are recorded in ~/.local/state/manage-music/state.json and are not offered again. A failed or interrupted import is not recorded, so it is offered again; the albums it left in beets with their files still in the download are named (beet rm -a id:N before importing it again, or the new import is a duplicate of them).

manage-music [-n] [-c] [-q] [--no-link] [--no-scan] [--audit] [--copy] [DIR...]
manage-music -p [-n] [-q] [--no-link] [--no-scan]
manage-music --cleanup [-n]
manage-music --clean-skipped [-n]
manage-music --list-sources | --edit-sources | --version

A subdirectory that is itself a source (transmission/completed and transmission/completed/lidarr, Lidarr's torrent category) is left to that source instead of being offered as an album.

With no DIR, every new subdirectory of the configured sources that has audio files and hasn't been written to in settle_minutes is ingested. A DIR given on the command line is ingested even if it was handled before (--copy to leave its files in place). -n lists what would be done, and -c marks every new directory as handled without importing it (to start from a clean slate). --audit ingests the albums in the library directory that beets doesn't know about, such as albums Lidarr imported before it was made a downloader only. Lidarr dropped some of those as loose tracks in the artist directory: they are moved to .manage-music-staging/<Artist> in the library (on the NAS, so instantly) and imported from there with --group-albums, which splits them into albums by their tags.

-p/--pick lists the new directories of the sources in fzf (with writing for those still being downloaded) and ingests the ones picked, each with its source's move/copy mode. Tab selects several, Ctrl-A adds the directories handled before (marked with their last result), Ctrl-N goes back to the new ones only, and the preview lists the directory's files. A picked directory is ingested even if it is still being written to or was handled before, like a DIR given on the command line.

--cleanup goes through every source directory with audio (handled or not, but not one still being written to) and offers to delete those whose tracks are all in the library already, one at a time: y, N, a (this one and all the rest) or q. A file is in the library when a track there has the same title (ignoring case, accents and anything in brackets), the same artist or album artist and a length within 2 s, so a FLAC that went in converted to m4a counts. Only tracks in the library directory count: an import that stopped before moving the files leaves tracks in beets that are still in the download, which would otherwise match themselves. Directories only partly in the library are listed and kept. A copy source is flagged: deleting it stops the torrent seeding. With -n or -q it only lists. It reads the beets database directly (read-only), which is instant.

--clean-skipped does the same for the source directories of albums you skipped at the beets prompt, which stay behind recorded as handled. Each is listed with its size, a few of its files and how many of its tracks are in the library, and deleted on y/a. Its state entry goes too, so a new download at the same path (another CD2) is new again. Only directories right in a source are offered, never the library ones --audit skipped, and not one still being written to, written to since it was skipped (a new download at that path), or skipped by a -q run (no confident match: ingest it attended instead); those are listed and kept. Skips recorded before 1.7.0 have no date, so for them only the being-written check applies.

--list-sources lists the configured sources: mode, how many new directories each has (or missing), and path. --edit-sources opens the config file they are in (~/.config/manage-music/config.toml, the whole file) in $VISUAL, $EDITOR or vi, checks it when the editor exits (valid TOML, every source with a path and a move or copy mode, offering to edit again otherwise) and lists the sources. --version prints the version, which is bumped with every change in behaviour.

Drift between beets, Lidarr and Navidrome

manage-music keeps new albums right. music-drift is for what is already in the library, with beets as the source of truth.

music-drift --check [--list] [--tsv FILE]
music-drift --fix [-n] [--limit N] [--track-case | --case] [QUERY]
music-drift --fix-ids [-n] [--limit N] [--reviewed FILE [--tolerance S]] [QUERY]
music-drift --fix-artists [-n] [--limit N]
music-drift --not-on-mb [-n] [--limit N] [--tsv FILE] [QUERY]
music-drift --rename-artist FOLDER NAME [-n]

--check changes nothing and takes about 20 s. It counts albums without MusicBrainz ids, Lidarr's unmapped files by what beets knows of them, albums Lidarr lacks files for, artists with files outside the library layout (their flat path), and Navidrome folders missing, with another number of tracks than beets, or there twice in different case. Artists rejected from the link farm are left out, and so are videos (.mp4, .m4v), which Navidrome doesn't read, and unmapped files in Non-Album, which Lidarr can't own. So are albums Lidarr lacks whose MusicBrainz type its metadata profile leaves out (singles, EPs, live albums with the standard profile): it can't own them either, and they are counted on their own line. Their types are asked of MusicBrainz once and kept in ~/.cache/manage-music/release-group-types.json. --list names every folder and --tsv writes them to a file. It exits 1 if there is any drift, 0 otherwise. Lidarr's Wanted list doesn't count: most of it is discography the library doesn't have.

~/.config/manage-music/drift-ignore.txt lists what is as it should be, such as albums MusicBrainz doesn't have: one library-relative path per line (a folder covers what is under it), # starts a comment. --check doesn't count them and --fix leaves them alone.

--fix retags albums with numeric or no MusicBrainz ids, numeric ones first, by the rule of the import check (mbmatch.py), if their path wouldn't change: a new path can come from capitalization, the year of a reissue or another sort name. Everything else is listed with the reason, and the summary counts them. The beets database is copied to ~/.local/state/manage-music/library-<date>.db first, and the folders are rescanned in Lidarr and Navidrome every 25 albums. It asks MusicBrainz one request a second, so the whole library takes hours. --limit stops after N albums retagged and -n only lists. --track-case also allows a retag that only changes the case of track titles, and --case one that changes the case of the album title or track titles. Not of the album artist: that would rename the artist's folder, which holds their other albums. The album is renamed through a folder of another name, .manage-music-case in the library, since on a share that ignores case beets can't give a file another case of its name (it adds .1, or keeps the old one). Files beets doesn't know of in the album folder follow. Albums with plays or playlist entries in Navidrome aren't renamed, and Navidrome keeps a missing row for each old name. For a new case of the album artist, use --rename-artist first.

--fix-ids is for an album Lidarr has but lacks files for. It looks in that album's release group, and if one release has the same tracks, or Lidarr prefers one of several, it writes only the MusicBrainz ids. Titles, years and paths stay as they are, so beet mbsync on such an album would make the changes this avoided. Albums with plays or playlist entries in Navidrome are left alone.

Titles can differ by a typo or a label. --reviewed takes a file of album ids and release ids, one pair a line, for albums you have compared by hand: the release must be in the album's release group with the same lengths, whatever the titles. --tolerance sets how many seconds a length may differ, 3 by default. A track MusicBrainz has no length for doesn't count, but one track must have it.

--fix-artists points Lidarr's artists at the folder holding all their files, if it is named after them (or after their name in ASCII, as beets spells it: Oresund Space Collective), without moving anything. Lidarr can add an artist during a rescan, with a flat path. The first artist of a collaboration doesn't qualify: that folder isn't theirs alone.

--not-on-mb looks for each album without MusicBrainz ids, not ignored yet, among MusicBrainz's release groups. A release group is like it if one of its artists has the name of one of the album's (or that sort name turned around: Takasaki, Akira), and its title is the same whole, without what is in brackets, or before a subtitle (Pt. is Part, Vol. Volume). The albums with nothing like them are added to the ignore list, in a section of their own, since Lidarr can't own them. The others are printed with what was found, and --tsv writes them to a file. An album MusicBrainz has under another artist name (an alias) is missed, so look over the section. It asks one or two requests an album.

--rename-artist gives an artist folder, such as F/Fields Of The Nephilim, the same name in another case (MusicBrainz's: Fields of the Nephilim). The albums in it get that album artist, and that track artist where only the case differs. The folder goes through a name of its own and each album through .manage-music-case, so that every folder and file has the case beets records. Then the farm link is remade, Lidarr's artist is pointed at the new folder and Navidrome scans it. Navidrome reads the farm through the share too, so the old folder stays there until a full scan (navidrome scan, about 8 minutes) marks its tracks missing. An artist with tracks outside albums, or with plays or playlist entries in Navidrome, is refused. Then --fix --case can retag their albums.

The Navidrome database is read at /mnt/disks/storage1/navidrome/data/navidrome.db, or db in [navidrome].

Setup

• Copy config.example.toml to ~/.config/manage-music/config.toml.
• fish completions: completions/manage-music.fish, linked into ~/.config/fish/completions by the sulaco playbook (scripts.yml). They complete the options, and a directory argument with the pending source directories as well as any path. -p needs fzf.
• Link music-drift next to manage-music, like it, and completions/music-drift.fish with the other completions.
• Needs uv, beet, docker with a running navidrome container, and key-based ssh as root to the NAS, with manage-linkfarm (Python 3 version) at linkfarm_script and python3 on the NAS.
• shntool, flac and cuetag (brew install shntool cuetools flac, where it is called cuetag.sh; both names work) to split FLAC images.
• Lidarr's Completed Download Handling is off (sulaco playbook, lidarr.yml), so its downloads stay in sabnzbd's downloads/music.