I think it’s quite important from the perspective of media preservation. We basically have a snapshot of music from a time where it was mostly Human-Made.
I agree, but on the flip side this will 100% be used to train new music generation models lol…
Wow this is so revolutionary.
Never in the history of the internet has music been available for free.
Not this easily with accurate tags and art it hasn’t.
They more often than torrents do have the wrong tags when its not English music. Took me way too many emails to google music and Spotify before they stopped screaming at me with ALL CAPS on one album and before removing the dots after album track names (1. Track Name), not to mention the ones named TRACK 1, etc.
Uhh get on a real site and it has been.
Help me out. Where?
Has anyone tried to self host this? Of course, hosting 300tb isn’t practical, so any solution would need to download the metadata and songs on demand.
That’s disgusting. Where would you find such torrents?
The album art torrent is a goldmine. Such a pain in the ass sometimes to find high quality album covers.
What I used to do is google the name of the album and append to it “cover itunes”. Usually I would find high quality images of the albums that way
Have they actually been indexed?
There is 200gb of just metadata
4TB if you include all with popularity=0 iirc
The fact that annas archive exists despite how fucked up everything is right now gives me hope. Every time I start to feel cynical about the future I remind myself that there’s people out there working to preserve the art and culture of our modern era with all the most powerful corporations and governments working against them, and they’re succeeding.
Anyway, thanks for coming to my Ted Talk.
100%
…and Archive.org
And Wikipedia.
People are saying it’s 300TB but this link is only 200GB why?
The 200GB is the metadata sqlite database only
God damn! That’s essentially just text, right? Or would it also include album cover art?
Basically, the id3 tags for the music files. However, Spotify uses several more nonstandard tags in their database, some of them are great to make playlists.
It includes cover art and also preview clips. In this blog post you can read what their database contains: https://annas-archive.li/blog/backing-up-spotify.html
Not released yet
Shit me, 200gb in metadata, 2.2TB in cover art before we even touch a piece of music! Wild
Its also crazy when you realise the amount of knowledge an experienced data analyst could gain from 200gb of metadata.
what a beautiful, simple and well designed website
None of these are audio torrents.
That’s not released yet.
cue Padme
‘And avoid it?’ — ‘To avoid it, right?!’
Now do Netflix, Prime, Paramount, HBO, Disney, Hulu and Apple and we’re golden.
Would be a magical day the day copyright dies.
When copyright dies so rich conglomerates can make money by monetizing out toys and collectibles and theme parks from smaller creators content without paying a dime back to them? Copyright is beneficial. Copyright is good. It needs reform, yes, but don’t mistake that with the concept being bad.
You should NOT actively do something, for money. You should EXPECT it.
Otherwise you’ll eventually try to maximize profits and get rid of everything that made whatever you did good in the first place.Abolishing copyright exclusive and only works if you abolish money with it. Otherwise you’re only benefiting the largest corporations, despite what you think, it won’t be the small guy winning
I’d just reduce copyright periods. Right now they are ridiculously long. No one should hold rights from 1930s works.
Already done. It’s called Torrent Streaming and lets you stream-on-demand anything that exists as a torrent without having to torrent anything yourself.
A client that can stream these Spotify torrents with an interface that works like Spotify (low bar, I know) will be awesome, but also including a database to match songs to artists so users can send money directly to the artists they listen to will make it revolutionary.
Did not see this coming when I built my 40TB NAS
Get to acquiring Seagate external HDDs and shucking them for your own 3.5" drive bays before the data centers get them
Sadly my wallet is on time out
The 20TB drives I was looking at in July are up 40% :/
Pc guys haven’t had a break in like 7 years. One component or another is the hottest item for one scam or another.
Frankenstein all the way, you just have to continuously build from whatever part is cheap at the moment.
At 100 now and it looks like I need to quadruple
I’m building my first Linux setup and have a NAS planned out. I’m so stoked. I got a raspberry pi kit from my dad as an Xmas gift yesterday.
I also used a raspberry pi (5). People here will advise against it but for me it’s been working fine so far. I can stream 4K with Jellyfin on my local network just fine. Read/write speeds aren’t great but good enough for me. I used a Pi hat with 5x SATA ports and I have 5x 8TB HDDs in a custom 3D printed enclosure and I’m using ZFS RAID z1. No complaints yet.
You learn a lot more than if you were to just slap an epyc in a box. Pi will reach you about encoding and balancing resources. I still use everything I learnt and some of the gear like the terra master only reason I don’t use it anymore is because I got free server stuff from work.
As far as I’ve read, the database is largely low bitrate files, and some AI. The value here is metadata and preservation of “rare” music.
deleted by creator
Nope, I would not call 160kbps Vorbis low bitrate, it’s roughly quality of 192kbps MP3. Only the ”popularity=0” stuff (so stuff with so few listens that Spotify does not keep record of) were re-encoded to 75kbps Opus, which as a modern codec is much better than it sounds like but of course re-encode is not great for already lossless stuff.
For purists there are those Tidal downloader sites available everywhere for free lossless music, even 24-bit hires FLAC.
Opus is what I’m encoding my working library to. I like ripping to flac (and archiving them as such), but the advantages to smaller file sizes for the working library are worth it for me. So far, I’m really liking the format.
I keep the archive on spinning hard drives, but the opus library on ssd (which makes browsing much quicker, and no unnecessary spinning up the hard drives.)
It’s not lossless but current ogg vorbis at 160kbps is absolutely transparent for the vast majority of people. That’s actually what I chose to keep my own collection, I mean, outside of the lossless albums that I absolutely want to flawlessly preserve.
How does it compare to 192 and 320 bps mp3?
I’d say in general it’s the same, but way lighter than 320 kbps mp3. It’s better than 192 kpbs mp3 and as good or better than 256 kbps mp3.
If you have really high end speakers you can hear difference between 160 Vorbis and 320 MP3, but between 160 Vorbis and 192 MP3 no way.
Am I losing my mind? All magnet links are metadata, no?
They havent released the music files yet
My very long game of avoiding spotify is finally paying off
Was there really much content on it that wasn’t already available in a torrent somewhere already?
I would be very surprised if it wasn’t, at best it’s stuff no one bothered with and somehow I expect that won’t get torrented much either.
Is this new? Aren’t most tracks already available in torrents?
Yep, most of tracks were already available on “various” sources, but this time they directly scraped the whole Spotify database.
It’s really nice from them to backup Spotify database on a distributed system, and for free ! This ensure Spotify business won’t be endanger in case of critical hardware failure.
So nice of them to help with Spotify’s off-site backup.
It’s new insofar as this is one big scrape. About 300TB iirc.
300tb is a lot, but its kind of crazy to think this entire company only needs 300tb storage arrays to function. I wonder how they handle things internally. I would imagine at least 1 backup server ready to go in HA. I wonder if they have multiple regions across the country that also serves up the same setup.
They need other 300TB to store all the ads.
“Are you an incel with few friends, no job, and a deep seated hate for melanin? COME JOIN ICE!”
Afaik 300 TB is just the most popular music and around a third of all tracks. The blog post on anna’s is quite entertaining tho.
Likely cloned Netflix’s “netflix in a box” design, where they drop a large 200TB+ NAS in thousands of different CDN datecenters with their most popular content cached so that total traffic is minimal across the internet at large.
Spotify mainly being music with very little video likely makes this even easier.
IIRC there’s still like 700TB of low popularity music missing, but it is only something like 0.4% of listens.
And they need a more storage overall because they have to set up datecenters around the world - doesn’t make sense to stream tens of millions of connections across the ocean. But that also gives all the backups one would need for “free”.There are 245 TB ssd drives now. You can almost fit that in a single drive.
deleted by creator
Oh I know, I work in the industry as well. Our company backups alone for workstations and servers is just under 1 petabyte. This is then replicated to an offsite location which is also out disaster recovery location, and also stored in long term storage in Azure. This is just backups, sooo much money for backups haha. Thats why I am shocked that this entire company can run off of 300tb which is a lot, but nothing when you think of it being the entire business model for them.
I think the craziest thing ive seen is we have these instruments that do genome testing and sequencing and they would create like 10tb worth of data per month. Every month they got there own 10tb drive handed to them to backup their stuff on there own on top of the ones we did for them.
Not mine, because I’m not famous enough for people to pirate my music lol. It would be flattering for me to be included in this batch of scraped music.
I’d steal your music
If your Spotify popularity is not 0, you probably are in the scraped archive.
Anna’s the GOAT
Sounds more like the pirate queen.
Fuck… Now that RAM prices are skyrocketing, we gonna see hoarders buy hundreds of TB of storage, leading to price hikes
The price for
restoredrefurbished HDDs has already gone up compared to a year or so agoRestored HDDs?
He probably means refurbished
That’s nothing compared to my old Napster collection
It would be awesome if we had an app that allowed to stream directly from such torrents, and had a user-made recommendation system to replace the discovery algorithm :D
Stremio + Torrentio does this for TV but I haven’t found an equivalent for music. Hoping to be proven wrong 🤞
Something as easy as stremio but for music. Connect to listenbrainz instead of trakt. Then only serve from the spotify collection because of their extensive metadata. With multi device sign in and syncing like stremio. Then a Kodi add on for the libreelec people.
i need a subscribe button for this
Whoever knows the answer, or when one is developed, someone please ping us all in this thread? thanks mate
Chatgpt is recommending an IPFS Cluster but I suspect this doesn’t solve the problem completely.
Let’s put it all on a Funkwhale server.
Sure, you set it up.
Dang. You called me out on my bullshit.
Well, I’ve set a funkwhale server up before and I’m not dealing with that bullshit again.
