Zurgling
Hey all,
You might have heard rumors of a secret new internal project called "Zyclops". We've had a few iterations, but it's now live and I'm ready to introduce you to it...
But first, a few announcements
- In preparation for some Black Friday attention, we've added 7 new nodes to our US datacenter, more than doubling our capacity. So, relocations to the US are back in stock! Test your suitability, and be sure to read the instructions first, to ensure your data / ENV vars are correctly migrated.
- We're cooking up some tasty Black Friday deals. If you've got suggestions, there's still time to drop them into this Discord thread, and watch this space for updates!
- We probably haven't said this before, but it helps our whole Discord community when users "boost" our server. If you've got spare "boosts" with your Discord Nitro subscription, boost them our way! 🚀
Tell me about Zyclops
First of all, let's explain what Zilean is... Users on DebridMediaManager (not necessarily elfies, just any users) can "share" a list of hashes from their library:
What this does, is effect, is to publish the selected items to https://hashlists.debridmediamanager.com/, which makes them available to other DMM users to add to their libraries.
Zilean is an indexer which scrapes all these public hashes, so that other tools (including Prowlarr, Riven, Plex_debrid, Comet and MediaFusion) can search the hashes when you ask them to find content.
Here's a diagram:
This has been a game-changer for many addons, since Zilean is really good at parsing titles into structured metadata, and is really efficient at searching, since all the data is already cached and indexed in a database (i.e., there's no correlating real-time queries against trackers when you search Zilean)
ElfHosted public Zilean instance now available
In fact, because Zilean is so efficient, we've established a public Zilean instance, so that anybody (including the Addons4Kodi community) can query it. The hope is that heavy users will prefer Zilean to scraping public Comet/Mediafusion, since the per-query resource impact is much higher on these addons, given they don't query just Zilean.
However...
The public hash database relies on DMM users choosing to manually share hashlists. And it's usually only a subset of a library which gets shared. How do we get more hashes?
Zyclops awakens!
Project Zyclops leverages some recent updates to Zilean and Zurg, within the ElfHosted ecosystem, to automatically share elfies' hashlists into a pool for our internal Zilean to scrape. This includes hashes not previously publically shared, potentially sourced from private trackers, and so not scrapeable from any public indexers.
Here's a diagram of how the internal Zilean works at ElfHosted:
You'll see that we're augmenting the public hashes with the hashlists from every ElfHosted user, automatically.
Note that all that's shared is hashes (see examples in the DMM git repo), nothing else, and these hashes are pooled/merged with existing public hashes, giving Zilean a larger dataset to index your queries against.
How much better is it?
At the last count, there were 79K hashes in the public Zilean, and 86K in the ElfHosted internal Zilean, .. roughly 10% more unique content at this time. As ElfHosted users increase, the pool of scrapable hashes continues to increase.
How do I use it?
You already are.
From a user perspective, your setup looks the same, but any Zilean searches you execute are executed against our internal, "augmented" Zilean!
This is amazing, how can I thank you?
We ❤️ our elfies, and we want to keep building cool toys / stacks for you! You can help us grow by leaving us a glowing review!