Over the past few days, I’ve witnessed a remarkable surge in the number of communities on browse.feddit.de. What started with 2k communities quickly grew to 4k, and now it has reached an astonishing 8k. While this exponential growth signifies a thriving platform, it also brings forth challenges such as increased fragmentation and the emergence of echo chambers. To tackle these issues, I propose the implementation of a Cross-Instance Automatic Multireddit feature within Lemmy. This feature aims to consolidate posts from communities with similar topics across all federated instances into a centralized location. By doing so, we can mitigate community fragmentation, counter the formation of echo chambers, and ultimately foster stronger community engagement. I welcome any insights or recommendations regarding the optimal implementation of this feature to ensure its effectiveness and success.
Why not make this purely client-side? Give me the option to merge what I see as like-minded feeds into one feed. Label it and be able to scroll it as one feed. All without the need for admins or instances to do more work?
Agree - Mastodon has lists, would like this too for Lemmy (even though I’ve not used them so far)
That’s how multi-reddit works, totally client-side, much better IMO than “forcing” the choice upon everyone at admin level.
Why don’t you read the issue? It’s in lemmy-ui, so it’s clearly client-side. So just because you want to waste your time going through hundreds of instances to find similar communities, do we have to force everyone else to do the same?
Maybe don’t take disagreement so personally?
I too would like to do this myself and not have AI or anyone else decide for me what content gets lumped together.
I predict that this is also an issue that will slowly resolve itself over time, as critical masses of users gradually coalesce around one community, or more…but only if the extras are distinct in some way…which would very specifically be made more difficult by the sort of programming you’re proposing.
I’m not saying there’s no merit in your suggestion, only that it may not be the one-size-fits-all solution that you seem to think it is
Not taking it personally.
I would also appreciate the ability to customize, but it would be helpful to begin with a curated list of instances for each topic.
But who gets to curate that? How who has to sift through all of the 8000 instances and figures out the topic of each of likely thiusands of communities?
Automated sounds as though it’s purely based off of the community name? How does it figure out the difference between table top gaming or video gaming or even slots/gambling?
What about football? American or the rest of the world?
What about politics? Is it left wing or right wing?
Seems like a cool idea at first, but when you get into the weeds it becomes a pretty complicated issue.
OptimusPrime is a Deception here, apparently.
Oh woah, dial it back, friend
I see very little discussion about the implications of this for moderation, and it feels to me like they get very sticky. With traditional human-curated multi-reddits, you as a subscriber must engage with the idea that you are choosing to aggregate multiple communities into a single feed, which is intuitive enough, the subscribed feed already works that way.
But by making it automatic, the software hides the fact that it’s pulling together discrete feeds from communities with different rules and different moderators. This feels very awkward to me. I’m all in favor of traditional multi-reddits, which can be used to create this sort of feed for yourself. I’m still on the train of “duplicate communities will sort themselves out if community discovery is made much easier and popular communities reliably show up at the top community searches, mostly irrespective of what instance the search was performed on” (obviously defederation takes precedence here).
I’m still on the train of “duplicate communities will sort themselves out if community discovery is made much easier and popular communities reliably show up at the top community searches, mostly irrespective of what instance the search was performed on”
Honestly, we’ve all seen this happen in every community that fragments for some reason. Whenever reddit banned a subreddit, 1,000 mini-subs would spawn, eventually coalescing into 1 or 2 main communities. Whenever a migration happens (Tumblrinos moving to reddit, redditors moving to various reddit-like things, forums moving to reddit, fark migrations, usenet, IRC, etc) they always fragment initially only to end up with a relatively few coalesced groups that hold similar ideas.
c/news might end up with 10,000 news subreddits (even one for each local newspaper!) but eventually it comes out to something like: c/news, c/leftnews c/rightnews c/ihateallnews with the most active (by far) being one of the generic ones. The others might technically exist, but if you search “news” and get one that has 10,000,000 subs and another that has… 3, well, you’re just not going to bother.
I think it would need to be voluntary; mods on different instances have to agree to be a joint mod group. If an instance decides to break up, they take their mods with them, etc. Might he complicated, but it seems doable.
If the mods were willing to agree to this, why would they not also agree to simply merge the communities? Frequently, when mods choose to maintain overlapping/competing communities/subs… it’s because there’s conflict about culture or moderation policies. This mechanism feels very duplicative of existing patterns, and limited in much the same ways when cooperation breaks down between mods of different communities.
There’s an advantage to splitting the load amongst multiple instances. If [email protected] has issues, the subscribers to community can still access [email protected].
Further, preventing centralization is a core focus of the fediverse, with all the pros and cons that come with it. The reddit migration is a testament to that philosophy.
If an in-fight happens between two instances’ communities, host1 won’t be able to boot host2’s entire existence, just their agreement to be in sync. Voluntary syncing of communities seems to me a natural extension of the federated philosophy.
There’s an advantage to splitting the load amongst multiple instances.
I don’t have links handy, but the devs have said multiple times that federation replication load is not a limiting factor. It’s the browsing load that matters, which is already spread across the instances that subscribers hail from. There isn’t a performance issue here to fix, at least around the current network size.
If [email protected] has issues, the subscribers to community can still access [email protected].
Again, user-account federation already provides this. If the host where the community resides is down, I can still read existing posts, and post and comment with users on my local instance. I don’t see significant further benefits in splitting the community hosting.
Voluntary syncing of communities seems to me a natural extension of the federated philosophy.
I don’t disagree with this, the challenge is that the federated philosphy makes everything it touches very complicated. A sprinkle of federation in the core of an ecosystem brings enormous resilience against the ecosystem getting coopted against the will of its users. A heaping spoonful of federation makes it an unusable mess. The complexity of federated design needs to bring very big benefits to “pull its weight” in complexity. The tradeoff looks positive for me for federated replication even though the cost in complexity is large, but I don’t for meta-communities. They seem like cosmetic improvements over the existing capabilities.
But fair enough, I see where you’re coming from. It’s not madness, on-balance it’s just not something I see providing value in proportion to its conceptual complexity.
If the host where the community resides is down, I can still read existing posts, and post and comment with users on my local instance.
Can you help me understand how that all works? I ran into this today where an instance wouldn’t let me submit due to performance issues, and I jumped instances to be able to provide some (potentially) important breaking news to “community.” If they were fully in-sync and one instance didn’t “own” the community, when things stabilized, my content would have replicated instead of being stranded on a lonely instance. As it stands, that content will never be part of the “real” community.
But fair enough, I see where you’re coming from. It’s not madness, on-balance it’s just not something I see providing value in proportion to its conceptual complexity.
Right on. I’m still new to the fediverse and am probably missing some foundational concepts still. Love where it’s headed though.
If the host where the community resides is down, I can still read existing posts, and post and comment with users on my local instance.
Can you help me understand how that all works?
I’ll preface this by saying I haven’t read the code and it’s certainly possible there are errors in my mental model, but my belief is:
- When the community-instance is up and it gets a post from a user local to the community… It write that data to its db and asynchronously sends federated messages to instances with users that sub the community. Those instances write it to their local DB.
- When the community-instance goes down and someone on a user-instence tries to read the post, it gets read from the local-db cached copy.
- While the community-instance is still down and user-a on a user-instance posts or comments, again it goes into the local DB and async sends a federation message to the community-instance… the message doesn’t immediately get through though.
- While the community-instance is down, if user-b on the same user-instance browses the community, they see user-a’s content from step 3 in the user-instances-db. If user-c on a different user-instance checks, they won’t see user-a’s content until the community-instance comes back up to process the federation messages from step 3.
- When the community-instance comes back up, it accepts the federation messages from step 3 and updates all subscribed instances so user-a’s content is viewable everywhere.
It’s possible I’m mistaken about 3-5, but if you had trouble posting… my guess would be that your own user-instance was struggling and slow. But if you’ve rules that out maybe the update behavior when the community-instance is down works differently than I think it does. The reading in step 2 I’m quite certain about though.
I’m pretty sure you can’t post to community-instance if they’re down/having issues to the local DB. The instance I’m on is very stable and every time I think it may be an issue, it turns out to be the other instance.
It sounds like a lot of the “sync” behavior I was thinking of is already built-in if we can just expand on it a bit.
For what it’s worth, I’m VERY much for this.
One of the pain points for those coming into this to fill the Reddit void is fragmentation. Beyond being a huge improvement in usability, information would be shared much more easily this way. For someone who spends a lot of time in IT/tech related subs, that’s very important to me.
I tend to agree with your take on this. I’m getting serious FOMO over here and over-subscribing because I don’t know which sub will be the one to “take off.”
Cross-instance “multireddits”, that are also automatic and topic-based. #1113
TL;DR: The suggestion is to implement an automatic multireddit feature in Lemmy that displays all posts from communities with the same name across federated instances. It aims to promote decentralization, avoid echo chambers, and ensure high availability. Community moderators would have the option to opt-in or opt-out their communities from being displayed. There are discussions about potential issues such as community name collisions, duplicates, abuse, and practical implementation. Some propose using a new link format, while others suggest providing users with a list of related communities.
I think what I’d prefer is a more supported version of Reddit crossposting… I’m brand new so stop me if Lemmy already does something like this. For example, if someone has a vegetarian recipe community, a more general vegetarian community might automate a feature to crosspost their content, with clear linkage to the source… But a new discussion thread as the default in the crossposting community.
This allows the different related communities to choose their own moderation and regulation, but can also allow communities to be content aggregators.
I can definitely see name-collisions being an issue, where communities on different instances have the same community “ID”, but aren’t actually about the same thing. I’m still overall in favor of the basic idea though.
I think it would be really nice to have a “fediverse map” for each server, to show where they’re connected to and what instances are endorsed back.
Would make finding new servers/communities easier too
There is a list of federated and blocked instances on the bottom of the page. https://latte.isnot.coffee/instances
Also this script is to see which instances block or are blocked by the instance you choose.
Another day another “When will the thing not meant to be Reddit be Reddit” post
The underlying problem here is the lemmy community being spread out across many instances, and this solution doesn’t really fix the underlying problem.
This is just speculation, but I think eventually 1-4 instances will grow much bigger than the rest. I think when this happens, communities will become much less fragmented and the problem will solve itself.
tl;dr while this is a good idea, I think if we just leave everything the way it is the problem will solve itself.
The goal of implementing this feature is to leverage the benefits of federation. If we wait until there is only a few big communities, the purpose of having federation becomes irrelevant. When an instance hosting one of those large communities shuts down, the community would have to migrate to the next major community.
By proactively implementing this feature, Lemmy can harness the advantages of federation while actively mitigating the challenges posed by community fragmentation and echo chambers. It provides a centralized hub that encourages cross-pollination of ideas, fosters community engagement, and ensures that valuable content is accessible to all users, regardless of the size or popularity of individual communities.
I disagree on two points:
- Fragmentation is a feature, not a bug. Echo chambers will always exist, but fragmentation is what keeps them contained to small pockets.
- A centralized hub would not necessarily foster community engagement. Seeing hundreds of comments on a post is often enough of a barrier.
see I’m not sure I see that as a problem. There are lots of reasons to spawn a new but similar community (bad community mods, bad server admins). There are lots of subreddits I avoided because they were just too big to get into any real info or discussion, just the same beginner questions asked over and over again.
The proposal does not necessarily imply merging all small communities with others. The implementation can provide an optional choice to community moderators, allowing them to decide whether they want their community to be included in the multireddit. This approach respects the autonomy of individual communities and acknowledges the reasons why new but similar communities may emerge, such as issues with community mods or server admins. By offering this flexibility, the feature can cater to the diverse needs and preferences of different communities while still providing the benefits of consolidating posts from communities with similar topics.
Who would control the multi-community, and how could a potential bad actor/community be dealt with?
I agree. Time will show which instance most people will go to. The smaller instances will slowly quiet down, while the larger ones will gain in popularity. The issue will definitely sort itself out over time. I’m not worried at all.
Sorry I don’t know how lemmy works, but when there is an enormous amount of traffic (i.e. lots of comments) on one post does it have a greater chance of crossing instances?
Isn’t a few large communities eating up the others like the opposite of what Lemmy is trying to do?
I think it’s only a problem if it congregates to 1 instead of 4 or so. If one of the 4 goes rogue or disappointing its users, people can easily just jump on a different one. Most servers will suck and that’s ok. Good ones will attract users.
If people are satisfied with them, I think that’s OK, and more efficient than having a zillion.
Problems will happen if we go too low, and bigger instances start de-federating. Some might be tempted to start monetizing like Reddit.
i keep hearing people call for this like its going to happen and be the only way things will be. Look at reddit, look at the history of some of these subs.
there will always be multiple copies of various communities. what software gives us the ability to do is sort and filter and tag (we need to add this) to our hearts content so instance admins and users have control over what comes across thier feeds.
Joined communities will have many of the same centralization problems reddit has now. I’ve seen this call mostly from users who were on reddit long after it was large. It seems many have no idea that almost every topic on reddit has 4-6 subs around it usually.
The example of federation most people have experience with is email. There will almost certainly be gmails and yahoos emerging over time, but they will have limited control compared to reddit, because if you don’t like the filtering/advertizing/whatever of one you’re free to leave for another
The email analogy breaks down when you consider that most email servers are run by big tech and cost a lot of money to upkeep
Or you can run your own email server for yourself and a few of your family.
There’s almost nothing in between gmail and some random person’s self hosted email server.
In terms of the fediverse,who the heck is willing to host a lemmy server for 1 million complete strangers? Not many people i think
If Lemmy takes off I wouldn’t be at all surprised if tech companies hosted instances that they monetize through advertising, and many people would be willing to have a home instance that showed them ads in exchange for high stability and potentially more user-friendly clients
I think you perfectly described my issues with comment sections on Reddit for the last few years. That attempt to appeal to an audience rather than further the discussion.
I used to love comment sections as much as, if not more than, the actual post on Reddit. It felt more like a conversation that had insight and humor. It got too big for it’s britches and became that soulless monolith.
I get an almost nostalgic vibe from this place. It’s nice.
Well, it’s the nature of almost any online community. Say the ‘popular’ thing and you’re lauded, even if a slightly less popular point is more valid / has better evidence.
There really is no good way to discourage this other than fostering a community which values the discourse over ‘popular’ thing. That’s difficult to do even offline.
I’m not sure it’s even possible to discourage it really. If you have any sort of user-user engagement system, whether up/downvotes or comments/shares or whatever, you’re going to have particular sentiments that are popular with particular audiences and get more of that engagement. If you take those features out, you’re going to lose engagement, pretty much definitionally.
I’ve always thought about creating some metric to weight users who create comments with the most engagement as higher. That leads to the most controversial or dividing comments rising though.
Some impartial judgement via mod points and or community awards to weigh valuable users would be nice.
The issue is any of these would be gamed, it might be possible today to use an AI model like ChatGPT but that’s got its own biases.
So for the moment I can’t think of a better system than upvote downvote.
Yeah, weighting ‘engagement’ higher is basically the youtube algorithm problem: you’d be attracting trolls most of all. You could probably devise something smarter, like weighting it to include all of most upvotes, fewest downvotes, and most comments; adding comments to it helps identify people who post positive but engaging things, but again that can lead to an echo chamber. Plus, it then under-weights new users compared to established ones, which can be unfortunate.
Yep. It turns out there is no such thing as a ‘balanced’ social network.
Which is analogous to life, depressingly enough.
I don’t think I’m understanding this right cause it sounds like you’re trying to make it more fun by adding more rules. If there are 20 groups that are all about pickles that’s fine they each like running things their own way. Eventually one group gets popular and that’s where the majority goes. I think your frustration could better be solved with something like tags where groups could choose to associate certain tags words that makes search easier like tag: pickles-fermenting-homemade-cucumbers and that could clear up search from people just wanting to share pickle Rick memes.
The thing is that say i want to see all the pickle content across the lemmy-verse, i want to just be able to subscribe to an umbrella “pickles” category and get all the c/pickles content from any instance my instance federates with without worrying that i missed a community or something. If there are 20 groups but i only know of 1 or 2, odds are that i’ll miss the biggest ones with a bunch of pickle content that i want. And I don’t want to have to manually go through, search for all the biggest instances, and subscribe to each pickle community one by one. Plus, say a new instance is started and it’s pickles community blows up. I’ll miss it because i already searched through and subscribed to all the pickles communities that were available when i joined. I’d rather default to subscribing to all the c/pickles communities my instance sees, and then if one instance is posting stuff i don’t want to see i could manually exclude them
Tldr I guess it depends what you think will take more effort to do manually. I think having to manually find and subscribe to every community i want from across the lemmyverse (and periodically run the search again to find new communities) would be way more work than subscribing to every c/pickles my instance can see and manually excluding the instances or instance-specific-communities i don’t want content from
I’m new to the whole concept of federation, so please let me know if I’m missing something fundamental, but with your proposal (subscribing to all c/pickles) works for when each instance has the same general rules for posts, but what about when one c/pickles on some server is actually about… dildos? (like r/trees was about pot, and r/marijuanaenthusiasts was about trees)?
Wouldn’t your feed be polluted with pics and reviews of dildos when you want wholesome and healthy pickle recipes?
That’s where manually being able to block specific instances would come in. So if you subscribe to every c/pickles your instance federates with but then didlos.lemmy/c/pickles starts causing problems, you could manually block that one instance. Or maybe you want to see dildos but not when you’re looking at pickles, you could choose to manually exclude didlos.lemmy/c/pickles from your c/pickles subscription
That makes sense, but sounds like it’d be a constant spam battle. Maybe I’m overthinking it, or just thinking about how I’d go about breaking/attacking something to try to figure out how to fix it.
Yeah it could be, but keep in mind it would only include communities from instances that your instance federates with. So if your instance has blocked another instance it wouldn’t be included in the group. This issue actually describes it really well and is better thought out than what i’ve described https://github.com/LemmyNet/lemmy-ui/issues/1113
I don’t know why you are bitching about rules and frustration.
I believe the best approach would be to have these multireddits automatically created for convenience. However, users should have the option to choose whether they want to see only the content from their instance’s community or from any number of communities, instead of being forced to view all of them.
I think that approach is needlessly complicated and would confuse people more than help them. There should be a way for individual users to merge the feeds from multiple communities in a multilemmy if they specifically choose to.
But I don’t see any way to create such multireddits/multilemmys automatically because there is no objective categorization. Everyone is going to have a different opinion about which communities fit together and which ones are similar but different enough to keep separate. Instead of forcing a generalist solution that makes everyone unhappy, just give people the tools to build their own solution that works best for them.
I would prefer something pre-made for convenience but that can be modified by each user to adjust to their preference. I’d rather have a generalist solution forced on me than have to spend countless hours grouping communities from hundreds of instances.
I think you’re vastly overestimating how many communities actually matter. At least 90% of communities will be ghost towns. It’s just too early to tell right now because the platform is basically in its first week of existence. You’re planning a solution to a problem that won’t exist.
I get it, some of the first comments I made when I joined were about how to combine communities across multiple instances. As I’ve spent more time here, I’ve begun to understand that it’s not actually a problem. The easy solution is to subscribe to the biggest community and/or one on your home instance.
Besides that, I don’t see much of a difference between manually going through hundreds of communities to add the ones you want, or going through hundreds of communities on a pre made list to remove the ones you don’t want.
Or just make it user-side. Let users create their own feed combinations. They’d still have to select a specific instance for posts.
Feeds would be consolidated but posts and comments will still be federated. And one user will be unaffected by how another user organizes their feed.
That seems like a good solution. Let me subscribe to a half dozen different game comms and put them together into one “games” list that shows up with pretty much the same interface as a single community, so I can browse just “games” content that I have subscribed to.
having subscribed to multiple games communities, the issue then becomes content duplication; the same trailer or article will get posted in three different communities, and I don’t actually want to see it three times in my feed. I’m not sure there’s a good way to solve that, though.
(I’m subscribed to multiple communities b/c I’m not sure which one will have the largest comments sections, and those are what I’m really interested in.)
Later on there could potentially be an “exclude duplicate links” option which will automatically filter out the duplicate posts which contain the exactly the same link and show only the post with the highest interaction count. Unsure if that would be valuable enough to implement as a feature though.
The issue I then see with that is well, which duplicate instance gets to be the Highlander? 😋 Does the system automatically show you the post with the most comments? The one that’s had its link clicked most? The one from the largest community?
Also, how do you prevent this from being hijacked? Say I post some bright, shiny new OC meme, only to have some bot immediately repost it with exactly the same title, etc…
Not that I have any solutions for any of this, mind…
Yeah these are all great points. It would definitely need to be workshopped to see if there are solutions to some of this or if other approaches are better.
My first thought would be that the top duplicate in the feed would be the one that would be shown (so based on up/down votes I guess). The others would be lower in the list anyway so you would only see those duplicates after the top one if they were there. Simply filtering them out and maybe having a way to show a list of duplicates on the visible one might be good. Then you could choose the comment section of the smaller duplicates of you want to read more comments or whatever. Still lots of unknowns but I feel like it could possibly work.
Of course, it’s a whole new platform, there’s bound to be all sorts of things that need ironing out. Tbh I just think it’s a good sign already that these are things that are being discussed (or are open for discussion at all, for that matter) and not just unilaterally decided.
Also, I hope my comment didn’t come across as being disparaging or anything… All my criticism was intended to be constructive, I promise
I personally like the duplicates because different communities have different comments.
I like the idea that tags could facilitate a sort of one-to-many and many-to-one relationship structure, where a pickle community could say it is related to “Canning” and “Cucumbers” among others, and potentially populate in the feed of someone interested in prepper stuff or in the feed of someone interested in vegetable gardening. I’m sure I’m not using the DB terms correctly but they feel indicative of the modular structure this could allow.
I like this flexibility better than the idea of a consolidation of subs into one themed silo, but I could be misunderstanding the structure of the mulit-reddit proposal if it allows for this.
Yes the fragmentation is a feature not a bug. There are dozens of reasons why people might want to leave a community and create their own alternative version. With blackjack and hookers.
Combining communities should be a front end feature… Allow users to merge their views if they want. But it should not be enforced at the backend or federation level.
Eventually there will be third party apps which can do this merging in their interface if someone wants it.
This seems like a devisive topic.
The fragmentation is not a feature to me, but something I view as detrimental.
Combining communities should be a front end feature… Allow users to merge their views if they want. But it should not be enforced at the backend or federation level.
Eventually there will be third party apps which can do this merging in their interface if someone wants it.
I agree with this. The grouping should be a front-end feature based on hashtags, as someone else mentioned, instead of the community names. Alternatively, there could be lists that you can simply copy and paste to create your own multireddit, eliminating the need for hashtags. However, considering that the original issue was already on the lemmy-ui, I’m not sure why you brought up the backend.
One thing I do agree with is that discoverability should be better. Can’t remember the url now but there’s a site which crawls all instances and creates a searchable list of all communities. This should seriously be built in. Have all instances generate a list of all communities which is broadcast, cached by all, and searchable by all.
This “paste a url of a community you already found into the search box” concept is not good enough.
Yeah, couldn’t agree more.
Cross-Instance Automatic Multireddit feature within Lemmy.Sounds great. It would be nice to see similar communities under a single heading.deleted by creator
I got a few ideas:
-
Let people make their own multi-communities, then publish them. The multis would get a separate category in search.
-
Let community moderators decide to join a list. For example:
- community@a and community@b exist.
- mod@a starts multi-community@a
- mod@b requests for community@b to join that multi
- mod@a accepts
- multi-community@a is now a shared feed of both communities.
-
I don’t see any problem. If the problem is the server admin is not able to moderate the communities, he can disable automatic community creation and define an approval process. He can also close dead communities, apoint moderators, merge proposals…
I read on another post in a different community that some servers have neo-nazis running them? If that is the case, no thanks, I don’t want that.
They don’t care what you want…
That’s easy to block those individual instances if needed (assuming they exist).
It’s possible, it’s one of the downsides of being federated. The instances are moderated independendently, there is no central authority that enforces law or morals. It’s great to avoid things like what happened to Reddit, but stuff like that can and will show up. You can always block instances to avoid seeing them, if you want.
Hmm, I don’t disagree with the fragmentation but that’s the nature of any new social platform. It’s also been proven out that eventually one or two communities for a topic will become the dominant one with the others falling into disuse.
Attempting to merge communities early or artificially will cause moderator strife as minor disagreements balloon. Especially in a multireddit community where no one mod(team) has absolute control.
I don’t have a reason from a technical point of view, but from a social one. Forcing communities and instances together early will only cause strife. After a few years where two communities have a track record and proven ‘behavior’ would the multireddit not cause issue.
Every moderator would have control only over the content displayed on their instances, and not on everyone else, as it should be. The argument about having one or two large communities is a recurring one. There is no reason to have federation if we are going to centralize communities in a couple of instances. Then, if one of those instances shuts down, everyone in those communities would have to migrate. The main benefit of federation is decentralization.
