P2P options

[recently_folded]

This thread is about the possibility of using distributed networks/P2P for fannish content. This something that cesperanza and lim (sorry, I have no idea now to make the fancy tumblr links) have been posting about on tumblr and sharing thoughts here, including in this comment and several that follow on from it.

From the master thread, three informational links:
https://webtorrent.io/
https://www.npmjs.com/package/dat-http
https://docs.datproject.org/http#serve-over-http

Comments

[prettyarbitrary]

My first concern with using P2P for a site like this would be the legality. In the case of a site that runs off a set of dedicated servers, that site is, first and foremost, beholden to the laws of the nation that hosts the machines. In the case of a P2P network-supported site, the last time I checked, individual users would be beholden to the laws of the place where they live--thus possibly preventing people from using the site because, for example, if they were in Russia they'd be breaking the law as 'servers' of LGBT content.

Now, that was a few years ago that I last looked into that, and the international quilt of internet policy has gotten a lot...weirder since then. But it's an angle we'd need to look into.

[oulfis]

I think this is an important aspect to think about -- gosh, I wish I could talk to like, five lawyers. But I think at least for, e.g., Russia, it would be sustainable for those users not to seed, and therefore not to permanently host/serve the material? I think (???) that simply viewing something through a P2P network, even though it does of course have to temporarily put that file on your computer, it would do so the same way viewing a website normally does.

When it comes to varying copyright laws, I think it gets trickier -- we can sustain, I think, pockets of lurkers-who-don't-seed, but we can't sustain "all of fandom is fragmented into country-specific walled gardens." Like, I, personally, live in two different countries and need to be able to Fandom from both. ...I started baselessly speculating about how this might be made to work, and then instead I went to go see if I am friends-of-friends with any lawyers. This feels solvable, if we can just get someone with the right expertise to be clever about it!

[teyla]

I think (???) that simply viewing something through a P2P network, even though it does of course have to temporarily put that file on your computer, it would do so the same way viewing a website normally does.

I'm by no means an expert (I literally started playing around with this stuff today), but the datproject that's linked above, and that Lim's been talking about, works by allowing you to download content onto your computer and thereby becoming a host in the P2P network. I think you can stop hosting as soon as you've finished the download, but while you're downloading (and whenever you're syncing your finished download to get new content), you're also hosting.

You'd definitely have to host if you want to your own content on the network. Once it's distributed far enough, you might be able to turn your original seed off, but initially you'd be the only one who can get this content out into the network at all. So fan creators in countries with very restrictive laws would definitely need a way to host anonymously.

[cesperanza]

Right, I think it's totally solvable; its all much more granular than the old bittorrent sites.

[pseudonym_loving_magpie]

You could have something that's administratively like a mastodon server, where the server owner can delete things from it if legally required, and process DMCA notifications, but part of the bandwidth for serving files from it would be provided by its members on a p2p basis. Since the server is a larger target and does DMCA stuff, the users are less likely to get in trouble (I'm not a lawyer and I don't know how much this would help). The members' computers serving files on behalf of the server would stop if the server blocked/deleted the file.

To avoid being a walled garden the servers could federate with servers in other countries, as they do with mastodon, but not serve files on behalf of the other server (the users can connect to the other servers directly)

People who don't want to have their posts controlled by someone else's server could run their own servers, so the server software should be made as easy to run as possible, and also there should be a way to back up your posts without running a server.

[demitas]

I've been investigating some possibilities on the tech end of this. As an experiment, I have a Scuttlebutt (https://www.scuttlebutt.nz) pub set up to play with (ping me for an invite, I have to set them up manually at the moment), so that we don't all have to be in person to connect. There was some talk of trying to get embedded links - so you could host something on a distributed network, and then embed the dat from a normal site that would use a homebase link (basically caches the dat on the server). (I'm not sure if this tech exists somewhere yet or if we're going to have to write it.)

I'm a programmer but this is well outside my usual purview and into sysadmin territory so far, so I'm a little out of my depth but trying to learn. :)

[oulfis]

I asked cesperanza to repost to DW (for posterity) a tumblr post that has particularly stuck in my head and makes me really excited about P2P for, at least, Image And Video Hosting Of Our Own. This is the quote I can't get out of my head:

it’s honestly whiplash to go from distributed computing and decentralised web stuff where they are wading through tools and software and approaches and ideas but don’t have the networks (the largest p2p search engine has like, 600 peer operators, approximately the size of an abandoned tumblr about the sock choices of Bucky Barnes) and fandom where the #1 concern is how to cope with the avalanche of content and people and sheer BYTES and ACTIVITY we produce. It’s like, fandom is constantly making these posts going ‘but what are we going to do with all this GOLD? We just can’t DEAL with this amount of GOLD. THIS GOLD IS A GODDAMN ENVIRONMENTAL HAZARD.’

(that quote is actually lim, whoops, sorry, not sure why I'd originally stored it in my memory as cesperanza!)

But: I consider myself a small-potatoes fic writer. Frankly, I don't even really consider myself a fic writer; I just read a lot and sometimes a fic falls out. But even I have fics with more than 600 views. 600 is NOTHING.

I was starting to get intrigued by P2P before lim blogged about it, since I started hearing people talk about scuttlebutt in the nerd-hippie circles I run in (my other secret life is a commune of gay engineers). We spent a whole evening shooting the shit about how nice it would be if the internet became the kind of internet that coolguy.website talks about. And then a week later lim blogged about Beaker and I went home and said: "Remember the dream internet? Fandom is moving there, so it's going to be real. Pack your bags." I expected it to be a gradual cultural shift, but maybe the time is ripe.

The thing that AO3 hasn't been able to do is image and video hosting, because it's just SO EXPENSIVE to host stuff that SO MANY PEOPLE WANT. But peer-to-peer hosting has the reverse problems of traditional problems: it's harder to host things that don't get viewed much. But I could easily host my own small-potatoes rarepair stuff, and if I suddenly got popular by definition at the point when others' demand to see my stuff exceeded my ability to easily host it myself, there would be enough people also around to help shoulder that burden.

I think the technology is still too far behind for the Social Network Of Our Own to make sense on something P2P, but I feel like if the octopus keeps growing tentacles, a P2P Hosting Of Our Own could connect with a Social Network Of Our Own via a clever apothecary chest of holding/ Open Linked Data Of Our Own, and that's the internet I want to live on.

[oulfis]

Oh, I should link The Future Will Be Technical because that's been on my mind a lot too!

[cesperanza]

Just FYI, I went to a lot of open source video conferences in like, IDK, 2009? 2011? trying to think of ways to get video in with the OTW, and I saw a lot of this technology in very very nascent forms. The people building this had the total opposite problem than us--they were hosting small video projects and wanted to get eyeballs/viewers. So that's why actually many of the systems work really fine for the small user, because they're thinking really pretty small p2p. Fandom had the opposite problem: you post a huge Juggernaut Ship Vid and crash all the servers as 10,000,000 people watch it, and also you go bankrupt. That sort of stuff becomes much much EASIER with p2p. What we need now is sort of features: streaming, embeds, interoperability, ease of setup, friendly user face. But we have the network and we have the work.

I totally imagine a future where fans are hosting all their own art and video work on dat sites and that stuff is being kept up by networks of fans who "seed" as well as "reblog" and "like" works that they love/the works of their friends. But everything is kept up collectively by fandom, large and small. Then these works are debuted/posted by embedding them within, say, a federated social networking system, a feed, and fans also store copies of their works in a deposit library archive owned by the OTW, which stores copies but doesn't serve them. That would make me very happy indeed; much redundancy, no corporate overlords, collective.

[jesse_the_k]

which stores copies but doesn't serve them.

LIKE AN ARCHIVE! Yes!

[snowgrouse]

I made the mistake of replying on the original post, so I'll just copypaste my original here. TL;DR: P2P is great, but if your country/law enforcement (and I'm not talking China/Iran; I'm talking Scandinavian and other European countries) think they're all evil, you're fucked.

***

I love, love, love the idea of the apothecary chest--don't get me wrong here--but as I was reading it, I was basically thinking "but this is how P2P works. This is how torrents work". And while I love them, that's a huge problem from the legal POV, considering how they are seen as very very shady by law enforcement all over the world already. And .dat domains are already being used for illegal purposes, so that's Darknet as far as the governments are concerned, whether we like it or not. I live in a country (Finland) where all the major ISPs throttle torrents like crazy, so I don't have high hopes of *anything* decentralised not being seen as legally dodgy, because it already is used for that on such a vast scale (and that sucks for us, because I really do love the decentralised idea :/). Let's assume that accessing P2P/Darknet-type places isn't illegal per se (technically, it isn't)--but if the cops come after you, perhaps on suspicion of something you haven't even done (say, a stalker trying to frame you for something, which does happen), the fact that your computer is linked up to stuff like that is not going to look good in court.

Obviously, that's a problem we'll always run into if we want to be free--with freedom will always come the people who abuse it. But when it's something that's already universally seen as dodgy by the law enforcement, the way P2P is, it's going to be much trickier for us to prove we aren't criminals, if that's the route we take. And I really wish it wasn't that way; I like torrents (apart from the fact that stuff can vanish on there, too, if there's nobody to seed that documentary on Afghan camel-drivers in 1800s Australia that you're desperate to see, and apart from the rampant sexism considering it's very much an alt-right heavy dudespace). I really would love to have that apothecary chest, I really would--but it has to be carefully thought out.
***

To add to that: I can see how I could use, say, a vpn or Tor to access those places, but if it requires extra software/tech knowledge/money/is legally dodgy, it's going to leave a lot of tech-clueless people (I'm assuming a majority of the US teenage girls who now use Tumblr) out. They can't even do torrents or know what a .rar file is and are baffled when I tell them about the days when we used to upload Who episodes to LJ comms. I'm not joking. I share public domain films all the time, and have to break down the most basic technical stuff to these younger fans over and over. So the final user interface would have to be very, very accessible even to the most tech-unsavvy grandma to work for most fans, I expect. And again, using a technology that doesn't already have a reputation so bad that it's already being blocked or at least monitored by law enforcement even in less police-state-y countries.

[probablyintraffic]

Many firewalls, especially university firewalls will block P2P by default as well.

Compounding the user experience issue is the fact that there would still have to be a central sort of search engine for the blogs which might or might not have to change its domain name once every three months, which is like you said sort of dodgy

My feeling is that an inclusive solution will not be decentralized.

[snowgrouse]

I don't know--technology changes fast, so you never know. But right now, decentralised/P2P already has such a bad rep from the past 20 years of its use having been primarily for dodgy reasons (or at least the public seeing it that way), which is my biggest concern. And I don't mean that in a "I would never use torrents!!1" sort of way, but I have friends who are terrified of downloading anything, even those copyright-free movies I've upped. And as I've bemoaned several times on the sites that my ISP caps, I can't afford a VPN and that would put me in a legally risky position. Imagine an anti or a stalker or a batshit ex trying to frame another fan for criminal activity (which does happen) and then the cops bust the fan but find nothing illegal per se--but they see that she's been using networks that are used for illegal filesharing. Dun dun duuun... This is the same reason that's often cited when fans and/or BDSM practitioners want anonymity--that their activities may be used against them in custody fights and such. So this is like a governmental/law enforcement version of the Apple store censorship of Tumblr--they want good, clean law-abiding citizens (snort) to not have any access to something that just might be illegal and then ban the whole medium/technology for parts of its content. Apple bans the Tumblr app because 20% of it is porn, governments put caps and surveillance on P2P/DAT sites because it's got kiddie porn and pirated stuff and drug deals on it. Ho hum. It ain't easy to start that decentralised anarcho-communist fan syndicate when you're next door to pedos...

[probablyintraffic]

Yes. I think some of these problems we have like anti censorship and sexually explicit content of minors will be exacerbated, not ameliorated by moving to P2P. Ultimately these persistent issues will have to have political solutions, not technical ones.

[recently_folded]

Good point. Sadly, given the current state of politics, I suspect that this will be even more unfavorable to fandom. I think we can productively spend some effort at least trying to put into place measures to reduce the kind of stuff that gets fandom thrown out with the anti-porn bathwater.

[demitas]

I actually didn't think of the apothecary chest of holding as P2P at all - if anything it would be a more centralized place to gather links. I was thinking it would just be set up in a database with literally just a collection of links to existing sites, descriptions, and tags - so just a text structure and some code that would go down the list of links looking for the first link and trying to return that to the caller. The biggest problem I can see with it is the hotlinking issue. It seems really very do-able dev-wise.

I totally can see your concern about P2P, but I think the apothecary chest wouldn't run into that issue.

[snowgrouse]

To be fair, I was replying to the conversation at a point where people had started gushing about P2P, and some of the ideas re: the chest were just very similar to how P2P works. Hence my concern. It could be done on the Web, sure.

[prettyarbitrary]

I think this is an important point. I specifically brought up P2P because of the potential issues with it. But the distributed/federated internet is a separate thing. They CAN work together, but they don't HAVE to. You can have one without the other. We CAN build a federated fannish social platform without going through P2P (and in fact it may even already exist, waiting for us to go madass and set up instances of it everywhere), although the potential for your Juggernaut Fanvid raining down 10,000,000 calls for bandwidth on your poor desktop PC-hosted basement web server... Someone must've thought of that issue, right? But my brain is already full of information it's sorting through desperately, so the non-P2P solution, or where to look for it, isn't coming to me right away.

[recently_folded]

Dropping a cross-ref here to
Fandom gets closer to the metal - issue 1: Hubzilla and pixelfed

[allochthon]

P2P, specifically, is also a very effective way to spread viruses and other infections.

The two most effective ways around that are
1) Only download from trusted users.
1a) and how many sites have had their user databases hacked in the last 5 years? This is not a reliable method.
1b) I, in a past life, was a cybersecurity professional, and just this week I clicked on a link on tumblr that infected me. No one is immune, all the time.

2) Checksum the files with a cryptographic hash.
2a) No one actually checks the hash.
2b) For those who do check the hash, the *software checking the hash* has to be unhacked. How do you guarantee that? You run it off of read-only media, like a dedicated USB key. Who is willing to do that?

caveat: I haven't looked into the current cutting-edge P2P, but this was the state the last time I did, a few years ago.

I think these things are probably solvable, but be aware these issues exist.

(I am all for a [social] network of our own. Please believe me. I don't mean to be a downer!)

[recently_folded]

No—not a downer. We need to look at potential pitfalls even as we're gathering up positive options. Security is a quagmire these days, so it's an area that we really need to be as forward-looking on as the issues surrounding potential abuse.