bug: Sharkey is not fully functioning within rootless podman

This happens cause it tries to access https://testing.coolwebsite.zone/files/8e193f52-14ba-4b06-b637-2aeaa8e6ebab to download one of the images in your exported file but since it doesn't exist/the server isn't there it fails with "CONNREFUSED" and errors out.

There is a try/catch but the queue processor generally just seems to cancel the queue items if one goes wrong.

Created by: ecco8443

The file it's referring to is actually the notes.json file uploaded for the import, not an image referenced in it. I can see the file in the local /files volume, and the server is definitely running. Is it somehow able to upload but not download files from itself?

It did work the one time with sudo, and images were successfully downloaded from the remote instance as well.

Created by: ecco8443

I've "solved" my problem, so I'm closing the issue, but I am honestly just as confused as I already was.

Thanks to the help in the Discord server, I was pointed to my firewall, which I hadn't really investigated before. I am using ufw as my firewall. From inside the Sharkey container, using wget to attempt to download the file failed with "connection refused", but using wget to retrieve a file from any other website did work. I had already tried disabling the firewall entirely to no avail, but after having this pointed out to me in the Discord, I investigated further and have come to two conclusions. I hope that for all three of the other people out there trying to run something like Sharkey in rootless podman, this might be helpful.

1: This is certainly a firewall issue, nothing to do with Sharkey (or even podman). After much pain trying to dig useful results out of Google, I edited /etc/ufw/sysctl.conf to uncomment these lines:

net/ipv4/ip_forward=1
net/ipv6/conf/default/forwarding=1
net/ipv6/conf/all/forwarding=1

Then I added this block to the bottom of the /etc/ufw/before.rules and /etc/ufw/before6.rules files, below the COMMIT line:

*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080
-A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 8443
-A PREROUTING -p udp --dport 443 -j REDIRECT --to-port 8443
-A OUTPUT -p tcp --dport 80 -j DNAT --to-destination :8080
-A OUTPUT -p tcp --dport 443 -j DNAT --to-destination :8443
-A OUTPUT -p udp --dport 443 -j DNAT --to-destination :8443
COMMIT

Because I had Caddy listening on its default ports, but the container bound to 8080 and 8443 due to rootless podman not having the ability to bind to ports below 1024, I needed to port forward, but for whatever reason, the redirect wasn't enough, so the DNAT lines were added, and this did allow wget to successfully retrieve the file from my server... but it no longer worked on any remote IP. The post import did go through and successfully imported all posts, but the images all failed to download.

This is way above my pay grade, so I never found a way around this, nor did I form an understanding of the problem or precisely what my "solution" did. Instead, I went back to brute force, trying every option to see what would happen. For some reason, changing the network driver to slirp4netns caused the error to change slightly... and after struggling with search results for a while again, I decided to try undoing the changes made to the ufw files, then setting net.ipv4.ip_unprivileged_port_start to 0 and binding Caddy directly to ports 80 and 443, and it worked perfectly. But I had forgotten to change the network driver back to pasta...

Well...

2: It turns out that pasta was the problem the whole time. I don't know how, I don't know why, I don't know where to even begin trying to figure out what the problem is, but it simply did not want to play ball I suppose. That's why entirely disabling the firewall also did not help when I tried it, because the firewall wasn't the problem, it was however pasta is interacting with the network.

The Caddy container (which is on its own, not part of any pod) is still running pasta, all my other containers and pods are running pasta, but for whatever reason, if a service needs to be able to go through the webserver to access its own files from the server's public IP rather than directly accessing its storage volume, pasta is out, but slirp4netns works. I didn't even need to add port_handler=slirp4netns to be able to see the correct IP addresses of visitors at the expense of throughput, because Caddy is using pasta and correctly forwards those addresses.

If anybody knows how to explain this, or how to properly set iptables rules to work the way I wanted to use unprivileged ports, please do chime in so that this problem may never happen again.

bug: Sharkey is not fully functioning within rootless podman

💡 Summary

🥰 Expected Behavior

🤬 Actual Behavior

📝 Steps to Reproduce

💻 Frontend Environment

🛰 Backend Environment (for server admin)

Do you want to address this bug yourself?

Designs

Child items 0

Activity