V

  • 3 Posts
  • 26 Comments
Joined 1 year ago
cake
Cake day: June 21st, 2023

help-circle

  • I could be misinformed, but this isn’t just limited to Spark as I understand it, I believe a lot (maybe all?) third-party clients do the same thing. They act as an intermediary between you and the server so they can deliver push notifications.

    However, as I understand it, Spark’s privacy policy outlines that they don’t read/scan the contents of your emails, and the use of app-specific passwords rather than your email password ensures they only have access to emails and nothing else.

    Pretty sure others such as Canary, Airmail, Edison, etc. all do/did the same thing, but it was the lack of clarity in Spark’s privacy policy that made them the main target for scrutiny. I think they’ve since cleared that up.

    I could be mistaken, though.



  • Before putting Pi-hole behind Traefik, it worked perfectly via :/admin. And the logs for Pi-hole now in Traefik show that it is up and working, and I get the login page. But just can’t get beyond it.

    The guides I’ve seen show how to structure the Traefik labels with and without the addprefix middleware, and both apparently work. So I’m wondering if by following several guides and taking bits from each, I’ve ended up overlooking something.

    I’ll try and expose 80 and see if it makes a difference, but like I say everything is up and running in the backend, I just can’t get past the login screen on the frontend.




  • Just a quick update on where I’m up to…

    I’ve managed to get all my containers working behind the Traefik reverse proxy with SSL. I’ve also deployed a Cloudflare DDNS container in Docker and have linked the external IP address of my Synology NAS to Cloudflare. I haven’t port forwarded 80 and 443, though, so it’s not accessible over the internet. So I’ve added local DNS into Pi-hole so I can access all the containers using subdomains.

    I’ve also deployed an Authelia container and have started running through my containers adding 2FA in front of them all.

    I should probably point out at this juncture, that if I encounter any errors, the HTTP 404 page that I get is a Cloudflare one - I assume that’s expected behaviour?

    So, the final three bits I’m struggling with now are:

    • Pi-hole behind the reverse proxy
    • Portainer behind the reverse proxy
    • Accessing Vaultwarden over the internet (because as soon as I leave my house, if the vault hasn’t synced then I don’t have access to all my passwords) - unless anybody has a better suggestion?

    Portainer - I have no idea how I do it, because I use it to manage my containers, so don’t have the config for Portainer in Portainer (obviously). So if I screw up the config, how am I getting back in to Portainer to fix it?

    And the far more troubling one is Pi-hole. I just cannot get that thing working behind the reverse proxy.

    I’ve followed a few different guides (though none of them are recent), and the below is the latest docker-compose I have. It will bring up the login page, but when I login it keeps returning me back to the login page - it won’t go to the main admin page.

    version: "3.7"
    
    services:
      pihole:
        container_name: pihole
        image: pihole/pihole:latest
        restart: unless-stopped
        networks:
          - medianet
          - npm_network
        ports:
          - 8008:80
          - 53:53/tcp
          - 53:53/udp
        environment:
          - TZ=Europe/London
          - WEBPASSWORD=xxxxxxxxxx
          - FTLCONF_LOCAL_IPV4=192.168.1.116
          - WEBTHEME=default-auto
          - DNSMASQ_LISTENING=ALL
          - VIRTUAL_HOST=pihole.mydomain.com
        volumes:
          - /path/to/pihole:/etc/pihole
          - /path/to/pihole/dnsmasq.d:/etc/dnsmasq.d
        cap_add:
          - NET_ADMIN
        labels:
          - traefik.enable=true
          - traefik.http.routers.pihole.entrypoints=http
          - traefik.http.routers.pihole.rule=Host(`pihole.mydomain.com`)
          - traefik.http.middlewares.pihole-https-redirect.redirectscheme.scheme=https
          - traefik.http.routers.pihole.middlewares=pihole-https-redirect
          - traefik.http.middlewares.pihole-addprefix.addprefix.prefix=/admin
          - traefik.http.routers.pihole.middlewares=pihole-addprefix
          - traefik.http.routers.pihole-secure.entrypoints=https
          - traefik.http.routers.pihole-secure.rule=Host(`pihole.mydomain.com`)
          - traefik.http.routers.pihole-secure.tls=true
          - traefik.http.routers.pihole-secure.service=pihole
          - traefik.http.services.pihole.loadbalancer.server.port=80
    
    networks:
      medianet:
        external: true
      npm_network:
        external: true
    

  • Update from this morning.

    So far I’ve got the Traefik dashboard and my Homepage container using subdomains. Pi-hole is still an issue that I need to figure out.

    I’ve decided to start on the SSL certificates and am following a couple of guides. Once I have those in place, I’ll start moving more containers behind subdomains.

    I might have to expose my NAS IP to the internet and link it via Cloudflare, because I use an ExpressVPN on my Mac at all times, and when it’s turned on I can’t access ***any ***of my subdomains - this is obviously because ExpressVPN use their own DNS entries and don’t use the ones I’ve set. That will probably prevent me from using Vaultwarden (which is the whole purpose for all of this in the first place) because if I’m on the VPN I won’t be able to access the Vaultwarden container.

    Unless anyone knows of a workaround for that?

    Next steps:

    • Get SSL working
    • Figure out how to access subdomains whilst on the VPN (or get a DDNS account, create an A record in Cloudflare and point it at the DDNS IP, and open up ports 80 and 443)
    • Spin up a Vaultwarden container via a subdomain
    • Put all my other services behind subdomains
    • Figure out how to get Pi-hole working via Traefik and subdomain
    • Figure out how to get Tailscale access to my containers when not on my LAN

  • Figured it out. It’s a weird setting on Netgear routers whereby you have to also update the MAC address. All been working well for the last few hours and getting queries running through Pi-hole.

    I’ve also got my Homepage container setup at http://home.mydomain.com and configured Traefik a little further so it’s now accessible from http://traefik.mydomain.com (no port).

    For the past few hours I’ve been struggling with getting Pi-hole behind Traefik and accessible using http://pihole.mydomain.com. Only works if I stick /admin on the end, which defeats the object of using a subdomain. Found a forum post suggesting to use Traefik’s addPrefix after declaring the Host as pihole.mydomain.com, which works great for accessing the login screen, but when you enter the password it just loops back to the login screen.

    Also tried a few other things that ultimately broke the Pi-hole container and took out my entire connection, as everything is dependent on Pi-hole for DNS! So need to figure out some kind of resiliency/backup for that (my router is using the NAS IP as it’s primary and only DNS server).

    So, some progress. I’ve set Pi-hope back to IP:port and I’m gonna focus on getting other containers behind Traefik and leave Pi-hole till last. Then and only then will I look at SSL certificates (unless it’s advised to do it earlier?)

    Any pointers on any of the above would be appreciated! And thanks again for getting me this far.





  • OK so made a start with this. Spun up a Pi-hole container, added mydomain.com as an A record in Local DNS, and created a CNAME for traefik.mydomain.com to point to mydomain.com.

    In Cloudflare, I removed the mydomain.com A record and the www CNAME record.

    Doing an nslookup on mydomain.com I get

    Non-authoritative answer:
    *** Can't find mydomain.com: No answer
    

    Which I guess is to be expected.

    However, when I then navigate to http://traefik.mydomain.com in my browser, I’m met with a Cloudflare error page: https://imgur.com/XhKOywo.

    Below is the docker-compose of my traefik container:

    traefik:
        container_name: traefik
        image: traefik:latest
        restart: unless-stopped
        networks:
          - medianet
        ports:
          - 80:80
          - 443:443
        expose:
          - 8080
        volumes:
          - /etc/localtime:/etc/localtime:ro
          - /var/run/docker.sock:/var/run/docker.sock:ro
          - /volume1/docker/traefik:/etc/traefik
          - /volume1/docker/traefik/access.log:/logs/access.log
          - /volume1/docker/traefik/traefik.log:/logs/traefik.log
          - /volume1/docker/traefik/acme/acme.json:/acme.json
        environment:
          - TZ=Europe/London
        labels:
          - traefik.enable=true
          - traefik.http.routers.traefik.rule=Host(`$TRAEFIK_DASHBOARD_HOST`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
          - traefik.http.routers.traefik.service=api@internal
          - traefik.http.routers.traefik.entrypoints=traefik
    

    My traefik.yml is also nice and basic at this point:

    global:
      sendAnonymousUsage: false
    
    entryPoints:
      web:
        address: ":80"
      traefik:
        address: "8080"
    
    api:
      dashboard: true
      insecure: true
    
    providers:
      docker:
        endpoint: "unix:///var/run/docker.sock"
        watch: true
        exposedByDefault: false
    
    log:
      filePath: traefik.log
      level: DEBUG
    
    accessLog:
      filePath: access.log
      bufferingSize: 100
    

    Any ideas what’s going wrong? I’m unclear on why the domain is still routing to Cloudflare.


  • Absolute superstar, thanks for your help so far. I’ll make a start on some of this tomorrow and see how far I get — either with Traefik or NPM.

    Do I need to do anything with the domain itself on Cloudflare at the moment? Or do I just leave it with its current A record pointing at an IP address (it was done as part of the setup in Cloudflare so I have no idea what that IP address is).

    Obviously that domain in reality will just sit there doing nothing.


  • I don’t plan on exposing any of this stuff to anybody other than me. I do plan on spinning up SearX but it’ll only be me using it. I’ve given up trying to convince my family to move away from Google to even DuckDuckGo or Startpage, so there’s no way I’ll convince them to use SearX!

    I think, therefore, for accessing away from home I’ll perhaps setup a subdomain that points to the IP of my Tailscale container — that means it’ll be accessible externally but only when I turn on the VPN.

    When I’m on my home network I have a VPN on my Mac anyway.



  • Thanks. I realise they’re all pretty basic questions. But brace yourself: more are on their way!

    So… no, I don’t want to give external access - I’m not running any services that anyone would want/need access to - other than perhaps my Jellyfin server, but not sure I even want anyone accessing that. So let’s assume for right now, no access to the outside world. Therefore, no port forwarding required.

    So to get access to my internal network from the domain, do I simply setup local DNS records in something like Pi-hole, to point mydomain.com to the internal IP or my NAS? Kind of like a network-wide equivalent of modding the /etc/hosts file on my machine?

    Perhaps a(nother) silly question but, what’s to stop me doing that now with a completely random domain name? Is there some kind of authentication I’d need to go through to prove that mydomain.com is, in fact, mine? Or does it simply not matter since it’s internal only?

    If I’ve understood correctly, then, I don’t need Cloudflare at all in my setup if there’s no external access? Nothing to proxy, nothing to protect?

    Assuming I get all of the above working and traffic routing to my containers, how would I then go about setting up SSL? Can that be done through Traefik rather than Cloudflare? Even if the domain isn’t external?


  • Thanks. There’s definitely stuff in here I want to do, I just need to figure out the order of play and break it down a bit.

    As per reply to another comment.

    Do I have to port forward 80 and 443 no matter what? Ideally I don’t want to forward anything.

    Do I need DDNS in here somewhere, i.e. create a DDNS and link it to my NAS, create an A record in Cloudflare to point my domain to the external IP of the DDNS? Is that how I get into my NAS from the domain without worrying about the IP changing? How do I then prevent anybody accessing the NAS admin on port 5000/5001, as well as anything else except the containers I expose via Traefik?


  • Thanks, I’d like to know more about how to go about this approach.

    I guess in my head, I want to achieve the following (however I go about it):

    • Access https://mydomain.com from outside my network and hit some kind of blank page that wouldn’t necessarily suggest to the public that anything exists here
    • Access https://mydomain.com from inside my network and hit a login page of some kind (Authelia or otherwise), to then gain access to the Homepage container running in Docker (essentially a dashboard to all my services)
    • Access https://secure.mydomain.com from outside my network and route through to the same as above, only this would be via the Tailscale IP address/container running on my stack to allow for remote access
    • Route all HTTP requests to HTTPS
    • Use the added protection that Cloudflare brings (orange clouds where possible)
    • SSL certificates for all services
    • Ability to turn up extra Docker containers and auto-obtain SSL certs for them Ensure that everything else on my NAS and network is secure/inaccessible other than the services I expose through Traefik.

    I have no idea where Cloudflare factors in (if at all), nor how Pi-hole factors in (if at all).

    Internal stuff I’ve been absolutely fine with. Stick a domain name, a reverse proxy and DNS in front of me and it’s like I’m learning how to code a Hello World app all over again.



  • Thanks. Yep, subdomains was what I’d planned on: traefik.mydomain.com to access the Traefik dashboard; home.mydomain.com to access the Homepage container. I was planning on spinning up an Authelia container as well to provide 2FA for the services I want protecting. I guess it’d also be nice to have some kind of landing page for traffic coming directly to www.mydomain.com or mydomain.com as well.

    Ideally I don’t want to port forward, so would I need to rely on Traefik to redirect the traffic from port 80 to port 443, and then proxy from port 443 to the required container? How do I therefore stop traffic from hitting the DSM admin on ports 5000/5001 for example?

    I need to figure out a starting point to get traffic from my domain into my NAS (safely) then start spinning up containers and have Traefik route them appropriately, then I can look at Pi-hole/local DNS and Tailscale. And then I guess SSL.