• 0 Posts
  • 21 Comments
Joined 2 years ago
cake
Cake day: June 7th, 2023

help-circle


  • Along with the things others have said (Backups, Linux, Docker, Networking) I’d also recommend getting comfortable with server and network security. A lot of this is wrapped up in the simple mantra “install your goddamn updates!” But, there is more to it than that. For example, if you go with Nextcloud, read through their hardening guide and seriously consider implementing all of the recommendation. Also think through how you intend to manage both the server and instance. If this is all local, then it is easier as you can keep SSH access to the server firewalled off from the internet. If you host part of your stuff “in the cloud”, you’ll want to start looking at limiting down access and using keys to login (which is good practice for all situations). Also, never use default credentials. You may also want to familiarize yourself with the logs provided by the applications and maybe setup some monitoring around them. I personally run Nextcloud and I feed all my logs into Splunk (you can run a free instance in a docker container). I have a number of dashboards I look at every morning to keep an eye on things. E.g. Failed/successful logins, traffic sources, URI requests, file access, etc. If your server is attached to the internet it will be under attack constantly. Fail2Ban on my wireguard container banned 112 IP addresses over the last 24 hours, for 3 failed attempts to login via SSH. Less commonly, attackers try to log in to my Nextcloud instance. And my WordPress site is under constant attack. If you choose to run Wordpress, be very careful about the plugins you choose to install, and then keep them up to date. Wordpress itself is reasonably secure, the plugins are a shit-show and worse when they aren’t kept up to date.




  • I see containers as having a couple of advantages:

    1. Separation of dependencies - while not as big of issue as it used to be, just knowing that you won’t end up with the requirements for one application conflicting with another is one less issue to worry about. Additionally, you can do anything you want to one container, without having an effect on another container. You don’t get stuck wanting to reboot or revert the system, but not wanting to break a different running service.
    2. Portability - Eventually, you are going to replace the OS of that VM (at least, you should). Moving a container to a new OS is dead simple. Re-installing an application on a new OS, moving data and configs can be anywhere from easy to a pain in the arse, depending on the software.
    3. Easier fall back - Have you ever upgraded an application and had everything go to shit? In my years working as a sysadmin, I lost way too many evenings to this sort of bullshit. And while VM snapshots should make reverting easy, sometimes it just didn’t work out that way. Containers force enough separation of applications that you can do just about anything to one container and not effect others.
    4. Less dependency on a single install - Have you ever had a system just get FUBAR, and after a few hours of digging the answer seems to be, just format the drive and start over? Maybe you tried some weird application out and the uninstall wasn’t really clean. By having all that crap happen in containers, you can isolate the damage. Nuke the container, nuke the image, and the base OS is still clean.
    5. Easier version testing - Want to try out upgrading to version 2 of an application, but worried that it may not be fully baked yet or the new configs may take a while to get right? Do it off in a separate container on a copy of the data. You can do this with VMs and snapshots; but, I find containers to be less overhead.

    That all said, if an application does not have an official container image, the added complexity of creating and maintaining your own image can be a significant downside. One of my use cases for containers is running game servers (e.g. Valheim). There isn’t an official image; so, I had to roll my own. The effort to set this up isn’t zero and, when trying to sort out an image for a new game, it does take me a while before I can start playing. And those images need to be updated when a new version of the game releases. Technically, you can update a running container in a lot of cases; but, I usually end up rebuilding it at some point anyway.

    I’d also note that, careful use of VMs and snapshots can replicate or mitigate most of the advantages I listed. I’ve done both (decade and a half as a sysadmin). But, part of that “careful use” usually meant spinning up a new VM for each application. Putting multiple applications on the same OS install was usually asking for trouble. Eventually, one of the applications would get borked and having the flexibility to just nuke the whole install saved a lot of time and effort. Going with containers removed the need to nuke the OS along with the application to get a similar effect.

    At the end of the day, though. It’s your box, you do what you are most comfortable with and want to support. If that’s a monolithic install, then go for it. While I, or other might find containers a better answer for us, maybe it isn’t for you.


  • My list of items I look for:

    • A docker image is available. Not some sort of make or build script which make gods know what changes to my system, even if the end result is a docker image. Just have a docker image out on Dockerhub or a Dockerfile as part of the project. A docker-compose.yaml file is a nice bonus.
    • Two factor auth. I understand this is hard, but if you are actually building something you want people to seriously use, it needs to be seriously secured. Bonus points for working with my YubiKey.
    • Good authentication logging. I may be an outlier on this one, but I actually look at the audit logs for my services. Having a log of authentication activity (successes and failures) is important to me. I use both fail2ban to block off IPs which get up to any fuckery and I manually blackhole entire ASNs when it seems they are sourcing a lot of attacks. Give me timestamps (in ISO8601 format, all other formats are wrong), IP address, username, success or failure (as a independent field, not buried in a message or other string) and any client information you can (e.g. User-Agent strings).
    • Good error logging. Look, I kinda suck, I’m gonna break stuff. When I do, it’s nice to have solid logging giving me an idea of what I broke and to provide a standardized error code to search on. It also means that, when I give up and post it as an issue to your github page, I can provide you with some useful context.

    As for that hackernews response, I’d categorically disagree with most of it.

    An app, self-contained, (essentially) a single file with minimal dependencies.

    Ya…no. Complex stuff is complex. And a lot of good stuff is complex. My main, self-hosted app is NextCloud. Trying to run that as some monolithic app would be brain-dead stupid. Just for the sake of maintainability, it is going to need to be a fairly sprawling list of files and folders. And it’s going to be dependent on some sort of web server software. And that is a very good place to NOT roll your own. Good web server software is hard, secure web server software is damn near impossible. Let the large projects (Apache/Nginx) handle that bit for you.

    Not something so complex that it requires docker.

    “Requires docker” may be a bit much. But, there is a reason people like to containerize stuff, it avoids a lot of problems. And supporting whatever random setup people have just sucks. I can understand just putting a project out as a container and telling people to fuck off with their magical snowflake setup. There is a reason flatpak is gaining popularity.
    Honestly, I see docker as a way to reduce complexity in my setup. I don’t have to worry about dependencies or having the right version of some library on my OS. I don’t worry about different apps needing different versions of the same library. I don’t need to maintain different virtual python environments for different apps. The containers “just work”. Hell, I regularly dockerize dedicated game servers just for my wife and I to play on.

    Not something that requires you to install a separate database.

    Oh goodie, let’s all create our own database formats and re-learn the lessons of the '90s about how hard databases actually are! No really, fuck off with that noise. If your app needs a small database backend, maybe try SQLite. But, some things just need a real database. And as with web servers, rolling your own is usually a bad plan.

    Not something that depends on redis and other external services.

    Again, sometimes you just need to have certain functionality and there is no point re-inventing the wheel every time. Breaking those discrete things out into other microservices can make sense. Sure, this means you are now beholden to everything that other service does; but, your app will never be an island. You are always going to be using libraries that other people wrote. Just try to avoid too much sprawl. Every dependency you spin up means your users are now maintaining an extra application. And you should probably build a bit of checking into your app to ensure that those dependencies are in sync. It really sucks to upgrade a service and have it fail, only to discover that one of it’s dependencies needed to be upgraded manually first, and now the whole thing is corrupt and needs to be restored from backup. Yes, users should read the release notes, they never do.
    The corollary here is to be careful about setting your users up for a supply chain attack. Every dependency or external library you add is one more place for your application to be attacked. And just because the actual vulnerability is in SomeCoolLib.js, it’s still your app getting hacked. You chose that library, you’re now beholden to everything it gets wrong.

    At the end of it all, I’d say the best app to write is the one you are interested in writing. The internet is littered with lots of good intentions and interesting starts. There is a lot less software which is actually feature complete and useful. If you lose interest, because you are so busy trying to please a whole bunch of idiots on the other side of the internet, you will never actually release anything. You do you, and fuck all the haters. If what you put out is interesting and useful, us users will show up and figure out how to use it. We’ll also bitch and moan, no matter how great your app is. It’s what users do. Do listen, feedback is useful. But, also remember that opinions are like assholes: everyone has one, and most of them stink.


  • BLUF: It’s been a mixed bag, but I would call it “worth it”.

    I’ve used Ubuntu a bit before. That’s what my home server runs on and has for years. Granted, most of it’s functions live in Docker containers. I also used both Debian (via Kali) and Ubuntu at work (yes, I know Ubuntu is Debian based, but it’s also big enough to have it’s own dedicated ecosystem). I work in Cybersecurity and use Linux based tools for image acquisition, digital forensics and data recovery. Kali makes for a great “it just works” system to validate vulnerabilities and poke at a network. And, between a lot of tools targeting Ubuntu and frameworks like SANS SIFT, Ubuntu gets used a lot. I also supported several Red Hat based servers at work for various tools. I’m far from an expert on Linux, but I can usually hold my own.

    In a lot of ways, Arch wasn’t an obvious choice for me. And I seriously considered going with Ubuntu (or another Debian based OS (e.g. PopOS)) at first. It’s worth mentioning that my primary use for my desktop is video games. So, that heavily effected my choices. That said, the reasons for choosing Arch ended up being:

    1. I have a SteamDeck and most of my games “just work” on it. With Arch being the flavor of Linux Valve is targeting, following their lead seemed like a good idea. I expected that a lot of effort to get games working on “Linux” would ultimately be focused on getting games working on Arch.
    2. I wanted a “minimal” system. I can be a bit of a control freak and privacy nut. I already self-host NextCloud, because I don’t want my pictures/data sitting on someone else’s computer. So, the “install only what you need” nature of Arch was appealing.
    3. I did do some testing of Ubuntu on my system and had driver issues (nVidia GPU) and some other problems I didn’t put the time into running down. In the end, it put me off Linux for a while before I came back to it and ran Arch.

    One of the things I did, which was really helpful, was a “try before you buy” setup. I was coming from Windows 10. And, as mentioned above, gaming was my main use case. So, that had to work for me to make the jump. Otherwise, I was going to milk Windows 10 for as long as possible and then figure things out when it went EOS. So, I installed Arch on a USB 3.0 thumbdrive and left my Windows OS partition alone. I also mounted my “Games” drive (M.2 SSD) and installed games to that. It was still NTFS, but that only created minor bumps in the road. Running that configuration for a couple months proved out that Arch was going to work for me.

    When it came time to fully change over, I formatted my Windows OS partition as ext4, setup the correct folder structure and rsync’d everything from the thumbdrive to it. So, everything was the way I’d had it for those couple months. I did have an issue that my BIOS refused to see the OS partition on the SATA SSD I used for my OS partition; but, that was MSI’s fault (I have an MSI motherboard). And that was resolved by changing where GRUB is located in my /boot partition.

    Overall, I’ve been happy with the choice I made. Arch hasn’t always been easy. Even the Official Install Guide seems to come from a RTFM perspective. But, if you’re willing to put the time into it, you will learn a lot or you won’t have a functional system. And you’ll end up with a system where you can fire up a packet capture and have a really good idea of what each and every packet is about. As for gaming, so far I’ve had exactly one game which didn’t run on Linux. That was Call of Duty 6, which I was considering giving a go to play with some folks I know. But, Activision’s Anti-Cheat software is a hard “no” on Linux. So, I had to pass on that. Otherwise, every game I have wanted to play either had native Linux support or worked via Proton/WINE.





  • Playtron’s CEO, Kirt McMaster, added, “We are thrilled to join forces with such a legendary games publisher as Square Enix. The PC we know is morphing into new forms that require a purpose built OS for gaming that meets the demands of powerful new gaming hardware such as handhelds and new players who have grown up in a mobile 1st world with much more sophisticated UX sensibilities. This investment will accelerate the development and deployment of GameOS and create new experiences for players around the globe.”

    Huh, so Playton’s CEO is an early example of an AI being used to replace people.


  • Have you considered just beige boxing a server yourself? My home server is a mini-ITX board from Asus running a Core i5, 32GB of RAM and a stack of SATA HDDs all stuffed in a smaller case. Nothing fancy, just hardware picked to fulfill my needs.

    Limiting yourself to bespoke systems means limiting yourself to what someone else wanted to build. The main downside to building it yourself is ensuring hardware comparability with the OS/software you want to run. If you are willing to take that on, you can tailor your server to just what you want.



  • No, but you are the target of bots scanning for known exploits. The time between an exploit being announced and threat actors adding it to commodity bot kits is incredibly short these days. I work in Incident Response and seeing wp-content in the URL of an attack is nearly a daily occurrence. Sure, for whatever random software you have running on your normal PC, it’s probably less of an issue. Once you open a system up to the internet and constant scanning and attack by commodity malware, falling out of date quickly opens your system to exploit.


  • Short answer: yes, you can self-host on any computer connected to your network.

    Longer answer:
    You can, but this is probably not the best way to go about things. The first thing to consider is what you are actually hosting. If you are talking about a website, this means that you are running some sort of web server software 24x7 on your main PC. This will be eating up resources (CPU cycles, RAM) which you may want to dedicated to other processes (e.g. gaming). Also, anything you do on that PC may have a negative impact on the server software you are hosting. Reboot and your server software is now offline. Install something new and you might have a conflict bringing your server software down. Lastly, if your website ever gets hacked, then your main PC also just got hacked, and your life may really suck. This is why you often see things like Raspberry Pis being used for self-hosting. It moves the server software on to separate hardware which can be updated/maintained outside a PC which is used for other purposes. And it gives any attacker on that box one more step to cross before owning your main PC. Granted, it’s a small step, but the goal there is to slow them down as much as possible.

    That said, the process is generally straight forward. Though, there will be some variations depending on what you are hosting (e.g. webserver, nextcloud, plex, etc.) And, your ISP can throw a massive monkey wrench in the whole thing, if they use CG-NAT. I would also warn you that, once you have a presence on the internet, you will need to consider the security implications to whatever it is you are hosting. With the most important security recommendation being “install your updates”. And not just OS updates, but keeping all software up to date. And, if you host WordPress, you need to stay on top of plugin and theme updates as well. In short, if it’s running on your system, it needs to stay up to date.

    The process generally looks something like:

    • Install your updates.
    • Install the server software.
    • Apply updates to the software (the installer may be an outdated version).
    • Apply security hardening based on guides from the software vendor.
    • Configure your firewall to forward the required ports (and only the required ports) from the WAN side to the server.
    • Figure out your external IP address.
    • Try accessing the service from the outside.

    Optionally, you may want to consider using a Dynamic DNS service (DDNS) (e.g. noip.com) to make reaching your server easier. But, this is technically optional, if you’re willing to just use an IP address and manually update things on the fly.

    Good luck, and in case I didn’t mention it, install your updates.




  • One of the more recent updates added in a lot of new Points of Interest and helped make the random maps less “WTF” and more reasonable. Though the replayability is down to personal preference. My wife and I will spin up a new world, play until we reach a point of “well, we’ve basically done everything” and then move on to a different game. And then come back to 7d2d when we want to scratch that “crafting, zombie survival, base defense” itch.

    In a lot of ways, 7d2d is like a comfy sweater. It’s not going to ever be radically different; but, it’s nice to wear for a bit. Personally, I’d be all for getting the gang back together. Unless you are all so saturated in other games to play, why not relive the fun? I’m also a sucker for survival/crafting games and have a Steam library full of them. There are only a couple which I keep going back to, Valheim and 7d2d being the top two. So, YMMV, but I’d say it’s a fun enough game and the official release is a good excuse to kill some zombies together again.