My recent projects

In my previous blog post I state, that I started programming in Go (or Golang). Since then, five month have passed by and I haven’t really written anything. But I haven’t been lazy, though. This post is meant to tell you a bit about my recent projects, that I’ve been working on.

Go is amazing! #

First of all a quick note on Go. I really fell in love with this language. The speed, the versatility, the ease of coding and the simple structure and concepts amaze me. I believe I’ve never written so much code in one go, as I have since I started learning Go - as well as for my private projects as well as for my professional business applications. I can’t, of course, disclose the details about my business applications, so I will spare them out in this post.

SoTBot - a Sea of Thieves themed Discord bot #

In April, Simon convinced me to start playing Sea of Thieves. I immediately fell in love and binge-played it for the first weeks.

It’s a pirate-themed sandbox game in which you set sail on the “Sea of Thieves”, run on adventures and voyages (PvE) and fight against other players for treasure and booty (PvP). The goal is to turn in these treasures in return of gold. This gold can then be used to buy cosmetics for your pirate, your ship or your pets. If you haven’t played it, give it a try!

While Simon and I were playing and talking about it on our Discord server, we jokingly thought about a Discord bot, that would monitor your SoT session and check your gold balance before and after the game. At the end, you should be provided with a “voyage summary”. Well, a couple of days later, the first version of “SoTBot” arrived in our Discord server and reported for service.

Even though SoT provided some API endpoints that allows us to fetch some user and account information, the API endpoints are not public. You have to log in with your MS Live account. The authentication is cookie based and the cookie is valid for only 14 days. Since the bot needs this cookie, to access the API, it has to be scraped from the SoT website. The first releases used Simon’s selenium-based docker image that scraped the site. Later I built a new version in NodeJS, using the Pupetteer framework but due to some useablity issue, we finally went with a Chrome extension.

Within the next weeks, the bot received lots of new features (check the documentation for all of them) and is now a vital part of our Discord server. If you are interested in the bot, you can find it here: https://github.com/wneessen/sotbot. The docker image with the scraper can be found here: https://github.com/wneessen/sotbot-token-extrator and finally the Chrome extension can be found here: https://github.com/wneessen/sotbot-rat-extension/tree/main/Chrome .

go-pf - a FreeBSD pf (packet filter) wrapper #

For a different project, I needed to automate firewall rules on a FreeBSD box. Since I was already working in Go for that project, I decided to write a wrapper around the FreeBSD pf (packet filter) interface. It allows to manage firewall rules on the fly, wrapping around the pfctl command. So far I’ve only covered the functions I really needed but it might grow over the next months. The code can be found on Github: https://github.com/wneessen/go-pf.

go-fileperm - check file permissions based on the current user in Go #

In the context of go-pf I also was in need of an easy way to check if a file is has specific permissions for the current user (similar to test -e/-r-/w in the Shell). Unfortunately there is no straightforward way to do so in Go. Therefore I’ve also created a module for that. It can be found on Github, as well: https://github.com/wneessen/go-fileperm

Static page generation with (Hu)Go #

For over 5 yeats, I am maintaining the website of a non-profit organisation (in my hometown). When I took over this task from the previous webmaster, the page was a collection of manually generated and maintained HTML pages. Since I didn’t want to manually write HTML on each page for every change that needed to be done, I decided to rebuild the whole site in Wordpress. Even though this worked pretty well, over the years I started to dislike Wordpress. Also with all the needed plugins, the site became horribly slow (4-5 seconds per requests - I’ve never really figured out which plugin caused this slowness). So some weeks ago I decided to get rid of the Wordpress.

I first wanted to go with a Jekyll based site (like this one). But I honestly never really enjoyed working with Jekyll. It works for me and especially for a little site like this, it is a good fit, but I wasn’t sure if it would also fit for a site like the non-profit website (200+ pages and posts). After some research, I quickly found Hugo. A static-page generator, similar to Jekyll but written in Go. Well, that was of course worth a try.

After the initial learning curve, I quickly got used to Hugo and really enjoyed working with it. Meanwhile I’ve migrated the old wordpress to Hugo completely and I am more than happy with the results. Working with Hugo is such a charme. It’s easy and quick as Go itself. Really, really cool! I am already considering, if I rebuild this page in Hugo as well… we’ll see.

js-mailer - a form mailer webservice for static websites #

One thing I couldn’t easily accomplish with Hugo was a form mailer. With Wordpress, since it’s PHP under the hood on the server side, this is pretty easy. But with Hugo, the pages are static HTML. So I had to use JavaScript and some kind of form mailer to send contact form requests. There are 3rd party services like https://formspree.io/ that are specialized on this, but since I don’t wanna invest money just to send forms and I also don’t trust the data privacy of such services, I decided to write my own little web service for it.

js-mailer is a very simple, yet powerful, web service (of course written in Go) that allows static websites to send forms via JavaScript’s fetch() or XMLHttpRequest APIs.

It follows a two-step workflow. First your JavaScript requests a token from the API using the /api/v1/token endpoint. If the request is valid and website is authorized to request a token, the API will respond with a TokenResponseJson. This holds some data, which needs to be included into your form as hidden inputs. It will also provide a submission URL endpoint /api/v1/send/<formid>/<token> that can be used as action in your form. Once the form is submitted, the API will then validate that all submitted data is correct and submit the form data to the configured recipients.

What started as a quick and dirty “I want to send HTML forms as mail” service, quickly transformed into a fully flegged web service. At the time of this post, the service supports the following features:

• Single-binary webservice
• Multi-form support
• Multiple recipients per form
• Only display form-fields that are configured in for the form in the resulting mail
• Check for required form fields
• Anti-SPAM functionality via built-in, auto-expiring and single-use security token feature
• Anti-SPAM functionality via honeypot fields
• Limit form access to specific domains
• Per-form mail server configuration
• hCaptcha support
• reCaptcha v2 support
• Form field type validation (text, email, number, bool)

The code is, you already guessed it, on Github: https://github.com/wneessen/js-mailer.

I’ve also dockerized it within Github, so you can just download and start the image: https://github.com/wneessen/js-mailer/pkgs/container/js-mailer