There are many ways to write a (RESTful) backend in Go. Most of the available tutorials are way too simple, with all the presented content fitting into a single file (or at most two-three). More complex examples are quite rare, and even most of them miss lots of things for the sake of reducing complexity. That’s one of the reasons I wrote Gorsk - to have a fully functional example of a RESTful backend (in Golang) utilizing best practices, idiomatic code, and minimal dependencies.
Gorsk V2 is released - read more about it
Before switching to Go a year ago, my job was building (RESTful) enterprise backend application by using Java, Spring, and SpringBoot. Even though it has many drawbacks compared to Go (which I might talk about in a future blog-post), there was definitely one advantage. The way you write and structure your application, read config files, inject dependencies is mostly predefined by Java, Spring, and SpringBoot. You don’t have to think about those, and other libraries can leverage this (such as Swagger, working without any annotations or modifications, unlike in Go).
On the other hand, in Go, you can write your apps in many different ways. Could you imagine a non-trivial Java app having a flat structure (single root folder), which is not an uncommon thing in Go?
Therefore package design in Go is very opinionated. Ignoring the flat structure for the sake of sanity, there are many opinions on how to structure your Go app:
-
MVC (Model-view-controller) oriented design
-
Many other, less known package designs
Besides package design (in Go), there are other challenges when writing an application from scratch. Most newcomers will search for things like:
-
How to write SQL queries and tests for it
-
How to write http middleware
-
How to validate requests
-
How to handle sessions
-
How to do proper logging
… You get the point. The idea behind gorsk was to give everyone an idea how you can do those using Go. Not mandatory that way, but once you see how it can be done, it is much easier to come up with your own solution.
Introducing Gorsk
Instead of spending time wiring up your project, thinking how and where to place HTTP handlers, how to inject dependencies, test your application services and database using table tests and mocking, handle sessions and more - use Gorsk as a base foundation for your application and start adding business logic, or get learn from it and copy only what you need/like to your application. That was the primary idea behind this project.
Ideas and motivation
I’ve always wanted to make a bigger OSS project, and I still do. My previous projects are all below 1k lines of code, which I consider to be quite small. Although gorsk is not a big project per se, it’s still my largest (open source) project so far (although I would like to make a useful library next, not an example repo).
Besides contributing to the open source community, I wanted to improve my skills to be able to build a web app from scratch using Go, as I get these requests here and there.
By spending last two months (of my free time devoted to programming) building gorsk I’ve learned plenty of new concepts in Go, web programming and generally sharpened my skills.
Building the project
Since we use similar package design at our work, I started working from that.
Using standard library I made a few handlers with Gorilla mux. I didn’t like how it looked - manually setting HTTP status to header and encoding response in JSON every time. I started searching for minimalistic Go web frameworks (not to be mistaken with monoliths such as Spring, Go frameworks like Gin are usually around few thousand lines of code) and settled with Gin (for now, will probably rewrite it in Echo - reasons mentioned in GitHub issue). Altough in Go world it is prefered to make most things using standard library only, so in this case net/http with a small wrapper arround it.
Gin claims to have outstanding performance, easily binds JSON, URL and Query params, validates the requests and responds in a single line. It features a nice router with grouping options, easily pluggable middleware and a powerful context.
Since I didn’t like the error handling (where I manually had to write status codes), I made a small function and custom error type for having proper HTTP status codes/messages.
Rest of the project went pretty straightforward, except for handling database queries.
First, I started with Gorm, as I’ve heard of it even before moving to Go. Since we’re using Datastore at work, so I was somehow familiar with ORM syntax in Go.
It can automatically migrate schema, handles created_at/updated_at/deleted_at by default and has a well-written documentation.
Once I needed to write a single join query, I felt stuck for few days. I’m pretty sure gorm is able to do this, but I didn’t manage to figure it out from the documentation and I moved on to sqlx.
Sqlx felt way too verbose. Apart from the way it handles embedded types, all my nullable fields needed to be sql.Null types (and then later checked whether they contain data or not to be properly marshaled into JSON), writing joins felt more complex than it needed to be and complex queries were a nightmare to write. I understand the desire not to use an ORM, but given the current state of Go SQL libraries, I’d rather use something not so verbose for my personal projects. This is still one of the things where Java comes as an obvious winner with JDBC, wrappers around it like JDBCTemplate, MyBatis around it and ORMs as Hibernate.
I found about go-pg, and after skimming through the docs I gave it a try. What I liked about it is how it handles null types, embedded types, and ease of writing simple joins. Although it does have some drawbacks compared to Gorm, I settled on it - for now. My opinion might change in the future.
Another thing that cost me way more time than planned is writing tests. Since I did lots of design changes at the beginning, I decided to write tests once I’m done with bigger changes, although it goes against TDD. Writing tests helped me see some bad design choices and fix few small bugs.
As the project grew I had more and more ideas on how to improve it, but since I wanted to publish it and this blog post, I’ll do the changes after posting it, getting feedback from others as well. Most of the improvements/issues are opened as issues on project’s GitHub repository.
Project Structure
The project structure is explained in details in THIS blog post, with some small changes taken from Package Oriented Design. and GoDD. Primarily the package design is architectured by tonto with some small changes done by me.
-
Root package contains things not related to code directly, e.g. docker-compose, CI/CD, readme, bash scripts etc. It should also contain vendor folder, Gopkg.toml and Gopkg.lock if dep is being used.
-
Main package ties together dependencies. Located in cmd/api/server.go, the main package instantiates routing, all configurations, connections etc and injects them as dependencies to services. Gorsk is structured as a monolith application but can be easily restructured to contain multiple microservices. An application may produce multiple binaries, therefore Gorsk uses the Go convention of placing main package as a subdirectory of the cmd package. As an example, scheduler application’s binary would be located under cmd/cron.
-
Rest of the code is located under /internal. Internal root contains domain types, e.g. User, Car, Company. This package only contains simple data types like User struct for holding user data or a UserService interface for fetching or saving user data.
-
Domain packages are located under the internal directory, in folders named after the domain. For example, car package is located in internal/car. All application/business logic is found here, and it connects to other ‘platforms’ such as a database, reporting, indexing, etc.
-
Platform folder contains various packages that provide support for things like databases, authentication or even marshaling. Most of the packages located under platform are decoupled by using interfaces. Every platform has its own package, for example, postgres, elastic, redis, memcache etc.
-
Mock package is shared across the entire application. Specific platforms have subpackages such as mockdb, mockindex etc. Since most of the dependencies are injected, it is trivial to mock them and pass the mock service as an argument.
-
Service package contains HTTP handlers. They only receive the requests, marshal and validate them using a helper package (request) and pass it to the corresponding services.
-
MW Package contains middleware related implementation, such as JWT authentication, CORS, perhaps request/error logging, security etc.
-
Config package contains application configurations, as well as the implementation to read the config files.
-
Swagger is a helper package containing go-swagger annotations for SwaggerUI generation.
Further steps
I don’t see myself as a good software developer. And that’s one of the reasons I love the open source community.
Developers better than me can easily notice drawbacks of this code, run into bugs by using etc. I’m eagerly waiting for well-versed criticism, in order to further sharpen my coding skills, improve this and the projects I contribute to.